[Caml-list] Unit testing Core Async

[Caml-list] Unit testing Core Async

[Kenneth Adam Miller]

[-- Attachment #1: Type: text/plain, Size: 1983 bytes --]

I've noticed that Core Async requites that a Scheduler.go () call be
placed-but that never returns. I have a Tcp.server that I'm creating, and I
like to use oUnit for my tests. Monads and all are beautiful, and Core is a
wonderful library, but I'm adamant that I have at least some minimal
functionality testing complete that demonstrates proper behavior as well as
intended usage.

What I'm wondering is the following: would there be a way to have the
scheduler.go call be placed in order to fire things off, but in another
thread have all the test code be dependent on the server's responses and
all of that, so that once completed, it can call Shutdown.shutdown?

I tried this out, and it introduced some issues.

First, I think that my shutdown call got executed before the unit test was
able to complete. This is because using Async's Deferred introduces some
complication if you want behavior to proceed sequentially as in without
building deeply nested callback chains. What I'm used to is asynchronous
send, and blocking receive that operates on a common execution chain. I
don't see any kind of Deferred.await that blocks until the instance
resolves (yes, there's upon, but that's just nesting again because it
returns another deferred.

Second, I think shutdown shuts *everything* down. What I need is just to
signal the completion of the job that was supposed to run, so that the
Scheduler.go returns in order to allow my unit tests to run to completion.

Third, I'm not certain about the semantics of Pipe/Reader/Writer. It's not
behaviorally like what I'm familiar with. For instance, callbacks may
return prematurely and only have part of a message. In ZMQ, what you send
is what you get. So that makes me concerned in regards to the Tcp.Server,
because right now what I need is for the Pipe to just allow blocking
receive so that I can make the threads coordinated, but I need the Tcp
Server to allow me to receive whole protobuf messages.

Can anyone please help me?

[-- Attachment #2: Type: text/html, Size: 2205 bytes --]

Re: [Caml-list] Unit testing Core Async

[Francois Berenger]

There is a google group for Janestreet Core:

https://groups.google.com/forum/#!forum/ocaml-core

You might get more luck in there and some Async specialists.

On 06/15/2015 06:33 PM, Kenneth Adam Miller wrote:
> I've noticed that Core Async requites that a Scheduler.go () call be
> placed-but that never returns. I have a Tcp.server that I'm creating,
> and I like to use oUnit for my tests. Monads and all are beautiful, and
> Core is a wonderful library, but I'm adamant that I have at least some
> minimal functionality testing complete that demonstrates proper behavior
> as well as intended usage.
>
> What I'm wondering is the following: would there be a way to have the
> scheduler.go call be placed in order to fire things off, but in another
> thread have all the test code be dependent on the server's responses and
> all of that, so that once completed, it can call Shutdown.shutdown?
>
> I tried this out, and it introduced some issues.
>
> First, I think that my shutdown call got executed before the unit test
> was able to complete. This is because using Async's Deferred introduces
> some complication if you want behavior to proceed sequentially as in
> without building deeply nested callback chains. What I'm used to is
> asynchronous send, and blocking receive that operates on a common
> execution chain. I don't see any kind of Deferred.await that blocks
> until the instance resolves (yes, there's upon, but that's just nesting
> again because it returns another deferred.
>
> Second, I think shutdown shuts *everything* down. What I need is just to
> signal the completion of the job that was supposed to run, so that the
> Scheduler.go returns in order to allow my unit tests to run to completion.
>
> Third, I'm not certain about the semantics of Pipe/Reader/Writer. It's
> not behaviorally like what I'm familiar with. For instance, callbacks
> may return prematurely and only have part of a message. In ZMQ, what you
> send is what you get. So that makes me concerned in regards to the
> Tcp.Server, because right now what I need is for the Pipe to just allow
> blocking receive so that I can make the threads coordinated, but I need
> the Tcp Server to allow me to receive whole protobuf messages.
>
> Can anyone please help me?

-- 
Regards,
Francois.
"When in doubt, use more types"

Re: [Caml-list] Unit testing Core Async

[Carl Eastlund]

[-- Attachment #1: Type: text/plain, Size: 3121 bytes --]

Internally at Jane Street -- and this may show up in some of the publicly
released code, but off the top of my head, I don't know what file to point
you at -- we run some of our async unit tests using
[Thread_safe.block_on_async_exn].  That will spin up the scheduler if
necessary, run the function you give it, block until the deferred is
determined, then return.  It does not shut down the async scheduler; we
generally don't do that until the program is done, so we would leave the
scheduler up from one test to another.  I don't know the entire rationale
behind this design, there may be a way to shut down the scheduler in
between tests, but in general it does not appear to be necessary.

As for partial reads, if you're concerned with receiving whole messages, I
think [Reader.read_one_chunk_at_a_time] can do what you need -- if you get
too little, just return [`Consumed (n_already_consumed, `Need
n_total_bytes_to_proceed)].

I hope this helps!

On Mon, Jun 15, 2015 at 12:33 PM, Kenneth Adam Miller <
kennethadammiller@gmail.com> wrote:

> I've noticed that Core Async requites that a Scheduler.go () call be
> placed-but that never returns. I have a Tcp.server that I'm creating, and I
> like to use oUnit for my tests. Monads and all are beautiful, and Core is a
> wonderful library, but I'm adamant that I have at least some minimal
> functionality testing complete that demonstrates proper behavior as well as
> intended usage.
>
> What I'm wondering is the following: would there be a way to have the
> scheduler.go call be placed in order to fire things off, but in another
> thread have all the test code be dependent on the server's responses and
> all of that, so that once completed, it can call Shutdown.shutdown?
>
> I tried this out, and it introduced some issues.
>
> First, I think that my shutdown call got executed before the unit test was
> able to complete. This is because using Async's Deferred introduces some
> complication if you want behavior to proceed sequentially as in without
> building deeply nested callback chains. What I'm used to is asynchronous
> send, and blocking receive that operates on a common execution chain. I
> don't see any kind of Deferred.await that blocks until the instance
> resolves (yes, there's upon, but that's just nesting again because it
> returns another deferred.
>
> Second, I think shutdown shuts *everything* down. What I need is just to
> signal the completion of the job that was supposed to run, so that the
> Scheduler.go returns in order to allow my unit tests to run to completion.
>
> Third, I'm not certain about the semantics of Pipe/Reader/Writer. It's not
> behaviorally like what I'm familiar with. For instance, callbacks may
> return prematurely and only have part of a message. In ZMQ, what you send
> is what you get. So that makes me concerned in regards to the Tcp.Server,
> because right now what I need is for the Pipe to just allow blocking
> receive so that I can make the threads coordinated, but I need the Tcp
> Server to allow me to receive whole protobuf messages.
>
> Can anyone please help me?
>



-- 
Carl Eastlund

[-- Attachment #2: Type: text/html, Size: 3746 bytes --]

Re: [Caml-list] Unit testing Core Async

[Kenneth Adam Miller]

[-- Attachment #1: Type: text/plain, Size: 3506 bytes --]

That helps-I was literally looking up a way to running unit tests on that
google group. I think that group helped a lot, I'll try and read what I can
there before I blast the list (I didn't know about it before hand).

On Mon, Jun 15, 2015 at 12:53 PM, Carl Eastlund <ceastlund@janestreet.com>
wrote:

> Internally at Jane Street -- and this may show up in some of the publicly
> released code, but off the top of my head, I don't know what file to point
> you at -- we run some of our async unit tests using
> [Thread_safe.block_on_async_exn].  That will spin up the scheduler if
> necessary, run the function you give it, block until the deferred is
> determined, then return.  It does not shut down the async scheduler; we
> generally don't do that until the program is done, so we would leave the
> scheduler up from one test to another.  I don't know the entire rationale
> behind this design, there may be a way to shut down the scheduler in
> between tests, but in general it does not appear to be necessary.
>
> As for partial reads, if you're concerned with receiving whole messages, I
> think [Reader.read_one_chunk_at_a_time] can do what you need -- if you get
> too little, just return [`Consumed (n_already_consumed, `Need
> n_total_bytes_to_proceed)].
>
> I hope this helps!
>
> On Mon, Jun 15, 2015 at 12:33 PM, Kenneth Adam Miller <
> kennethadammiller@gmail.com> wrote:
>
>> I've noticed that Core Async requites that a Scheduler.go () call be
>> placed-but that never returns. I have a Tcp.server that I'm creating, and I
>> like to use oUnit for my tests. Monads and all are beautiful, and Core is a
>> wonderful library, but I'm adamant that I have at least some minimal
>> functionality testing complete that demonstrates proper behavior as well as
>> intended usage.
>>
>> What I'm wondering is the following: would there be a way to have the
>> scheduler.go call be placed in order to fire things off, but in another
>> thread have all the test code be dependent on the server's responses and
>> all of that, so that once completed, it can call Shutdown.shutdown?
>>
>> I tried this out, and it introduced some issues.
>>
>> First, I think that my shutdown call got executed before the unit test
>> was able to complete. This is because using Async's Deferred introduces
>> some complication if you want behavior to proceed sequentially as in
>> without building deeply nested callback chains. What I'm used to is
>> asynchronous send, and blocking receive that operates on a common execution
>> chain. I don't see any kind of Deferred.await that blocks until the
>> instance resolves (yes, there's upon, but that's just nesting again because
>> it returns another deferred.
>>
>> Second, I think shutdown shuts *everything* down. What I need is just to
>> signal the completion of the job that was supposed to run, so that the
>> Scheduler.go returns in order to allow my unit tests to run to completion.
>>
>> Third, I'm not certain about the semantics of Pipe/Reader/Writer. It's
>> not behaviorally like what I'm familiar with. For instance, callbacks may
>> return prematurely and only have part of a message. In ZMQ, what you send
>> is what you get. So that makes me concerned in regards to the Tcp.Server,
>> because right now what I need is for the Pipe to just allow blocking
>> receive so that I can make the threads coordinated, but I need the Tcp
>> Server to allow me to receive whole protobuf messages.
>>
>> Can anyone please help me?
>>
>
>
>
> --
> Carl Eastlund
>

[-- Attachment #2: Type: text/html, Size: 4421 bytes --]

Re: [Caml-list] Unit testing Core Async

[Kenneth Adam Miller]

[-- Attachment #1: Type: text/plain, Size: 3931 bytes --]

Oh, as far as shutting the scheduler down, I just wanted the scheduler to
return from the go (); I need my unit tests to execute until completion but
afterward just finish. I think the solution you pointed me to was exactly
what I was looking for. :)

On Mon, Jun 15, 2015 at 12:56 PM, Kenneth Adam Miller <
kennethadammiller@gmail.com> wrote:

> That helps-I was literally looking up a way to running unit tests on that
> google group. I think that group helped a lot, I'll try and read what I can
> there before I blast the list (I didn't know about it before hand).
>
> On Mon, Jun 15, 2015 at 12:53 PM, Carl Eastlund <ceastlund@janestreet.com>
> wrote:
>
>> Internally at Jane Street -- and this may show up in some of the publicly
>> released code, but off the top of my head, I don't know what file to point
>> you at -- we run some of our async unit tests using
>> [Thread_safe.block_on_async_exn].  That will spin up the scheduler if
>> necessary, run the function you give it, block until the deferred is
>> determined, then return.  It does not shut down the async scheduler; we
>> generally don't do that until the program is done, so we would leave the
>> scheduler up from one test to another.  I don't know the entire rationale
>> behind this design, there may be a way to shut down the scheduler in
>> between tests, but in general it does not appear to be necessary.
>>
>> As for partial reads, if you're concerned with receiving whole messages,
>> I think [Reader.read_one_chunk_at_a_time] can do what you need -- if you
>> get too little, just return [`Consumed (n_already_consumed, `Need
>> n_total_bytes_to_proceed)].
>>
>> I hope this helps!
>>
>> On Mon, Jun 15, 2015 at 12:33 PM, Kenneth Adam Miller <
>> kennethadammiller@gmail.com> wrote:
>>
>>> I've noticed that Core Async requites that a Scheduler.go () call be
>>> placed-but that never returns. I have a Tcp.server that I'm creating, and I
>>> like to use oUnit for my tests. Monads and all are beautiful, and Core is a
>>> wonderful library, but I'm adamant that I have at least some minimal
>>> functionality testing complete that demonstrates proper behavior as well as
>>> intended usage.
>>>
>>> What I'm wondering is the following: would there be a way to have the
>>> scheduler.go call be placed in order to fire things off, but in another
>>> thread have all the test code be dependent on the server's responses and
>>> all of that, so that once completed, it can call Shutdown.shutdown?
>>>
>>> I tried this out, and it introduced some issues.
>>>
>>> First, I think that my shutdown call got executed before the unit test
>>> was able to complete. This is because using Async's Deferred introduces
>>> some complication if you want behavior to proceed sequentially as in
>>> without building deeply nested callback chains. What I'm used to is
>>> asynchronous send, and blocking receive that operates on a common execution
>>> chain. I don't see any kind of Deferred.await that blocks until the
>>> instance resolves (yes, there's upon, but that's just nesting again because
>>> it returns another deferred.
>>>
>>> Second, I think shutdown shuts *everything* down. What I need is just to
>>> signal the completion of the job that was supposed to run, so that the
>>> Scheduler.go returns in order to allow my unit tests to run to completion.
>>>
>>> Third, I'm not certain about the semantics of Pipe/Reader/Writer. It's
>>> not behaviorally like what I'm familiar with. For instance, callbacks may
>>> return prematurely and only have part of a message. In ZMQ, what you send
>>> is what you get. So that makes me concerned in regards to the Tcp.Server,
>>> because right now what I need is for the Pipe to just allow blocking
>>> receive so that I can make the threads coordinated, but I need the Tcp
>>> Server to allow me to receive whole protobuf messages.
>>>
>>> Can anyone please help me?
>>>
>>
>>
>>
>> --
>> Carl Eastlund
>>
>
>

[-- Attachment #2: Type: text/html, Size: 5089 bytes --]