Re: [Caml-list] Non-blocking IO interface design

[Yaron Minsky <yminsky@janestreet.com>]

On Tue, Apr 10, 2012 at 4:21 AM, Daniel Bünzli
<daniel.buenzli@erratique.ch> wrote:
> Anil,
>
> Thanks for the analysis.
>
>> The I/O loop is being called twice for the non-blocking version, as it receives
>> the `Await signal, does the Unix syscall, and then jumps into decode_src. Presumably
>> a full non-blocking version would have to register with a select handler if it
>> gets an EAGAIN at this point,
>
>
> Yes.
>
>> In terms of the number of system calls, the non-blocking one is more efficient,
>> as it uses a 16KB buffer versus the 4K reads done by the blocking version.
>
>
> Yes, the 4K reads are a limitation of pervasives channels. For each mechanism I used the largest buffer that the OCaml runtime uses.
>
>> Looking at the two decoders in src/se.ml, it looks like the non-blocking one
>> allocates closures on every loop, which the blocking one doesn't. This is so it
>> can store the continuation in d.k for the next loop.
>
>
> Yes, that's a side effect of writing in continuation passing style in general since continuations are often partially applied functions.

I believe this particular performance issue is fixed in the upcoming
4.0 release, based on some work by OCamlPro.

>> So to summarise, instead of storing a continuation closure, it would probably be better
>> to explicitly store the state in d.k to minimise allocation?
>
>
> Maybe, but keep in mind that s-expressions are very simple to parse. It may be obvious in this case but depending on what you decode defining/storing the state may become complex. Cps is an easy and general way to solve the problem while keeping the whole thing reasonably readable. But do you maybe see another pattern that I don't ?
>
>> The library looks very useful by the way: I have exactly the same issue with several
>> Lwt-only protocol libraries we're developing at the moment. Would love to use yours before
>> the first release of them to make them more independent of the underlying I/O mechanism...
>
>
> That would be nice, I'm glad if you can somehow reuse the pattern.
>
>
> Best,
>
> Daniel
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa-roc.inria.fr/wws/info/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>