Re: [Caml-list] Asynchronous IO programming in OCaml

[Goswin von Brederlow <goswin-v-b@web.de>]

Jérémie Dimino <jeremie@dimino.org> writes:

> On Thu, Oct 28, 2010 at 11:00:59AM +0200, Goswin von Brederlow wrote:
>> NUMA has memory dedicated to each core. Each core can access its memory
>> fast. Accessing some other cores memory on the other hand is slower (and
>> will have to fight for access with that core).
>
> And so that is a bad solution for Lwt since it runs on one system
> thread.

Does that matter? The copying would be going on in the kernel and use
multiple cores even if the read requests all came from one core I think.

>> Nah, if you can detect the error then handing is easy. The problem is
>> detecting. Write() itself can fail and that is easy to detect. But
>> write() succeeding doesn't mean the data has been written
>> successfully. You can still get an error after that which you only get
>> on fsync().
>
> Take for example buffered channels, when you would have to flush the
> buffer, you would launch a thread calling write and let the program
> continue to write onto the channel, assuming that the write succeed. But
> if one write fails, you would get the error latter. And worst, if the
> program stop using the buffer after the flush, it will never get the
> error.
>
> Jérémie

True. Your program will have to handle errors.

Normaly I would say that a flush/sync on a channel/file has to block the
thread and continue only on error or success. Otherwise you would be
talking more about barriers and simulate them with flush/sync (since
userspace lacks an interface for them). Doing error handling with
barriers is indeed more tricky.

But I was more thinking of having async writes with error reporting. Say
you have 1000 "threads" that need to write something. They all call
my_write(). The my_write() function issues an async write request,
blocks the "thread" and continues some other "thread". Once the write
request has complete the "thread" is woken up and gets a success or
failure message. "thread" here doesn't mean an actual system
thread. More a logical thread like handling of a specific connection.

Even more async would be to pass a callback to the my_write() that gets
called on success or failure and have my_write() return directly. Again
error hadnling is easy since you have the callback to handle it. So
my_write would look something like this:

let my_write fd off str fn = ...
val my_write : Unix.file_descr Int64.t string (Unix.error option -> unit) -> unit
(** Write string <str> to file <fd> at offset <off> and call <fn> with
    None on success or Some error on failure. *)

The program can then decide to continue (do something else after
my_write) or to block (put the rest of the code into the callback and
return).

MfG
        Goswin