Re: [Caml-list] Re: Efficient I/O with threads

[Gerd Stolpmann <info@gerd-stolpmann.de>]

Am Dienstag, den 24.05.2005, 21:42 -0400 schrieb Yaron Minsky:
> An addendum.  One thing that was pointed out to me in some private
> emails was that buffering could solve the problem on the reading side
> as well.  That is true, as far as it  goes --- that's why I said that
> I can't think of a _clean_ way of handling it.  One of the nice things
> about ocaml IO channels is that they handle buffering, and it seems a
> shame to have to reimplement buffering on top of them.
> 
> Put another way, the problem with input/output channels appears to be
> that the buffering is done on the wrong side of the lock.  You
> shouldn't have to do any locking to do IO when the request can be
> satisfied from the buffer.  The fact that IO channels always require
> you to acquire the lock means that the performance is crappy unless
> you bundle up writes by yourself.
> 
> Fixing this is perhaps too deep of a change to drive into the OCaml
> system at this point.  Is this a problem that is addressed by the I/O
> channels provided by any other library such as extlib?

I just looked into the sources of the OCaml runtime. The additional work
to lock/unlock the I/O channels is very, very small, just a
pthread_mutex_lock and a pthread_mutex_unlock for every operation. What
counts more is the general overhead for the multi-threading machinery.
For every blocking system call a lot of additional overhead is
necessary.

As an alternative, you can try the object channels of Ocamlnet. With

let in_ch =
  Netchannels.lift_in (`Raw (new Netchannels.input_descr in_fd))

and

let out_ch =
  Netchannels.lift_out (`Raw (new Netchannels.output_descr out_fd))

you get object channels over the file descriptors in_fd, out_fd that
implement buffers by O'Caml code and work much like the built-in
channels. These channels aren't protected against concurrent usage, and
may be more light-weight because of this.

Gerd
-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
------------------------------------------------------------