Efficient I/O with threads

Efficient I/O with threads

[Yaron Minsky]

We've been running into some interesting problems building highly
efficient I/O routines in threaded code in ocaml, and I'm curious if
anyone else has some thoughts on this.  The basic problem seems to be
that the locking and unlocking of the IO channels seems to take a
large fraction of the execution time.

A little bit of background first.  The data type we're outputting is
basically a simple s-expression, with the following type:

type sexp = Atom of string | List of sexp list

We write out an s-expression by writing a tag-byte to determine
whether the s-expression is an atom or a string.  If the s-expression
is an atom, we then write a 4-byte int, which is the length of the
string, and then the string.  If the s-expression is a list, we write
an atom which is the number of s-expression that are contained, and
then write those s-expressions.

It's very easy to write parsing and marshalling for this type of wire
protocol, but that code turns out to be quite inefficient, because you
end up making too many calls to the input and output functions, and
each one of those calls requires releasing and acquiring locks.  I
just can't think of a clean way of implementing a reader for this kind
of protocol.  (a writer could be done by writing stuff to a buffer
first, and then writing the whole buffer out at the socket at once.)

Any thoughts?
Yaron

Re: Efficient I/O with threads

[Yaron Minsky]

An addendum.  One thing that was pointed out to me in some private
emails was that buffering could solve the problem on the reading side
as well.  That is true, as far as it  goes --- that's why I said that
I can't think of a _clean_ way of handling it.  One of the nice things
about ocaml IO channels is that they handle buffering, and it seems a
shame to have to reimplement buffering on top of them.

Put another way, the problem with input/output channels appears to be
that the buffering is done on the wrong side of the lock.  You
shouldn't have to do any locking to do IO when the request can be
satisfied from the buffer.  The fact that IO channels always require
you to acquire the lock means that the performance is crappy unless
you bundle up writes by yourself.

Fixing this is perhaps too deep of a change to drive into the OCaml
system at this point.  Is this a problem that is addressed by the I/O
channels provided by any other library such as extlib?

Yaron

On 5/24/05, Yaron Minsky <yminsky@gmail.com> wrote:
> We've been running into some interesting problems building highly
> efficient I/O routines in threaded code in ocaml, and I'm curious if
> anyone else has some thoughts on this.  The basic problem seems to be
> that the locking and unlocking of the IO channels seems to take a
> large fraction of the execution time.
> 
> A little bit of background first.  The data type we're outputting is
> basically a simple s-expression, with the following type:
> 
> type sexp = Atom of string | List of sexp list
> 
> We write out an s-expression by writing a tag-byte to determine
> whether the s-expression is an atom or a string.  If the s-expression
> is an atom, we then write a 4-byte int, which is the length of the
> string, and then the string.  If the s-expression is a list, we write
> an atom which is the number of s-expression that are contained, and
> then write those s-expressions.
> 
> It's very easy to write parsing and marshalling for this type of wire
> protocol, but that code turns out to be quite inefficient, because you
> end up making too many calls to the input and output functions, and
> each one of those calls requires releasing and acquiring locks.  I
> just can't think of a clean way of implementing a reader for this kind
> of protocol.  (a writer could be done by writing stuff to a buffer
> first, and then writing the whole buffer out at the socket at once.)
> 
> Any thoughts?
> Yaron
>

Re: [Caml-list] Re: Efficient I/O with threads

[Nicolas Cannasse]

[snip]

> Fixing this is perhaps too deep of a change to drive into the
> OCaml system at this point.  Is this a problem that is
> addressed by the I/O channels provided by any other library
> such as extlib?

I can maybe answer on that one.

Extlib IO channels provide "high-level" channels. A channel is just a record
of lambdas that are used to read and write to it. There are implementations
for reading and writing from caml low level channels, but also to input and
output directly from a string. You can also create your own channels by
providing the appropriate API functions ( 3 functions for input channels :
read / input / close  and 4 functions for output channels : write / output /
flush / close ).

This approach means that you can easily wrap one channel with another. For
example there is a Base64 module that takes a channel as parameter and
returns a channel that will either perform encoding or decoding in B64 and
read/write to the underlying channel. The same approach could be used to add
a buffer for either reading or writing.

ExtLib IO channels are focused more on usability than performances. Using
them require a very small overhead compared to using direct caml channels
but is more flexible (you can later retarget your output to a string, or
wrap it with a compression or encoding library) and if you're performing IO
on disk it should not be so much different in terms of performances.

Here's the module documentation :
http://ocaml-lib.sourceforge.net/doc/IO.html

Nicolas

Re: [Caml-list] Re: Efficient I/O with threads

[Gerd Stolpmann]

Am Dienstag, den 24.05.2005, 21:42 -0400 schrieb Yaron Minsky:
> An addendum.  One thing that was pointed out to me in some private
> emails was that buffering could solve the problem on the reading side
> as well.  That is true, as far as it  goes --- that's why I said that
> I can't think of a _clean_ way of handling it.  One of the nice things
> about ocaml IO channels is that they handle buffering, and it seems a
> shame to have to reimplement buffering on top of them.
> 
> Put another way, the problem with input/output channels appears to be
> that the buffering is done on the wrong side of the lock.  You
> shouldn't have to do any locking to do IO when the request can be
> satisfied from the buffer.  The fact that IO channels always require
> you to acquire the lock means that the performance is crappy unless
> you bundle up writes by yourself.
> 
> Fixing this is perhaps too deep of a change to drive into the OCaml
> system at this point.  Is this a problem that is addressed by the I/O
> channels provided by any other library such as extlib?

I just looked into the sources of the OCaml runtime. The additional work
to lock/unlock the I/O channels is very, very small, just a
pthread_mutex_lock and a pthread_mutex_unlock for every operation. What
counts more is the general overhead for the multi-threading machinery.
For every blocking system call a lot of additional overhead is
necessary.

As an alternative, you can try the object channels of Ocamlnet. With

let in_ch =
  Netchannels.lift_in (`Raw (new Netchannels.input_descr in_fd))

and

let out_ch =
  Netchannels.lift_out (`Raw (new Netchannels.output_descr out_fd))

you get object channels over the file descriptors in_fd, out_fd that
implement buffers by O'Caml code and work much like the built-in
channels. These channels aren't protected against concurrent usage, and
may be more light-weight because of this.

Gerd
-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
------------------------------------------------------------