Hello !

Many thanks all for your answers !

Managing to have the almost same performance whether in mutithreaded
environment or not (even not using threads for this particular task) is
something I would like to have anyway.

I'll give a try to big buffers using Using.read. Any example code around ?
And then why not try iao !

Memory mapping of the file could be done using BigArray or do I have to
write C code ?

Rémi

On Tue, Feb 17, 2009 at 11:26, Mark Shinwell <mshinwell@janestcapital.com>wrote:

> On Tue, Feb 17, 2009 at 10:07:05AM +0000, Sylvain Le Gall wrote:
> > On 17-02-2009, Rémi Dewitte <remi@gide.net> wrote:
> > You are using input_char and standard IO channel. This is a good choice
> > for non-threaded program. But in your case, I will use Unix.read with a
> > big buffer (32KB to 4MB) and change your program to use it. As
> > benchmarked by John Harrop, you are spending most of your time in
> > caml_enter|leave_blocking section.
>
> This isn't quite right actually -- the profile is deceiving.  It is true
> that there are a lot of calls to enter/leave_blocking_section, but you're
> actually being killed by the overhead of an independent locking strategy
> in the channel-based I/O calls.  I've measured this using some hackery
> with a hex editor.  When you call input_char, you acquire and then release
> another lock which is specific to these calls (the global runtime lock is
> often not released here).  This process isn't especially cheap, so it would
> be better to use one of the other channel calls to read data in larger
> blocks.
>
> Mark
>