Re: [Caml-list] forking, threads and events

[Gerd Stolpmann <info@gerd-stolpmann.de>]

Am Dienstag, den 10.10.2006, 22:16 +1000 schrieb Erik de Castro Lopo:
> Hi all,
> 
> I'm thinking about to start a new project which has some rather 
> critical requirements. It will be compiled with the native compiler 
> initially targeting Linux/Unix but eventually also windoze.
> 
> The app consists of a main engine which spawns many short lived 
> child threads or processes. The children go away, do their work and 
> then pass their results back to the main engine. Many of the children 
> will spawn another process and read the child process output via a 
> pipe and many of the children will block on I/O. A small portion of 
> the children may be I/O bound, but there is not way of telling which 
> beforehand.
> 
> Since I would like to maximize the throughput on multi-core and 
> multi-processor machines I am thinking of using a mix of forking and 
> threading.

You know that there is no fork on Windows?

>  For communications, I was thinking of using the Event
> module for communication between threads, but I don't think that
> works for forked process (pipes maybe?).
> 
> Anybody have any advice for this project? Any war stories from similar
> projects? Any readings they can recommend?

If there wasn't the Windows requirement I could recommend this:

You may have a look at ocamlnet2. It includes the netplex library that
manages parent/children relationships between either forked processes or
threads. You can use SunRPC for communication (or whatever you like, but
SunRPC support is included). The bad news is that it does not run on the
native Windows port, not even in the threaded variant (because there's
no socketpair...).

Ah yes, I have a war story, but cannot tell it now. I can only say it is
used for a really big commercial project.

Download it here:

http://www.ocaml-programming.de/packages/ocamlnet-2.2test13.tar.gz

An online manual is here:

http://ocamlnet.sourceforge.net/manual-2.2/

Gerd
-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
Phone: +49-6151-153855                  Fax: +49-6151-997714
------------------------------------------------------------