Re: [Caml-list] The Future Possibility of Concurrent Garbage Collection?

[Gerd Stolpmann <info@gerd-stolpmann.de>]

Am Freitag, den 15.09.2006, 12:24 -0400 schrieb Mike Lin:
> Slightly OT question: I often want to parallelize algorithms in
> computational biology in which (a) the parallel computations take a
> long time (seconds/minutes) to complete, (b) they use a very large
> heap (gigs) of immutable data, and (c) they don't really need to
> synchronize at intermediate points in the computation. This seems best
> accomplished with fork(). What would be your favorite way to collect
> the results from the child processes?
> Right now I have a hacked up thing to marshal them through pipes. The
> parent process reads the values from the pipes serially, which is
> obviously sub-optimal, but I was too lazy to write a select() loop. Is
> this what you would do or can you think of a better way?
> For my purposes (embarassingly parallelizable computational biology),
> a convenient and type-safe little library for doing this would satisfy
> 80% of my SMP needs.

Use my sunrpc library. It allows you to do asynchronous calls. One
process can call the other worker processes and wait until all the
results are back. Of course, the calls are type-safe.

There is even now a manager for forking subprocesses called netplex.
I've recently used these libraries to develop a highly parallelized
system that runs on a cluster of seven machines.

Both libraries (rpc and netplex) are part of the not yet released
ocamlnet2 package. You find the sources here:

https://gps.dynxs.de/svn/lib-ocamlnet2/trunk/

A test release is here (this version is used for the mentioned cluster
and very stable):

http://ocaml-programming.de/packages/ocamlnet-2.2test11.tar.gz

Gerd

-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
Phone: +49-6151-153855                  Fax: +49-6151-997714
------------------------------------------------------------