From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on yquem.inria.fr X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=disabled version=3.1.3 X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from discorde.inria.fr (discorde.inria.fr [192.93.2.38]) by yquem.inria.fr (Postfix) with ESMTP id 35130BC50 for ; Fri, 15 Sep 2006 18:44:49 +0200 (CEST) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.171]) by discorde.inria.fr (8.13.6/8.13.6) with ESMTP id k8FGimco023757 for ; Fri, 15 Sep 2006 18:44:48 +0200 Received: from [84.58.143.74] (helo=gate.lan.gerd-stolpmann.de) by mrelayeu.kundenserver.de (node=mrelayeu4) with ESMTP (Nemesis), id 0ML21M-1GOGnz1qjJ-0002l6; Fri, 15 Sep 2006 18:44:43 +0200 Received: from flakew.lan.gerd-stolpmann.de (fw.lan.gerd-stolpmann.de [192.168.1.1]) by gate.lan.gerd-stolpmann.de (Postfix) with ESMTP id 1DE4CC35C; Fri, 15 Sep 2006 18:44:43 +0200 (CEST) Subject: Re: [Caml-list] The Future Possibility of Concurrent Garbage Collection? From: Gerd Stolpmann To: Mike Lin Cc: caml-list In-Reply-To: <2a1a1a0c0609150924j43c719e7q729f19a2d0d61e77@mail.gmail.com> References: <891bd3390609150729k27b7acf8rc9b12f1e08eae93@mail.gmail.com> <2a1a1a0c0609150924j43c719e7q729f19a2d0d61e77@mail.gmail.com> Content-Type: text/plain Date: Fri, 15 Sep 2006 18:44:41 +0200 Message-Id: <1158338681.15398.12.camel@localhost.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.4.1 Content-Transfer-Encoding: 7bit X-Provags-ID: kundenserver.de abuse@kundenserver.de login:a6865a839c0178d9aa0ce41878507ea2 X-Miltered: at discorde with ID 450AD880.001 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! X-Spam: no; 0.00; gerd:01 stolpmann:01 computations:01 heap:01 hacked:01 sub-optimal:01 type-safe:01 type-safe:01 ocamlnet:01 ocamlnet-:01 gerd:01 stolpmann:01 viktoriastr:01 64293:01 darmstadt:01 Am Freitag, den 15.09.2006, 12:24 -0400 schrieb Mike Lin: > Slightly OT question: I often want to parallelize algorithms in > computational biology in which (a) the parallel computations take a > long time (seconds/minutes) to complete, (b) they use a very large > heap (gigs) of immutable data, and (c) they don't really need to > synchronize at intermediate points in the computation. This seems best > accomplished with fork(). What would be your favorite way to collect > the results from the child processes? > Right now I have a hacked up thing to marshal them through pipes. The > parent process reads the values from the pipes serially, which is > obviously sub-optimal, but I was too lazy to write a select() loop. Is > this what you would do or can you think of a better way? > For my purposes (embarassingly parallelizable computational biology), > a convenient and type-safe little library for doing this would satisfy > 80% of my SMP needs. Use my sunrpc library. It allows you to do asynchronous calls. One process can call the other worker processes and wait until all the results are back. Of course, the calls are type-safe. There is even now a manager for forking subprocesses called netplex. I've recently used these libraries to develop a highly parallelized system that runs on a cluster of seven machines. Both libraries (rpc and netplex) are part of the not yet released ocamlnet2 package. You find the sources here: https://gps.dynxs.de/svn/lib-ocamlnet2/trunk/ A test release is here (this version is used for the mentioned cluster and very stable): http://ocaml-programming.de/packages/ocamlnet-2.2test11.tar.gz Gerd -- ------------------------------------------------------------ Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany gerd@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ------------------------------------------------------------