From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail1-relais-roc.national.inria.fr (mail1-relais-roc.national.inria.fr [192.134.164.82]) by walapai.inria.fr (8.13.6/8.13.6) with ESMTP id pAU9b5Zu013895 for ; Wed, 30 Nov 2011 10:37:05 +0100 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AuABAMH41U6GnQCBlGdsb2JhbABEFqptIgEBAQEHDQkJFAMigXIBAQU6PxALGC4USYgiuB2KPWMElFaSEg X-IronPort-AV: E=Sophos;i="4.69,596,1315173600"; d="scan'208";a="133299682" Received: from shiva.jussieu.fr ([134.157.0.129]) by mail1-smtp-roc.national.inria.fr with ESMTP/TLS/DHE-RSA-AES256-SHA; 30 Nov 2011 10:36:35 +0100 Received: from hydrogene.pps.jussieu.fr (hydrogene.pps.jussieu.fr [134.157.168.1]) by shiva.jussieu.fr (8.14.4/jtpda-5.4) with ESMTP id pAU9Zwtb083990 ; Wed, 30 Nov 2011 10:36:11 +0100 (CET) X-Ids: 168 Received: from strontium.pps.jussieu.fr (strontium.pps.jussieu.fr [134.157.168.38]) by hydrogene.pps.jussieu.fr (Postfix) with ESMTPS id 83A36C01CE; Wed, 30 Nov 2011 10:35:57 +0100 (CET) Received: from vouillon by strontium.pps.jussieu.fr with local (Exim 4.77) (envelope-from ) id 1RVga5-00054n-Bv; Wed, 30 Nov 2011 10:35:57 +0100 Date: Wed, 30 Nov 2011 10:35:57 +0100 From: Jerome Vouillon To: Martin Jambon Cc: OCaml Mailing List Message-ID: <20111130093557.GA19471@pps.jussieu.fr> References: <4ED59AF6.1030506@ens-lyon.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4ED59AF6.1030506@ens-lyon.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Miltered: at jchkmail.jussieu.fr with ID 4ED5F8FE.004 by Joe's j-chkmail (http : // j-chkmail dot ensmp dot fr)! X-j-chkmail-Enveloppe: 4ED5F8FE.004/134.157.168.1/hydrogene.pps.jussieu.fr/hydrogene.pps.jussieu.fr/ Subject: Re: [Caml-list] Nproc: process pools for OCaml (request for suggestions) Hi, On Tue, Nov 29, 2011 at 06:54:46PM -0800, Martin Jambon wrote: > I would like to publicize Nproc, which is an implementation of process > pools for OCaml based on fork, pipes, Marshal and Lwt: > > https://github.com/MyLifeLabs/nproc > > Using Nproc involves: > > 1. Creating a pool of N processes, N being chosen by the user. > 2. Running tasks: > a. Submitting a task (f, x) of any type. > b. Defining what to do when the result becomes available. Marco Danelutto and Roberto Di Cosmo have written a small library to perform parallel maps and folds on multi-core machines. This is complementary to your library. You should look at the implementation: communication is performed by marshalling to a shared memory area for better performances, and pipes are used only for synchronization. https://gitorious.org/parmap/ I have also written a more low-level library, where you can control which process runs each task. This is useful when the processes have to work with a lot of data that you don't want to duplicate. You can get the code with the following command: darcs clone http://www.pps.jussieu.fr/~vouillon/coinst/darcs/dev/ The file of interest is task.ml. -- Jerome