From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail4-relais-sop.national.inria.fr (mail4-relais-sop.national.inria.fr [192.134.164.105]) by walapai.inria.fr (8.13.6/8.13.6) with ESMTP id p7HH8biX008783 for ; Wed, 17 Aug 2011 19:08:41 +0200 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AlEFANr0S05KfVK2kGdsb2JhbABBAZlbg12LMAgUAQEBAQkJDQcUBCGBWQImBgErCgMBAw0FNgInIAEFAQFWh1CYPgqPDIUmiSgCAwYIg0gBghJfBJlHhiU8gT+CIw X-IronPort-AV: E=Sophos;i="4.68,240,1312149600"; d="scan'208";a="105690548" Received: from mail-wy0-f182.google.com ([74.125.82.182]) by mail4-smtp-sop.national.inria.fr with ESMTP/TLS/RC4-SHA; 17 Aug 2011 19:08:40 +0200 Received: by wyh15 with SMTP id 15so1438036wyh.27 for ; Wed, 17 Aug 2011 10:08:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:date:from:to:cc:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=YsD5vLu98Lb6+ds3W6rKdOwQa5icVWHRURAtOWMPgX0=; b=p81qOI2yBSZR/t5JShfGYvmIJ1FbenHH43dlZB3Gw12fWDzTbNuOCIjhggwsu5I/gE w3xXfrMWCVYnoOJt5ATumOZCzH3W+RP9ow+E3H5z5gymtUCSmqbsekFGn932SlFY7PJ8 3LKRSVl+V1fDQM4dZX/dWCevxZg16EDJ8AkFQ= Received: by 10.216.168.137 with SMTP id k9mr1075519wel.46.1313600920513; Wed, 17 Aug 2011 10:08:40 -0700 (PDT) Received: from voyager (183.36.0.109.rev.sfr.net [109.0.36.183]) by mx.google.com with ESMTPS id u51sm904844wec.29.2011.08.17.10.08.38 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 17 Aug 2011 10:08:39 -0700 (PDT) Sender: Roberto Di Cosmo Received: from dicosmo by voyager with local (Exim 4.72) (envelope-from ) id 1QtjdT-0002bE-FO; Wed, 17 Aug 2011 19:10:35 +0200 Date: Wed, 17 Aug 2011 19:10:35 +0200 From: Roberto Di Cosmo To: caml-list@inria.fr Cc: roberto@dicosmo.org, marcod@di.unipi.it Message-ID: <20110817171035.GA8231@voyager> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.20 (2009-06-14) Subject: [Caml-list] [ANN]: Parmap Dear all, a few lines to announce the availability of a minimalistic library which can be useful to exploit your multicore processor with minimal modifications to your OCaml programs. In a nutshell ------------- If you want to use your many cores to accelerate an operation which happens to be a map, fold or map/fold (map-reduce), just use Parmap's parmap, parfold and parmapfold primitives in place of the standard List.map and friends, and specify the number of subprocesses to use by the optional parameter ncores. For example, in the classical Mandelbrot example present in the example directory, the line Parmap.parmap pixel tasks ~ncores:i allows to spawn i separate processes, each working on 1/ith of the list tasks. Rationale --------- The principle of Parmap is very simple: when you call one of the three available primitives, map, fold, and mapfold , your OCaml sequential program forks in n subprocesses (you choose the n), and each subprocess performs the computation on the 1/n of the data, returing the results through a shared memory area to the parent process, that resumes execution once all the children have terminated, and the data has been recollected. This means that you *must* run your program on a *single* multicore machine. Repeat after us: Parmap is not meant to run on a cluster, see one of the many available (re)implementations of the map-reduce schema for that. By forking the parent process on a sigle machine, the children get access, for free, to all the data structures already built, even the imperative ones, and as far as your computation inside the map/fold does not produce side effects that need to be preserved, the final result will be the same as performing the sequential operation, the only difference is that you might get it faster. Of course, if you happen to have open channels, or files, or other connections that should only be used by the parent process, your program may behave in a very wierd way: as an example, *do not* open a graphic window before calling a Parmap primitive, and *do not* use this library if your program is multi-threaded! The OCaml code is quite simple and does not rely on any external C library: all the magic is done by your operating system's fork and memory mapping mechanisms. One could gain some speed by implementing a marshal/unmarshal operation directly on bigarrays, but we did not do this yet. How to get it ------------- Project home: https://gitorious.org/parmap To compile and install: git clone git://gitorious.org/parmap/parmap.git make make install Enjoy -- Marco Danelutto and Roberto Di Cosmo P.S.: special thanks to Pierre Chambart for useful discussions on this code