From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail1-relais-roc.national.inria.fr (mail1-relais-roc.national.inria.fr [192.134.164.82]) by walapai.inria.fr (8.13.6/8.13.6) with ESMTP id p7IAEhjb012234 for ; Thu, 18 Aug 2011 12:14:43 +0200 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AkAEAIDlTE5KfVK2kGdsb2JhbABBAZkohBKLLQgUAQEBAQkJDQcUBCGBQAEBAQEDEgImBgEUFwIIAwEDDAEFBRgNDAITFCABBQEBCRgTEhCHUAKbRwqPDIUHiSgCAwYIg0gBghJfBJlHg0OCYjyBP4Ij X-IronPort-AV: E=Sophos;i="4.68,244,1312149600"; d="scan'208";a="116174904" Received: from mail-wy0-f182.google.com ([74.125.82.182]) by mail1-smtp-roc.national.inria.fr with ESMTP/TLS/RC4-SHA; 18 Aug 2011 12:14:37 +0200 Received: by wyh15 with SMTP id 15so2297650wyh.27 for ; Thu, 18 Aug 2011 03:14:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=nsUAyD4XX62Pl70l1upYfnQNLDuFSkkrT9SExXDBS/s=; b=cHvJeW8uultwgp66a7jPJEyc9eAwWrfuEqP0IqAteLFQi1TjGttm4aQIDwDh0HVUgq Ke5wuj5p6kSP6oaVn3XmkUcaUezerQb1lPrPkQ4AYtE9wcnwrThk/j3PyMgCkOPt9kLH CoSzmO/PLZVSkGceuWoNdgrg8K1cB3t9pDoGI= Received: by 10.227.32.73 with SMTP id b9mr1744925wbd.107.1313662477402; Thu, 18 Aug 2011 03:14:37 -0700 (PDT) Received: from voyager (183.36.0.109.rev.sfr.net [109.0.36.183]) by mx.google.com with ESMTPS id fy12sm1621736wbb.66.2011.08.18.03.14.34 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 18 Aug 2011 03:14:35 -0700 (PDT) Sender: Roberto Di Cosmo Received: from dicosmo by voyager with local (Exim 4.72) (envelope-from ) id 1QtzeI-0003Iv-Rv; Thu, 18 Aug 2011 12:16:30 +0200 Date: Thu, 18 Aug 2011 12:16:30 +0200 From: Roberto Di Cosmo To: Ashish Agarwal Cc: caml-list@inria.fr, marcod@di.unipi.it Message-ID: <20110818101630.GA12561@voyager> References: <20110817171035.GA8231@voyager> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) Subject: Re: [Caml-list] [ANN]: Parmap Dear Ashish, thanks for the report; these issues are fixed in the latest commit on https://gitorious.org/parmap (bfdf714) Let me know if you find any more issues All the best --Roberto On Wed, Aug 17, 2011 at 03:25:53PM -0400, Ashish Agarwal wrote: > Thanks. This is very exciting. I tried a simple test but get an error: > > # #require "extlib";; > # #require "parmap";; > # Parmap.parfold (+) (List.of_enum (1 -- 1000)) 0 ~ncores:1;; > Signal -10 > > The function List.of_enum is from Batteries. I use it just to create a long > list of integers. > > Also, you need to add extlib to the META file. > > > > On Wed, Aug 17, 2011 at 1:10 PM, Roberto Di Cosmo wrote: > > Dear all, > a few lines to announce the availability of a minimalistic library > which > can be useful to exploit your multicore processor with minimal > modifications to > your OCaml programs. > > In a nutshell > ------------- > > If you want to use your many cores to accelerate an operation which happens > to > be a map, fold or map/fold (map-reduce), just use Parmap's parmap, parfold > and > parmapfold primitives in place of the standard List.map and friends, and > specify > the number of subprocesses to use by the optional parameter ncores. > > For example, in the classical Mandelbrot example present in the example > directory, > the line > > Parmap.parmap pixel tasks ~ncores:i > > allows to spawn i separate processes, each working on 1/ith of the list > tasks. > > Rationale > --------- > > The principle of Parmap is very simple: when you call one of the three > available > primitives, map, fold, and mapfold , your OCaml sequential program forks > in n > subprocesses (you choose the n), and each subprocess performs the > computation on > the 1/n of the data, returing the results through a shared memory area > to the > parent process, that resumes execution once all the children have > terminated, > and the data has been recollected. > > This means that you *must* run your program on a *single* multicore > machine. > Repeat after us: Parmap is not meant to run on a cluster, see one of the > many > available (re)implementations of the map-reduce schema for that. > > By forking the parent process on a sigle machine, the children get > access, for > free, to all the data structures already built, even the imperative ones, > and as > far as your computation inside the map/fold does not produce side effects > that > need to be preserved, the final result will be the same as > performing the > sequential operation, the only difference is that you might get it faster. > > Of course, if you happen to have open channels, or files, or other > connections > that should only be used by the parent process, your program may behave > in a > very wierd way: as an example, *do not* open a graphic window before > calling a > Parmap primitive, and *do not* use this library if your program > is > multi-threaded! > > The OCaml code is quite simple and does not rely on any external C > library: all > the magic is done by your operating system's fork and memory mapping > mechanisms. > One could gain some speed by implementing a marshal/unmarshal operation > directly > on bigarrays, but we did not do this yet. > > > How to get it > ------------- > > Project home: https://gitorious.org/parmap > > To compile and install: > > git clone git://gitorious.org/parmap/parmap.git > make > make install > > Enjoy > > -- Marco Danelutto and Roberto Di Cosmo > > P.S.: special thanks to Pierre Chambart for useful discussions on this code > > > > > > > -- > Caml-list mailing list. Subscription management and archives: > https://sympa-roc.inria.fr/wws/info/caml-list > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-bugs > > > -- --Roberto Di Cosmo ------------------------------------------------------------------ Professeur En delegation a l'INRIA PPS E-mail: roberto@dicosmo.org Universite Paris Diderot WWW : http://www.dicosmo.org Case 7014 Tel : ++33-(0)1-57 27 92 20 5, Rue Thomas Mann F-75205 Paris Cedex 13 Identica: http://identi.ca/rdicosmo FRANCE. Twitter: http://twitter.com/rdicosmo ------------------------------------------------------------------ Attachments: MIME accepted Word deprecated, http://www.rfc1149.net/documents/whynotword ------------------------------------------------------------------ Office location: Bureau 6C08 (6th floor) 175, rue du Chevaleret, XIII Metro Chevaleret, ligne 6 ------------------------------------------------------------------