Re: [Caml-list] [ANN]: Parmap

[Ashish Agarwal <agarwal1975@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 3578 bytes --]

Thanks. This is very exciting. I tried a simple test but get an error:

# #require "extlib";;
# #require "parmap";;
# Parmap.parfold (+) (List.of_enum (1 -- 1000)) 0 ~ncores:1;;
Signal -10

The function List.of_enum is from Batteries. I use it just to create a long
list of integers.

Also, you need to add extlib to the META file.



On Wed, Aug 17, 2011 at 1:10 PM, Roberto Di Cosmo <roberto@dicosmo.org>wrote:

> Dear all,
>       a few lines to announce the availability of a minimalistic library
> which
> can be useful to exploit your multicore processor with minimal
> modifications to
> your OCaml programs.
>
> In a nutshell
> -------------
>
> If you want to use your many cores to accelerate an operation which happens
> to
> be a map, fold or map/fold (map-reduce), just use Parmap's parmap, parfold
> and
> parmapfold primitives in place of the standard List.map and friends, and
> specify
> the number of subprocesses to use by the optional parameter ncores.
>
> For example, in the classical Mandelbrot example present in the example
> directory,
> the line
>
>        Parmap.parmap pixel tasks ~ncores:i
>
> allows to spawn i separate processes, each working on 1/ith of the list
> tasks.
>
> Rationale
> ---------
>
> The principle of Parmap is very simple: when you call one of the three
> available
> primitives, map, fold, and  mapfold , your OCaml  sequential program forks
>  in n
> subprocesses (you choose the n), and each subprocess performs the
> computation on
> the  1/n of the data, returing  the results through a  shared memory area
> to the
> parent process, that resumes  execution once all  the children  have
> terminated,
> and the data has been recollected.
>
> This means that you *must* run your program on a *single* multicore
> machine.
> Repeat after us: Parmap is not meant to run on a cluster, see one of the
> many
> available (re)implementations of the map-reduce schema for that.
>
> By forking the parent process  on a sigle  machine, the children get
> access, for
> free, to all the data structures already built, even the imperative ones,
> and as
> far as your computation  inside the map/fold  does not produce side effects
> that
> need  to be  preserved, the  final result will   be the same  as
>  performing the
> sequential operation, the only difference is that you might get it faster.
>
> Of course, if you happen  to have open  channels, or files, or other
> connections
> that should only be  used by the parent  process, your program  may behave
> in  a
> very wierd way: as an example, *do  not* open a  graphic window before
> calling a
> Parmap primitive, and   *do   not*  use  this  library   if  your  program
>    is
> multi-threaded!
>
> The OCaml code is quite simple and does not rely on any  external C
> library: all
> the magic is done by your operating system's fork and memory mapping
> mechanisms.
> One could gain some speed by implementing a marshal/unmarshal operation
> directly
> on bigarrays, but we did not do this yet.
>
>
> How to get it
> -------------
>
> Project home: https://gitorious.org/parmap
>
> To compile and install:
>
>  git clone git://gitorious.org/parmap/parmap.git
>  make
>  make install
>
> Enjoy
>
> -- Marco Danelutto and Roberto Di Cosmo
>
> P.S.: special thanks to Pierre Chambart for useful discussions on this code
>
>
>
>
>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa-roc.inria.fr/wws/info/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>
>

[-- Attachment #2: Type: text/html, Size: 4581 bytes --]