* Re: [Caml-list] true parallelism / threads
2009-02-20 16:40 true parallelism / threads Atmam Ta
@ 2009-02-20 16:57 ` Yoann Padioleau
2009-02-20 16:59 ` Hezekiah M. Carty
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Yoann Padioleau @ 2009-02-20 16:57 UTC (permalink / raw)
To: Atmam Ta; +Cc: caml-list
Atmam Ta <atmamta2@gmail.com> writes:
> Hi,
>
> I am trying to evaluate ocaml for a project involving large scale numerical
> calculations. We would need parallel processing, i.e. a library that
> distributes jobs accross multiple processors within a machine and accross
> multiple PCs.
> Speed and easy programability are important. I have tried to search this issue
> first, but the postings I found were usually negative and 4-5 years old. On the
> other hand, I see a number of libraries in the Hump that by now might be taking
> care of these things.
>
> My question is: is ocaml good for parallel processing / hreaded computation,
> are there (mature) libraries or tools that let developers make use of multicore
> and multimachine environments?
MPI ...
http://pauillac.inria.fr/~xleroy/software.html#ocamlmpi
>
Then it's quite easy to define your own helpers on top of that.
Here is for example my poor's man google map-reduce in ocaml:
http://aryx.kicks-ass.org/~pad/darcs/commons/distribution.ml
> cheers,
> Atmam
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [Caml-list] true parallelism / threads
2009-02-20 16:40 true parallelism / threads Atmam Ta
2009-02-20 16:57 ` [Caml-list] " Yoann Padioleau
@ 2009-02-20 16:59 ` Hezekiah M. Carty
2009-02-20 17:01 ` Will M. Farr
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Hezekiah M. Carty @ 2009-02-20 16:59 UTC (permalink / raw)
To: Atmam Ta; +Cc: caml-list
2009/2/20 Atmam Ta <atmamta2@gmail.com>:
> Hi,
>
> I am trying to evaluate ocaml for a project involving large scale numerical
> calculations. We would need parallel processing, i.e. a library that
> distributes jobs accross multiple processors within a machine and accross
> multiple PCs.
> Speed and easy programability are important. I have tried to search this
> issue first, but the postings I found were usually negative and 4-5 years
> old. On the other hand, I see a number of libraries in the Hump that by now
> might be taking care of these things.
>
> My question is: is ocaml good for parallel processing / hreaded computation,
> are there (mature) libraries or tools that let developers make use of
> multicore and multimachine environments?
>
> cheers,
> Atmam
There are several libraries available which seem to be reasonably
usable in their current state.
Distributed processing across multiple machines:
- OCAMLMPI - http://pauillac.inria.fr/~xleroy/software.html
- OCamlP3l - http://camlp3l.inria.fr/eng.htm
- BSML - http://frederic.loulergue.eu/research/bsmllib/bsml-0.4beta.html
Fork-based parallelism for exploiting multiple cores/processors locally:
- Prelude.ml - http://github.com/kig/preludeml/tree/master
There is also JoCaml (http://jocaml.inria.fr/), which is an extension
of OCaml itself. JoCaml has examples for various distributed
processing methods.
Hez
--
Hezekiah M. Carty
Graduate Research Assistant
University of Maryland
Department of Atmospheric and Oceanic Science
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [Caml-list] true parallelism / threads
2009-02-20 16:40 true parallelism / threads Atmam Ta
2009-02-20 16:57 ` [Caml-list] " Yoann Padioleau
2009-02-20 16:59 ` Hezekiah M. Carty
@ 2009-02-20 17:01 ` Will M. Farr
2009-02-20 17:35 ` Markus Mottl
2009-02-20 19:03 ` Gerd Stolpmann
4 siblings, 0 replies; 6+ messages in thread
From: Will M. Farr @ 2009-02-20 17:01 UTC (permalink / raw)
To: Atmam Ta; +Cc: caml-list
Atmam,
I've had some luck using OCaml with MPI (using the OCamlMPI library at
http://caml.inria.fr/cgi-bin/hump.en.cgi?contrib=401 ). That may not
satisfy your needs as far as multi-core goes, but perhaps it will. I
can't speak to the speed of the interface (my operations were
compute-bound on the individual processors, not communication bound,
so any OCaml overhead on the MPI communication was lost in the noise),
but it was definitely easy to use. At the extreme easy-to-use end,
you can simply send arbitrary OCaml values over the MPI channels; for
more performance, you can use the functions specific to common types
(float arrays, int arrays, etc) to speed up the operations.
As far as single-core OCaml speed goes, I find that it is always
within a factor of 2 of C for straight-line loops (i.e. matrix-vector
multiply, etc), and usually *much* faster whenever more complicated
data structures are involved (maps, binary trees, etc), unless you
really sweat blood with the C implementation.
Hope this helps!
Will
2009/2/20 Atmam Ta <atmamta2@gmail.com>:
> Hi,
>
> I am trying to evaluate ocaml for a project involving large scale numerical
> calculations. We would need parallel processing, i.e. a library that
> distributes jobs accross multiple processors within a machine and accross
> multiple PCs.
> Speed and easy programability are important. I have tried to search this
> issue first, but the postings I found were usually negative and 4-5 years
> old. On the other hand, I see a number of libraries in the Hump that by now
> might be taking care of these things.
>
> My question is: is ocaml good for parallel processing / hreaded computation,
> are there (mature) libraries or tools that let developers make use of
> multicore and multimachine environments?
>
> cheers,
> Atmam
>
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Caml-list] true parallelism / threads
2009-02-20 16:40 true parallelism / threads Atmam Ta
` (2 preceding siblings ...)
2009-02-20 17:01 ` Will M. Farr
@ 2009-02-20 17:35 ` Markus Mottl
2009-02-20 19:03 ` Gerd Stolpmann
4 siblings, 0 replies; 6+ messages in thread
From: Markus Mottl @ 2009-02-20 17:35 UTC (permalink / raw)
To: Atmam Ta; +Cc: caml-list
2009/2/20 Atmam Ta <atmamta2@gmail.com>:
> My question is: is ocaml good for parallel processing / hreaded computation,
> are there (mature) libraries or tools that let developers make use of
> multicore and multimachine environments?
For heavy-duty linear algebra you might want to use Lacaml:
http://ocaml.info/home/ocaml_sources.html#lacaml
It interfaces almost all functions in BLAS and LAPACK and allows
executing multiple computations in several threads in parallel on
multi-core machines. If you combine this with some tool for
distributed computation (e.g. MPI-based, etc.), you should get what
you need.
Regards,
Markus
--
Markus Mottl http://www.ocaml.info markus.mottl@gmail.com
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [Caml-list] true parallelism / threads
2009-02-20 16:40 true parallelism / threads Atmam Ta
` (3 preceding siblings ...)
2009-02-20 17:35 ` Markus Mottl
@ 2009-02-20 19:03 ` Gerd Stolpmann
4 siblings, 0 replies; 6+ messages in thread
From: Gerd Stolpmann @ 2009-02-20 19:03 UTC (permalink / raw)
To: Atmam Ta; +Cc: caml-list
Am Freitag, den 20.02.2009, 10:40 -0600 schrieb Atmam Ta:
> Hi,
>
> I am trying to evaluate ocaml for a project involving large scale
> numerical calculations. We would need parallel processing, i.e. a
> library that distributes jobs accross multiple processors within a
> machine and accross multiple PCs.
> Speed and easy programability are important. I have tried to search
> this issue first, but the postings I found were usually negative and
> 4-5 years old. On the other hand, I see a number of libraries in the
> Hump that by now might be taking care of these things.
>
> My question is: is ocaml good for parallel processing / hreaded
> computation, are there (mature) libraries or tools that let developers
> make use of multicore and multimachine environments?
ocamlnet contains a mature sunrpc implementation, and a framework for
multi-processing. It is used in professional cluster environments, e.g.
by the Wink people searcher.
See here for a commented example:
http://blog.camlcity.org/blog/parallelmm.html
Gerd
--
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany
gerd@gerd-stolpmann.de http://www.gerd-stolpmann.de
Phone: +49-6151-153855 Fax: +49-6151-997714
------------------------------------------------------------
^ permalink raw reply [flat|nested] 6+ messages in thread