Re: [Caml-list] Concurrent/parallel programming

[Francois Berenger <berenger@riken.jp>]

On 01/08/2014 05:03 AM, Roberto Di Cosmo wrote:
> Dear Yotam,
>       speeding up a computation by parallelising part of it, as you are probably
> finding out the hard way, is not an easy task.
>
> Being the main developer of Parmap, I am quite curious to know where the
> "complete failure" you are mentioning actually comes from, and I would like
> to put you in touch with Francois Berenger, that has been using Parmap in
> production for quite a while with excellent results.

Absolutely!

I usually develop a sequential program working on a big list
and using List.iter or List.map.

When I am happy with the sequential program, I add an nprocs parameter
to use the Parmap equivalents when nprocs > 1.

The only problems I have encountered were out of memory
in case the nprocs forks did not fit on my machine when
I handled too big data.

I usually handle independent pieces of data, for example
many proteins or chemical molecules.

More answers follow below.

> All the best
>
> --
> Roberto
>
>
> On Tue, Jan 07, 2014 at 02:54:33PM -0500, Yotam Barnoy wrote:
>> Hi List
>>
>> So far, I've been programming in ocaml using only sequential programs. In my
>> last project, which was an implementation of a large machine learning
>> algorithm, I tried to speed up computation using a little bit of parallelism
>> with ParMap, and it was a complete failure. It's possible that more time would
>> have yielded better results, but I just didn't have the time to invest in it
>> given how bad the initial results were.
>>
>> My question is, what are the options right now as far as parallelism is
>> concerned? I'm not talking about cooperative multitasking, but about really
>> taking advantage of multiple cores. I'm well aware of the runtime lock and I'm
>> ok with message passing between processes or a shared area in memory, but I'd
>> rather have something more high level than starting up several processes,
>> creating a named pipe or a socket, and trying to pass messages through that.

There are MPI bindings for OCaml.

>> Also, I assume that using a shared area in memory involves some C code? Am I
>> wrong about that?

Gerd Stolpman's impressive ocamlnet library has nice pure OCaml wrappers 
for all this I believe.

http://projects.camlcity.org/projects/ocamlnet.html

>> I was expecting Core's Async to fill this role, but realworldocaml is fuzzy on
>> this topic, apparently preferring to dwell on cooperative multitasking (which
>> is fine but not what I'm looking for), and I couldn't find any other
>> documentation that was clearer.

Cf. parallel Vs. concurrent in there:

http://chimera.labs.oreilly.com/books/1230000000929/ch01.html#sec_terminology

I believe Parmap and MPI are for parallelism (what you want
very probably), while Async/Lwt and OCaml threads are for concurrency
(used in webservers, GUIs, etc.).

-- 
Best regards,
Francois Berenger.