Re: [Caml-list] Concurrent/parallel programming

[Gerd Stolpmann <info@gerd-stolpmann.de>]

[-- Attachment #1: Type: text/plain, Size: 4966 bytes --]

Am Dienstag, den 07.01.2014, 21:35 -0500 schrieb Yotam Barnoy:
> Thanks for the links and info, everyone.
> 
> Gerd, would it be possible to put netmulticore up on opam? I realize
> this is a sensitive topic for you, but from what I can tell, this is
> the most comprehensive and efficient solution to parallel programming
> that exists for ocaml, so why not make it as easily available as
> possible?

Netmulticore is part of Ocamlnet, and I guess you can get it by simply
installing ocamlnet with opam. I'm not the maintainer, so I don't know
whether it is included or not.

> Also, regarding the ocaml_modify() and ocaml_initialize() regression
> in 4.01 -- yikes! Is there any way to get support back for writing
> outside of the ocaml heap? This seems like a pretty big deal! (Sorry
> if I'm arriving late for whatever discussion took place about this
> before).

There is a workaround in place - so far the OS supports it: thanks to
Xavier the symbols caml_modify and caml_initialize are declared as weak
in 4.01, allowing them to be redefined in executables. Netmulticore
redefines these symbols with their pre-4.01 functions.

This isn't optimal yet, because the old write barriers are a bit slower,
and because this introduces a very low-level dependency on the current
version of Ocaml. Nevertheless, it works for now. (Ideas for a better
solution are highly welcome.)

Gerd




> 
> -Yotam
> 
> 
> 
> On Tue, Jan 7, 2014 at 3:35 PM, Gerd Stolpmann
> <info@gerd-stolpmann.de> wrote:
>         There is Netmulticore, a part of Ocamlnet:
>         
>         http://projects.camlcity.org/projects/dl/ocamlnet-3.7.3/doc/html-main/Intro.html#ch_comp
>         
>         It utilizes shared memory, and accesses it from several
>         processes. There
>         is no C code involved, but it is fairly unsafe nevertheless,
>         because you
>         need to stick to certain programming rules, or the program
>         crashes.
>         However, if you manage to encapsulate all the unsafe things,
>         it is a
>         good option. I'm using Netmulticore in the Plasma Map/Reduce
>         implementation.
>         
>         Gerd
>         
>         Am Dienstag, den 07.01.2014, 14:54 -0500 schrieb Yotam Barnoy:
>         > Hi List
>         >
>         >
>         > So far, I've been programming in ocaml using only sequential
>         programs.
>         > In my last project, which was an implementation of a large
>         machine
>         > learning algorithm, I tried to speed up computation using a
>         little bit
>         > of parallelism with ParMap, and it was a complete failure.
>         It's
>         > possible that more time would have yielded better results,
>         but I just
>         > didn't have the time to invest in it given how bad the
>         initial results
>         > were.
>         >
>         >
>         > My question is, what are the options right now as far as
>         parallelism
>         > is concerned? I'm not talking about cooperative
>         multitasking, but
>         > about really taking advantage of multiple cores. I'm well
>         aware of the
>         > runtime lock and I'm ok with message passing between
>         processes or a
>         > shared area in memory, but I'd rather have something more
>         high level
>         > than starting up several processes, creating a named pipe or
>         a socket,
>         > and trying to pass messages through that. Also, I assume
>         that using a
>         > shared area in memory involves some C code? Am I wrong about
>         that?
>         >
>         >
>         > I was expecting Core's Async to fill this role, but
>         realworldocaml is
>         > fuzzy on this topic, apparently preferring to dwell on
>         cooperative
>         > multitasking (which is fine but not what I'm looking for),
>         and I
>         > couldn't find any other documentation that was clearer.
>         >
>         >
>         > Thanks
>         >
>         > Yotam
>         >
>         
>         
>         --
>         ------------------------------------------------------------
>         Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
>         My OCaml site:          http://www.camlcity.org
>         Contact details:        http://www.camlcity.org/contact.html
>         Company homepage:       http://www.gerd-stolpmann.de
>         ------------------------------------------------------------
> 
> 

-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
My OCaml site:          http://www.camlcity.org
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 490 bytes --]