From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by sympa.inria.fr (Postfix) with ESMTPS id A77CF7EE99 for ; Wed, 8 Jan 2014 03:35:32 +0100 (CET) Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of yotambarnoy@gmail.com) identity=pra; client-ip=209.85.128.43; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="yotambarnoy@gmail.com"; x-sender="yotambarnoy@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail3-smtp-sop.national.inria.fr: domain of yotambarnoy@gmail.com designates 209.85.128.43 as permitted sender) identity=mailfrom; client-ip=209.85.128.43; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="yotambarnoy@gmail.com"; x-sender="yotambarnoy@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-qe0-f43.google.com) identity=helo; client-ip=209.85.128.43; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="yotambarnoy@gmail.com"; x-sender="postmaster@mail-qe0-f43.google.com"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Au4DAG+4zFLRVYArlWdsb2JhbABZg0NVsRuIVoEHCBYOAQEBAQcNCQkSKoIlAQEBAwFAARQHEQwBAwELBgULAzgiAREBBQEcBhMJEodUAQMJCA2dV4xcgwmRLQoZJw1khCEBBQyOUiYHhDcEiUOMa4FpgTCOeBgphHce X-IPAS-Result: Au4DAG+4zFLRVYArlWdsb2JhbABZg0NVsRuIVoEHCBYOAQEBAQcNCQkSKoIlAQEBAwFAARQHEQwBAwELBgULAzgiAREBBQEcBhMJEodUAQMJCA2dV4xcgwmRLQoZJw1khCEBBQyOUiYHhDcEiUOMa4FpgTCOeBgphHce X-IronPort-AV: E=Sophos;i="4.95,621,1384297200"; d="scan'208";a="43911511" Received: from mail-qe0-f43.google.com ([209.85.128.43]) by mail3-smtp-sop.national.inria.fr with ESMTP/TLS/RC4-SHA; 08 Jan 2014 03:35:31 +0100 Received: by mail-qe0-f43.google.com with SMTP id jy17so1223418qeb.16 for ; Tue, 07 Jan 2014 18:35:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=FEpl75kUrHS4k8XsEHCdgyv4HX3wXsjzhEZZE8612Qo=; b=RJuBzZdZXPNq2FQCSHVX6htZvjuX2jKwraEulUZvbukxJ23rXyt9PpLvwMf1mSe3tp YtrCNPNoo57FGSyUpZktAdG0uvzwyp35gyQ2LTPzMJvuLbzPs9HClX5hqyy/yMAjd1tX NbwvDlwWQhbdQapALwTrrTiP8uKwsZPzWvo53/cWRLXt0fbBT4ZUjvVTn5PbK1hPYDFK 4EGjKOgUYbNrJb0h4WOO9+hE1+iLwXwI/3vssezJ0tApuZLqsizoQGlaEidhxogFBE9k LYMfRbtTVBl1OsQ8zuJnrUdUskyAP/Zi7Ar0rb4crNbpQKBgCWjjt3t/qjQsELXfFky2 ZlFQ== X-Received: by 10.49.25.46 with SMTP id z14mr10818749qef.20.1389148529947; Tue, 07 Jan 2014 18:35:29 -0800 (PST) MIME-Version: 1.0 Received: by 10.224.95.8 with HTTP; Tue, 7 Jan 2014 18:35:09 -0800 (PST) In-Reply-To: <1389126940.2692.4.camel@zotac> References: <1389126940.2692.4.camel@zotac> From: Yotam Barnoy Date: Tue, 7 Jan 2014 21:35:09 -0500 Message-ID: To: Gerd Stolpmann Cc: Ocaml Mailing List Content-Type: multipart/alternative; boundary=047d7b5d9b99aafa8604ef6c582b Subject: Re: [Caml-list] Concurrent/parallel programming --047d7b5d9b99aafa8604ef6c582b Content-Type: text/plain; charset=ISO-8859-1 Thanks for the links and info, everyone. Gerd, would it be possible to put netmulticore up on opam? I realize this is a sensitive topic for you, but from what I can tell, this is the most comprehensive and efficient solution to parallel programming that exists for ocaml, so why not make it as easily available as possible? Also, regarding the ocaml_modify() and ocaml_initialize() regression in 4.01 -- yikes! Is there any way to get support back for writing outside of the ocaml heap? This seems like a pretty big deal! (Sorry if I'm arriving late for whatever discussion took place about this before). -Yotam On Tue, Jan 7, 2014 at 3:35 PM, Gerd Stolpmann wrote: > There is Netmulticore, a part of Ocamlnet: > > > http://projects.camlcity.org/projects/dl/ocamlnet-3.7.3/doc/html-main/Intro.html#ch_comp > > It utilizes shared memory, and accesses it from several processes. There > is no C code involved, but it is fairly unsafe nevertheless, because you > need to stick to certain programming rules, or the program crashes. > However, if you manage to encapsulate all the unsafe things, it is a > good option. I'm using Netmulticore in the Plasma Map/Reduce > implementation. > > Gerd > > Am Dienstag, den 07.01.2014, 14:54 -0500 schrieb Yotam Barnoy: > > Hi List > > > > > > So far, I've been programming in ocaml using only sequential programs. > > In my last project, which was an implementation of a large machine > > learning algorithm, I tried to speed up computation using a little bit > > of parallelism with ParMap, and it was a complete failure. It's > > possible that more time would have yielded better results, but I just > > didn't have the time to invest in it given how bad the initial results > > were. > > > > > > My question is, what are the options right now as far as parallelism > > is concerned? I'm not talking about cooperative multitasking, but > > about really taking advantage of multiple cores. I'm well aware of the > > runtime lock and I'm ok with message passing between processes or a > > shared area in memory, but I'd rather have something more high level > > than starting up several processes, creating a named pipe or a socket, > > and trying to pass messages through that. Also, I assume that using a > > shared area in memory involves some C code? Am I wrong about that? > > > > > > I was expecting Core's Async to fill this role, but realworldocaml is > > fuzzy on this topic, apparently preferring to dwell on cooperative > > multitasking (which is fine but not what I'm looking for), and I > > couldn't find any other documentation that was clearer. > > > > > > Thanks > > > > Yotam > > > > -- > ------------------------------------------------------------ > Gerd Stolpmann, Darmstadt, Germany gerd@gerd-stolpmann.de > My OCaml site: http://www.camlcity.org > Contact details: http://www.camlcity.org/contact.html > Company homepage: http://www.gerd-stolpmann.de > ------------------------------------------------------------ > --047d7b5d9b99aafa8604ef6c582b Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Thanks for the links and info, everyone.

Gerd,= would it be possible to put netmulticore up on opam? I realize this is a s= ensitive topic for you, but from what I can tell, this is the most comprehe= nsive and efficient solution to parallel programming that exists for ocaml,= so why not make it as easily available as possible?

Also, regarding the ocaml_modify() and ocaml_initialize() regression in= 4.01 -- yikes! Is there any way to get support back for writing outside of= the ocaml heap? This seems like a pretty big deal! (Sorry if I'm arriv= ing late for whatever discussion took place about this before).

-Yotam


On Tue,= Jan 7, 2014 at 3:35 PM, Gerd Stolpmann <info@gerd-stolpmann.de&g= t; wrote:
There is Netmulticore, a part of Ocamlnet:
http://projects.camlcity.org/pr= ojects/dl/ocamlnet-3.7.3/doc/html-main/Intro.html#ch_comp

It utilizes shared memory, and accesses it from several processes. There
is no C code involved, but it is fairly unsafe nevertheless, because you
need to stick to certain programming rules, or the program crashes.
However, if you manage to encapsulate all the unsafe things, it is a
good option. I'm using Netmulticore in the Plasma Map/Reduce
implementation.

Gerd

Am Dienstag, den 07.01.2014, 14:54 -0500 schrieb Yotam Barnoy:
> Hi List
>
>
> So far, I've been programming in ocaml using only sequential progr= ams.
> In my last project, which was an implementation of a large machine
> learning algorithm, I tried to speed up computation using a little bit=
> of parallelism with ParMap, and it was a complete failure. It's
> possible that more time would have yielded better results, but I just<= br> > didn't have the time to invest in it given how bad the initial res= ults
> were.
>
>
> My question is, what are the options right now as far as parallelism > is concerned? I'm not talking about cooperative multitasking, but<= br> > about really taking advantage of multiple cores. I'm well aware of= the
> runtime lock and I'm ok with message passing between processes or = a
> shared area in memory, but I'd rather have something more high lev= el
> than starting up several processes, creating a named pipe or a socket,=
> and trying to pass messages through that. Also, I assume that using a<= br> > shared area in memory involves some C code? Am I wrong about that?
>
>
> I was expecting Core's Async to fill this role, but realworldocaml= is
> fuzzy on this topic, apparently preferring to dwell on cooperative
> multitasking (which is fine but not what I'm looking for), and I > couldn't find any other documentation that was clearer.
>
>
> Thanks
>
> Yotam
>

--
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany =A0 =A0gerd@gerd-stolpmann.de
My OCaml site: =A0 =A0 =A0 =A0 =A0http://www.camlcity.org
Contact details: =A0 =A0 =A0 =A0http://www.camlcity.org/contact.html
Company homepage: =A0 =A0 =A0 http://www.gerd-stolpmann.de
------------------------------------------------------------

--047d7b5d9b99aafa8604ef6c582b--