From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by walapai.inria.fr (8.13.6/8.13.6) with ESMTP id p2OFcimY028659 for ; Thu, 24 Mar 2011 16:38:44 +0100 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AhwCAL4Bi03U4xEIkGdsb2JhbACERKEFFAEBAQEJCQ0HFAMiiE2obpEZAoElg0t3BIgqiBc X-IronPort-AV: E=Sophos;i="4.63,237,1299452400"; d="scan'208";a="94786398" Received: from moutng.kundenserver.de ([212.227.17.8]) by mail2-smtp-roc.national.inria.fr with ESMTP; 24 Mar 2011 16:38:39 +0100 Received: from office1.lan.sumadev.de (dslb-188-097-066-234.pools.arcor-ip.net [188.97.66.234]) by mrelayeu.kundenserver.de (node=mreu2) with ESMTP (Nemesis) id 0MbLW4-1QLPOk3KXE-00IrBy; Thu, 24 Mar 2011 16:38:38 +0100 Received: from [192.168.1.111] (546BE640.cm-12-4d.dynamic.ziggo.nl [84.107.230.64]) by office1.lan.sumadev.de (Postfix) with ESMTPA id 706785F701; Thu, 24 Mar 2011 16:38:38 +0100 (CET) From: Gerd Stolpmann To: Joel Reymont Cc: Alexy Khrabrov , caml-list In-Reply-To: <6C1E9880-E792-4496-A906-34D8B7ABEBC1@gmail.com> References: <4587C749-DEFE-47B9-8D9A-44FE5980B58F@gmail.com> <1300978675.8429.227.camel@thinkpad> <6C1E9880-E792-4496-A906-34D8B7ABEBC1@gmail.com> Content-Type: text/plain; charset="UTF-8" Date: Thu, 24 Mar 2011 16:38:36 +0100 Message-ID: <1300981116.8429.256.camel@thinkpad> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-Provags-ID: V02:K0:4o4BQ6059VCrjiQAukBE0frF5PDG4z/gf65gj7UQ6eN y8xRXdKklY1DQU++i13NAbl8Ior1YnnluaQoGP1elRHpfrB8CK T0GwqeUPu3qHUwatEzgKr97MdYPZEF7455Kau4eqMpqvZF/0BS upXlWH3qQN5/qL78jPhNvXTAEkj1aZx1zcMPmpyDR8pLHc7UNe rFwepxvjNtqZjjD0mjcOQ== Subject: Re: [Caml-list] Efficient OCaml multicore -- roadmap? Am Donnerstag, den 24.03.2011, 15:03 +0000 schrieb Joel Reymont: > I'm using a combination of OCaml and ZeroMQ which allows for many combinations of workers, routers, dispatchers, etc. > > Works for me and I recommend it. Right, for server architectures you can use middleware like message queues. This is data-driven parallelism - works nicely when your data allows it, and you don't need special features of the language runtime. It becomes more complicated when your data doesn't allow this structure (e.g. think of a matrix multiplication), or when you need very short reaction times (in the low microseconds range), or the computations are comparatively cheap, and the communication overhead would dominate. A few months ago I started an experiment: I wanted to speed up a function that sorts a large array of strings, and parallelized it. It turned out that all sort of communication and synchronization are already that expensive so that I finally only got a speedup on a quad core for very large data sets (in the multi-gigabyte range). As sorting is O(n log n) every data copy (which is O(n)) is already measurable, and the distribution of the input data to the workers and the gathering of the result was that costly that I first got an improvement for such large data sets. Gerd > > On Mar 24, 2011, at 2:57 PM, Gerd Stolpmann wrote: > > > So, here is my answer to your latter question: I've started a library > > for using independent processes as workers that can communicate via > > explicit shared memory (see > > https://godirepo.camlcity.org/svn/lib-ocamlnet2/trunk/code/src/netmulticore/, and https://godirepo.camlcity.org/svn/lib-ocamlnet2/trunk/code/examples/multicore/chameneos.ml for a coding sample). This seems at least to be a solution in the Unix world for coarse-grained parallelism that is not dominated by the communication overhead between the processes. > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > > > -- ------------------------------------------------------------ Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany gerd@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ------------------------------------------------------------