From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by yquem.inria.fr (Postfix) with ESMTP id 82AF7BBAF for ; Sat, 13 Mar 2010 15:29:40 +0100 (CET) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av8BAPsvm0vVhjEXkGdsb2JhbACbBwEBAQEJCQwHEwMfuFGEewQ X-IronPort-AV: E=Sophos;i="4.49,632,1262559600"; d="scan'208";a="46633739" Received: from ihsmtp01voda.lis.interhost.com (HELO ihsmtp01cons.lis.interhost.com) ([213.134.49.23]) by mail2-smtp-roc.national.inria.fr with ESMTP; 13 Mar 2010 15:29:40 +0100 Received: from [192.168.1.64] ([95.136.106.136]) by ihsmtp01cons.lis.interhost.com with Microsoft SMTPSVC(6.0.3790.3959); Sat, 13 Mar 2010 14:29:39 +0000 Message-ID: <4B9BA14F.2030805@inescporto.pt> Date: Sat, 13 Mar 2010 14:29:35 +0000 From: Hugo Ferreira User-Agent: Thunderbird 2.0.0.23 (X11/20090817) MIME-Version: 1.0 To: Richard Jones Cc: caml-list@yquem.inria.fr Subject: Re: [Caml-list] Shared memory parallel application: kernel threads References: <4B9A2BCB.3040607@inescporto.pt> <20100313135623.GA17113@annexia.org> In-Reply-To: <20100313135623.GA17113@annexia.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 13 Mar 2010 14:29:39.0871 (UTC) FILETIME=[9D62B2F0:01CAC2B9] X-Spam: no; 0.00; ocaml:01 read-only:01 ocaml:01 ocamlnet:01 model:01 gigabyte:98 threads:01 threads:01 wrote:01 caml-list:01 data:02 data:02 api:02 kernel:02 seems:03 Hi Richard, Richard Jones wrote: > Also add my 2 cents here: > > At least look at OCaml Ancient for sharing the data. You possibly may > not use it, but it was designed pretty much for what you have in mind. > The README should be informative: > > http://merjis.com/developers/ancient > http://merjis.com/_file/ancient-readme.txt > Sylvain Le Gall already pointed this out. I have looked at the readme and checked that I have it in GODI (version 0.8, but I think it will do). > (You should also look at the API and source). You didn't mention how > large your read-only data set is, but OCaml Ancient should be able to > handle 100s of gigabytes, assuming a 64 bit machine. We used to use > it with 10s of gigabyte data sets without any issues. > Don't think I will need so much. > As others have said, don't use threads to launch your jobs. Look at > one of the fork-based libraries. Going to experiment with the ocamlnet stuff (version 3). > In addition to the ones mentioned, > take a look at PreludeML, which is internally very simple and should > give you a good start if you decide to write your own: > > http://github.com/kig/preludeml > Ok. > If you want to spread the jobs over multiple machines, then OCaml MPI > is probably the way to go. > I will be using a single 8-CPU machine for the experiments. For the problem at hand shared the memory model seems to be a better fit than messaging. Thanks, Hugo F. > Rich. >