From: Richard Jones <rich@annexia.org>
To: Hugo Ferreira <hmf@inescporto.pt>
Cc: caml-list@yquem.inria.fr
Subject: Re: [Caml-list] Shared memory parallel application: kernel threads
Date: Sat, 13 Mar 2010 13:56:23 +0000 [thread overview]
Message-ID: <20100313135623.GA17113@annexia.org> (raw)
In-Reply-To: <4B9A2BCB.3040607@inescporto.pt>
Also add my 2 cents here:
At least look at OCaml Ancient for sharing the data. You possibly may
not use it, but it was designed pretty much for what you have in mind.
The README should be informative:
http://merjis.com/developers/ancient
http://merjis.com/_file/ancient-readme.txt
(You should also look at the API and source). You didn't mention how
large your read-only data set is, but OCaml Ancient should be able to
handle 100s of gigabytes, assuming a 64 bit machine. We used to use
it with 10s of gigabyte data sets without any issues.
As others have said, don't use threads to launch your jobs. Look at
one of the fork-based libraries. In addition to the ones mentioned,
take a look at PreludeML, which is internally very simple and should
give you a good start if you decide to write your own:
http://github.com/kig/preludeml
If you want to spread the jobs over multiple machines, then OCaml MPI
is probably the way to go.
Rich.
--
Richard Jones
Red Hat
next prev parent reply other threads:[~2010-03-13 13:56 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-03-12 11:55 Hugo Ferreira
2010-03-12 12:34 ` [Caml-list] " Gerd Stolpmann
2010-03-12 13:36 ` Hugo Ferreira
2010-03-12 14:30 ` Sylvain Le Gall
2010-03-12 14:54 ` [Caml-list] " Hugo Ferreira
2010-03-12 23:59 ` Philippe Wang
2010-03-13 9:12 ` Hugo Ferreira
2010-03-13 13:56 ` Richard Jones [this message]
2010-03-13 14:29 ` [Caml-list] " Hugo Ferreira
2010-03-13 15:10 ` Richard Jones
2010-03-13 15:37 ` Hugo Ferreira
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20100313135623.GA17113@annexia.org \
--to=rich@annexia.org \
--cc=caml-list@yquem.inria.fr \
--cc=hmf@inescporto.pt \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).