caml-list - the Caml user's mailing list
 help / color / mirror / Atom feed
From: "Jon Harrop" <jon@ffconsultancy.com>
To: "'Pierre-Alexandre Voye'" <ontologiae@gmail.com>,
	"'caml-list'" <caml-list@inria.fr>
Subject: RE: [Caml-list] Case study in optimization: porting a compiler from OCaml to F#
Date: Sun, 17 Mar 2013 12:06:06 -0000	[thread overview]
Message-ID: <037b01ce2307$d54e6130$7feb2390$@ffconsultancy.com> (raw)
In-Reply-To: <CANnJ5GeEmvJni+Kf06wXf14s4zZXhLug9JRSfAW8XazQc6dmLQ@mail.gmail.com>

Pierre-Alexandre Voye wrote:
> So you could maybe use Parmap.map ?
> Parmap.parmap ~ncores:4 funct (Parmap.L elem_list)

What happens if the inner function returns results via mutation? I assume you must rearrange the code to return all results explicitly and they will then be deep copied (which destroys scalability due to limited shared memory bandwidth on multicores).

Does it do load balancing? I assume not given that ncores is hardcoded.

Does a parmap with ncores=4 inside a parmap with ncores=4 create 16 processes?

Does it deep copy inputs and/or outputs? I assume so, at least for outputs, because you cannot write results in-place without a shared mutable heap.

Does parmap have a large constant overhead? I assume so if it is forking processes.

Another solution is to prefork and explicitly communicate all inputs using message passing but this is equally problematic. You have to rearrange the code. Deep copying inputs also destroys scalability.

Cheers,
Jon.



  reply	other threads:[~2013-03-17 12:06 UTC|newest]

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-03-13 17:04 Jon Harrop
2013-03-13 17:14 ` julien verlaguet
2013-03-13 19:19   ` Jon Harrop
2013-03-13 19:28     ` oliver
2013-03-14 15:05       ` oliver
2013-03-14 15:15         ` oliver
2013-03-15 13:34     ` Pierre-Alexandre Voye
2013-03-17 12:06       ` Jon Harrop [this message]
2013-03-19  1:50         ` Francois Berenger
2013-03-20 20:54           ` Jon Harrop
2013-03-20 22:35             ` Roberto Di Cosmo
2013-03-21  4:13               ` Mike Lin
2013-03-21  7:35                 ` Roberto Di Cosmo
2013-03-21 20:07                   ` Roberto Di Cosmo
2013-03-19 12:47         ` Jean-Marc Alliot
2013-03-20  9:32           ` Roberto Di Cosmo
2013-03-19  1:37     ` Francois Berenger
2013-03-13 17:55 ` Alain Frisch
2013-03-13 19:44   ` Jon Harrop
2013-03-13 21:02     ` Alain Frisch
2013-03-13 18:27 ` oliver
2013-03-13 20:00   ` Jon Harrop

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='037b01ce2307$d54e6130$7feb2390$@ffconsultancy.com' \
    --to=jon@ffconsultancy.com \
    --cc=caml-list@inria.fr \
    --cc=ontologiae@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).