Re: [Caml-list] Case study in optimization: porting a compiler from OCaml to F#

[Francois Berenger <berenger@riken.jp>]

I have observed and measured perfect scalability with up to 4 cores
of an OCaml program using Parmap.
With more than 4 cores, the scalability was degrading.

I think the scalability of the program depends only on the
granularity of the tasks. The tasks were coarse in my case.

F.

On 03/17/2013 09:06 PM, Jon Harrop wrote:
> Pierre-Alexandre Voye wrote:
>> So you could maybe use Parmap.map ?
>> Parmap.parmap ~ncores:4 funct (Parmap.L elem_list)
>
> What happens if the inner function returns results via mutation? I assume you must rearrange the code to return all results explicitly and they will then be deep copied (which destroys scalability due to limited shared memory bandwidth on multicores).
>
> Does it do load balancing? I assume not given that ncores is hardcoded.
>
> Does a parmap with ncores=4 inside a parmap with ncores=4 create 16 processes?
>
> Does it deep copy inputs and/or outputs? I assume so, at least for outputs, because you cannot write results in-place without a shared mutable heap.
>
> Does parmap have a large constant overhead? I assume so if it is forking processes.
>
> Another solution is to prefork and explicitly communicate all inputs using message passing but this is equally problematic. You have to rearrange the code. Deep copying inputs also destroys scalability.
>
> Cheers,
> Jon.
>
>
>