From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <berenger@riken.jp>
X-Original-To: caml-list@sympa.inria.fr
Delivered-To: caml-list@sympa.inria.fr
Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83])
	by sympa.inria.fr (Postfix) with ESMTPS id 4CA6F7EE8A
	for <caml-list@sympa.inria.fr>; Tue, 19 Mar 2013 02:50:05 +0100 (CET)
Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender
  authenticity information available from domain of
  berenger@riken.jp) identity=pra; client-ip=134.160.33.161;
  receiver=mail2-smtp-roc.national.inria.fr;
  envelope-from="berenger@riken.jp";
  x-sender="berenger@riken.jp"; x-conformance=sidf_compatible
Received-SPF: Pass (mail2-smtp-roc.national.inria.fr: domain of
  berenger@riken.jp designates 134.160.33.161 as permitted
  sender) identity=mailfrom; client-ip=134.160.33.161;
  receiver=mail2-smtp-roc.national.inria.fr;
  envelope-from="berenger@riken.jp";
  x-sender="berenger@riken.jp"; x-conformance=sidf_compatible;
  x-record-type="v=spf1"
Received-SPF: Pass (mail2-smtp-roc.national.inria.fr: domain of
  postmaster@postman.riken.jp designates 134.160.33.161 as
  permitted sender) identity=helo; client-ip=134.160.33.161;
  receiver=mail2-smtp-roc.national.inria.fr;
  envelope-from="berenger@riken.jp";
  x-sender="postmaster@postman.riken.jp";
  x-conformance=sidf_compatible; x-record-type="v=spf1"
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: As8AAFnDR1GGoCGhmWdsb2JhbABDiC+6I4JugXYOAQEBAQETCwcUKIIkAQEFIxVAEQsYAgIFFgsCAgkDAgECAUUTCAEBF4d5r2qCQJAWgSOMN4E9FoIXgRMDiHONa4V9jh6BWw
X-IPAS-Result: As8AAFnDR1GGoCGhmWdsb2JhbABDiC+6I4JugXYOAQEBAQETCwcUKIIkAQEFIxVAEQsYAgIFFgsCAgkDAgECAUUTCAEBF4d5r2qCQJAWgSOMN4E9FoIXgRMDiHONa4V9jh6BWw
X-IronPort-AV: E=Sophos;i="4.84,868,1355094000"; 
   d="scan'208";a="8211494"
Received: from postman1.riken.jp (HELO postman.riken.jp) ([134.160.33.161])
  by mail2-smtp-roc.national.inria.fr with ESMTP; 19 Mar 2013 02:50:04 +0100
Received: from postman.riken.jp (postman1.riken.jp [127.0.0.1])
	by postman.riken.jp (Postfix) with SMTP id D3ED632C00E5
	for <caml-list@inria.fr>; Tue, 19 Mar 2013 10:50:00 +0900 (JST)
Received: from [172.27.98.103] (rikad98.riken.jp [134.160.214.98])
	by postman.riken.jp (Postfix) with ESMTPA id 9FBE632A0047
	for <caml-list@inria.fr>; Tue, 19 Mar 2013 10:50:00 +0900 (JST)
Message-ID: <5147C448.5030803@riken.jp>
Date: Tue, 19 Mar 2013 10:50:00 +0900
From: Francois Berenger <berenger@riken.jp>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130106 Thunderbird/17.0.2
MIME-Version: 1.0
To: caml-list@inria.fr
References: <00ba01ce200c$d23f1910$76bd4b30$@ffconsultancy.com> <CABkc=AM91=+O-BOt_uUvW7T4YG2vJ_WU7-BtQQHnpL-yuVR34g@mail.gmail.com> <010801ce201f$afd6ad80$0f840880$@ffconsultancy.com> <CANnJ5GeEmvJni+Kf06wXf14s4zZXhLug9JRSfAW8XazQc6dmLQ@mail.gmail.com> <037b01ce2307$d54e6130$7feb2390$@ffconsultancy.com>
In-Reply-To: <037b01ce2307$d54e6130$7feb2390$@ffconsultancy.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-PMX-Version: 5.6.0.2009776, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2013.3.19.13921
Subject: Re: [Caml-list] Case study in optimization: porting a compiler from
 OCaml to F#

I have observed and measured perfect scalability with up to 4 cores
of an OCaml program using Parmap.
With more than 4 cores, the scalability was degrading.

I think the scalability of the program depends only on the
granularity of the tasks. The tasks were coarse in my case.

F.

On 03/17/2013 09:06 PM, Jon Harrop wrote:
> Pierre-Alexandre Voye wrote:
>> So you could maybe use Parmap.map ?
>> Parmap.parmap ~ncores:4 funct (Parmap.L elem_list)
>
> What happens if the inner function returns results via mutation? I assume you must rearrange the code to return all results explicitly and they will then be deep copied (which destroys scalability due to limited shared memory bandwidth on multicores).
>
> Does it do load balancing? I assume not given that ncores is hardcoded.
>
> Does a parmap with ncores=4 inside a parmap with ncores=4 create 16 processes?
>
> Does it deep copy inputs and/or outputs? I assume so, at least for outputs, because you cannot write results in-place without a shared mutable heap.
>
> Does parmap have a large constant overhead? I assume so if it is forking processes.
>
> Another solution is to prefork and explicitly communicate all inputs using message passing but this is equally problematic. You have to rearrange the code. Deep copying inputs also destroys scalability.
>
> Cheers,
> Jon.
>
>
>