From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by sympa.inria.fr (Postfix) with ESMTPS id 3B24E7EE80 for ; Wed, 20 Mar 2013 21:54:53 +0100 (CET) Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of jon@ffconsultancy.com) identity=pra; client-ip=84.93.230.244; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="jon@ffconsultancy.com"; x-sender="jon@ffconsultancy.com"; x-conformance=sidf_compatible Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of jon@ffconsultancy.com) identity=mailfrom; client-ip=84.93.230.244; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="jon@ffconsultancy.com"; x-sender="jon@ffconsultancy.com"; x-conformance=sidf_compatible Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of postmaster@avasout03.plus.net) identity=helo; client-ip=84.93.230.244; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="jon@ffconsultancy.com"; x-sender="postmaster@avasout03.plus.net"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AlEDAA8hSlFUXeb0gWdsb2JhbABEiBiqaXqRLYFSFg4BARYmKIIkAQEFCAIZSxcBAwIJEQQBAQECAgkaAwICGSMJAQkIAgQBEgsFAgmHZwMTCLABiGQDiWWBI4w2gSsSBoIngRMDjiyJU5JtgWo X-IPAS-Result: AlEDAA8hSlFUXeb0gWdsb2JhbABEiBiqaXqRLYFSFg4BARYmKIIkAQEFCAIZSxcBAwIJEQQBAQECAgkaAwICGSMJAQkIAgQBEgsFAgmHZwMTCLABiGQDiWWBI4w2gSsSBoIngRMDjiyJU5JtgWo X-IronPort-AV: E=Sophos;i="4.84,880,1355094000"; d="scan'208";a="8561569" Received: from avasout03.plus.net ([84.93.230.244]) by mail2-smtp-roc.national.inria.fr with ESMTP; 20 Mar 2013 21:54:52 +0100 Received: from XPS ([87.112.110.254]) by avasout03 with smtp id Dwuq1l0015VNGGP01wurGY; Wed, 20 Mar 2013 20:54:52 +0000 X-CM-Score: 0.00 X-CNFS-Analysis: v=2.0 cv=Df3JXIRW c=1 sm=1 a=fP/72gr2kmGa62nm52o1XQ==:17 a=9NSLKdjcVD4A:10 a=Xub9RBUEA-sA:10 a=Kvk-SOs2Z7YA:10 a=IkcTkHD0fZMA:10 a=r2vSxAw-AAAA:8 a=cdQsuJNHsHMA:10 a=ZOzjf2MOAAAA:8 a=CjxXgO3LAAAA:8 a=vuDvQPjcJ8dN9IRidosA:9 a=QEXdDO2ut3YA:10 a=fP/72gr2kmGa62nm52o1XQ==:117 X-AUTH: jdh302:2500 Reply-To: From: "Jon Harrop" To: "'Francois Berenger'" , References: <00ba01ce200c$d23f1910$76bd4b30$@ffconsultancy.com> <010801ce201f$afd6ad80$0f840880$@ffconsultancy.com> <037b01ce2307$d54e6130$7feb2390$@ffconsultancy.com> <5147C448.5030803@riken.jp> In-Reply-To: <5147C448.5030803@riken.jp> Date: Wed, 20 Mar 2013 20:54:59 -0000 Organization: Flying Frog Consultancy Ltd. Message-ID: <069d01ce25ad$30bd3800$9237a800$@ffconsultancy.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQGTusbNW31EwOb0tRgwTltM6UzJNgH30vQ2ArLJNnABxXIViQDiwtZpAmLg1tOY1i88sA== Content-Language: en-gb Subject: RE: [Caml-list] Case study in optimization: porting a compiler from OCaml to F# Not just the granularity. Also the communication including any communicatio= n involved in scatter and gather phases. That differs a lot more between OC= aml and F#. Fork does copy-on-write but (IIRC) the GC can incur unnecessary= copying but, more importantly, requires the gather phase to deep copy resu= lts back to the original process. In contrast, data can be passed by refere= nce in F#. Would be very interesting to benchmark this... Cheers, Jon. -----Original Message----- From: caml-list-request@inria.fr [mailto:caml-list-request@inria.fr] On Beh= alf Of Francois Berenger Sent: 19 March 2013 01:50 To: caml-list@inria.fr Subject: Re: [Caml-list] Case study in optimization: porting a compiler fro= m OCaml to F# I have observed and measured perfect scalability with up to 4 cores of an O= Caml program using Parmap. With more than 4 cores, the scalability was degrading. I think the scalability of the program depends only on the granularity of t= he tasks. The tasks were coarse in my case. F. On 03/17/2013 09:06 PM, Jon Harrop wrote: > Pierre-Alexandre Voye wrote: >> So you could maybe use Parmap.map ? >> Parmap.parmap ~ncores:4 funct (Parmap.L elem_list) > > What happens if the inner function returns results via mutation? I assume= you must rearrange the code to return all results explicitly and they will= then be deep copied (which destroys scalability due to limited shared memo= ry bandwidth on multicores). > > Does it do load balancing? I assume not given that ncores is hardcoded. > > Does a parmap with ncores=3D4 inside a parmap with ncores=3D4 create 16 p= rocesses? > > Does it deep copy inputs and/or outputs? I assume so, at least for output= s, because you cannot write results in-place without a shared mutable heap. > > Does parmap have a large constant overhead? I assume so if it is forking = processes. > > Another solution is to prefork and explicitly communicate all inputs usin= g message passing but this is equally problematic. You have to rearrange th= e code. Deep copying inputs also destroys scalability. > > Cheers, > Jon. > > > -- Caml-list mailing list. Subscription management and archives: https://sympa.inria.fr/sympa/arc/caml-list Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs