From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by sympa.inria.fr (Postfix) with ESMTPS id 5F3B77EE80 for ; Wed, 20 Mar 2013 23:35:05 +0100 (CET) Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of rdicosmo@gmail.com) identity=pra; client-ip=74.125.82.179; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="rdicosmo@gmail.com"; x-sender="rdicosmo@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail2-smtp-roc.national.inria.fr: domain of rdicosmo@gmail.com designates 74.125.82.179 as permitted sender) identity=mailfrom; client-ip=74.125.82.179; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="rdicosmo@gmail.com"; x-sender="rdicosmo@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-we0-f179.google.com) identity=helo; client-ip=74.125.82.179; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="rdicosmo@gmail.com"; x-sender="postmaster@mail-we0-f179.google.com"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqIMAFA4SlFKfVKzgmdsb2JhbABEhT6tSYxxhTIDAYFSFg4BAQsJFBIGJIIkAQEFOgYBFBkLAQMIBAEFBREEAQEBCQQaBw8FIAEFAQEJAQkOEwkJCYdnAw8EAwWkII8thHsnAwqJVQEFDAmNRAZTUhiCWWEDll+BIIpMgzU/gVmCVYFpCRc X-IPAS-Result: AqIMAFA4SlFKfVKzgmdsb2JhbABEhT6tSYxxhTIDAYFSFg4BAQsJFBIGJIIkAQEFOgYBFBkLAQMIBAEFBREEAQEBCQQaBw8FIAEFAQEJAQkOEwkJCYdnAw8EAwWkII8thHsnAwqJVQEFDAmNRAZTUhiCWWEDll+BIIpMgzU/gVmCVYFpCRc X-IronPort-AV: E=Sophos;i="4.84,880,1355094000"; d="scan'208";a="8568955" Received: from mail-we0-f179.google.com ([74.125.82.179]) by mail2-smtp-roc.national.inria.fr with ESMTP/TLS/RC4-SHA; 20 Mar 2013 23:35:04 +0100 Received: by mail-we0-f179.google.com with SMTP id u3so1797096wey.38 for ; Wed, 20 Mar 2013 15:35:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:sender:date:from:to:cc:subject:message-id:references :mime-version:content-type:content-disposition:in-reply-to :user-agent; bh=sZ5XG1wBZU0NJXoffpETzhXFjGVfVDBRVD2xI1b3hfE=; b=F9d+Z9cXKSTT1fndq4p1lbVY7Cf+pwEQ3Gren61TcoRJWWSmSgLzpnW2bDSyGWyQBt xeKV1Y31R8080+dJL0/S3CQoTd2tZMO5kNW2LDhmE4XvSAyWmrChlwoLhldL4HLUtbJc ZT61ivK8UZgPmDWTfQ5FuW4D0d2+wFdJmNpxylcxVbnXwv9vJ/lEBV0Psa0mU/atEoM0 /gJQWV9AGeBSzGqD6IzuZk8qE4r6QMPzfnUgpxT5yT0i2xqnj8ZB7f5ty4xM8O5kEHOM ZiLUm62/IkDyItpZLjkhtr9ONWgXSJWELuDku7i7kLqd6GW/9EiCHwh08PFMZ5QtqboC nz7g== X-Received: by 10.181.13.175 with SMTP id ez15mr1343875wid.8.1363818904620; Wed, 20 Mar 2013 15:35:04 -0700 (PDT) Received: from voyager (bny92-3-81-56-44-163.fbx.proxad.net. [81.56.44.163]) by mx.google.com with ESMTPS id ex15sm1410219wid.5.2013.03.20.15.35.02 (version=TLSv1 cipher=RC4-SHA bits=128/128); Wed, 20 Mar 2013 15:35:03 -0700 (PDT) Sender: Roberto Di Cosmo Received: from dicosmo by voyager with local (Exim 4.72) (envelope-from ) id 1UIRb3-0007Lq-Fq; Wed, 20 Mar 2013 23:35:01 +0100 Date: Wed, 20 Mar 2013 23:35:01 +0100 From: Roberto Di Cosmo To: Jon Harrop Cc: 'Francois Berenger' , caml-list@inria.fr Message-ID: <20130320223501.GA28114@voyager> References: <00ba01ce200c$d23f1910$76bd4b30$@ffconsultancy.com> <010801ce201f$afd6ad80$0f840880$@ffconsultancy.com> <037b01ce2307$d54e6130$7feb2390$@ffconsultancy.com> <5147C448.5030803@riken.jp> <069d01ce25ad$30bd3800$9237a800$@ffconsultancy.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <069d01ce25ad$30bd3800$9237a800$@ffconsultancy.com> User-Agent: Mutt/1.5.20 (2009-06-14) Subject: Re: [Caml-list] Case study in optimization: porting a compiler from OCaml to F# Hi Jon, a concrete set of well justified benchmarks could serve the cause more than any abstract discussion; please feel free to set it up, run it, and analyze the results. Having spent quite a bit of energy on Parmap, after it started as a sort of a one-afternoon project, and with the experience of the now very old OCamlP3l library that started much of this at the end of the '90s (including a detour through an experimental reimplementation in Haskell), I definitely took the St Thomas stance with this kind of issues :-) -- Roberto On Wed, Mar 20, 2013 at 08:54:59PM -0000, Jon Harrop wrote: > > Not just the granularity. Also the communication including any communication involved in scatter and gather phases. That differs a lot more between OCaml and F#. Fork does copy-on-write but (IIRC) the GC can incur unnecessary copying but, more importantly, requires the gather phase to deep copy results back to the original process. In contrast, data can be passed by reference in F#. > > Would be very interesting to benchmark this... > > Cheers, > Jon. > > -----Original Message----- > From: caml-list-request@inria.fr [mailto:caml-list-request@inria.fr] On Behalf Of Francois Berenger > Sent: 19 March 2013 01:50 > To: caml-list@inria.fr > Subject: Re: [Caml-list] Case study in optimization: porting a compiler from OCaml to F# > > I have observed and measured perfect scalability with up to 4 cores of an OCaml program using Parmap. > With more than 4 cores, the scalability was degrading. > > I think the scalability of the program depends only on the granularity of the tasks. The tasks were coarse in my case. > > F. > > On 03/17/2013 09:06 PM, Jon Harrop wrote: > > Pierre-Alexandre Voye wrote: > >> So you could maybe use Parmap.map ? > >> Parmap.parmap ~ncores:4 funct (Parmap.L elem_list) > > > > What happens if the inner function returns results via mutation? I assume you must rearrange the code to return all results explicitly and they will then be deep copied (which destroys scalability due to limited shared memory bandwidth on multicores). > > > > Does it do load balancing? I assume not given that ncores is hardcoded. > > > > Does a parmap with ncores=4 inside a parmap with ncores=4 create 16 processes? > > > > Does it deep copy inputs and/or outputs? I assume so, at least for outputs, because you cannot write results in-place without a shared mutable heap. > > > > Does parmap have a large constant overhead? I assume so if it is forking processes. > > > > Another solution is to prefork and explicitly communicate all inputs using message passing but this is equally problematic. You have to rearrange the code. Deep copying inputs also destroys scalability. > > > > Cheers, > > Jon. > > > > > > > > > -- > Caml-list mailing list. Subscription management and archives: > https://sympa.inria.fr/sympa/arc/caml-list > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-bugs > > > -- > Caml-list mailing list. Subscription management and archives: > https://sympa.inria.fr/sympa/arc/caml-list > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-bugs -- Roberto Di Cosmo ------------------------------------------------------------------ Professeur En delegation a l'INRIA PPS E-mail: roberto@dicosmo.org Universite Paris Diderot WWW : http://www.dicosmo.org Case 7014 Tel : ++33-(0)1-57 27 92 20 5, Rue Thomas Mann F-75205 Paris Cedex 13 Identica: http://identi.ca/rdicosmo FRANCE. Twitter: http://twitter.com/rdicosmo ------------------------------------------------------------------ Attachments: MIME accepted, Word deprecated http://www.gnu.org/philosophy/no-word-attachments.html ------------------------------------------------------------------ Office location: Bureau 320 (3rd floor) Batiment Sophie Germain Avenue de France Metro Bibliotheque Francois Mitterrand, ligne 14/RER C ----------------------------------------------------------------- GPG fingerprint 2931 20CE 3A5A 5390 98EC 8BFC FCCA C3BE 39CB 12D3