From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by sympa.inria.fr (Postfix) with ESMTPS id 77C317EE80 for ; Thu, 21 Mar 2013 08:36:15 +0100 (CET) Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of rdicosmo@gmail.com) identity=pra; client-ip=209.85.214.182; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="rdicosmo@gmail.com"; x-sender="rdicosmo@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail2-smtp-roc.national.inria.fr: domain of rdicosmo@gmail.com designates 209.85.214.182 as permitted sender) identity=mailfrom; client-ip=209.85.214.182; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="rdicosmo@gmail.com"; x-sender="rdicosmo@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-ob0-f182.google.com) identity=helo; client-ip=209.85.214.182; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="rdicosmo@gmail.com"; x-sender="postmaster@mail-ob0-f182.google.com"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AmsCAHi3SlHRVda2gmdsb2JhbABEhUatTJIpgVAIFg4BAQsJFBIGJIIkAQEFQAEUGQsBAwgEAQUFCwMDBAEBAQ0aByISAQUBCgEJCAYTCQkJh2cDDwcFpCiPLYRpJwMKiVUBBQyMMIEeBoElGIM6A5ZggR+KTIM1FimBWYJVO4EuCRc X-IPAS-Result: AmsCAHi3SlHRVda2gmdsb2JhbABEhUatTJIpgVAIFg4BAQsJFBIGJIIkAQEFQAEUGQsBAwgEAQUFCwMDBAEBAQ0aByISAQUBCgEJCAYTCQkJh2cDDwcFpCiPLYRpJwMKiVUBBQyMMIEeBoElGIM6A5ZggR+KTIM1FimBWYJVO4EuCRc X-IronPort-AV: E=Sophos;i="4.84,884,1355094000"; d="scan'208";a="8602074" Received: from mail-ob0-f182.google.com ([209.85.214.182]) by mail2-smtp-roc.national.inria.fr with ESMTP/TLS/RC4-SHA; 21 Mar 2013 08:35:59 +0100 Received: by mail-ob0-f182.google.com with SMTP id va7so2564620obc.13 for ; Thu, 21 Mar 2013 00:35:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=GWhkg9ptnbVdgXQ+hmpCZf3pDd2hdtYUX8jqq63Sv+8=; b=wVu1cAi1xLwMcHNiINmc4M4VZ7eojKrqeELLMyxZtFWWJbS2umaoZz1r/2NMiL61wB wmaTIt4guG9UnsWLJrUF0nTJp2LUz6/zHOeusf8zrE+l+pOdYOVo+lU0tRU8GvL9N6L7 PE6uq+M3N1qh64Hv9m8PfH7X6YYGQ9BfrWaUhla4ddLpr/uFKb89K08eoZxbdx6cjLCs 6T6UCjpaB7j+d6e2yO81ppTpcxRurbt9KM9LKg2aZHHRATKfekBLAkFtJyINcTN2Ty+T KV4BWijArWi1oY2RmUwWy5Zvb2iuIxZUa2CiIGluHDrzj6xJSmzNyiiYEjstMlLM0iOs ZswA== MIME-Version: 1.0 X-Received: by 10.182.46.228 with SMTP id y4mr6241733obm.13.1363851357965; Thu, 21 Mar 2013 00:35:57 -0700 (PDT) Sender: rdicosmo@gmail.com Received: by 10.60.131.42 with HTTP; Thu, 21 Mar 2013 00:35:57 -0700 (PDT) In-Reply-To: References: <00ba01ce200c$d23f1910$76bd4b30$@ffconsultancy.com> <010801ce201f$afd6ad80$0f840880$@ffconsultancy.com> <037b01ce2307$d54e6130$7feb2390$@ffconsultancy.com> <5147C448.5030803@riken.jp> <069d01ce25ad$30bd3800$9237a800$@ffconsultancy.com> <20130320223501.GA28114@voyager> Date: Thu, 21 Mar 2013 08:35:57 +0100 X-Google-Sender-Auth: xFv3aqFzCMuKNooW-ANe6WXlXfQ Message-ID: From: Roberto Di Cosmo To: Mike Lin Cc: Jon Harrop , Francois Berenger , "caml-list@inria.fr" Content-Type: text/plain; charset=ISO-8859-1 Subject: Re: [Caml-list] Case study in optimization: porting a compiler from OCaml to F# Hi Mike, that is something that one could add rather easily... just open an issue here: https://github.com/rdicosmo/parmap 2013/3/21 Mike Lin : > Just a comment FWIW, I personally moved away from Parmap because it doesn't > deal well with exceptions raised in the child processes. I know the > brokenness of exception marshalling (PR#0001961) complicates this, but I > came up with something I felt was pretty reasonable in my iteration of the > same concept: > https://github.com/mlin/forkwork > Of course I didn't reimplement some of Parmap's fancier features, but this > solution has been "just right" for my applications. I hope this can inspire > a future improvement to exception handling in Parmap. > Mike > > > > > On Wed, Mar 20, 2013 at 3:35 PM, Roberto Di Cosmo > wrote: >> >> Hi Jon, >> a concrete set of well justified benchmarks could serve >> the cause more than any abstract discussion; please feel >> free to set it up, run it, and analyze the results. >> >> Having spent quite a bit of energy on Parmap, after it >> started as a sort of a one-afternoon project, and with >> the experience of the now very old OCamlP3l library that >> started much of this at the end of the '90s (including a >> detour through an experimental reimplementation in Haskell), >> I definitely took the St Thomas stance with this kind of issues :-) >> >> -- >> Roberto >> >> On Wed, Mar 20, 2013 at 08:54:59PM -0000, Jon Harrop wrote: >> > >> > Not just the granularity. Also the communication including any >> > communication involved in scatter and gather phases. That differs a lot more >> > between OCaml and F#. Fork does copy-on-write but (IIRC) the GC can incur >> > unnecessary copying but, more importantly, requires the gather phase to deep >> > copy results back to the original process. In contrast, data can be passed >> > by reference in F#. >> > >> > Would be very interesting to benchmark this... >> > >> > Cheers, >> > Jon. >> > >> > -----Original Message----- >> > From: caml-list-request@inria.fr [mailto:caml-list-request@inria.fr] On >> > Behalf Of Francois Berenger >> > Sent: 19 March 2013 01:50 >> > To: caml-list@inria.fr >> > Subject: Re: [Caml-list] Case study in optimization: porting a compiler >> > from OCaml to F# >> > >> > I have observed and measured perfect scalability with up to 4 cores of >> > an OCaml program using Parmap. >> > With more than 4 cores, the scalability was degrading. >> > >> > I think the scalability of the program depends only on the granularity >> > of the tasks. The tasks were coarse in my case. >> > >> > F. >> > >> > On 03/17/2013 09:06 PM, Jon Harrop wrote: >> > > Pierre-Alexandre Voye wrote: >> > >> So you could maybe use Parmap.map ? >> > >> Parmap.parmap ~ncores:4 funct (Parmap.L elem_list) >> > > >> > > What happens if the inner function returns results via mutation? I >> > > assume you must rearrange the code to return all results explicitly and they >> > > will then be deep copied (which destroys scalability due to limited shared >> > > memory bandwidth on multicores). >> > > >> > > Does it do load balancing? I assume not given that ncores is >> > > hardcoded. >> > > >> > > Does a parmap with ncores=4 inside a parmap with ncores=4 create 16 >> > > processes? >> > > >> > > Does it deep copy inputs and/or outputs? I assume so, at least for >> > > outputs, because you cannot write results in-place without a shared mutable >> > > heap. >> > > >> > > Does parmap have a large constant overhead? I assume so if it is >> > > forking processes. >> > > >> > > Another solution is to prefork and explicitly communicate all inputs >> > > using message passing but this is equally problematic. You have to rearrange >> > > the code. Deep copying inputs also destroys scalability. >> > > >> > > Cheers, >> > > Jon. >> > > >> > > >> > > >> > >> > >> > -- >> > Caml-list mailing list. Subscription management and archives: >> > https://sympa.inria.fr/sympa/arc/caml-list >> > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners >> > Bug reports: http://caml.inria.fr/bin/caml-bugs >> > >> > >> > -- >> > Caml-list mailing list. Subscription management and archives: >> > https://sympa.inria.fr/sympa/arc/caml-list >> > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners >> > Bug reports: http://caml.inria.fr/bin/caml-bugs >> >> -- >> Roberto Di Cosmo >> >> ------------------------------------------------------------------ >> Professeur En delegation a l'INRIA >> PPS E-mail: roberto@dicosmo.org >> Universite Paris Diderot WWW : http://www.dicosmo.org >> Case 7014 Tel : ++33-(0)1-57 27 92 20 >> 5, Rue Thomas Mann >> F-75205 Paris Cedex 13 Identica: http://identi.ca/rdicosmo >> FRANCE. Twitter: http://twitter.com/rdicosmo >> ------------------------------------------------------------------ >> Attachments: >> MIME accepted, Word deprecated >> http://www.gnu.org/philosophy/no-word-attachments.html >> ------------------------------------------------------------------ >> Office location: >> >> Bureau 320 (3rd floor) >> Batiment Sophie Germain >> Avenue de France >> Metro Bibliotheque Francois Mitterrand, ligne 14/RER C >> ----------------------------------------------------------------- >> GPG fingerprint 2931 20CE 3A5A 5390 98EC 8BFC FCCA C3BE 39CB 12D3 >> >> -- >> Caml-list mailing list. Subscription management and archives: >> https://sympa.inria.fr/sympa/arc/caml-list >> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners >> Bug reports: http://caml.inria.fr/bin/caml-bugs > > -- --Roberto Di Cosmo ------------------------------------------------------------------ Professeur En delegation a l'INRIA PPS E-mail: roberto@dicosmo.org Universite Paris Diderot WWW : http://www.dicosmo.org Case 7014 Tel : ++33-(0)1-57 27 92 20 5, Rue Thomas Mann F-75205 Paris Cedex 13 FRANCE. ------------------------------------------------------------------ Attachments: MIME accepted Word deprecated, http://www.rfc1149.net/documents/whynotword ------------------------------------------------------------------ Office location: Bureau 6C15 (6th floor) 175, rue du Chevaleret, XIII Metro Chevaleret, ligne 6 ------------------------------------------------------------------