From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by sympa.inria.fr (Postfix) with ESMTPS id 678F27FD92 for ; Sat, 11 Jun 2016 01:15:14 +0200 (CEST) IronPort-PHdr: 9a23:2b8N2hLwbwym0D/7gtmcpTZWNBhigK39O0sv0rFitYgUIv7xwZ3uMQTl6Ol3ixeRBMOAu6MC1Lqd6f6ocFdDyKjCmUhKSIZLWR4BhJdetC0bK+nBN3fGKuX3ZTcxBsVIWQwt1Xi6NU9IBJS2PAWK8TWM5DIfUi/yKRBybrysXNWC3oLoh6vopdX6WEZhunmUWftKNhK4rAHc5IE9oLBJDeIP8CbPuWZCYO9MxGlldhq5lhf44dqsrtY4q3wD89pozcNLUL37cqIkVvQYSW1+ayFmrPHs4CLfRA2O/X8VTC0/iAZFBgvE6lmuV43tvy75t+xV1yyTPMmwRrcxD2eM9aBuHS7hkiABfxs49nrUm4QknadapgmitjR9yojZe52POfdiOKjaeIVJFiJ6Qs9NWnkZUcuHZIwVAr9EZL4Aog== Authentication-Results: mail2-smtp-roc.national.inria.fr; spf=None smtp.pra=yminsky@janestreet.com; spf=Pass smtp.mailfrom=yminsky@janestreet.com; spf=None smtp.helo=postmaster@mxout3.mail.janestreet.com Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of yminsky@janestreet.com) identity=pra; client-ip=38.105.200.229; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="yminsky@janestreet.com"; x-sender="yminsky@janestreet.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail2-smtp-roc.national.inria.fr: domain of yminsky@janestreet.com designates 38.105.200.229 as permitted sender) identity=mailfrom; client-ip=38.105.200.229; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="yminsky@janestreet.com"; x-sender="yminsky@janestreet.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of postmaster@mxout3.mail.janestreet.com) identity=helo; client-ip=38.105.200.229; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="yminsky@janestreet.com"; x-sender="postmaster@mxout3.mail.janestreet.com"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0BEAQBMSVtXc+XIaSZehBR9BqoJhReMAIF5JIVzAoEiBzgUAQEBAQEBAQERAQoWCVCCMIIVAQEBBBIRHQEBLAsBCwQLCwYEAQEBAgIJGgMCAiEBEgEFAQoBAwEFCAYTEgkHh3QDFwMLn0iBMT4xilRnhEIBAQWIEgMKg34BAQEBAQEBAQEBAQEBAQEBAQEBAQEUCBBxhSaETYJDgWcWgwGCWoZKDJFaNIYEhiqBeoI3jGmHfoYuEh6BDx6CMA0cgWdSAYoHAQEB X-IPAS-Result: A0BEAQBMSVtXc+XIaSZehBR9BqoJhReMAIF5JIVzAoEiBzgUAQEBAQEBAQERAQoWCVCCMIIVAQEBBBIRHQEBLAsBCwQLCwYEAQEBAgIJGgMCAiEBEgEFAQoBAwEFCAYTEgkHh3QDFwMLn0iBMT4xilRnhEIBAQWIEgMKg34BAQEBAQEBAQEBAQEBAQEBAQEBAQEUCBBxhSaETYJDgWcWgwGCWoZKDJFaNIYEhiqBeoI3jGmHfoYuEh6BDx6CMA0cgWdSAYoHAQEB X-IronPort-AV: E=Sophos;i="5.26,452,1459807200"; d="scan'208";a="221994181" Received: from mxout3.mail.janestreet.com ([38.105.200.229]) by mail2-smtp-roc.national.inria.fr with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 11 Jun 2016 01:14:54 +0200 Received: from tot-qpr-mailcore2.delacy.com ([172.27.56.106] helo=tot-qpr-mailcore2) by mxout3.mail.janestreet.com with esmtps (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.82) (envelope-from ) id 1bBVdc-0004fu-TF for caml-list@inria.fr; Fri, 10 Jun 2016 19:14:52 -0400 X-JS-Flow: external Received: by tot-qpr-mailcore2 with JS-mailcore (0.1) (envelope-from ) id BXW0ns-AAAEGQ-a1; 2016-06-10 19:14:52.859723-04:00 Received: from mail-yw0-f197.google.com ([209.85.161.197]) by mxgoog1.mail.janestreet.com with esmtps (UNKNOWN:AES128-GCM-SHA256:128) (Exim 4.72) (envelope-from ) id 1bBVdc-0006so-OD for caml-list@inria.fr; Fri, 10 Jun 2016 19:14:52 -0400 Received: by mail-yw0-f197.google.com with SMTP id x189so191113284ywe.2 for ; Fri, 10 Jun 2016 16:14:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=janestreet.com; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-transfer-encoding; bh=KFGPP3OqUKoL6PoMb2DLVecPtwZTrt3bYctvn5+plwY=; b=i5M3QjJS0zhEtuUqg83koB943BcmqKX1x9TCRFIYmqi1fkMFV/BfwuPSk63fwguLH1 e6t0AQ7w4I7/uiBWnb3UMAg1YwoAAVODnuhpazAnTngF1CSouW86/04tzpTo7CaO0cIL OnBS6e+nVcMFM31yyXIYK+HH5A/d7svLo3dSU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-transfer-encoding; bh=KFGPP3OqUKoL6PoMb2DLVecPtwZTrt3bYctvn5+plwY=; b=ZK9W0+HHer+D1Ry4ERqnaKylp1kmKqHTukWXl4BrjcVvsjtfzkSJxtbrfwFLSmzaEq 8Iruoza32oLzqBjBqQ9/85Z3Rm6RTuLsq66W6NT2/3y7OWJD4+1wPb50RHBwfdiCzszg LiFoAtbqd6mg6i4FoPwJ2boe/Cj78IJ94ukCdKPPZz/X3096XoEzKB2RZqj++hNq3P3c YFKQd5K2ZVWcD4Buk820LIUEnPHByrQdvdwSA0Jcuaq8DDAHS0Kj2Ct8oq/M3VzZLCTx 6UOSXUETsdU53Vn2SjaPFYpgZtHjoxzdG/5Kxi/5MOwde2GCni46909vbGWwu6nY7seJ jweg== X-Gm-Message-State: ALyK8tKxu2eb9r3QMEphUP8esGpBITnWUXgSXdooXYju1paud0AofGeTY7vZeF0uBGp+SXHIgVJTSmNvO3vZqyDMa1PAs5Sh7QNVqWS2yGsfdmORm2vDSPAYY1481ibSW08bqzLXtjP2HdbNiZozenz1/w0EOxfxKipAouUUMtQ= X-Received: by 10.13.205.69 with SMTP id p66mr2600546ywd.255.1465600492301; Fri, 10 Jun 2016 16:14:52 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.13.205.69 with SMTP id p66mr2600527ywd.255.1465600491949; Fri, 10 Jun 2016 16:14:51 -0700 (PDT) Received: by 10.37.52.131 with HTTP; Fri, 10 Jun 2016 16:14:51 -0700 (PDT) In-Reply-To: References: <00c801d1c357$95598ca0$c00ca5e0$@ffconsultancy.com> Date: Fri, 10 Jun 2016 19:14:51 -0400 Message-ID: From:Yaron Minsky To:Stanislav Artemkin Cc:Jon Harrop , Gabriel Scherer , caml users , Damien Doligez Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-JS-Processed-by: mailcore Subject: Re: [Caml-list] Measuring GC latencies for OCaml program 10ms and better, more like it. You have to know what you're doing, but you can do as well as C++ in OCaml for many workloads. Just don't allocate anything. And even without any extreme style changes, you can do a hell of a lot better than 10ms, depending on exactly what you're doing. y On Fri, Jun 10, 2016 at 5:34 PM, Stanislav Artemkin wr= ote: > Very interesting! It seems it was completely broken C++ solution. Wish I > could use OCaml for current project, but we still have to use C++ to get > microsecond latencies. > > Do I correctly understand that OCaml is suitable for latencies ~10ms and > worse? > > Also, there is still an issue with multithreading. Did you use any existi= ng > solution? > > Thank you > > On Sat, Jun 11, 2016 at 12:35 AM, Jon Harrop wrot= e: >> >> >> Very interesting, thank you! >> >> We just implemented a substantial client and server system for the finan= ce >> sector with the "low" latency server written in OCaml. I have done this >> before in other languages and seen it done in many more languages. The O= Caml >> is by far the consistently-fastest solution I have ever seen. Orders of >> magnitude faster than the last C++ solution I saw. In particular, compar= ed >> to Java and .NET where we see substantial latencies from the GC at around >> 100ms, with OCaml there is no visible peak at high latency due to the GC= at >> all. And this project was implemented to a very short deadline with no t= ime >> for optimisation at all. >> >> On a related note, we used Jane St.'s Core and Async libraries as well as >> Cohttp and found them all to be *phenomenally* efficient and robust. >> >> In case anyone is interested, the only pain point I had was the >> development environment. I actually prototyped all my hard code in >> simplified F# in Visual Studio on Windows and then ported to OCaml. Emacs >> and Merlin crash and hang a lot for me: maybe 50 times per day. Hence my >> other post. :-) >> >> In terms of the language, OCaml was very well suited to this task. Lots = of >> purely functional data structures forming in-memory databases that can be >> queried in different ways and have many different versions of them store= d in >> different places at different times. Perhaps the main language feature I >> missed from F# was (surprisingly!) reflection. My F# client code uses >> reflection to serialize and deserialize messages. With no reflection I >> couldn't do that in OCaml so I used reflection in F# to autogenerate the >> OCaml code. >> >> Cheers, >> Jon. >> >> -----Original Message----- >> From: caml-list-request@inria.fr [mailto:caml-list-request@inria.fr] On >> Behalf Of Gabriel Scherer >> Sent: 30 May 2016 20:48 >> To: caml users >> Cc: Damien Doligez >> Subject: [Caml-list] Measuring GC latencies for OCaml program >> >> Dear caml-list, >> >> You may be interested in the following blog post, in which I give >> instructions to measure the worst-case latencies incurred by the GC: >> >> Measuring GC latencies in Haskell, OCaml, Racket >> >> http://prl.ccs.neu.edu/blog/2016/05/24/measuring-gc-latencies-in-haskell= -ocaml-racket/ >> >> In summary, the commands to measure latencies look something like: >> >> % build the program with the instrumented runtime >> ocamlbuild -tag "runtime_variant(i)" myprog.native >> >> % run with instrumentation enabled >> OCAML_INSTR_FILE=3D"ocaml.log" ./main.native >> >> % visualize results from the raw log >> $(OCAML_SOURCES)/tools/ocaml-instr-graph ocaml.log >> $(OCAML_SOURCES)/tools/ocaml-instr-report ocaml.log >> >> While the OCaml GC has had a good incremental mode with low latencies for >> most workloads for a long time, the ability to instrument it to actually >> measure latencies is still in its infancy: it is a side-result of Damien >> Doligez's stint at Jane Street last year, and >> 4.03.0 is the first release in which this work is available. >> >> A practical consequence of this youth is that the "user experience" of >> actually performing these measurements is currently very bad. The GC >> measurements are activated in an instrumented runtime variant (OCaml >> supports having several variants of the runtime available, and deciding >> which one to use for a specific program at link-time), which is the right >> design choice, but right now this variant is not built by default by the >> compiler distribution -- building it is a configure-time option disabled= by >> default. This means that, as a user interested in doing the measurements, >> you have to compile an alternative OCaml compiler. >> Furthermore, processing the raw instrumented log requires tool that are >> also in their infancy, and are currently included in the compiler >> distribution sources but not installed -- so you have to have a source >> checkout available to use them. In contrast, GHC's instrumentation is >> enabled by just passing the option "+RTS -s" to the Haskell program of >> interest; this is superbly convenient and something we should aim at. >> >> I discussed with Damien whether we should enable building the instrument= ed >> runtime by default (for example pass the --with-instrumented-runtime opt= ion >> to the opam switches people are using, and encourage distributions to us= e it >> in their packages as well). Of course there is a cost/benefit trade-off: >> currently virtually nobody is using this instrumentation, but enabling i= t by >> default would increase the compilation time of the compiler distribution= for >> everyone. (On my machine it only adds 5 seconds to total build time.) >> >> I personally think that we should aim for a rock-solid experience for >> profiling and instrumenting OCaml program enabled by default=C2=B9. It i= s worth >> making it slightly longer for anyone to install the compiler if we can m= ake >> it vastly easier to measure GC pauses in our program when the need arises >> (even if it's not very often). But that is a discussion that should be h= ad >> before making any choice. >> >> Regarding the log analysis tools, before finding about Damien's >> included-but-not installed tools (a shell and an awk script, in the fine= st >> Unix tradition) I built a quick&dirty OCaml script to do some measuremen= ts, >> which can be found in the benchmark repository below. It would not be mu= ch >> more work to grow this in a reusable library to extract the current log >> format into a structured data structure -- the format is undocumented but >> the provided scripts in tools/ have enough information to infer the >> structure. Such a script/library would, of course, remain tightly couple= d to >> the OCaml version, but I think it could be useful to have it packaged for >> users to play with. >> >> >> https://gitlab.com/gasche/gc-latency-experiment/blob/master/parse_ocaml_= log.ml >> >> =C2=B9: We cannot expect users to effectively write performant code if t= hey >> don't have the tool support for it. The fact that lazyness in Haskell ma= kes >> it harder for users to reason about efficiency or memory usage has made = the >> avaibility of excellent performance tooling *necessary*, where it is mer= ely >> nice-to-have in OCaml. Rather ironically, Haskell tooling is now much be= tter >> than OCaml's in this area, to the point that it can be easier to write >> efficient code in Haskell. >> >> Three side-notes on profiling tools: >> >> 1. `perf record --call-graph=3Ddwarf` works fine for ocamlopt binaries >> (no need for a frame-pointers switch), and this is documented: >> >> https://ocaml.org/learn/tutorials/performance_and_profiling.html#Usingpe= rfonLinux >> >> 2. Thomas Leonard has done excellent work on domain-specific profiling >> tools for Mirage, and this is the kind of tool support that I think >> should be available to anyone out of the box. >> http://roscidus.com/blog/blog/2014/08/15/optimising-the-unikernel/ >> >> http://roscidus.com/blog/blog/2014/10/27/visualising-an-asynchronous-mon= ad/ >> >> 3. There is currently more debate than anyone could wish for around >> a pull request of Mark Shinwell for runtime support for dynamic call >> graph construction and its use for memory profiling. >> https://github.com/ocaml/ocaml/pull/585 >> >> 4. Providing a good user experience for performance or space profiling >> is a fundamentally harder problem than for GC pauses. It may >> require specially-compiled versions of the libraries used by your >> program, and thus a general policy/agreement across the >> ecosystem. Swapping a different runtime at link-time is very easy. >> >> -- >> Caml-list mailing list. Subscription management and archives: >> https://sympa.inria.fr/sympa/arc/caml-list >> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners >> Bug reports: http://caml.inria.fr/bin/caml-bugs=3D >> >> >> -- >> Caml-list mailing list. Subscription management and archives: >> https://sympa.inria.fr/sympa/arc/caml-list >> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners >> Bug reports: http://caml.inria.fr/bin/caml-bugs > >