From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by sympa.inria.fr (Postfix) with ESMTPS id BA0BA7FDE7 for ; Fri, 10 Jun 2016 22:35:11 +0200 (CEST) IronPort-PHdr: 9a23:Q2OBdhC6km1kI5CA6WZqUyQJP3N1i/DPJgcQr6AfoPdwSP/8rsbcNUDSrc9gkEXOFd2CrakU2qyJ7Ou5BTJIyK3CmU5BWaQEbwUCh8QSkl5oK+++Imq/EsTXaTcnFt9JTl5v8iLzG0FUHMHjew+a+SXqvnYsExnyfTB4Ov7yUtaLyZ/nhqbtpNaKP1sArQH+SI0xBS3+lR/WuMgSjNkqAYcK4TyNnEF1ff9Lz3hjP1OZkkW0zM6x+Jl+73YY4Kp5pIZoGJ/3dKUgTLFeEC9ucyVsvJWq5i/4UBCX63AAfmITmxtOS0iZvVCpFqv252G1meN3wiiXOYXNRrA5Qzm4oO0jHBDhgj0GOjp/62rXh9Z9lopUpRugo1p0xIuCJMnfe/F3e6eVYMgXX3EOFI4FXCVEBsa4bpATJ+sHJ+dR6Yfn8Qg0oAO6FDWrUbftzTlgiHH92qshzuA9GAfNzUorGNdY41rOq9CgfoUVV6iVxbPSyi2JJ6dU3jHV5obDdRY6vPWQVLR7YIzazkx5RFCNtUmZtYGwZ2Dd7e8KqWXOqrc5WA== Authentication-Results: mail2-smtp-roc.national.inria.fr; spf=None smtp.pra=jon@ffconsultancy.com; spf=None smtp.mailfrom=jon@ffconsultancy.com; spf=None smtp.helo=postmaster@avasout02.plus.net Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of jon@ffconsultancy.com) identity=pra; client-ip=212.159.14.17; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="jon@ffconsultancy.com"; x-sender="jon@ffconsultancy.com"; x-conformance=sidf_compatible Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of jon@ffconsultancy.com) identity=mailfrom; client-ip=212.159.14.17; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="jon@ffconsultancy.com"; x-sender="jon@ffconsultancy.com"; x-conformance=sidf_compatible Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of postmaster@avasout02.plus.net) identity=helo; client-ip=212.159.14.17; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="jon@ffconsultancy.com"; x-sender="postmaster@avasout02.plus.net"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0DoAgCRI1tXfhEOn9RehBR9AaoOY5IuFw2FbQQCAoErPBABAQEBAQEBAREBAQkNCQkhL4IwghUBAQEECAIZBUYLDAEDAgkRBAEBAwIJGgMCAhkjCQEDAQUIAgQBEgsFAgmHewMbCq0nhiSGIwOECAEBAQEBAQQBAQEBAQEBIIEBiXOEKoJfOBOCRwWGRQySDgGBWoQpiluMaY9rNYQObQGKBwEBAQ X-IPAS-Result: A0DoAgCRI1tXfhEOn9RehBR9AaoOY5IuFw2FbQQCAoErPBABAQEBAQEBAREBAQkNCQkhL4IwghUBAQEECAIZBUYLDAEDAgkRBAEBAwIJGgMCAhkjCQEDAQUIAgQBEgsFAgmHewMbCq0nhiSGIwOECAEBAQEBAQQBAQEBAQEBIIEBiXOEKoJfOBOCRwWGRQySDgGBWoQpiluMaY9rNYQObQGKBwEBAQ X-IronPort-AV: E=Sophos;i="5.26,451,1459807200"; d="scan'208";a="221987231" Received: from avasout02.plus.net ([212.159.14.17]) by mail2-smtp-roc.national.inria.fr with ESMTP/TLS/DHE-RSA-AES128-SHA; 10 Jun 2016 22:35:10 +0200 Received: from XPS ([80.189.16.107]) by avasout02 with smtp id 58b91t0022Jc0sY018bAQ0; Fri, 10 Jun 2016 21:35:10 +0100 X-CM-Score: 0.00 X-CNFS-Analysis: v=2.1 cv=COHXJkfD c=1 sm=1 tr=0 a=IT7DEMVM+saEYhBIg8ZHbA==:117 a=IT7DEMVM+saEYhBIg8ZHbA==:17 a=L9H7d07YOLsA:10 a=9cW_t1CCXrUA:10 a=s5jvgZ67dGcA:10 a=IkcTkHD0fZMA:10 a=AiHQdgjdAAAA:8 a=p0WdMEafAAAA:8 a=YG86fJyiAAAA:8 a=vlzoIh7UAAAA:8 a=NEAV23lmAAAA:8 a=ZOzjf2MOAAAA:8 a=CjxXgO3LAAAA:8 a=fze9awyfiyrGaAG0yQcA:9 a=ovo7klGmaOa6z-C5:21 a=VO0OmiOJaAJPn0Ib:21 a=QEXdDO2ut3YA:10 a=5kWise4xawAA:10 a=M-fgipkQCoj_B38W52kE:22 a=KNX8SZJeBz7yKKbDRToh:22 a=2rTcZijYBW9OfK4jDYag:22 a=iH0rUA9h8FOUXzGSNSYd:22 a=Bn2pgwyD2vrAyMmN8A2t:22 a=1Mhi-5-LkjG4w5oc0yAU:22 a=_E8DXSXWC0jInl61zbkG:22 X-AUTH: jdh302@:2500 Reply-To: From: "Jon Harrop" To: "'Gabriel Scherer'" , "'caml users'" Cc: "'Damien Doligez'" References: In-Reply-To: Date: Fri, 10 Jun 2016 21:35:09 +0100 Organization: Flying Frog Consultancy Ltd. Message-ID: <00c801d1c357$95598ca0$c00ca5e0$@ffconsultancy.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQISLKA/Zy9ItN2+dVFL0ec6gqjD+p9iUolw Content-Language: en-gb Subject: RE: [Caml-list] Measuring GC latencies for OCaml program Very interesting, thank you! We just implemented a substantial client and server system for the finance = sector with the "low" latency server written in OCaml. I have done this bef= ore in other languages and seen it done in many more languages. The OCaml i= s by far the consistently-fastest solution I have ever seen. Orders of magn= itude faster than the last C++ solution I saw. In particular, compared to J= ava and .NET where we see substantial latencies from the GC at around 100ms= , with OCaml there is no visible peak at high latency due to the GC at all.= And this project was implemented to a very short deadline with no time for= optimisation at all. On a related note, we used Jane St.'s Core and Async libraries as well as C= ohttp and found them all to be *phenomenally* efficient and robust. In case anyone is interested, the only pain point I had was the development= environment. I actually prototyped all my hard code in simplified F# in Vi= sual Studio on Windows and then ported to OCaml. Emacs and Merlin crash and= hang a lot for me: maybe 50 times per day. Hence my other post. :-) In terms of the language, OCaml was very well suited to this task. Lots of = purely functional data structures forming in-memory databases that can be q= ueried in different ways and have many different versions of them stored in= different places at different times. Perhaps the main language feature I m= issed from F# was (surprisingly!) reflection. My F# client code uses reflec= tion to serialize and deserialize messages. With no reflection I couldn't d= o that in OCaml so I used reflection in F# to autogenerate the OCaml code. Cheers, Jon. -----Original Message----- From: caml-list-request@inria.fr [mailto:caml-list-request@inria.fr] On Beh= alf Of Gabriel Scherer Sent: 30 May 2016 20:48 To: caml users Cc: Damien Doligez Subject: [Caml-list] Measuring GC latencies for OCaml program Dear caml-list, You may be interested in the following blog post, in which I give instructi= ons to measure the worst-case latencies incurred by the GC: Measuring GC latencies in Haskell, OCaml, Racket http://prl.ccs.neu.edu/blog/2016/05/24/measuring-gc-latencies-in-haskell-= ocaml-racket/ In summary, the commands to measure latencies look something like: % build the program with the instrumented runtime ocamlbuild -tag "runtime_variant(i)" myprog.native % run with instrumentation enabled OCAML_INSTR_FILE=3D"ocaml.log" ./main.native % visualize results from the raw log $(OCAML_SOURCES)/tools/ocaml-instr-graph ocaml.log $(OCAML_SOURCES)/tools/ocaml-instr-report ocaml.log While the OCaml GC has had a good incremental mode with low latencies for m= ost workloads for a long time, the ability to instrument it to actually mea= sure latencies is still in its infancy: it is a side-result of Damien Dolig= ez's stint at Jane Street last year, and 4.03.0 is the first release in which this work is available. A practical consequence of this youth is that the "user experience" of actu= ally performing these measurements is currently very bad. The GC measuremen= ts are activated in an instrumented runtime variant (OCaml supports having = several variants of the runtime available, and deciding which one to use fo= r a specific program at link-time), which is the right design choice, but r= ight now this variant is not built by default by the compiler distribution = -- building it is a configure-time option disabled by default. This means t= hat, as a user interested in doing the measurements, you have to compile an= alternative OCaml compiler. Furthermore, processing the raw instrumented log requires tool that are als= o in their infancy, and are currently included in the compiler distribution= sources but not installed -- so you have to have a source checkout availab= le to use them. In contrast, GHC's instrumentation is enabled by just passi= ng the option "+RTS -s" to the Haskell program of interest; this is superbl= y convenient and something we should aim at. I discussed with Damien whether we should enable building the instrumented = runtime by default (for example pass the --with-instrumented-runtime option= to the opam switches people are using, and encourage distributions to use = it in their packages as well). Of course there is a cost/benefit trade-off:= currently virtually nobody is using this instrumentation, but enabling it = by default would increase the compilation time of the compiler distribution= for everyone. (On my machine it only adds 5 seconds to total build time.) I personally think that we should aim for a rock-solid experience for profi= ling and instrumenting OCaml program enabled by default=C2=B9. It is worth = making it slightly longer for anyone to install the compiler if we can make= it vastly easier to measure GC pauses in our program when the need arises = (even if it's not very often). But that is a discussion that should be had = before making any choice. Regarding the log analysis tools, before finding about Damien's included-bu= t-not installed tools (a shell and an awk script, in the finest Unix tradit= ion) I built a quick&dirty OCaml script to do some measurements, which can = be found in the benchmark repository below. It would not be much more work = to grow this in a reusable library to extract the current log format into a= structured data structure -- the format is undocumented but the provided s= cripts in tools/ have enough information to infer the structure. Such a scr= ipt/library would, of course, remain tightly coupled to the OCaml version, = but I think it could be useful to have it packaged for users to play with. https://gitlab.com/gasche/gc-latency-experiment/blob/master/parse_ocaml_l= og.ml =C2=B9: We cannot expect users to effectively write performant code if they= don't have the tool support for it. The fact that lazyness in Haskell make= s it harder for users to reason about efficiency or memory usage has made t= he avaibility of excellent performance tooling *necessary*, where it is mer= ely nice-to-have in OCaml. Rather ironically, Haskell tooling is now much b= etter than OCaml's in this area, to the point that it can be easier to writ= e efficient code in Haskell. Three side-notes on profiling tools: 1. `perf record --call-graph=3Ddwarf` works fine for ocamlopt binaries (no need for a frame-pointers switch), and this is documented: https://ocaml.org/learn/tutorials/performance_and_profiling.html#Usingp= erfonLinux 2. Thomas Leonard has done excellent work on domain-specific profiling tools for Mirage, and this is the kind of tool support that I think should be available to anyone out of the box. http://roscidus.com/blog/blog/2014/08/15/optimising-the-unikernel/ http://roscidus.com/blog/blog/2014/10/27/visualising-an-asynchronous-m= onad/ 3. There is currently more debate than anyone could wish for around a pull request of Mark Shinwell for runtime support for dynamic call graph construction and its use for memory profiling. https://github.com/ocaml/ocaml/pull/585 4. Providing a good user experience for performance or space profiling is a fundamentally harder problem than for GC pauses. It may require specially-compiled versions of the libraries used by your program, and thus a general policy/agreement across the ecosystem. Swapping a different runtime at link-time is very easy. -- Caml-list mailing list. Subscription management and archives: https://sympa.inria.fr/sympa/arc/caml-list Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs=3D