From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by sympa.inria.fr (Postfix) with ESMTPS id D63A97F613 for ; Wed, 14 Sep 2016 04:51:44 +0200 (CEST) Authentication-Results: mail3-smtp-sop.national.inria.fr; spf=None smtp.pra=pratikfegade@gmail.com; spf=Pass smtp.mailfrom=pratikfegade@gmail.com; spf=None smtp.helo=postmaster@mail-pa0-f62.google.com Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of pratikfegade@gmail.com) identity=pra; client-ip=209.85.220.62; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="pratikfegade@gmail.com"; x-sender="pratikfegade@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail3-smtp-sop.national.inria.fr: domain of pratikfegade@gmail.com designates 209.85.220.62 as permitted sender) identity=mailfrom; client-ip=209.85.220.62; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="pratikfegade@gmail.com"; x-sender="pratikfegade@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-pa0-f62.google.com) identity=helo; client-ip=209.85.220.62; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="pratikfegade@gmail.com"; x-sender="postmaster@mail-pa0-f62.google.com"; x-conformance=sidf_compatible IronPort-PHdr: =?us-ascii?q?9a23=3AtI5GDRHPF9LP+EHqG+6nxZ1GYnF86YWxBRYc798d?= =?us-ascii?q?s5kLTJ74pcWwAkXT6L1XgUPTWs2DsrQf2rOQ7PmrBjRIoc7Y9itTKNoUD15NoP?= =?us-ascii?q?5VtjRoONSCB0z/IayiRA0BN+MGamVY+WqmO1NeAsf0ag6aiHSz6TkPBke3blIt?= =?us-ascii?q?dazLE4Lfx/66y/q1s8WKJV4Z3XziOPgtdF329VyX7ZhOx9M6a+4Y8VjgmjNwYe?= =?us-ascii?q?NYxGdldxq4vi3XwYOOxqNl6DlaoPk79sRNAu3QdqU8SqFEXnx9azhmrOWijxTI?= =?us-ascii?q?TBOO630ASS1W10MQW0mW2irnRY/8uTfWse902S/SfZGnDON8ZTP3yKxiTRugoS?= =?us-ascii?q?gILTM2+X3Wjco4qKNAvFr1oBd2xoqRaY6aLvdxd7/cetVfQWtaQu5eUCtOBo6z?= =?us-ascii?q?dYwSF/FHNuFd+drTvVwL+AOkDBWtB+3ixTZEi1f52KQ716IqFgSVj0QbA9sSvS?= =?us-ascii?q?GM/53OP6AIXLXwkfDF?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A0CDAwCKuthXgD7cVdFdHAEBBAEBCgEBG?= =?us-ascii?q?AEFAQsBgw4BAQEBAXV8pm2GEo1VJoV4AoFPPBABAQEBAQEBAQEBARIBAQsLCQk?= =?us-ascii?q?XMUEMgWUEARUFghABAQEDARIRHQEbEgsBAwELBgUYDR0CAiMRAQUBChIGJRCID?= =?us-ascii?q?QEDDwgOoXCBMj4yiz2BaoJeBYNqChknAwpTglYBAQEBAQEBAQEBAQEBAQEBAQE?= =?us-ascii?q?BFgIGEIQVhAQIgk6HQoJaBYZmDIgxikOGJYtpjSaMWII5MYERNTNCghGCFxw0A?= =?us-ascii?q?YEChQqBQAEBAQ?= X-IPAS-Result: =?us-ascii?q?A0CDAwCKuthXgD7cVdFdHAEBBAEBCgEBGAEFAQsBgw4BAQE?= =?us-ascii?q?BAXV8pm2GEo1VJoV4AoFPPBABAQEBAQEBAQEBARIBAQsLCQkXMUEMgWUEARUFg?= =?us-ascii?q?hABAQEDARIRHQEbEgsBAwELBgUYDR0CAiMRAQUBChIGJRCIDQEDDwgOoXCBMj4?= =?us-ascii?q?yiz2BaoJeBYNqChknAwpTglYBAQEBAQEBAQEBAQEBAQEBAQEBFgIGEIQVhAQIg?= =?us-ascii?q?k6HQoJaBYZmDIgxikOGJYtpjSaMWII5MYERNTNCghGCFxw0AYEChQqBQAEBAQ?= X-IronPort-AV: E=Sophos;i="5.30,332,1470693600"; d="scan'208";a="193046569" Received: from mail-pa0-f62.google.com ([209.85.220.62]) by mail3-smtp-sop.national.inria.fr with ESMTP/TLS/AES128-GCM-SHA256; 14 Sep 2016 04:51:42 +0200 Received: by mail-pa0-f62.google.com with SMTP id bd2so328638pad.0; Tue, 13 Sep 2016 19:51:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:message-id:in-reply-to:references:subject :mime-version; bh=EqUqYNn0RJvpmdKd/E8mPbUTatiZmyqSm9H2SHRW0L8=; b=CafoIML+7dXejw8btZGr5AYOdvqs2Qag9nYAWJdIuz5hOOsxPt4wiU4RCUcKUCQbqM /OLb1Wzik4+2pWnr7WBtq4zu6nZpzgNBLSj/Q13QYq6PuotOIEA/qEGA7837XXagXyWm xZtjvxV6K70fhqO7hSzLp4CuA8AvvX1OaZFSYSwQ5XfR5D7IgukqVUouxpCbJm875Rs4 HN+L75Y/ncoJkJkHT1WbLj7qj6pcZfj0fUKpitHEuwdEVucX0CPOztWSks3xZouXxbSg qgaEtoOBDXQ58XwePU51yLiUG5/JlG1Gj0CzteOA1uHeUeenGLMZYe7jfvkD24AK93bj cWRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:message-id:in-reply-to :references:subject:mime-version; bh=EqUqYNn0RJvpmdKd/E8mPbUTatiZmyqSm9H2SHRW0L8=; b=jM6ReBq5Edcy8PK1Eaw53jdrnvUYJ8DulofqXCOHlUnqaCJHGHAQZNVTFG3FpkN3u9 bAYbj/7PiD5tLBDQyxILen999FpWMc3ve0Qb9+P2as6+3BwWkNzyK7QrIeWiU3o+qjiv cWuL+ncMn3UAMSuXsfLqnKKD0sYagCi/rwE4fXpXFYxS+bTBOQvCeLc5w/D9qG5M2/b0 mhgEBlibfvu9uh2PpR+1805R6TJXWAeuNKmoTOMWFi868sNXm3s2OgXB6H375RLgLhmv /ZYy6hYZmKTVobbydbOrjLsvYpoBdoaFKm+5fN0evjs4TPMfIAtcRQqaXg84nSAV9Xci KMWQ== X-Gm-Message-State: AE9vXwNPpSNFS8+gGNYPtBqf17NcHoekewn34LW8JKuE9eGBa/fchFbjRCNcrlfQiTdamflj78zT X-Received: by 10.36.225.5 with SMTP id n5mr49641ith.2.1473821500402; Tue, 13 Sep 2016 19:51:40 -0700 (PDT) X-Google-Already-Archived: Yes X-Google-Already-Archived-Group-Id: 18375bed8f X-Google-Doc-Id: f385d687520e8 X-Google-Thread-Id: 5e52f3344d7eb2c4 X-Google-Message-Url: http://groups.google.com/group/ocaml-aggregation-list/msg/f385d687520e8 X-Google-Thread-Url: http://groups.google.com/group/ocaml-aggregation-list/t/5e52f3344d7eb2c4 X-Google-Web-Client: true Date: Tue, 13 Sep 2016 19:51:38 -0700 (PDT) From: pratikfegade@gmail.com To: Ocaml Aggregation List Cc: caml-list@inria.fr, damien.doligez@inria.fr, gabriel.scherer@gmail.com Message-Id: In-Reply-To: References: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_87_1382013517.1473821499121" X-Google-Token: ELr24r4FQNSd6yCKF9E0 X-Google-IP: 67.171.69.83 Subject: Re: [Caml-list] Measuring GC latencies for OCaml program ------=_Part_87_1382013517.1473821499121 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi, I am not sure if this is the right place to ask this but this being the on= ly reference to the instrumented runtime I found, I am going to go ahead an= d ask. I am trying to use the instrumented runtime as you described but it does no= t seem to work. I cannot compile the program with the instrumented runtime.= I have compiled the compiler with the --with-instrumented-runtime flag (I = have ocamlruni) in my path. I get the following error while trying to compi= le the way you suggested: Configuration "true: quiet, runtime_variant(i)", line 1, characters 13-31: Warning: tag "runtime_variant" does not expect a parameter, but is used wit= h parameter "i" Configuration "true: quiet, runtime_variant(i)", line 1, characters 13-31: Warning: the tag "runtime_variant(i)" is not used in any flag declaration, = so it will have no effect; it may be a typo. Otherwise use `mark_tag_used` = in your myocamlbuild.ml to disable this warning. Finished, 4 targets (0 cached) in 00:00:00. I also tried to compile directly to native code using the command ocamlopt -runtime-variant i main.ml which does not give an error which I assume to mean that the instrumented r= untime exists on my system. I cannot however find any log file nor does run= ning the executable give any information on stdout or stderr. Am I missing something here? Thanks, Pratik Fegade. On Monday, May 30, 2016 at 3:49:21 PM UTC-4, Gabriel Scherer wrote: > Dear caml-list, >=20 > You may be interested in the following blog post, in which I give > instructions to measure the worst-case latencies incurred by the GC: >=20 > Measuring GC latencies in Haskell, OCaml, Racket > http://prl.ccs.neu.edu/blog/2016/05/24/measuring-gc-latencies-in-haskel= l-ocaml-racket/ >=20 > In summary, the commands to measure latencies look something like: >=20 > % build the program with the instrumented runtime > ocamlbuild -tag "runtime_variant(i)" myprog.native >=20 > % run with instrumentation enabled > OCAML_INSTR_FILE=3D"ocaml.log" ./main.native >=20 > % visualize results from the raw log > $(OCAML_SOURCES)/tools/ocaml-instr-graph ocaml.log > $(OCAML_SOURCES)/tools/ocaml-instr-report ocaml.log >=20 > While the OCaml GC has had a good incremental mode with low latencies > for most workloads for a long time, the ability to instrument it to > actually measure latencies is still in its infancy: it is > a side-result of Damien Doligez's stint at Jane Street last year, and > 4.03.0 is the first release in which this work is available. >=20 > A practical consequence of this youth is that the "user experience" of > actually performing these measurements is currently very bad. The GC > measurements are activated in an instrumented runtime variant (OCaml > supports having several variants of the runtime available, and > deciding which one to use for a specific program at link-time), which > is the right design choice, but right now this variant is not built by > default by the compiler distribution -- building it is > a configure-time option disabled by default. This means that, as > a user interested in doing the measurements, you have to compile an > alternative OCaml compiler. > Furthermore, processing the raw instrumented log requires tool that > are also in their infancy, and are currently included in the compiler > distribution sources but not installed -- so you have to have a source > checkout available to use them. In contrast, GHC's instrumentation is > enabled by just passing the option "+RTS -s" to the Haskell program of > interest; this is superbly convenient and something we should aim at. >=20 > I discussed with Damien whether we should enable building the > instrumented runtime by default (for example pass > the --with-instrumented-runtime option to the opam switches people are > using, and encourage distributions to use it in their packages as > well). Of course there is a cost/benefit trade-off: currently > virtually nobody is using this instrumentation, but enabling it by > default would increase the compilation time of the compiler > distribution for everyone. (On my machine it only adds 5 seconds to > total build time.) >=20 > I personally think that we should aim for a rock-solid experience for > profiling and instrumenting OCaml program enabled by default=C2=B9. It is > worth making it slightly longer for anyone to install the compiler if > we can make it vastly easier to measure GC pauses in our program when > the need arises (even if it's not very often). But that is > a discussion that should be had before making any choice. >=20 > Regarding the log analysis tools, before finding about Damien's > included-but-not installed tools (a shell and an awk script, in the > finest Unix tradition) I built a quick&dirty OCaml script to do some > measurements, which can be found in the benchmark repository below. It > would not be much more work to grow this in a reusable library to > extract the current log format into a structured data structure -- the > format is undocumented but the provided scripts in tools/ have enough > information to infer the structure. Such a script/library would, of > course, remain tightly coupled to the OCaml version, but I think it > could be useful to have it packaged for users to play with. >=20 > https://gitlab.com/gasche/gc-latency-experiment/blob/master/parse_ocaml= _log.ml >=20 > =C2=B9: We cannot expect users to effectively write performant code if th= ey > don't have the tool support for it. The fact that lazyness in Haskell > makes it harder for users to reason about efficiency or memory usage > has made the avaibility of excellent performance tooling *necessary*, > where it is merely nice-to-have in OCaml. Rather ironically, Haskell > tooling is now much better than OCaml's in this area, to the point > that it can be easier to write efficient code in Haskell. >=20 > Three side-notes on profiling tools: >=20 > 1. `perf record --call-graph=3Ddwarf` works fine for ocamlopt binaries > (no need for a frame-pointers switch), and this is documented: > https://ocaml.org/learn/tutorials/performance_and_profiling.html#Usin= gperfonLinux >=20 > 2. Thomas Leonard has done excellent work on domain-specific profiling > tools for Mirage, and this is the kind of tool support that I think > should be available to anyone out of the box. > http://roscidus.com/blog/blog/2014/08/15/optimising-the-unikernel/ > http://roscidus.com/blog/blog/2014/10/27/visualising-an-asynchronous= -monad/ >=20 > 3. There is currently more debate than anyone could wish for around > a pull request of Mark Shinwell for runtime support for dynamic call > graph construction and its use for memory profiling. > https://github.com/ocaml/ocaml/pull/585 >=20 > 4. Providing a good user experience for performance or space profiling > is a fundamentally harder problem than for GC pauses. It may > require specially-compiled versions of the libraries used by your > program, and thus a general policy/agreement across the > ecosystem. Swapping a different runtime at link-time is very easy. >=20 > --=20 > Caml-list mailing list. Subscription management and archives: > https://sympa.inria.fr/sympa/arc/caml-list > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-bugs ------=_Part_87_1382013517.1473821499121--