From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by sympa.inria.fr (Postfix) with ESMTPS id 19ABC7F614 for ; Wed, 14 Sep 2016 10:39:43 +0200 (CEST) Authentication-Results: mail3-smtp-sop.national.inria.fr; spf=None smtp.pra=gabriel.scherer@gmail.com; spf=Pass smtp.mailfrom=gabriel.scherer@gmail.com; spf=None smtp.helo=postmaster@mail-it0-f68.google.com Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of gabriel.scherer@gmail.com) identity=pra; client-ip=209.85.214.68; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="gabriel.scherer@gmail.com"; x-sender="gabriel.scherer@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail3-smtp-sop.national.inria.fr: domain of gabriel.scherer@gmail.com designates 209.85.214.68 as permitted sender) identity=mailfrom; client-ip=209.85.214.68; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="gabriel.scherer@gmail.com"; x-sender="gabriel.scherer@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-it0-f68.google.com) identity=helo; client-ip=209.85.214.68; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="gabriel.scherer@gmail.com"; x-sender="postmaster@mail-it0-f68.google.com"; x-conformance=sidf_compatible IronPort-PHdr: =?us-ascii?q?9a23=3AV22oKhExs1LbEIkoA6bym51GYnF86YWxBRYc798d?= =?us-ascii?q?s5kLTJ74psywAkXT6L1XgUPTWs2DsrQf2rOQ7PmrADRdqdbZ6TZZIcQKD0dEwe?= =?us-ascii?q?wt3CUYSPafDkP6KPO4JwcbJ+9lEGFfwnegLEJOE9z/bVCB6le77DoVBwmtfVEt?= =?us-ascii?q?fre9JIfegoyN2vyo/NWLOkMT1WP7Puo5dU3r5UWJ749N0NMkcv5wgjLy4VJwM9?= =?us-ascii?q?xMwm1pIV/B1z3d3eyXuKBZziJLpvg6/NRBW6ipN44xTLhfESh0ezttvJ6jnVD5?= =?us-ascii?q?QACO/noRVHkN2loNWlCdrULMZYrqqibxqsZ63SCbO4W2EeF1Cnye6PJzUhb0iS?= =?us-ascii?q?QJOjEw/Gz/hcl5jaYdqxWk9DJlxIuBT4ifLvtzeuvmdtMXX2dbFpJeXiZbA464?= =?us-ascii?q?KZAED+cbMPxwoIz0pl9Iphy7U1r/TNjzwyNF0yellZYx1P4sRESbhQE=3D?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A0BIAwD6C9lXekTWVdFcHAEBBAEBCgEBF?= =?us-ascii?q?wEBBAEBCgEBgw8BAQEBAXVtDwemZiqMMIUPggMmhXQCAgKBRQc6EgEBAQEBAQE?= =?us-ascii?q?BAQEBEgEBCQsUGS+CMgQBFQWCEAEBAQMBEhEdARsSCwEDAQsGBQsNDR0CAiEBA?= =?us-ascii?q?REBBQEKEhkSEIgNAQMPCA6hbIEyPjKLPYFqgl4Fg2oKGScDClOCVgEBAQEBBQE?= =?us-ascii?q?BAQEBAQEBFwIGEIYhhE+CQ4IqglWCWgWGZgyIMIoPNYFlhECGRYJpgjyNJohMh?= =?us-ascii?q?AyCOBMegREPFgY9R4FsK4IGLTQBhgyBQAEBAQ?= X-IPAS-Result: =?us-ascii?q?A0BIAwD6C9lXekTWVdFcHAEBBAEBCgEBFwEBBAEBCgEBgw8?= =?us-ascii?q?BAQEBAXVtDwemZiqMMIUPggMmhXQCAgKBRQc6EgEBAQEBAQEBAQEBEgEBCQsUG?= =?us-ascii?q?S+CMgQBFQWCEAEBAQMBEhEdARsSCwEDAQsGBQsNDR0CAiEBAREBBQEKEhkSEIg?= =?us-ascii?q?NAQMPCA6hbIEyPjKLPYFqgl4Fg2oKGScDClOCVgEBAQEBBQEBAQEBAQEBFwIGE?= =?us-ascii?q?IYhhE+CQ4IqglWCWgWGZgyIMIoPNYFlhECGRYJpgjyNJohMhAyCOBMegREPFgY?= =?us-ascii?q?9R4FsK4IGLTQBhgyBQAEBAQ?= X-IronPort-AV: E=Sophos;i="5.30,333,1470693600"; d="scan'208,217";a="193078098" Received: from mail-it0-f68.google.com ([209.85.214.68]) by mail3-smtp-sop.national.inria.fr with ESMTP/TLS/AES128-GCM-SHA256; 14 Sep 2016 10:39:40 +0200 Received: by mail-it0-f68.google.com with SMTP id 186so899314itf.1; Wed, 14 Sep 2016 01:39:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=uXw1YISvgcJCFvt/vi0Gq1uJl1NBM9auO9oS4I15tVk=; b=by2XlBs+t2v6d0uxB3bdKVhX5X3NpcEUqUlE8xN3pNpa9ZiYx6UYMkE6W45JCG3lsB eTjQYyxpJJUQjpwh6OXtHw5rGNtVhP/NGXR5w3uQDnpAtZPClY2/AOudbkeysvmehywT M8jm9Hw/e7C/Cmz5pmyJZ09GwDbH7Eg3iKz9uSjYxAgmJ6YILIiDRhPZ/5cDCEGBe3L4 vjClhVhRVyuev45V1P/cgyF2RKmGXry8Wp6Whcet6yz67/QE9mzLZlBd4I3eqZIj8S6H +569Km3FiuoR8QP9OBej8IDujhsuTLMtpwU57dvnK5jv2g3qh4sv7eXllF9NJsJXpc27 lH5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=uXw1YISvgcJCFvt/vi0Gq1uJl1NBM9auO9oS4I15tVk=; b=Cp8cQdkPPR2t4nm3UWXmAaeBvU7+rmOYe3jZkFUdi1nVRKYhwHResZ4BE82Sb1puH0 gwYqnRUmMXHLryrBY9SrbbIyAOO7twzDoONtdpuTNW6qOQ1XlEpANszFcbgg2hnJfkCj GC96wtiK4nvyrDrhQyu43EA484TAemLgXnpVhP7+y2ZYW3f8mDgUMmLbYOK5hfXp5ahT SG57qXeUxvAHAIyyhdzlSWpCMpwiuhAtBBThYMQrBXi8tHRbZmnYhu4M6SXlzyCssvZN X0BvLk1H2rgfF+L+LeyjYadSS9E8pKnHXiCr074d15VWiB0Cl3IdwPnZvlldOOrPy2q+ jACQ== X-Gm-Message-State: AE9vXwM4SeDOJmy6F7g1lMXHUMQPmbd/r4YJNOgZwOoejgJRwKOcyZ4qzpGNUOE5x3CO47xg9BFUK3jeMB8RQw== X-Received: by 10.107.12.166 with SMTP id 38mr4342893iom.159.1473842378942; Wed, 14 Sep 2016 01:39:38 -0700 (PDT) MIME-Version: 1.0 Received: by 10.79.28.17 with HTTP; Wed, 14 Sep 2016 01:38:58 -0700 (PDT) In-Reply-To: References: From: Gabriel Scherer Date: Wed, 14 Sep 2016 17:38:58 +0900 Message-ID: To: pratikfegade@gmail.com Cc: Ocaml Aggregation List , caml users , Damien Doligez Content-Type: multipart/alternative; boundary=001a113f928073af78053c73ab9a Subject: Re: [Caml-list] Measuring GC latencies for OCaml program --001a113f928073af78053c73ab9a Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable I suspect a mistake in your OCaml environment configuration. The runtime_variant(...) parametrized tag was contributed by whitequark for OCaml 4.02.2, so it should be available in all OCaml versions that also provide the instrumented runtime (that is, 4.03.0 and future versions). Once you compile an executable with the instrumented runtime, you should explicitly set the OCAML_INSTR_FILE=3D"ocaml.log" variable if you want logging to happen. On Wed, Sep 14, 2016 at 11:51 AM, wrote: > Hi, > > I am not sure if this is the right place to ask this but this being the > only reference to the instrumented runtime I found, I am going to go ahead > and ask. > > I am trying to use the instrumented runtime as you described but it does > not seem to work. I cannot compile the program with the instrumented > runtime. I have compiled the compiler with the --with-instrumented-runtime > flag (I have ocamlruni) in my path. I get the following error while trying > to compile the way you suggested: > > Configuration "true: quiet, runtime_variant(i)", line 1, characters 13-31: > Warning: tag "runtime_variant" does not expect a parameter, but is used > with parameter "i" > Configuration "true: quiet, runtime_variant(i)", line 1, characters 13-31: > Warning: the tag "runtime_variant(i)" is not used in any flag declaration, > so it will have no effect; it may be a typo. Otherwise use `mark_tag_used` > in your myocamlbuild.ml to disable this warning. > Finished, 4 targets (0 cached) in 00:00:00. > > I also tried to compile directly to native code using the command > > ocamlopt -runtime-variant i main.ml > > which does not give an error which I assume to mean that the instrumented > runtime exists on my system. I cannot however find any log file nor does > running the executable give any information on stdout or stderr. > > Am I missing something here? > > Thanks, > Pratik Fegade. > > On Monday, May 30, 2016 at 3:49:21 PM UTC-4, Gabriel Scherer wrote: > > Dear caml-list, > > > > You may be interested in the following blog post, in which I give > > instructions to measure the worst-case latencies incurred by the GC: > > > > Measuring GC latencies in Haskell, OCaml, Racket > > http://prl.ccs.neu.edu/blog/2016/05/24/measuring-gc- > latencies-in-haskell-ocaml-racket/ > > > > In summary, the commands to measure latencies look something like: > > > > % build the program with the instrumented runtime > > ocamlbuild -tag "runtime_variant(i)" myprog.native > > > > % run with instrumentation enabled > > OCAML_INSTR_FILE=3D"ocaml.log" ./main.native > > > > % visualize results from the raw log > > $(OCAML_SOURCES)/tools/ocaml-instr-graph ocaml.log > > $(OCAML_SOURCES)/tools/ocaml-instr-report ocaml.log > > > > While the OCaml GC has had a good incremental mode with low latencies > > for most workloads for a long time, the ability to instrument it to > > actually measure latencies is still in its infancy: it is > > a side-result of Damien Doligez's stint at Jane Street last year, and > > 4.03.0 is the first release in which this work is available. > > > > A practical consequence of this youth is that the "user experience" of > > actually performing these measurements is currently very bad. The GC > > measurements are activated in an instrumented runtime variant (OCaml > > supports having several variants of the runtime available, and > > deciding which one to use for a specific program at link-time), which > > is the right design choice, but right now this variant is not built by > > default by the compiler distribution -- building it is > > a configure-time option disabled by default. This means that, as > > a user interested in doing the measurements, you have to compile an > > alternative OCaml compiler. > > Furthermore, processing the raw instrumented log requires tool that > > are also in their infancy, and are currently included in the compiler > > distribution sources but not installed -- so you have to have a source > > checkout available to use them. In contrast, GHC's instrumentation is > > enabled by just passing the option "+RTS -s" to the Haskell program of > > interest; this is superbly convenient and something we should aim at. > > > > I discussed with Damien whether we should enable building the > > instrumented runtime by default (for example pass > > the --with-instrumented-runtime option to the opam switches people are > > using, and encourage distributions to use it in their packages as > > well). Of course there is a cost/benefit trade-off: currently > > virtually nobody is using this instrumentation, but enabling it by > > default would increase the compilation time of the compiler > > distribution for everyone. (On my machine it only adds 5 seconds to > > total build time.) > > > > I personally think that we should aim for a rock-solid experience for > > profiling and instrumenting OCaml program enabled by default=C2=B9. It = is > > worth making it slightly longer for anyone to install the compiler if > > we can make it vastly easier to measure GC pauses in our program when > > the need arises (even if it's not very often). But that is > > a discussion that should be had before making any choice. > > > > Regarding the log analysis tools, before finding about Damien's > > included-but-not installed tools (a shell and an awk script, in the > > finest Unix tradition) I built a quick&dirty OCaml script to do some > > measurements, which can be found in the benchmark repository below. It > > would not be much more work to grow this in a reusable library to > > extract the current log format into a structured data structure -- the > > format is undocumented but the provided scripts in tools/ have enough > > information to infer the structure. Such a script/library would, of > > course, remain tightly coupled to the OCaml version, but I think it > > could be useful to have it packaged for users to play with. > > > > https://gitlab.com/gasche/gc-latency-experiment/blob/ > master/parse_ocaml_log.ml > > > > =C2=B9: We cannot expect users to effectively write performant code if = they > > don't have the tool support for it. The fact that lazyness in Haskell > > makes it harder for users to reason about efficiency or memory usage > > has made the avaibility of excellent performance tooling *necessary*, > > where it is merely nice-to-have in OCaml. Rather ironically, Haskell > > tooling is now much better than OCaml's in this area, to the point > > that it can be easier to write efficient code in Haskell. > > > > Three side-notes on profiling tools: > > > > 1. `perf record --call-graph=3Ddwarf` works fine for ocamlopt binaries > > (no need for a frame-pointers switch), and this is documented: > > https://ocaml.org/learn/tutorials/performance_and_profiling.html# > UsingperfonLinux > > > > 2. Thomas Leonard has done excellent work on domain-specific profiling > > tools for Mirage, and this is the kind of tool support that I think > > should be available to anyone out of the box. > > http://roscidus.com/blog/blog/2014/08/15/optimising-the-unikernel/ > > http://roscidus.com/blog/blog/2014/10/27/visualising-an- > asynchronous-monad/ > > > > 3. There is currently more debate than anyone could wish for around > > a pull request of Mark Shinwell for runtime support for dynamic call > > graph construction and its use for memory profiling. > > https://github.com/ocaml/ocaml/pull/585 > > > > 4. Providing a good user experience for performance or space profiling > > is a fundamentally harder problem than for GC pauses. It may > > require specially-compiled versions of the libraries used by your > > program, and thus a general policy/agreement across the > > ecosystem. Swapping a different runtime at link-time is very easy. > > > > -- > > Caml-list mailing list. Subscription management and archives: > > https://sympa.inria.fr/sympa/arc/caml-list > > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > > Bug reports: http://caml.inria.fr/bin/caml-bugs > --001a113f928073af78053c73ab9a Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
I suspect a mistake in your OCaml environment configu= ration. The runtime_variant(...) parametrized tag was contributed by whiteq= uark for OCaml 4.02.2, so it should be available in all OCaml versions that= also provide the instrumented runtime (that is, 4.03.0 and future versions= ).

Once you compile an executable with the instrumented runtim= e, you should explicitly set the OCAML_INSTR_FILE=3D"ocaml.log" v= ariable if you want logging to happen.
=
On Wed, Sep 14, 2016 at 11:51 AM, <pr= atikfegade@gmail.com> wrote:
myocamlbuild.ml to disable this warning.
Finished, 4 targets (0 cached) in 00:00:00.

I also tried to compile directly to native code using the command

ocamlopt -runtime-variant i main.ml

which does not give an error which I assume to mean that the instrumented r= untime exists on my system. I cannot however find any log file nor does run= ning the executable give any information on stdout or stderr.

Am I missing something here?

Thanks,
Pratik Fegade.

On Monday, May 30, 2016 at 3:49:21 PM UTC-4, Gabriel Scherer wrote:
> Dear caml-list,
>
> You may be interested in the following blog post, in which I give
> instructions to measure the worst-case latencies incurred by the GC: >
>=C2=A0 =C2=A0Measuring GC latencies in Haskell, OCaml, Racket
>=C2=A0 =C2=A0http://prl.ccs.neu.edu/blog/2016/05/24/measuring-gc-latencies-= in-haskell-ocaml-racket/
>
> In summary, the commands to measure latencies look something like:
>
>=C2=A0 =C2=A0 =C2=A0% build the program with the instrumented runtime >=C2=A0 =C2=A0 =C2=A0ocamlbuild -tag "runtime_variant(i)" mypr= og.native
>
>=C2=A0 =C2=A0 =C2=A0% run with instrumentation enabled
>=C2=A0 =C2=A0 =C2=A0OCAML_INSTR_FILE=3D"ocaml.log" ./main.nat= ive
>
>=C2=A0 =C2=A0 =C2=A0% visualize results from the raw log
>=C2=A0 =C2=A0 =C2=A0$(OCAML_SOURCES)/tools/ocaml-instr-graph ocaml= .log
>=C2=A0 =C2=A0 =C2=A0$(OCAML_SOURCES)/tools/ocaml-instr-report ocam= l.log
>
> While the OCaml GC has had a good incremental mode with low latencies<= br> > for most workloads for a long time, the ability to instrument it to
> actually measure latencies is still in its infancy: it is
> a side-result of Damien Doligez's stint at Jane Street last year, = and
> 4.03.0 is the first release in which this work is available.
>
> A practical consequence of this youth is that the "user experienc= e" of
> actually performing these measurements is currently very bad. The GC > measurements are activated in an instrumented runtime variant (OCaml > supports having several variants of the runtime available, and
> deciding which one to use for a specific program at link-time), which<= br> > is the right design choice, but right now this variant is not built by=
> default by the compiler distribution -- building it is
> a configure-time option disabled by default. This means that, as
> a user interested in doing the measurements, you have to compile an
> alternative OCaml compiler.
> Furthermore, processing the raw instrumented log requires tool that
> are also in their infancy, and are currently included in the compiler<= br> > distribution sources but not installed -- so you have to have a source=
> checkout available to use them. In contrast, GHC's instrumentation= is
> enabled by just passing the option "+RTS -s" to the Haskell = program of
> interest; this is superbly convenient and something we should aim at.<= br> >
> I discussed with Damien whether we should enable building the
> instrumented runtime by default (for example pass
> the --with-instrumented-runtime option to the opam switches people are=
> using, and encourage distributions to use it in their packages as
> well). Of course there is a cost/benefit trade-off: currently
> virtually nobody is using this instrumentation, but enabling it by
> default would increase the compilation time of the compiler
> distribution for everyone. (On my machine it only adds 5 seconds to
> total build time.)
>
> I personally think that we should aim for a rock-solid experience for<= br> > profiling and instrumenting OCaml program enabled by default=C2=B9. It= is
> worth making it slightly longer for anyone to install the compiler if<= br> > we can make it vastly easier to measure GC pauses in our program when<= br> > the need arises (even if it's not very often). But that is
> a discussion that should be had before making any choice.
>
> Regarding the log analysis tools, before finding about Damien's
> included-but-not installed tools (a shell and an awk script, in the
> finest Unix tradition) I built a quick&dirty OCaml script to do so= me
> measurements, which can be found in the benchmark repository below. It=
> would not be much more work to grow this in a reusable library to
> extract the current log format into a structured data structure -- the=
> format is undocumented but the provided scripts in tools/ have enough<= br> > information to infer the structure. Such a script/library would, of
> course, remain tightly coupled to the OCaml version, but I think it
> could be useful to have it packaged for users to play with.
>
>=C2=A0 =C2=A0https= ://gitlab.com/gasche/gc-latency-experiment/blob/master/parse_ocam= l_log.ml
>
> =C2=B9: We cannot expect users to effectively write performant code if= they
> don't have the tool support for it. The fact that lazyness in Hask= ell
> makes it harder for users to reason about efficiency or memory usage > has made the avaibility of excellent performance tooling *necessary*,<= br> > where it is merely nice-to-have in OCaml. Rather ironically, Haskell > tooling is now much better than OCaml's in this area, to the point=
> that it can be easier to write efficient code in Haskell.
>
> Three side-notes on profiling tools:
>
> 1. `perf record --call-graph=3Ddwarf` works fine for ocamlopt binaries=
>=C2=A0 =C2=A0(no need for a frame-pointers switch), and this is documen= ted:
>=C2=A0 =C2=A0 =C2=A0https://ocaml.org/learn/tutorials/performance_and_profiling.= html#UsingperfonLinux
>
> 2. Thomas Leonard has done excellent work on domain-specific profiling=
>=C2=A0 =C2=A0 tools for Mirage, and this is the kind of tool support th= at I think
>=C2=A0 =C2=A0 should be available to anyone out of the box.
>=C2=A0 =C2=A0 =C2=A0 http://ro= scidus.com/blog/blog/2014/08/15/optimising-the-unikernel/
>=C2=A0 =C2=A0 =C2=A0 = http://roscidus.com/blog/blog/2014/10/27/visualising-an-asynchron= ous-monad/
>
> 3. There is currently more debate than anyone could wish for around
>=C2=A0 =C2=A0 a pull request of Mark Shinwell for runtime support for d= ynamic call
>=C2=A0 =C2=A0 graph construction and its use for memory profiling.
>=C2=A0 =C2=A0 =C2=A0 https://github.com/ocaml/ocaml/= pull/585
>
> 4. Providing a good user experience for performance or space profiling=
>=C2=A0 =C2=A0 is a fundamentally harder problem than for GC pauses. It = may
>=C2=A0 =C2=A0 require specially-compiled versions of the libraries used= by your
>=C2=A0 =C2=A0 program, and thus a general policy/agreement across the >=C2=A0 =C2=A0 ecosystem. Swapping a different runtime at link-time is v= ery easy.
>
> --
> Caml-list mailing list.=C2=A0 Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list > Beginner's list: http://groups.yahoo.com/group= /ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

--001a113f928073af78053c73ab9a--