From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by sympa.inria.fr (Postfix) with ESMTPS id 1337A7F919 for ; Thu, 19 May 2016 21:04:30 +0200 (CEST) IronPort-PHdr: 9a23:DGnQJx1IFtXtldvesmDT+DRfVm0co7zxezQtwd8ZsegeLPad9pjvdHbS+e9qxAeQG96LurQf0aGP6fGocFdDyKjCmUhKSIZLWR4BhJdetC0bK+nBN3fGKuX3ZTcxBsVIWQwt1Xi6NU9IBJS2PAWK8TWM5DIfUi/yKRBybrysXNWC3oLsjavrocybSj4LrQT+SIs6FA+xowTVu5teqqpZAYF19CH0pGBVcf9d32JiKAHbtR/94sCt4MwrqHwI6LoJvvRNWqTifqk+UacQTHF/azh0t4XXskzoTRGO/WcdW2NethtODwnKpEX+X5H9syTSue902S3cNsrzG+MaQzOnuohiQgXphSNPDDU5/XvakIQkg6tRuhOso1pkyI7ZeoyPHPV7d6LZO9gdQDwSDY5qSyVdD9bkPMM0BO0bMLMd9tGlqg== Authentication-Results: mail2-smtp-roc.national.inria.fr; spf=None smtp.pra=gabriel.scherer@gmail.com; spf=Pass smtp.mailfrom=gabriel.scherer@gmail.com; spf=None smtp.helo=postmaster@mail-io0-f174.google.com Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of gabriel.scherer@gmail.com) identity=pra; client-ip=209.85.223.174; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="gabriel.scherer@gmail.com"; x-sender="gabriel.scherer@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail2-smtp-roc.national.inria.fr: domain of gabriel.scherer@gmail.com designates 209.85.223.174 as permitted sender) identity=mailfrom; client-ip=209.85.223.174; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="gabriel.scherer@gmail.com"; x-sender="gabriel.scherer@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-io0-f174.google.com) identity=helo; client-ip=209.85.223.174; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="gabriel.scherer@gmail.com"; x-sender="postmaster@mail-io0-f174.google.com"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0DUAQACDT5Xeq7fVdFeg1U4fgapQZBYgXUehXMCgTMHORMBAQEBAQEBAREBAQkLCwkfMYItghYBAQMBEhEdARsdAQMMBgMCCzcCAiEBAREBBQEcBhMIEweHcgEDDwgOlGSPQoExPjGLO4FqglgFiCUKGScNUoNWAQEBAQYBAQEBAQEZAgYQhhWDSoEDgkOCKYJTglkFl38ygVaEKoYngXmBaU6HLIU3RYcchikSHoEPIQGCOYIQIDKIBgEBAQ X-IPAS-Result: A0DUAQACDT5Xeq7fVdFeg1U4fgapQZBYgXUehXMCgTMHORMBAQEBAQEBAREBAQkLCwkfMYItghYBAQMBEhEdARsdAQMMBgMCCzcCAiEBAREBBQEcBhMIEweHcgEDDwgOlGSPQoExPjGLO4FqglgFiCUKGScNUoNWAQEBAQYBAQEBAQEZAgYQhhWDSoEDgkOCKYJTglkFl38ygVaEKoYngXmBaU6HLIU3RYcchikSHoEPIQGCOYIQIDKIBgEBAQ X-IronPort-AV: E=Sophos;i="5.26,335,1459807200"; d="scan'208,217";a="218959818" Received: from mail-io0-f174.google.com ([209.85.223.174]) by mail2-smtp-roc.national.inria.fr with ESMTP/TLS/AES128-GCM-SHA256; 19 May 2016 21:04:28 +0200 Received: by mail-io0-f174.google.com with SMTP id t40so27442050ioi.0 for ; Thu, 19 May 2016 12:04:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=SdQmefnkONRLLNt0nh+jl00Rnl14Bwd5A7g9feCN0Ig=; b=w8gTGgQ6kSxDBxspz1sp1erBsovuXXH2z7i5XI3QBinl0W3l15/KhJ6Y4sES9wFKyf +KPDCobC7IKTKv5j3ci+8khC1Be4rOq0cY6OeUBiuZql21POvtLp75XwRE47HS4kTBcr gS1IclbHlzyK7iTflD/0CXa9oj0TSgRsIKitr7iv7SwEF4VX0AS/oKXjyyFQn0LvbYro xeN8XxvcbB2O6fyR96I0xFkn7zjmgW6NVgX2a+l13wefPTUTxHAWr0ayn25Nt3RnJtws zNTN1Sd7+17T4axyHTAt66PUI4sm24WZl2UugVP6sBSYqHpt6nO8NQbY1EpeYhz6pgtQ tqrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=SdQmefnkONRLLNt0nh+jl00Rnl14Bwd5A7g9feCN0Ig=; b=ZeutW3wowskakNY+IIPV8XnfsBKDLxe6fd++PuZwNpsstYgKiREDGbqdpZh5aHIY/9 Mh2+jydiV/7ColuPpZXRaseg1A50ykZQt6Rg23wQO9DdLCpovSJQ9QNXLpeck5H8ahtT 5DvCZpNR9tQCG+H2t/7Lkk5rzy0dSniI2cSDVNb5Gr1L1jZAtwTKhLXfR0408OZSEsqY 95UrkMJLkm1BmgmEpCNbMgsIh4TfnuhE57leSGfmiumuAN7MYVCOa6F9sO9ToXO0VBQ9 qmWymifU07oTHeg8WyZH9H7PkgR4qaPSIlveZTwAEuA1rlBZ90033Jr3KzP4CsZNlETs gZkQ== X-Gm-Message-State: AOPr4FUa4EtFNwC0gEBpNlCYRVvs9/+EuybGEXurxX7vbMUm/txdj8qLEN7DsDFBoYnPgvBSsfGp94l98X9SCQ== X-Received: by 10.36.64.8 with SMTP id n8mr4191579ita.21.1463684667602; Thu, 19 May 2016 12:04:27 -0700 (PDT) MIME-Version: 1.0 Received: by 10.79.89.195 with HTTP; Thu, 19 May 2016 12:03:48 -0700 (PDT) In-Reply-To: <43B63E1B-768C-4C2F-ADCB-C2C994A99EB6@gmail.com> References: <43B63E1B-768C-4C2F-ADCB-C2C994A99EB6@gmail.com> From: Gabriel Scherer Date: Thu, 19 May 2016 15:03:48 -0400 Message-ID: To: Christian Lindig Cc: caml users , antonbachin@yahoo.com Content-Type: multipart/alternative; boundary=001a1143dd0aad2155053336a4a0 Subject: Re: [Caml-list] Coverage profiling with bisect_ppx, Oasis, and OCamlbuild - best practice? --001a1143dd0aad2155053336a4a0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Inside the OCaml compiler codebase, profiling and debugging builds are handled by using a different extension suffix: .p.cmx and .d.cmx, for example. In my experience, this way of doing things is rather fragile, because it is easy to get in a situation where we would expect the compiler to go read foo.bar.cmi or foo.bar.cmx but it instead reads foo.cmi or foo.cmx (or doesn't find them and fails or continues silently without cross-module information). In my experience, using "-tag debug" instead is less convenient but more robust, and this is what I recommend. (This problem could be solved by having a richer language for telling the compiler where to read files from and where to produce output files. I think this would be of independent interest, have other benefits, but it would also be a lot of work.) Note that if you go the "separate ocamlbuild calls" route, you do not necessarily need to clean between each build, you can use different build directories instead: ocamlbuild -tag debug -build-dir _build_debug targets.otarget ocamlbuild -tag "package(bisect-ppx)" -build-dir _build_coverage targets.otarget ... In particular, this approach gives you incremental rebuilds during development -- the clean-heavy solution does not. Note that to appease ocamlbuild hygiene checker you then need to explicitly mention that the *other* build directories should be ignored. (It used to be ignored by default in older OCaml versions, but 0.9.2 will complain about them.) You can use a generic rule in your _tags file for that: <_build*>: -traverse On Thu, May 19, 2016 at 2:46 PM, Christian Lindig wrote: > > Bisect_ppx (https://github.com/aantron/bisect_ppx) is a great tool for > instrument code for coverage profiling. I=E2=80=99d like to explore the b= est way to > use this in a bigger project that is built with Oasis/OCamlbuild. In > particular, I=E2=80=99d like to find a high-level way to specify that I w= ant > variants of binaries produced, analogous to > > binary.native > binary.byte > binary.coverage (say, for native binary with coverage > instrumentation) > > without duplicating too much code in the _oasis file. It is my goal to > produce such binaries for installation and hence would like to avoid doing > this sequentially like in > > make opt; make clean; make byte; make clean > > but rather have one target that produces all variants. If this is > impossible, I=E2=80=99d revert to building variants sequentially. The rea= son I > expect this to be difficult is that an object file produced with and > without instrumentation looks the same and thus it might be difficult for > the build system to understand that these need the be rebuilt. > > Code instrumented for coverage needs to be compiled with the corresponding > PPX. OCamlfind and OCamlbuild abstracts this nicely. For compiling > example.ml it=E2=80=99s enough to say: > > ocamlbuild =E2=80=94use-ocamlfind -pkg bisect_ppx example.native > > It is less obvious how to organise this in a bigger project. (For now I > add rules > > <**/*.ml{,i,y}>: pkg_bisect_ppx > <**/*.native>: pkg_bisect_ppx > > to the _tags file generated by Oasis.) I believe the problem already > arises with profiling and debugging builds and I would be happy to learn > how OCamlbuild can be directed via myocamlbuild.ml to do this. > > =E2=80=94 Christian > > --001a1143dd0aad2155053336a4a0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Inside the OCaml compiler codebase, profiling and deb= ugging builds are handled by using a different extension suffix: .p.cmx and= .d.cmx, for example. In my experience, this way of doing things is rather = fragile, because it is easy to get in a situation where we would expect the= compiler to go read foo.bar.cmi or foo.bar.cmx but it instead reads foo.cm= i or foo.cmx (or doesn't find them and fails or continues silently with= out cross-module information). In my experience, using "-tag debug&quo= t; instead is less convenient but more robust, and this is what I recommend= .

(This problem could be solved by having a richer langua= ge for telling the compiler where to read files from and where to produce o= utput files. I think this would be of independent interest, have other bene= fits, but it would also be a lot of work.)

Not= e that if you go the "separate ocamlbuild calls" route, you do no= t necessarily need to clean between each build, you can use different build= directories instead:

=C2=A0 ocamlbuild -tag debug -build= -dir _build_debug targets.otarget
=C2=A0 ocamlbuild -tag &quo= t;package(bisect-ppx)" -build-dir _build_coverage targets.otarget
= =C2=A0 ...

In particular, this approach gives you increme= ntal rebuilds during development -- the clean-heavy solution does not.
=

Note that to appease ocamlbuild hygiene checker y= ou then need to explicitly mention that the *other* build directories shoul= d be ignored. (It used to be ignored by default in older OCaml versions, bu= t 0.9.2 will complain about them.) You can use a generic rule in your _tags= file for that:

=C2=A0=C2=A0 <_build*>: -traverse

On = Thu, May 19, 2016 at 2:46 PM, Christian Lindig <lindig@gmail.com> wrote:

Bisect_ppx (https://github.com/aantron/bisect_ppx) is a gre= at tool for instrument code for coverage profiling. I=E2=80=99d like to exp= lore the best way to use this in a bigger project that is built with Oasis/= OCamlbuild. In particular, I=E2=80=99d like to find a high-level way to spe= cify that I want variants of binaries produced, analogous to

=C2=A0 =C2=A0 =C2=A0 =C2=A0 binary.native
=C2=A0 =C2=A0 =C2=A0 =C2=A0 binary.byte
=C2=A0 =C2=A0 =C2=A0 =C2=A0 binary.coverage (say, for native binary with co= verage instrumentation)

without duplicating too much code in the _oasis file. It is my goal to prod= uce such binaries for installation and hence would like to avoid doing this= sequentially like in

=C2=A0 =C2=A0 =C2=A0 =C2=A0 make opt; make clean; make byte; make clean

but rather have one target that produces all variants. If this is impossibl= e, I=E2=80=99d revert to building variants sequentially. The reason I expec= t this to be difficult is that an object file produced with and without ins= trumentation looks the same and thus it might be difficult for the build sy= stem to understand that these need the be rebuilt.

Code instrumented for coverage needs to be compiled with the corresponding = PPX. OCamlfind and OCamlbuild abstracts this nicely. For compiling example.ml = it=E2=80=99s enough to say:

=C2=A0 =C2=A0 =C2=A0 =C2=A0 ocamlbuild =E2=80=94use-ocamlfind -pkg bisect_p= px example.native

It is less obvious how to organise this in a bigger project. (For now I add= rules

=C2=A0 =C2=A0 =C2=A0 =C2=A0 <**/*.ml{,i,y}>: pkg_bisect_ppx
=C2=A0 =C2=A0 =C2=A0 =C2=A0 <**/*.native>: pkg_bisect_ppx

to the _tags file generated by Oasis.) I believe the problem already arises= with profiling and debugging builds and I would be happy to learn how OCam= lbuild can be directed via myocamlbuild.ml to do this.

=E2=80=94 Christian


--001a1143dd0aad2155053336a4a0--