Re: [Caml-list] [ANN] Bisect_ppx 1.0.0 – Modernized code coverage for OCaml

[Gerd Stolpmann <info@gerd-stolpmann.de>]

[-- Attachment #1: Type: text/plain, Size: 5940 bytes --]

Yes, we used BISECT_MARK a lot to resolve this, but it is a cumbersome
and incomplete solution. Well, error paths are special: some errors must
not occur at all, and some only in certain configurations, and some
errors can be tested out, and others cannot.

What I have in mind, is something like:

 - a new tag NEVER - it is an error if the code is executed.
   NEVER automatically extends to inner blocks. E.g.

   if error then begin 
     (* NEVER *)
     if debug_enabled then debug_log "error";
     failwith "error"
   end

   An "assert false" automatically counts as NEVER.

   In contrast, a MARK/VISIT only ignores code and AFAIK it doesn't
   extend to inner blocks.

 - conditional sections that can be turned off and on via
   command-line switches, and that either count as positive
   (normal) sections or negative sections like NEVER.

   Something like: (* BEGIN(name) *) ... (* END(name) *)

   and then a command-line switch (or env variable)

   -section name=(ignore|expect|never)

The background for conditional sections is that (1) sometimes certain
features are only testable in certain configurations, and (2) you want
to have some control for managing testing - let's first focus on feature
X but ignore Y for a while. In particular for larger code bases this is
very helpful.

But anyway, thanks for listening. Bisect was very helpful for testing
code.

Gerd


Am Mittwoch, den 10.02.2016, 13:33 -0600 schrieb Anton Bachin:
> Assuming this is what Gerd is indeed looking for, I can confirm that
> Bisect_ppx
> still supports it. See
> 
> 
> https://github.com/rleonid/bisect_ppx/blob/master/doc/advanced.md#unreachable-code
> 
> 
> We preferred to document only BISECT_VISIT (which does the same
> thing), in order
> to have only option. But both Bisect_ppx and Bisect support
> BISECT_MARK and
> BISECT_VISIT.
> 
> > On Feb 10, 2016, at 13:23, Gabriel Scherer
> > <gabriel.scherer@gmail.com> wrote:
> > 
> > 
> > 
> > On Wed, Feb 10, 2016 at 2:00 PM, Gerd Stolpmann
> > <info@gerd-stolpmann.de> wrote:
> >         That's interesting news. In my last job I used (the old)
> >         bisect
> >         frequently.
> >         
> >         What I really would love to see is negative coverage: Mark
> >         code sections
> >         where you expect that they are never executed. That could be
> >         a simple
> >         "assert false" but also more extensive error handling.
> >         Ideally, the tool
> >         would automatically recognize certain patterns.
> > 
> > 
> > The old bisect supports special (*BISECT-MARK*) comment to mark dead
> > code for this reason. Do you have something else in mind?
> >  
> >         That's especially useful when you have a management that is
> >         after high
> >         coverage numbers...
> >         
> >         Gerd
> >         
> >         
> >         Am Mittwoch, den 10.02.2016, 09:33 -0600 schrieb Anton
> >         Bachin:
> >         > Hello,
> >         >
> >         > We would like to announce the release of Bisect_ppx 1.0.0,
> >         a code coverage tool
> >         > for OCaml with appealing reports:
> >         >
> >         >   https://github.com/rleonid/bisect_ppx
> >         >
> >         >
> >         > You can see a live coverage report here:
> >         >
> >         >   http://rleonid.github.io/bisect_ppx/coverage/
> >         >
> >         > Reports can also be submitted to Coveralls.io using
> >         ocveralls [1]. See an
> >         > example here [2].
> >         >
> >         >
> >         > Bisect_ppx is a fork of the original Bisect by Xavier
> >         Clerc, with extensive
> >         > further development. Differences from Bisect, and from
> >         earlier versions of
> >         > Bisect_ppx, include:
> >         >
> >         > - the nicer and more legible HTML reports,
> >         > - more thorough instrumentation, now including nested
> >         functions and or-patterns,
> >         > - improved compatibility with other PPX rewriters,
> >         > - an Ocamlbuild plugin,
> >         > - many bugfixes, and
> >         > - usage, performance, and documentation improvements.
> >         >
> >         >
> >         > Bisect_ppx was originally forked to update and maintain
> >         Bisect's PPX mode, with
> >         > the OCaml community moving to PPX. Bisect_ppx does not
> >         have Bisect's Camlp4
> >         > dependency. We do not believe that the original Bisect is
> >         being actively
> >         > maintained.
> >         >
> >         >
> >         > Regards,
> >         > Anton & Leonid
> >         >
> >         >
> >         > P.S. If you are working on a project that uses Bisect_ppx,
> >         please let us know!
> >         >
> >         >
> >         > [1]: https://github.com/sagotch/ocveralls
> >         > [2]:
> >         https://coveralls.io/builds/4913198/source?filename=src%
> >         2Fsyntax%2FinstrumentPpx.ml
> >         >
> >         >
> >         
> >         --
> >         ------------------------------------------------------------
> >         Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
> >         My OCaml site:          http://www.camlcity.org
> >         Contact details:        http://www.camlcity.org/contact.html
> >         Company homepage:       http://www.gerd-stolpmann.de
> >         ------------------------------------------------------------
> >         
> > 
> > 
> 

-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
My OCaml site:          http://www.camlcity.org
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 473 bytes --]