If there is a good, general, alternative approach for this, we can support it in Bisect_ppx. Unfortunately, I don’t know enough about Menhir to be able to propose anything specific at this point. Best, Anton > On Mar 6, 2016, at 16:53, Gabriel Scherer wrote: > > This is an interesting question and, as far as I know, there is no good solution using existing versions of the interacting tools. > > Below very simple patch that will add (*BISECT-IGNORE*) in front of every line of code generated by Menhir, except those written by the programmer (the "strecthes" in Menhir-speak). It applies cleanly on top of the latest released Menhir archive, > http://gallium.inria.fr/~fpottier/menhir/menhir-20160303.tar.gz > > The patch as-is is obviously a hack: it would need to be a configuration option when running menhir, and hard-coding Bisect (or bisect_ppx)'s syntax into Menhir is not elegant. One could try to have a configuration option to let users write a fixed string (or comment) at the beginning of each generated code line, but I'm not sure whether François Pottier (in cc:) would consider this is elegant enough. François, would you comment on whether this is a direction that seems acceptable to you? > > (Bisect support ignoring entire regions at once by using (*BISECT-IGNORE-BEGIN*) and (*BISECT-IGNORE-END*); we could try to implement that instead of a per-line change, but I suspect that it would be slightly harder to implement (you have to hook the beginning of input, end of input, and around each user-code insertion) for no real gain.) > > Toggling code-coverage semantics by inserting comments is not a very nice interface (although rather logical when you think of the level of generality required), so it's a bit frustrating that parser generators would have to play at this level. It would be better to have a more structured, unified interface supported by all the code-coverage tools, but to my knowledge no such thing exists. > > > From d595ba5149a314c56623e1735af7678f5f62d525 Mon Sep 17 00:00:00 2001 > From: Gabriel Scherer > > Date: Sun, 6 Mar 2016 17:43:14 -0500 > Subject: [PATCH] output (*BISECT-IGNORE*) in front of each > non-programmer-written line > > EXPERIMENTAL PATCH: this should of course be turned into an explicit option > --- > src/printer.ml | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/src/printer.ml b/src/printer.ml > index ea978bc..714bb08 100644 > --- a/src/printer.ml > +++ b/src/printer.ml > @@ -46,6 +46,7 @@ let rawnl f = > > let nl f = > rawnl f; > + output_string f "(*BISECT-IGNORE*)"; > output_substring f whitespace 0 !indentation > > let indent ofs producer f x = > -- > 2.5.0 > > > > On Sun, Mar 6, 2016 at 2:53 PM, Vincent Jacques > wrote: > Hello, > > Does somebody have experience measuring test coverage of generated lexers/parsers? > > I'm using ocamllex/ocamlyacc [1] (but I can switch to Menhir [2]) to generate a lexer/parser. In my tests, I simply check that some input strings give the ASTs I expect. > > I usually use Bisect [3] to make sure that my tests cover the code I intended to cover, but in that configuration, Bisect is lost between the .mll/.mly files and the generated .ml files and produces useless reports. > > How would you measure test coverage in that case? > > Thanks, > > [1] http://caml.inria.fr/pub/docs/manual-ocaml/lexyacc.html > [2] http://gallium.inria.fr/~fpottier/menhir/ > [3] http://bisect.x9c.fr/ > -- > Vincent Jacques > http://vincent-jacques.net > > "S'il n'y a pas de solution, c'est qu'il n'y a pas de problème" > Devise Shadock > > <0001-output-BISECT-IGNORE-in-front-of-each-non-programmer.patch>