[Caml-list] Implementing include "file" statement in menhir

["Richard W.M. Jones" <rich@annexia.org>]

[Resend, apologies if you get this twice, but I sent it
earlier and that seems to have disappeared.]

I'm writing a parser which needs to have a C-like include directive.
There's an old thread on this describing a rather complicated way to
do this for ocamllex:
https://groups.google.com/forum/#!topic/fa.caml/_v_k4WTQV_Q

I thought I'd have a go at writing an include statement in menhir, and
I did come up with something which works but it's quite a large hack.
What I did is documented below, but I wonder if someone can think of a
simpler way to do this?  Also two related questions:

How do you pass extra parameters to menhir's generated parser
functions?

Is there a nice way to export values into menhir's generated
parser.mli file?

----

The concept behind my include statement uses the following grammar:

  %token INCLUDE
  %token <string> STRING
  %start file
  %%
  file: list(stmt) ;

  stmt:
      | INCLUDE STRING
      {
        let filename = $2 in
        let fp = open_in filename in
        let lexbuf = Lexing.from_channel fp in
        lexbuf.lex_curr_p <- { lexbuf.lex_curr_p with pos_fname = filename };
	(* Recursively call Parser.file: *)
        file Lexer.read lexbuf;
        close_in fp;
      }
      | ... other statements ...
      ;


Unfortunately as written the above code cannot work because it
introduces a circular dependency between the Parser and the Lexer
modules (normally the Lexer module depends on the Parser, and so the
Parser cannot use any functions from the Lexer module).

To break the cycle we have to add:

  %{
  let lexer_read = ref None
  %}

and replace Lexer.read with:

  let reader =
     match !lexer_read with None -> assert false | Some r -> r in
  file reader lexbuf;

Then to initialize lexer_read, we have to export it by doing this
hack:

  menhir parser.mly
  echo 'val lexer_read : (Lexing.lexbuf -> token) option ref' >> parser.mli

and we can set it from the main program.

Rich.