Hi,

I had implemented a include mechanisms at the lexer level using ocamllex for the Caph programming language.
This is here (undocumented unfortunately) (related stuff starts at line 128):

https://github.com/jserot/caph/blob/master/compiler/lexer.mll <https://github.com/jserot/caph/blob/master/compiler/lexer.mll>

Just in case it can help.


Jocelyn

> Le 21 janv. 2020 à 07:48, Richard W.M. Jones <rich@annexia.org> a écrit :
> 
> [Resend, apologies if you get this twice, but I sent it
> earlier and that seems to have disappeared.]
> 
> I'm writing a parser which needs to have a C-like include directive.
> There's an old thread on this describing a rather complicated way to
> do this for ocamllex:
> https://groups.google.com/forum/#!topic/fa.caml/_v_k4WTQV_Q
> 
> I thought I'd have a go at writing an include statement in menhir, and
> I did come up with something which works but it's quite a large hack.
> What I did is documented below, but I wonder if someone can think of a
> simpler way to do this?  Also two related questions:
> 
> How do you pass extra parameters to menhir's generated parser
> functions?
> 
> Is there a nice way to export values into menhir's generated
> parser.mli file?
> 
> ----
> 
> The concept behind my include statement uses the following grammar:
> 
>  %token INCLUDE
>  %token <string> STRING
>  %start file
>  %%
>  file: list(stmt) ;
> 
>  stmt:
>      | INCLUDE STRING
>      {
>        let filename = $2 in
>        let fp = open_in filename in
>        let lexbuf = Lexing.from_channel fp in
>        lexbuf.lex_curr_p <- { lexbuf.lex_curr_p with pos_fname = filename };
> 	(* Recursively call Parser.file: *)
>        file Lexer.read lexbuf;
>        close_in fp;
>      }
>      | ... other statements ...
>      ;
> 
> 
> Unfortunately as written the above code cannot work because it
> introduces a circular dependency between the Parser and the Lexer
> modules (normally the Lexer module depends on the Parser, and so the
> Parser cannot use any functions from the Lexer module).
> 
> To break the cycle we have to add:
> 
>  %{
>  let lexer_read = ref None
>  %}
> 
> and replace Lexer.read with:
> 
>  let reader =
>     match !lexer_read with None -> assert false | Some r -> r in
>  file reader lexbuf;
> 
> Then to initialize lexer_read, we have to export it by doing this
> hack:
> 
>  menhir parser.mly
>  echo 'val lexer_read : (Lexing.lexbuf -> token) option ref' >> parser.mli
> 
> and we can set it from the main program.
> 
> Rich.