Another nasty with ocamlyacc: magic needed?

[skaller <skaller@users.sourceforge.net>]

I have what looks like a showstopper (meaning black magic
seems to be the only solution).

I need two *distinct* parsers which use the same set
of tokens. This is currently impossible AFAICS without
magic (distinct -- in a different module entirely)

Roughly, the problem is that Ocamlyacc generates the
tokens, and they go in the parser.mli file. So any
second parser has to depend on it. However the only
way to *invoke* the parser is to pass it inside a token.
the interface for that parser must be defined AFTER
the parser.mli since it needs to know the token types.
But it has to be known INSIDE the actual parser,
to invoke it. There is no way to extend the interface
Ocamlyacc generates for a parser (you can add
extra helper functions but they don't go in
the interface).

Thus -- it is impossible. IMHO the traditional yacc
style design is poor, but it works fine for C because
there is no problem with recursive dependencies in
interfaces (header files).

In Ocaml, there is such a problem, and so the yacc design
basically fails -- it isn't suitable for Ocaml.

But i need to work around this somehow, does anyone
have any smart ideas how to fix the problem, without
abandoning Ocamlyacc?

If I could abstract the type of the secondary parser
so it didn't need to expose the knowledge of tokens,
it would fix the problem I think -- then the type could
be defined BEFORE the ocamlyacc parser, but the implementation
defined AFTER it (using Ocaml classes and class types).

Just to be clear the situation is:

user_statement:
  | USER_KEYWORD 
    { 
      let srt, kw = $1 in
      let sr = slift srt in
      print_endline ("USER KEYWORD " ^ kw);
      `AST_nop (sr,kw)
    }

Instead of `AST_nop I need to call a function
which parses more of the input, probably
passing it the current lexbuf as well.
The lexer knows this information and could
put it into the USER_KEYWORD token .. if it
were possible to define the type of the attribute
of the USER_KEYWORD token.. but that can only
be done in one place .. prior to the parser mli
file.

So I think I may have to use Obj.magic .. or completely
replace Ocamlyacc.


-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net