[Caml-list] On ocamlyacc and ocamllex

["Vesa Karvonen" <vesa.karvonen@housemarque.fi>]

Hi,

Ocamlyacc with the standard Parsing module seems to generate non-reentrant
parsers, because the env is stored as a global variable in the Parsing module.
(BTW: Having a reentrant Parsing module would also eliminate the need for the
clear_parser function.)

Ocamllex with the Lexing module generates a lexer that seems to be difficult
to extend without using global variables. You may wonder what I mean with
extending the lexer. I would like to make it so that the lexer would record
the positions of line breaks so that I could directly give line number and
column information in error messages. I would like to effectively augment the
lexbuf with additional fields and write new functions for the augmented lexbuf
(functions manipulating the new fields).

Has someone written extensible and reentrant lexing and parsing modules for
Ocaml?

Is someone interested in such lexer and parser modules?

I was thinking about writing replacements for the Lexing and Parsing modules
that would use the (lex_)engine and parse_engine external functions. The
replacements would effectively be parameterized layers on top of some of the
facilities in Lexing and Parsing modules. In addition I also thought about
writing a program that modifies the files generated by ocamllex and ocamlyacc
so that they can be used with the replacement Lexing and Parsing modules.

Are the some hidden problems with the above approach?

Alternatively, the tools and modules of the Ocaml distribution could be
modified, but this isn't necessarily an easy route.

As my project schedule is tight and I may be able to live with the global
variables, I don't necessarily want to spend time on this, but if there are
others interested in such modules, I might put in some extra time to implement
them.

...

Another issue with ocamllex and ocamlyacc (and lex/flex and yacc/bison) is
that the dependencies between the generated lexer and parser are not quite
optimal. Currently the generated lexer is dependent on the parser, because the
parser generates the token type. This means that each time the grammar is
modified, but not the token definitions, the lexer is recompiled. This could
be avoided by making it so that the token type is defined in a separate
module.

Currently: lexer -> parser.token
Effect: lexer recompiled when grammar or token type is modified

Ideally: lexer -> token <- parser
Effect: lexer recompiled only when token type is modified



-------------------
Bug reports: http://caml.inria.fr/bin/caml-bugs  FAQ: http://caml.inria.fr/FAQ/
To unsubscribe, mail caml-list-request@inria.fr  Archives: http://caml.inria.fr