Re: [Caml-list] On ocamlyacc and ocamllex

[Christian Lindig <lindig@eecs.harvard.edu>]

On Sun, Sep 23, 2001 at 12:09:14AM +0300, Vesa Karvonen wrote:
> I would like to make it so that the lexer would record the positions
> of line breaks so that I could directly give line number and column
> information in error messages.

I agree that more flexible lexer and parser generators would be nice and
have myself lobbied for them in the past. On the other hand I have
always found my way with the existing ones which probably is the reason
that we still use them. 

The particular problem can be solved outside of Lex and Yacc: in the
Quick C-- compiler we have a mutable Sourcemap.map data type that
records the connection between character positions and
(file,line,column) triples. The scanner call a function Sourcemap.nl for
every newline that it encounters and to build up the connection. Later
the map can be used to find the (file,line,column) position for every
character offset. This method has the advantage that it can deal with
input streams that are created from different source files using a
pre-processor. You can find the module as part of the ocamlerror tool
that annotates stack traces with source code positions:

http://www.eecs.harvard.edu/~lindig/software/download/ocamlerror.tar.gz

> Another issue with ocamllex and ocamlyacc (and lex/flex and
> yacc/bison) is that the dependencies between the generated lexer and
> parser are not quite optimal. Currently the generated lexer is
> dependent on the parser, because the parser generates the token type.
> This means that each time the grammar is modified, but not the token
> definitions, the lexer is recompiled. This could be avoided by making
> it so that the token type is defined in a separate module.

This is a general problem with make: when you edit a comment, a file is
touched and all dependent files must be recompiled. Knowing nothing
about OCaml, Make must assume that a touched file has changed in a
significant way. You can help Make out by comparing files explicitly
(untested - you get the idea): foo.mli is only updated, if the token
type has changed.

foo.mli foo.ml:         bar.mly
                        ocamlyacc bar.mly
                        cp bar.ml  foo.ml
                        cmp -s bar.mli foo.mli || cp bar.mli foo.mli 

-- Christian    

-- 
Christian Lindig          Harvard University - EECS 
lindig@eecs.harvard.edu   33 Oxford St, MD 242, Cambridge MA 02138
phone: (617) 496-7157     http://www.eecs.harvard.edu/~lindig/   
                          
-------------------
Bug reports: http://caml.inria.fr/bin/caml-bugs  FAQ: http://caml.inria.fr/FAQ/
To unsubscribe, mail caml-list-request@inria.fr  Archives: http://caml.inria.fr