Re: [Caml-list] Lexing.lexeme_start_p broken?

[skaller <skaller@users.sourceforge.net>]

On Tue, 2004-09-21 at 18:26, Damien Doligez wrote:
> On Sep 20, 2004, at 16:44, skaller wrote:

> The token supplying function is supposed to _update_ the lexbuf, if
> you want the parser to report the correct locations.  ocamllex does
> some of the work by updating the char count, 

Not in my case it doesn't. 
The lexing function isn't an ocamllex lexer.
So I'd have to update the char count too.

I actually am using ocamllex, but I drive it manually
to collect a token list, then feed the list to the parser.

Hmmm.. OK, how is this for an idea:

Suppose we add to the lexbuf a mutable field of type:

	lexbuf -> loc

which returns the location information the parser needs
given a lexbuf. The parser then fetches the location
information by calling this function on the lexbuf
from which it was obtained.

I can then provide a function which accepts my own
state object and curry it. This way, I don't have
to keep updating the lexbuf, and the parser cannot
see the lexbuf details. My routine might be
expensive -- but it only gets called once when
there is a parse error, not every token.

Whilst I don't think this is a perfect solution,
it does seem to partially decouple the parser from
the lexbuf by at least abstracting it using a function.

Would this interfere with the Ocaml bootstrap?

*** a better solution might be to pass this function
directly the the parser, thereby decoupling it
entirely from the lexer. However that changes the
type of parser functions. That can easily be fixed
though -- just make a compatibility wrapper which 
calls the full parser function, passing a default
function value.

If there was any interest, I could probably provide
a design which provided proper decoupling, whilst
retaining compatibility using wrappers and defaults.
[But I imagine it could also be done easily by someone
on the Ocaml team -- and throw in a user state object
at the same time please, as has been done for the lexer :]

The parser does need to get tokens, and it may need
location information, but it should not
depend on an object whose principle purpose is 
to support lexing.

In theory this is also true for generated lexers:
they shouldn't depend on any lexbufs. However for
performance reasons, abstracting a character source
probably isn't tolerable.


-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners