Re: [Caml-list] lexer library

["Nicolas Pouillard" <nicolas.pouillard@gmail.com>]

Excerpts from Jon Harrop's message of Sun Aug 19 00:34:44 +0200 2007:
> On Saturday 18 August 2007 19:39:52 skaller wrote:
> > I am interested in building a dynamically extensible lexer.. can
> > anyone recommend any libraries or tools that would help?
> >
> > I could use Dypgen (extensible GLR parser) for this, but it is
> > probably too slow.
> >
> > A system using combinators (rather than strings) would be preferred,
> > and Unicode support would be nice too. Ideally the lexer automaton
> > could be saved with Marshal, in which case the time to build the
> > automaton isn't important.
> 
> The new camlp4 looks like it supports this but I'm damned if I can figure out 
> how you use it. There is a beautiful tutorial on the Wiki that fizzles out 
> just when it gets exciting.
> 

Not  really,  with  the new camlp4 you can change the lexer and the token type
(thanks  to  the  system  module).  One can also dynamically add lexer filters
that  transforms  the  token stream. However the provided default lexer is not
extensible (it's an ocamllex one).

-- 
Nicolas Pouillard aka Ertai