[Caml-list] (Release) ocamllex tutorial (ver 0.1)

[Caml-list] (Release) ocamllex tutorial (ver 0.1)

[SooHyoung Oh]

If one doesn't have knowledge for lexcial analyzer or parsing theory, it's
not sufficient for an user to use "ocamllex" and "ocamlyacc" after reading
manuals of these utilities only.

One has to read "lex"/"flex" manual and "yacc"/"bison" manual before using
"ocamllex" and "ocamlyacc".

So, I thought it would be helpful if ocaml community has "ocamllex" and
"ocamlyacc" tutorials.

This is my first contribute to "ocamllex tutorial" and I'm preparing the
companion "ocamlyacc tutorial", too.

Any comment will be appreciated.

--- Abstract ---

This is a tutorial on how to use ocamllex which is distributed with Ocaml
language.

This tutorial borrowed lots of part from flex manual, and some part from
Chap. 12 Lexer and parser generators (ocamllex, ocamlyacc).

The companion tutorial for ocamlyacc will be available in one or two weeks
later.

Please mail all comments and suggestions to <shoh at compiler dot kaist dot
ac dot kr>

This tutorial is work-in-progress. The latest version can be found at
http://pllab.kaist.ac.kr/~shoh/ocaml/ocamllex_ocamlyacc/ocamllex-tutorial/index.html.

Last updated: 2004-08-24

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] (Release) ocamllex tutorial (ver 0.1)

[skaller]

On Tue, 2004-08-24 at 18:07, SooHyoung Oh wrote:

> Any comment will be appreciated.

It looks very good!

> This tutorial is work-in-progress. The latest version can be found at
> http://pllab.kaist.ac.kr/~shoh/ocaml/ocamllex_ocamlyacc/ocamllex-tutorial/index.html.

First -- you haven't explained how to use the new argument
feature. A good example would be to take the non-reentrant
line counting example and show how to make it re-entrant
by passing an object containing the counters.

To justify not using a global variable, you can extend
the example to allow for recursively counting #include
files (or something similar :)

The second thing I'd like to see here is an example
showing the common technique of lexing an identifier
and then using a lookup table to see if it is keyword:
this is done because (a) it is possible and (b) it
simplifies the lexer which otherwise needs a huge
number of states.

The third thing I'd like to see is a more functional example
where the lexer isn't just executing code for side effects.
All your examples have the lexer tail calling itself: but
most real lexing applications do not do this, instead
you call the lexer repeatedly from a driver loop.

In particular Ocamlyacc calls Ocamllex to lex one
token at a time.

Finally -- since your tutorial is Ocaml specific,
I'd like to see some information on lexbufs.
Are they functional? Can I write a backtracking
functional parser that just 'goes backwards'
in the input stream using a lexbuf to allow
re-lexing some input? 

How do i cope with the fact
that lexbufs think they're reading named
files with lines in them .. what if this isn't so?

This is actually a design fault which is unfortunately
propagated to Ocamlyacc.

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] (Release) ocamllex tutorial (ver 0.1)

[Nicolas Rougnon-Glasson]

another interesting example is the use of several lexer functions (like 
in the ocaml lexer itself...), to analyse properly string literals and 
nested comments.

skaller wrote:

>On Tue, 2004-08-24 at 18:07, SooHyoung Oh wrote:
>
>  
>
>>Any comment will be appreciated.
>>    
>>
>
>It looks very good!
>
>  
>
>>This tutorial is work-in-progress. The latest version can be found at
>>http://pllab.kaist.ac.kr/~shoh/ocaml/ocamllex_ocamlyacc/ocamllex-tutorial/index.html.
>>    
>>
>
>First -- you haven't explained how to use the new argument
>feature. A good example would be to take the non-reentrant
>line counting example and show how to make it re-entrant
>by passing an object containing the counters.
>
>To justify not using a global variable, you can extend
>the example to allow for recursively counting #include
>files (or something similar :)
>
>The second thing I'd like to see here is an example
>showing the common technique of lexing an identifier
>and then using a lookup table to see if it is keyword:
>this is done because (a) it is possible and (b) it
>simplifies the lexer which otherwise needs a huge
>number of states.
>
>The third thing I'd like to see is a more functional example
>where the lexer isn't just executing code for side effects.
>All your examples have the lexer tail calling itself: but
>most real lexing applications do not do this, instead
>you call the lexer repeatedly from a driver loop.
>
>In particular Ocamlyacc calls Ocamllex to lex one
>token at a time.
>
>Finally -- since your tutorial is Ocaml specific,
>I'd like to see some information on lexbufs.
>Are they functional? Can I write a backtracking
>functional parser that just 'goes backwards'
>in the input stream using a lexbuf to allow
>re-lexing some input? 
>
>How do i cope with the fact
>that lexbufs think they're reading named
>files with lines in them .. what if this isn't so?
>
>This is actually a design fault which is unfortunately
>propagated to Ocamlyacc.
>
>  
>


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners