[Caml-list] ocamllex/yacc and camlp4

[Caml-list] ocamllex/yacc and camlp4

[John Goerzen]

Hi,

Quick question: why are ocamllex and ocamlyacc not implemented with
camlp4?  They seem to be doing exactly what camlp4 is there for, and I
think would serve as great camlp4 examples (plus being able to extend
*their* syntax could be very powerful indeed.)

Thanks,
John

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] ocamllex/yacc and camlp4

[Luc Maranget]

> Hi,
> 
> Quick question: why are ocamllex and ocamlyacc not implemented with
> camlp4?  They seem to be doing exactly what camlp4 is there for, and I
> think would serve as great camlp4 examples (plus being able to extend
> *their* syntax could be very powerful indeed.)
> 
> Thanks,
> John
> 
> -------------------
> To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
> Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Hello,

First there is history, ocamllex and ocamlyacc predate camlp4, thus
they were not written with camlp4 initially.

Second there is bootstrap. Since the lexer and parser of ocamlc itself
are written with ocamllex/ocamlyacc, Making these tools to depend on
camlp4 would include camlp4 in the bootstrap cycle of ocamlc.
The resulting situation would complicate bootstraping ocamlc.

Of course there could be camlp4 versions of ocamllex/ocamlyacc in
addition to ocamllex/ocamlyacc versions of ocamllex/ocamlyacc. Well,
nobody ever thought about doing that, I guess.


-- Luc Maranget

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] ocamllex/yacc and camlp4

[Pierre Weis]

Hi,

> > Quick question: why are ocamllex and ocamlyacc not implemented with
> > camlp4?  They seem to be doing exactly what camlp4 is there for, and I
> > think would serve as great camlp4 examples (plus being able to extend
> > *their* syntax could be very powerful indeed.)
> > 
> > Thanks,
> > John
> 
> Hello,
> 
> First there is history, ocamllex and ocamlyacc predate camlp4, thus
> they were not written with camlp4 initially.
> 
> Second there is bootstrap. Since the lexer and parser of ocamlc itself
> are written with ocamllex/ocamlyacc, Making these tools to depend on
> camlp4 would include camlp4 in the bootstrap cycle of ocamlc.
> The resulting situation would complicate bootstraping ocamlc.
> 
> Of course there could be camlp4 versions of ocamllex/ocamlyacc in
> addition to ocamllex/ocamlyacc versions of ocamllex/ocamlyacc. Well,
> nobody ever thought about doing that, I guess.
> 
> -- Luc Maranget

I would add two points to Luc's answer:

 1) ocamllex and ocamlyacc implementation technologies are damned fast
and it is difficult to compete with them using streams.

 2) Semantics differences between Yacc and functionnal parsing are
large and complex, so that implementing the precise Yacc semantics
with its reduce/reduce and shift/reduce conflicts and the default
conflicts resolution that Yacc also implements could not be a trivial
task.

Last but not least, the actual ocamllex/ocamlyacc implementations work
pretty well, so that there is no clear necessity to rewrite them.

In conclusion: pure Camlp4 implementation of ocamllex/ocamlyacc is
still an interesting and challenging progamming task for the next few
years, if you (or someone else) had the will and time to provide two
``great camlp4 examples'' to the rest of us...

Happy hacking :)

Pierre Weis

INRIA, Projet Cristal, Pierre.Weis@inria.fr, http://pauillac.inria.fr/~weis/


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] ocamllex/yacc and camlp4

[Shawn Wagner]

On Wed, Jun 16, 2004 at 08:56:16AM +0200, Luc Maranget wrote:
> 
> Of course there could be camlp4 versions of ocamllex/ocamlyacc in
> addition to ocamllex/ocamlyacc versions of ocamllex/ocamlyacc. Well,
> nobody ever thought about doing that, I guess.

I remember hearing of such a thing for ocamllex. In fact, it seems to be in
the 3.07 distribution: ocaml-3.07/camlp4/etc/pa_ocamllex.ml. Doesn't appear
to have been built or installed, though.

-- 
Shawn Wagner
shawnw@speakeasy.org

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] ocamllex/yacc and camlp4

[William Lovas]

On Thu, Jun 17, 2004 at 12:48:01AM +0200, Pierre Weis wrote:
> In conclusion: pure Camlp4 implementation of ocamllex/ocamlyacc is
> still an interesting and challenging progamming task for the next few
> years, if you (or someone else) had the will and time to provide two
> ``great camlp4 examples'' to the rest of us...

Is such a thing even possible?  I thought ocamllex/ocamlyacc produced
parsers for LALR(1) languages while camlp4 produced parsers for LL(k)
languages.  These two sets do not bear a subset relation in either
direction, so i would think it impossible to completely implement either
tool in terms of the other.  Have i misunderstood something?

cheers,
William

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] ocamllex/yacc and camlp4

[Alain Frisch]

On Thu, 17 Jun 2004, Pierre Weis wrote:

>  1) ocamllex and ocamlyacc implementation technologies are damned fast
> and it is difficult to compete with them using streams.

As I understand it, the question was about implementing ocamllex/ocamlyacc
frontend (parsing the specifications) and backend (producing OCaml code)
using Camlp4, not about implementing lexers/parsers using Camlp4 grammars.

I wrote a pa_ocamllex some time ago (and it is included in the OCaml
distribution), but it is no longer in sync with new ocamllex features
(capture variables).

Using Camlp4 to parse the specifications has several advantages: simpler
build procedure (no intermediate file to be generated), support for
alternative syntaxes (e.g. the revised syntax).


-- Alain

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] ocamllex/yacc and camlp4

[skaller]

On Thu, 2004-06-17 at 08:48, Pierre Weis wrote:

>  1) ocamllex and ocamlyacc implementation technologies are damned fast
> and it is difficult to compete with them using streams.

They're not so fast when your problem exceeds the constraints
which determine what they're good at. All my lexers generate
an in memory token list for this reason.

> Last but not least, the actual ocamllex/ocamlyacc implementations work
> pretty well, so that there is no clear necessity to rewrite them.

I think there is: they haven't worked so well for any of the
parsers I've had to write -- not even the Felix parser which
is specifically designed to be unambiguious LALR(1) 
and Ocamlyaccable.

The parser can't take a state argument, it can't accept
a token type, the generated interface can't be added to
by the client which is necessary when you need to define
a function callable by the lexer and parser which depends on the
type of a token, you can't use a meta-grammar notation with
the obvious interpretation (a* makes a list, a? an option).

Considerable effort is required to decouple the
faulty interface which makes the parser depend on
a lexbuf.

LALR(1) is very hard to work with, and often the easiest
workaround is by doing some lookahead in the tokeniser:
the coupling of the parser and lexer make this difficult.
Felix lexer/parser needs about 8 files: more than any
other part of the compiler.

Also Ocamllex is only an 8 bit lexer which isn't
that useful these days where XML/Web stuff demands UTF-8
encoded Unicode.

> In conclusion: pure Camlp4 implementation of ocamllex/ocamlyacc is
> still an interesting and challenging progamming task for the next few
> years, if you (or someone else) had the will and time to provide two
> ``great camlp4 examples'' to the rest of us...
> 
> Happy hacking :)

Ulex already integrates lexing and provides UTF-8,
Camomile already provides 32 bit lexers. Code exists.
The problem here isn't hacking the code, but getting
INRIA to agree to sit down and work with the community
on designing an interface specification for a facility
good enough to put directly in the standard distro.
Once that were agreed I'm quite sure the non-INRIA
community would rapidly provide an implementation.

As an added incentive: an integrated lexer automatically
provides a superior alternative to Str and PCRE, and if
you do a parser as well, an even better system: quite a
lot of 'regexp' stuff should actually be done by a real
parser.

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] ocamllex/yacc and camlp4

[skaller]

On Thu, 2004-06-17 at 12:04, William Lovas wrote:
> On Thu, Jun 17, 2004 at 12:48:01AM +0200, Pierre Weis wrote:
> > In conclusion: pure Camlp4 implementation of ocamllex/ocamlyacc is
> > still an interesting and challenging progamming task for the next few
> > years, if you (or someone else) had the will and time to provide two
> > ``great camlp4 examples'' to the rest of us...
> 
> Is such a thing even possible?  I thought ocamllex/ocamlyacc produced
> parsers for LALR(1) languages while camlp4 produced parsers for LL(k)
> languages.  These two sets do not bear a subset relation in either
> direction, so i would think it impossible to completely implement either
> tool in terms of the other.  Have i misunderstood something?

Yes: there is no need for camlp4 to generate lexers and
parsers replacing those you'd build with ocamllex/ocamlyacc,
the need is to replace the ocamllex/ocamlyacc parsers used
to scan your lexer/grammar specifications -- and that
can easily be done by camlp4 (and for lexers certainly
has been done several times eg ulex).

Having read such specifications 'inline' in Ocaml code,
actually generating the parsers/lexers to implement
the semantics can easily be done by camlp4 by simply
calling an external function.

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

[Caml-list] Startconditions in ocamllex

[oliver]

Hello,


in the original lex it is possible to use startconditions.
Is that possible in ocamllex too?
There is no description of start conditions in the
ocamllex-section of the Ocaml-manual.

So I'm not sure if that feature is implemented and not documented,
or if it is not implemented.

And: which syntax to use? Is it the same like in lex?

Ciao,
   Oliver

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Startconditions in ocamllex

[Luc Maranget]

> Hello,
> 
> 
> in the original lex it is possible to use startconditions.
> Is that possible in ocamllex too?
> There is no description of start conditions in the
> ocamllex-section of the Ocaml-manual.
> 
> So I'm not sure if that feature is implemented and not documented,
> or if it is not implemented.
> 
> And: which syntax to use? Is it the same like in lex?
> 
> Ciao,
>    Oliver

Start conditions, whatever they are, are not implemented in ocamllex.

Cheers,

--Luc

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Startconditions in ocamllex

[Hendrik Tews]

   > in the original lex it is possible to use startconditions.
   > Is that possible in ocamllex too?
   
   Start conditions, whatever they are, are not implemented in ocamllex.
   
You can simulate startconditions with several entry points and a
global variable. See my message
http://caml.inria.fr/archives/200309/msg00027.html (just ask
again if that is not clear enough).

Bye,

Hendrik

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners