Stream parser/make_lexer

Stream parser/make_lexer

[Sen Horak]

Hi,

I tried building a small parser using Genlexer.make_parser and the
stream parser through the camlp4 preprocessor. I'm using the ocaml
compiler 3.08.1.

let lexer=make_lexer ["begin";"end";...]

let parse = parser
      [<'Kwd "name=";'String t>] -> wm#setname t
  |  ...

let istream = of_channel stdin

let ilexer = lexer istream in
  parse ilexer

The program exits with a Stream.Failure exception. The "stream
builders" documentation clearly says that we are not to mix the of_*
functions to build streams with <> streams - so I guess this is
expected. Although I've come across some examples that use of_string
to build the char stream and use the lexer on it.

How then does one build a stream compatible with the stream parser
connected to stdin?

Also, is the make_lexer/stream parser way of building a parser
deprecated? I find it a convenient way to build small parsers quickly.
Or do most ocaml programmers just use ocamllex/ocamlyacc for
everything?

SH

Re: [Caml-list] Stream parser/make_lexer

[Oliver Bandel]

On Sun, Jun 26, 2005 at 12:12:40PM +0200, Sen Horak wrote:
[...]
> Also, is the make_lexer/stream parser way of building a parser
> deprecated? I find it a convenient way to build small parsers quickly.
> Or do most ocaml programmers just use ocamllex/ocamlyacc for
> everything?

I sometimes used the Stream-Module.
But in the last time I switched more to ocamllex/ocamlyacc.
The reason is, that I find them a good tool for many things.
But I didn't tried the camlp4-stuff. It may be more easy to
write complicated parsers with thema and maybe more elegant.

But one reason to use ocamllex/ocamlyacc is that they are
similar to lex/yacc, and therfore, if programming in a
commercial field, porting the Ocaml-software back to C
(even if not really happy about this), is easier.

So during the next days and weeks I think I will develop
some software twice: in C and the same program also in OCaml.
This will yield insights to how much easier and safer it is to develop
in OCaml than in C.

I just yesterday got a "Bus error" because one function
was not/bad implemented in the C version. I didn't ported
that stuff to OCaml now, but it is planned.
And OCaml had warned me that there is a problem.

(But, btw: using -Wall from gcc would have warned me, but
 OCaml wouldn't compile the code...)
 
So, using ocamllex/ocymlyacc has some advantages that are
not in the programming field itself... but in the working
environment / where you want to use your software.

It's really hard to start with OCaml and then restrictions
say that you have to use C instead... but that's how it is.

Ciao,
   Oliver

Re: [Caml-list] Stream parser/make_lexer

[Sen Horak]

Ok, but that still doesn't answer my question (on how to connect the
stdin channel to a <> compatible char stream).

> I sometimes used the Stream-Module.
> But in the last time I switched more to ocamllex/ocamlyacc.
> The reason is, that I find them a good tool for many things.
> But I didn't tried the camlp4-stuff. It may be more easy to
> write complicated parsers with thema and maybe more elegant.

The documentation says that the parser interface is no longer
supported by proper ocaml, so we need to preprocess with ocampp4.
 
> But one reason to use ocamllex/ocamlyacc is that they are
> similar to lex/yacc, and therfore, if programming in a
> commercial field, porting the Ocaml-software back to C
> (even if not really happy about this), is easier.

The parser interface seems to be a nice thing if you want to "script"
in ocaml - that is, if you want to write quick and dirty solutions to
problems that you would ordinarily do in Perl/PYthon etc.

Of course, if you want to write a more complex parser, it's better to
do it in lex/yacc.

SH

Re: [Caml-list] Stream parser/make_lexer

[Virgile Prevosto]

Hello,
2005/6/26, Sen Horak <sen.horak@gmail.com>:
> let parse = parser
>       [<'Kwd "name=";'String t>] -> wm#setname t
>   |  ...
> 
> let istream = of_channel stdin
> The program exits with a Stream.Failure exception. The "stream
> builders" documentation clearly says that we are not to mix the of_*
> functions to build streams with <> streams - so I guess this is
> expected. Although I've come across some examples that use of_string
> to build the char stream and use the lexer on it.
> 

I can be wrong, but I think that the documentation prohibits stuff like
let s = Stream.of_string "foo" in [< s; s >], that is building a
stream with both the of_* functions and the [< >] notation. This is
not the case here, since [< >] is used only for pattern matching.

Concerning your original problem, I think that the main issue is that
"name=" is not a valid keyword for lexer. Try to split it in "name"
and "=" (cf documentation of Genlex.token), i.e. do something like
let lexer = Genlex.make_lexer ["name"; "="; ...]
let parse = parser [< 'Kwd "name"; 'Kwd "="; 'String t >] -> ... 

-- 
E tutto per oggi, a la prossima volta
Virgile

Re: [Caml-list] Stream parser/make_lexer

[Oliver Bandel]

On Sun, Jun 26, 2005 at 03:25:36PM +0200, Sen Horak wrote:
> Ok, but that still doesn't answer my question (on how to connect the
> stdin channel to a <> compatible char stream).


Yes, sorry, I was a littlebid off-topic.
But you asked, if the stream-parsers are
deprecated.
And you also asked, if most people are using ocamllex/ocamlyacc.
So I thought it was for your interest too,
to hear such opinions about selecting ocamllex/ocamlyacc.

For later I will try the other parer-stuff too, but for now,
where I look to get some projects running, it's best to do it
such a "compatible" way.
So rapid development can be done in OCaml very fast. And when all
goes well, the software will be ported back to C, if not possible
to show the people that it's better to let it as it is (and in OCaml).

If this did not interest you, sorry.



> 
> > I sometimes used the Stream-Module.
> > But in the last time I switched more to ocamllex/ocamlyacc.
> > The reason is, that I find them a good tool for many things.
> > But I didn't tried the camlp4-stuff. It may be more easy to
> > write complicated parsers with thema and maybe more elegant.
> 
> The documentation says that the parser interface is no longer
> supported by proper ocaml, so we need to preprocess with ocampp4.

Well ocampp4? The first time I hear from ocampp4.


>  
> > But one reason to use ocamllex/ocamlyacc is that they are
> > similar to lex/yacc, and therfore, if programming in a
> > commercial field, porting the Ocaml-software back to C
> > (even if not really happy about this), is easier.
> 
> The parser interface seems to be a nice thing if you want to "script"
> in ocaml - that is, if you want to write quick and dirty solutions to
> problems that you would ordinarily do in Perl/PYthon etc.

Ah, ok. Fine. ;-)

> 
> Of course, if you want to write a more complex parser, it's better to
> do it in lex/yacc.

Ah, ok. So then I decided the right way. :)

Sometimes it seems to mee, that people who are writing about
the stream-parsers on this list, would ´choose this because it's
good for doing hard parser stuff.
So I maybe misunderstood this.

As I planned (for a long time, but now started to lift up my ass and
start with it) to implement own languages (Domain Specific Languages),
so I need a tool for doing the lex-/parse-stuff.

I will start with simple things, but I'm shure it will be complex
short time later... :)

And as maybe said before, I'm happy to have OCaml on my side. :)

BTW: is there something like a converter from OCaml to C, so
that I can write OCaml-code but for people who wants it in C
it can be converted automatically?
(Well, they should install ocaml instead... but....)

Ciao,
   Oliver

Re: [Caml-list] Stream parser/make_lexer

[Sen Horak]

Hi,

Thanks for this. Your answer fixed the problem.

Additionally, i was wrongly using 'String instead of 'Ident. 'String
it appears, is
for quoted text - which of course is in the manual. My bad.

SH

On 6/26/05, Virgile Prevosto <virgile.prevosto@gmail.com> wrote:
> Hello,
> 2005/6/26, Sen Horak <sen.horak@gmail.com>:
> > let parse = parser
> >       [<'Kwd "name=";'String t>] -> wm#setname t
> >   |  ...
> >
> > let istream = of_channel stdin
> > The program exits with a Stream.Failure exception. The "stream
> > builders" documentation clearly says that we are not to mix the of_*
> > functions to build streams with <> streams - so I guess this is
> > expected. Although I've come across some examples that use of_string
> > to build the char stream and use the lexer on it.
> >
> 
> I can be wrong, but I think that the documentation prohibits stuff like
> let s = Stream.of_string "foo" in [< s; s >], that is building a
> stream with both the of_* functions and the [< >] notation. This is
> not the case here, since [< >] is used only for pattern matching.
> 
> Concerning your original problem, I think that the main issue is that
> "name=" is not a valid keyword for lexer. Try to split it in "name"
> and "=" (cf documentation of Genlex.token), i.e. do something like
> let lexer = Genlex.make_lexer ["name"; "="; ...]
> let parse = parser [< 'Kwd "name"; 'Kwd "="; 'String t >] -> ...
> 
> --
> E tutto per oggi, a la prossima volta
> Virgile
> 
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>