[Caml-list] Using menhir for a camlp4 quotation: a compilation issue

[Caml-list] Using menhir for a camlp4 quotation: a compilation issue

[Philippe Veber]

[-- Attachment #1: Type: text/plain, Size: 2060 bytes --]

Dear camlers,

I'm currently trying to implement a camlp4 quotation whose contents will be
parsed using ocamllex and menhir. Everything went unexpectedly well until I
tried to add antiquotations. Following Tiphaine Turpin's format syntax
extension (https://forge.ocamlcore.org/projects/format/), an antiquotation
is parsed as a token in ocamllex, and this token is defined in my parser as:

%token <Camlp4.PreCast.Syntax.Ast.expr> EXPR

The thing is when I try to compile my project with ocamlbuild I get the
following error:

[gesundheit:~/w/ocaml-r 14:46]$make
ocaml setup.ml -build
Finished, 0 targets (0 cached) in 00:00:00.
+ menhir --ocamlc 'ocamlfind ocamlc -I src/syntax' --infer
src/syntax/r_lang_parser_y.mly
File "src/syntax/r_lang_parser_y.mly", line 7, characters 40-44:
Warning: the token EXPR is unused.
File "src/syntax/r_lang_parser_y.mly", line 7, characters 8-38:
Error: Unbound module Camlp4
...

Now I can easily (but manually) fix this by adding the missing -I option
for camlp4 libs:

[gesundheit:~/w/ocaml-r/_build 15:22]$menhir --ocamlc 'ocamlfind ocamlc -I
src/syntax -I +camlp4' --infer src/syntax/r_lang_parser_y.mly
File "src/syntax/r_lang_parser_y.mly", line 7, characters 40-44:
Warning: the token EXPR is unused.
[gesundheit:~/w/ocaml-r/_build 15:22]$

My question is how do I tell ocamlbuild to add this option directly? I fear
the response is "you cannot", if I refer to the piece of code in ocamlbuild
that generates the call to menhir:

let menhir mly env build =
  let mly = env mly in
  let menhir = if !Options.ocamlyacc = N then V"MENHIR" else
!Options.ocamlyacc in
  Ocaml_compiler.prepare_compile build mly;
  Cmd(S[menhir;
        A"--ocamlc"; Quote(S[!Options.ocamlc; ocaml_include_flags mly]);
        T(tags_of_pathname mly++"ocaml"++"parser"++"menhir");
        A"--infer"; Px mly])

I fail to see in this function a hook which I could use to inject the
missing argument. Would any one know a workaround to use tokens that have
hold a value whose type is not defined in the standard library?

Cheers,
  Philippe.

[-- Attachment #2: Type: text/html, Size: 4474 bytes --]

Re: [Caml-list] Using menhir for a camlp4 quotation: a compilation issue

[Gabriel Scherer]

I'm not familiar with this part of ocamlbuild, but a look at
ocamlbuild/ocaml_tools.ml suggest that the different rule named
"menhir_modular" (which creates stuff from a file with extension
".mlypack" rather than ".mly", and which I have never used) has a
slightly more expression -ocamlc option, that includes (T
ocamlc_tags), which includes in particular the tags for the .mlypack
file.

You could try to see if, naming your grammar file .mlypack and adding
a "foo.mlypack: package(camlp4.lib)" to your _tags, you at least get a
satisfying menhir invocation. From there, you could either find out
what this "modular menhir" mode is and how to use it in your case, or
suggest patching ocamlbuild to use this (T ocamlc_tags) also in simple
menhir invocations.

On Thu, Sep 20, 2012 at 3:45 PM, Philippe Veber
<philippe.veber@gmail.com> wrote:
> Dear camlers,
>
> I'm currently trying to implement a camlp4 quotation whose contents will be
> parsed using ocamllex and menhir. Everything went unexpectedly well until I
> tried to add antiquotations. Following Tiphaine Turpin's format syntax
> extension (https://forge.ocamlcore.org/projects/format/), an antiquotation
> is parsed as a token in ocamllex, and this token is defined in my parser as:
>
> %token <Camlp4.PreCast.Syntax.Ast.expr> EXPR
>
> The thing is when I try to compile my project with ocamlbuild I get the
> following error:
>
> [gesundheit:~/w/ocaml-r 14:46]$make
> ocaml setup.ml -build
> Finished, 0 targets (0 cached) in 00:00:00.
> + menhir --ocamlc 'ocamlfind ocamlc -I src/syntax' --infer
> src/syntax/r_lang_parser_y.mly
> File "src/syntax/r_lang_parser_y.mly", line 7, characters 40-44:
> Warning: the token EXPR is unused.
> File "src/syntax/r_lang_parser_y.mly", line 7, characters 8-38:
> Error: Unbound module Camlp4
> ...
>
> Now I can easily (but manually) fix this by adding the missing -I option for
> camlp4 libs:
>
> [gesundheit:~/w/ocaml-r/_build 15:22]$menhir --ocamlc 'ocamlfind ocamlc -I
> src/syntax -I +camlp4' --infer src/syntax/r_lang_parser_y.mly
> File "src/syntax/r_lang_parser_y.mly", line 7, characters 40-44:
> Warning: the token EXPR is unused.
> [gesundheit:~/w/ocaml-r/_build 15:22]$
>
> My question is how do I tell ocamlbuild to add this option directly? I fear
> the response is "you cannot", if I refer to the piece of code in ocamlbuild
> that generates the call to menhir:
>
> let menhir mly env build =
>   let mly = env mly in
>   let menhir = if !Options.ocamlyacc = N then V"MENHIR" else
> !Options.ocamlyacc in
>   Ocaml_compiler.prepare_compile build mly;
>   Cmd(S[menhir;
>         A"--ocamlc"; Quote(S[!Options.ocamlc; ocaml_include_flags mly]);
>         T(tags_of_pathname mly++"ocaml"++"parser"++"menhir");
>         A"--infer"; Px mly])
>
> I fail to see in this function a hook which I could use to inject the
> missing argument. Would any one know a workaround to use tokens that have
> hold a value whose type is not defined in the standard library?
>
> Cheers,
>   Philippe.
>
>

Re: [Caml-list] Using menhir for a camlp4 quotation: a compilation issue

[Philippe Veber]

[-- Attachment #1: Type: text/plain, Size: 3955 bytes --]

Thanks for your suggestion Gabriel! If I change the extension of my parser
to mlypack, ocamlbuild fails to use it correctly:

[gesundheit:~/w/ocaml-r 17:05]$make
ocaml setup.ml -build
Finished, 0 targets (0 cached) in 00:00:00.
+ ocamlfind ocamlc -c -g -package camlp4 -package camlp4.extend -package
camlp4.lib -package camlp4.quotations -package camlp4.quotations.o -syntax
camlp4o -I src/syntax -o src/syntax/r_lang_lexer.cmo src/syntax/
r_lang_lexer.ml
File "src/syntax/r_lang_lexer.mll", line 4, characters 2-22:
Error: Unbound module R_lang_parser_y

It's not clear to me either what the mlypack are supposed to be. I'll try
to patch the non-modular menhir function in ocamlbuild/ocaml_tools.ml as
you suggest.

Thanks again!
ph.

2012/9/20 Gabriel Scherer <gabriel.scherer@gmail.com>

> I'm not familiar with this part of ocamlbuild, but a look at
> ocamlbuild/ocaml_tools.ml suggest that the different rule named
> "menhir_modular" (which creates stuff from a file with extension
> ".mlypack" rather than ".mly", and which I have never used) has a
> slightly more expression -ocamlc option, that includes (T
> ocamlc_tags), which includes in particular the tags for the .mlypack
> file.
>
> You could try to see if, naming your grammar file .mlypack and adding
> a "foo.mlypack: package(camlp4.lib)" to your _tags, you at least get a
> satisfying menhir invocation. From there, you could either find out
> what this "modular menhir" mode is and how to use it in your case, or
> suggest patching ocamlbuild to use this (T ocamlc_tags) also in simple
> menhir invocations.
>
> On Thu, Sep 20, 2012 at 3:45 PM, Philippe Veber
> <philippe.veber@gmail.com> wrote:
> > Dear camlers,
> >
> > I'm currently trying to implement a camlp4 quotation whose contents will
> be
> > parsed using ocamllex and menhir. Everything went unexpectedly well
> until I
> > tried to add antiquotations. Following Tiphaine Turpin's format syntax
> > extension (https://forge.ocamlcore.org/projects/format/), an
> antiquotation
> > is parsed as a token in ocamllex, and this token is defined in my parser
> as:
> >
> > %token <Camlp4.PreCast.Syntax.Ast.expr> EXPR
> >
> > The thing is when I try to compile my project with ocamlbuild I get the
> > following error:
> >
> > [gesundheit:~/w/ocaml-r 14:46]$make
> > ocaml setup.ml -build
> > Finished, 0 targets (0 cached) in 00:00:00.
> > + menhir --ocamlc 'ocamlfind ocamlc -I src/syntax' --infer
> > src/syntax/r_lang_parser_y.mly
> > File "src/syntax/r_lang_parser_y.mly", line 7, characters 40-44:
> > Warning: the token EXPR is unused.
> > File "src/syntax/r_lang_parser_y.mly", line 7, characters 8-38:
> > Error: Unbound module Camlp4
> > ...
> >
> > Now I can easily (but manually) fix this by adding the missing -I option
> for
> > camlp4 libs:
> >
> > [gesundheit:~/w/ocaml-r/_build 15:22]$menhir --ocamlc 'ocamlfind ocamlc
> -I
> > src/syntax -I +camlp4' --infer src/syntax/r_lang_parser_y.mly
> > File "src/syntax/r_lang_parser_y.mly", line 7, characters 40-44:
> > Warning: the token EXPR is unused.
> > [gesundheit:~/w/ocaml-r/_build 15:22]$
> >
> > My question is how do I tell ocamlbuild to add this option directly? I
> fear
> > the response is "you cannot", if I refer to the piece of code in
> ocamlbuild
> > that generates the call to menhir:
> >
> > let menhir mly env build =
> >   let mly = env mly in
> >   let menhir = if !Options.ocamlyacc = N then V"MENHIR" else
> > !Options.ocamlyacc in
> >   Ocaml_compiler.prepare_compile build mly;
> >   Cmd(S[menhir;
> >         A"--ocamlc"; Quote(S[!Options.ocamlc; ocaml_include_flags mly]);
> >         T(tags_of_pathname mly++"ocaml"++"parser"++"menhir");
> >         A"--infer"; Px mly])
> >
> > I fail to see in this function a hook which I could use to inject the
> > missing argument. Would any one know a workaround to use tokens that have
> > hold a value whose type is not defined in the standard library?
> >
> > Cheers,
> >   Philippe.
> >
> >
>

[-- Attachment #2: Type: text/html, Size: 5257 bytes --]

Re: [Caml-list] Using menhir for a camlp4 quotation: a compilation issue

[Benoit Montagu]

The typical use case for mlypack is the following: you have several 
parser files (say, A.mly and B.mly) that menhir needs to combine (yes, 
menhir can do that), and you want menhir to create a parser C.ml/C.mli.
To do so, you create C.mlypack, whose contents is:
A
B

And then ocamlbuild should work.

In your case, r_lang_parser.mlypack should contain
Foo

where Foo.mly is your menhir file.

I don't know about how to pass extra arguments to menhir.

-- Benoit

Re: [Caml-list] Using menhir for a camlp4 quotation: a compilation issue

[Philippe Veber]

[-- Attachment #1: Type: text/plain, Size: 1072 bytes --]

Good, I could make it work thanks to your combined suggestions! More
precisely:

1. rename r_lang_parser.mly to r_lang_parser_aux.mly
2. create r_lang_parser.mlypack which contents is a single line:
R_lang_parser_aux
3. add a line in the _tags file (the package(camlp4.lib) syntax did not
work for me, I used the tag pkg_camlp4.lib which is created in the oasis
ocamlbuild plugin).

So it does appear that the function to call menhir in ocamlbuild lacks an
argument. I'll make a report in mantis for that.

Thank you both!
ph.

2012/9/20 Benoit Montagu <benoit.montagu@m4x.org>

> The typical use case for mlypack is the following: you have several parser
> files (say, A.mly and B.mly) that menhir needs to combine (yes, menhir can
> do that), and you want menhir to create a parser C.ml/C.mli.
> To do so, you create C.mlypack, whose contents is:
> A
> B
>
> And then ocamlbuild should work.
>
> In your case, r_lang_parser.mlypack should contain
> Foo
>
> where Foo.mly is your menhir file.
>
> I don't know about how to pass extra arguments to menhir.
>
> -- Benoit
>

[-- Attachment #2: Type: text/html, Size: 1469 bytes --]