Ocamlbuild + menhir + separate token file

Ocamlbuild + menhir + separate token file

[hmf]

[-- Attachment #1: Type: text/plain, Size: 3258 bytes --]

Hello,

While working on some parser stuff I had some compilation errors.
I am attempting to use a lexer whose tokens are shared by two
parsers implemented using Menhir. This has already been done
and for the life of me I cannot reproduce the results in [1].

So I have used menhir's demos (calc-param) [2] to try and set-up
a small and very simple build process. The source code consists of:

calc.ml
lexer.mll
tokens.mly
parser.mly

where tokens.mly are shared by the lexer and parser.

My _tag file is:
true: use_menhir, debug, profile
"tokens.mly": only_tokens
"parser.mly": token_base, ext_tokens, use_tokens

My myocamlbuild file is:
open Ocamlbuild_plugin;;
open Command;;

dispatch begin function
| After_rules ->

     (* Menhir --explain flag *)
     flag ["ocaml"; "parser" ; "menhir" ; "use_menhir" ]
     (S[A"--explain"]);

     (* Menhir --only-tokens flag *)
     flag ["ocaml"; "parser"; "menhir"; "only_tokens" ] (A"--only-tokens");
     flag ["ocaml"; "menhir_ocamldep"; "only_tokens" ] (A"--only-tokens");

     (* Menhir --external-tokens and --base flag *)
     flag ["ocaml"; "parser"; "menhir"; "ext_tokens" ]
(S[A"--external-tokens";
                                                          A"Tokens";
                                                          A"tokens.mly"; ]);
     flag ["ocaml"; "menhir_ocamldep"; "ext_tokens" ]
(S[A"--external-tokens";
                                                         A"Tokens";
                                                         A"tokens.mly"; ]);

     flag ["ocaml"; "parser"; "menhir"; "token_base" ] (S[A"--base";
                                                          A"parser"; ]);
     flag ["ocaml"; "menhir_ocamldep"; "token_base" ] (S[A"--base";
                                                         A"parser"; ]);

     dep ["ocaml"; "menhir_ocamldep"; "use_tokens"] ["fake.ml"];
     dep ["ocaml"; "parser"; "menhir"; "use_tokens"] ["fake.cmi"];

| _ -> ()
end;;

And I use a target file:
calc.d.byte

and invoke ocamlbuild using a shell script with:
ocamlbuild -verbose 100 all.otarget

The build process always ends with:

+ /home/hugof/ocaml/godi/bin/ocamlc.opt -c -g -o calc.cmo calc.ml
File "calc.ml", line 35, characters 2-13:
Unbound module Parser.Make
Command exited with code 2.

And no parser dependency file is generated.
In fact in this case no parser.mly file is even compiled.

I have tried quite a few variations, but none seem to work.
I addition to this I have noticed that Ocambuild repeats the
use of flags that may cause errors. In the line below is an example
where flag "--only-tokens" is repeated 3 times. Is this correct?

# Target: tokens.mly.depends, tags: { debug, extension:mly,
file:tokens.mly, menhir_ocamldep, ocaml, ocamldep, only_tokens, profile,
quiet, traverse, use_menhir }
/home/hugof/ocaml/godi/bin/menhir --only-tokens --raw-depend --ocamldep
'/home/hugof/ocaml/godi/bin/ocamldep.opt --only-tokens -pp --only-tokens
-modules' tokens.mly > tokens.mly.depends

Can any kind soul give me a hand with this seemingly trivial exercise?

TIA,
Hugo Ferreira.


[1]
http://caml.inria.fr/pub/ml-archives/caml-list/2007/04/d3b9765dd69a010219144200d925ea39.en.html
[2] http://cristal.inria.fr/~fpottier/menhir/menhir-20071212.tar.gz

[-- Attachment #2: all.itarget --]
[-- Type: text/plain, Size: 13 bytes --]

calc.d.byte

[-- Attachment #3: build_all.sh --]
[-- Type: application/x-shellscript, Size: 74 bytes --]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: myocamlbuild.ml --]
[-- Type: text/x-ocaml; name="myocamlbuild.ml", Size: 1337 bytes --]

open Ocamlbuild_plugin;;
open Command;;

dispatch begin function
| After_rules ->

    (* Menhir --explain flag *)
    flag ["ocaml"; "parser" ; "menhir" ; "use_menhir" ]
    (S[A"--explain"]);

    (* Menhir --only-tokens flag *) 
    flag ["ocaml"; "parser"; "menhir"; "only_tokens" ] (A"--only-tokens");
    flag ["ocaml"; "menhir_ocamldep"; "only_tokens" ] (A"--only-tokens");

    (* Menhir --external-tokens and --base flag *)
    flag ["ocaml"; "parser"; "menhir"; "ext_tokens" ] (S[A"--external-tokens";
                                                         A"Tokens";
                                                         A"tokens.mly"; ]);
    flag ["ocaml"; "menhir_ocamldep"; "ext_tokens" ] (S[A"--external-tokens";
                                                        A"Tokens";
                                                        A"tokens.mly"; ]);

    flag ["ocaml"; "parser"; "menhir"; "token_base" ] (S[A"--base";
                                                         A"parser"; ]);
    flag ["ocaml"; "menhir_ocamldep"; "token_base" ] (S[A"--base";
                                                        A"parser"; ]);

    dep ["ocaml"; "menhir_ocamldep"; "use_tokens"] ["fake.ml"];
    dep ["ocaml"; "parser"; "menhir"; "use_tokens"] ["fake.cmi"];

| _ -> ()
end;;


[-- Attachment #5: _tags --]
[-- Type: text/plain, Size: 113 bytes --]

true: use_menhir, debug, profile
"tokens.mly": only_tokens
"parser.mly": token_base, ext_tokens, use_tokens