Camlp4: example/parse_files.ml

Camlp4: example/parse_files.ml

[Hendrik Tews]

Hi,

I have a few questions to the parse_files example.

The example starts with building the parser and printer via the
following functor application

    module Caml =
      Camlp4.Printers.OCaml.Make
	(Camlp4OCamlParser.Make
	  (Camlp4OCamlRevisedParser.Make
	    (Camlp4.OCamlInitSyntax.Make(Ast)(Gram)(Quotation))));;


1. Can I omit the printer functor if I am only interested in
   parsing files (and not printing) and simplify to

    module Caml =
       Camlp4OCamlParser.Make
	(Camlp4OCamlRevisedParser.Make
	  (Camlp4.OCamlInitSyntax.Make(Ast)(Gram)(Quotation)))

2. Ast, Gram, Quotation come from Camlp4.PreCast. Do I miss
   something if I simplify further to

    module Caml =
       Camlp4OCamlParser.Make
	(Camlp4OCamlRevisedParser.Make(Camlp4.PreCast.Syntax))

3. I looked into the registration of syntax extentions (Register
   functors) and how the syntax extention callbacks are called
   in Register.iter_and_take_callbacks. I have the impression
   that in camlp4 not the above nested functor application is
   performed but the Make functors are sequentially applied to
   the same Syntax functor. Therefore, can I further simplify to 

    module Caml = Camlp4.PreCast.Syntax
    let () = 
      let module M1 = Camlp4OCamlRevisedParser.Make(Caml) in
      let module M2 = Camlp4OCamlParser.Make(Caml)
      in
	()

   ?

Bye,

Hendrik

Re: [Caml-list] Camlp4: example/parse_files.ml

[Nicolas Pouillard]

[-- Attachment #1: Type: text/plain, Size: 1762 bytes --]

Excerpts from Hendrik Tews's message of Thu Apr 03 01:13:41 +0200 2008:
> Hi,
> 
> I have a few questions to the parse_files example.
> 
> The example starts with building the parser and printer via the
> following functor application
> 
>     module Caml =
>       Camlp4.Printers.OCaml.Make
>     (Camlp4OCamlParser.Make
>       (Camlp4OCamlRevisedParser.Make
>         (Camlp4.OCamlInitSyntax.Make(Ast)(Gram)(Quotation))));;
> 
> 
> 1. Can I omit the printer functor if I am only interested in
>    parsing files (and not printing) and simplify to
> 
>     module Caml =
>        Camlp4OCamlParser.Make
>     (Camlp4OCamlRevisedParser.Make
>       (Camlp4.OCamlInitSyntax.Make(Ast)(Gram)(Quotation)))

Yes I think so.

> 2. Ast, Gram, Quotation come from Camlp4.PreCast.

Right.

> Do I miss
>    something if I simplify further to
> 
>     module Caml =
>        Camlp4OCamlParser.Make
>     (Camlp4OCamlRevisedParser.Make(Camlp4.PreCast.Syntax))

You  are  applying  original  and  revised parser, so if the current parser is
empty that's ok. Otherwise the extension can fail.

> 3. I looked into the registration of syntax extentions (Register
>    functors) and how the syntax extention callbacks are called
>    in Register.iter_and_take_callbacks. I have the impression
>    that in camlp4 not the above nested functor application is
>    performed but the Make functors are sequentially applied to
>    the same Syntax functor. Therefore, can I further simplify to 
> 
>     module Caml = Camlp4.PreCast.Syntax
>     let () = 
>       let module M1 = Camlp4OCamlRevisedParser.Make(Caml) in
>       let module M2 = Camlp4OCamlParser.Make(Caml)
>       in
>     ()

Right, assuming you're starting with an empty grammar.

-- 
Nicolas Pouillard aka Ertai

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 194 bytes --]

Re: [Caml-list] Camlp4: example/parse_files.ml

[Hendrik Tews]

"Nicolas Pouillard" <nicolas.pouillard@gmail.com> writes:

   > Do I miss
   >    something if I simplify further to
   > 
   >     module Caml =
   >        Camlp4OCamlParser.Make
   >     (Camlp4OCamlRevisedParser.Make(Camlp4.PreCast.Syntax))

   You are applying original and revised parser, so if the
   current parser is empty that's ok. Otherwise the extension can
   fail.

Right, I missed that: With this simplification I am not starting
with a fresh and empty grammar but with whatever has accumulated
in PreCast.Syntax.

I have a followup question. With the functorial interface I can
build and use several grammars at the same time. For instance

    module FreshGrammar(Unit : sig end) =
      Camlp4.OCamlInitSyntax.Make
        (Camlp4.PreCast.Ast)(Camlp4.PreCast.Gram)(Camlp4.PreCast.Quotation)

    module G1 = FreshGrammar(struct end)
    module M1 = Camlp4OCamlRevisedParser.Make(G1)
    module M2 = Camlp4OCamlParser.Make(G1)

    module G2 = FreshGrammar(struct end)
    module M3 = Camlp4OCamlRevisedParser.Make(G2)
    module M4 = Camlp4MacroParser.Make(G2)  

Now G1.parse_implem parses Ocaml syntax, while G2.parse_implem
parses revised syntax with macros.

Can I do the same with a syntax extention that is only loaded at
runtime via Dynlink? Assume a syntax extention X that is only
available at runtime and not at compiletime. After loading X with
Dynlink, all I can do is invoke the callback of X, but this will
add the grammar extension of X to Camlp4.PreCast.Syntax. 

Is there a way to obtain two functions parse_implem (as above)
that mix X with different syntaxes (eg Ocaml + X and 
Revised + X)? For that, I believe, it would be necessary to
extract parse_implem from a syntax module such that it survives
clearing that Syntax.

Bye,

Hendrik

Re: [Caml-list] Camlp4: example/parse_files.ml

[Nicolas Pouillard]

[-- Attachment #1: Type: text/plain, Size: 2228 bytes --]

Excerpts from Hendrik Tews's message of Thu Apr 03 23:24:17 +0200 2008:
> "Nicolas Pouillard" <nicolas.pouillard@gmail.com> writes:
> 
>    > Do I miss
>    >    something if I simplify further to
>    > 
>    >     module Caml =
>    >        Camlp4OCamlParser.Make
>    >     (Camlp4OCamlRevisedParser.Make(Camlp4.PreCast.Syntax))
> 
>    You are applying original and revised parser, so if the
>    current parser is empty that's ok. Otherwise the extension can
>    fail.
> 
> Right, I missed that: With this simplification I am not starting
> with a fresh and empty grammar but with whatever has accumulated
> in PreCast.Syntax.

Yes.

> 
> I have a followup question. With the functorial interface I can
> build and use several grammars at the same time. For instance

Yes.

>     module FreshGrammar(Unit : sig end) =
>       Camlp4.OCamlInitSyntax.Make
>         (Camlp4.PreCast.Ast)(Camlp4.PreCast.Gram)(Camlp4.PreCast.Quotation)
> 
>     module G1 = FreshGrammar(struct end)
>     module M1 = Camlp4OCamlRevisedParser.Make(G1)
>     module M2 = Camlp4OCamlParser.Make(G1)
> 
>     module G2 = FreshGrammar(struct end)
>     module M3 = Camlp4OCamlRevisedParser.Make(G2)
>     module M4 = Camlp4MacroParser.Make(G2)  
> 
> Now G1.parse_implem parses Ocaml syntax, while G2.parse_implem
> parses revised syntax with macros.

Yes.

> Can I do the same with a syntax extention that is only loaded at
> runtime via Dynlink? Assume a syntax extention X that is only
> available at runtime and not at compiletime. After loading X with
> Dynlink, all I can do is invoke the callback of X, but this will
> add the grammar extension of X to Camlp4.PreCast.Syntax. 


You  can  do  it  but  you  will to resort to some kind of registering of your
extensions, as it's done in camlp4 for the default grammar.

> Is there a way to obtain two functions parse_implem (as above)
> that mix X with different syntaxes (eg Ocaml + X and 
> Revised + X)? For that, I believe, it would be necessary to
> extract parse_implem from a syntax module such that it survives
> clearing that Syntax.

Yes   but   your   extension  X  has  to  be  a  functor,  then  you  can  get
X(OCamlOriginal), X(OCamlRevised).

-- 
Nicolas Pouillard aka Ertai

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 194 bytes --]

Re: [Caml-list] Camlp4: example/parse_files.ml

[Hendrik Tews]

"Nicolas Pouillard" <nicolas.pouillard@gmail.com> writes:

   > Can I do the same with a syntax extention that is only
   > loaded at runtime via Dynlink? Assume a syntax extention X
   > that is only available at runtime and not at compiletime.
   > After loading X with Dynlink, all I can do is invoke the
   > callback of X, but this will add the grammar extension of X
   > to Camlp4.PreCast.Syntax.

   You can do it but you will to resort to some kind of
   registering of your extensions, as it's done in camlp4 for the
   default grammar.

   > Is there a way to obtain two functions parse_implem (as
   > above) that mix X with different syntaxes (eg Ocaml + X and
   > Revised + X)? For that, I believe, it would be necessary to
   > extract parse_implem from a syntax module such that it
   > survives clearing that Syntax.

   Yes but your extension X has to be a functor, then you can get
   X(OCamlOriginal), X(OCamlRevised).

But this only works if I can statically link against X. There is
no way to access the functor X through the Dynlink interface.
With the standard interface of Camlp4 syntax extensions the
loaded module applies X only to PreCast.Syntax. This could be
repeated but one would have to clear PreCast.Syntax in between.

Changing the Camlp4 interface one could chieve that X is applied
to two or maybe three times (to PreCast.Syntax1 ... Syntax3). But
how about the general case, where X is applied to n Syntax
modules and n depends on the input? I believe this is impossible.

Bye,

Hendrik

Re: [Caml-list] Camlp4: example/parse_files.ml

[Nicolas Pouillard]

[-- Attachment #1: Type: text/plain, Size: 2051 bytes --]

Excerpts from Hendrik Tews's message of Fri Apr 04 16:01:57 +0200 2008:
> "Nicolas Pouillard" <nicolas.pouillard@gmail.com> writes:
> 
>    > Can I do the same with a syntax extention that is only
>    > loaded at runtime via Dynlink? Assume a syntax extention X
>    > that is only available at runtime and not at compiletime.
>    > After loading X with Dynlink, all I can do is invoke the
>    > callback of X, but this will add the grammar extension of X
>    > to Camlp4.PreCast.Syntax.
> 
>    You can do it but you will to resort to some kind of
>    registering of your extensions, as it's done in camlp4 for the
>    default grammar.
> 
>    > Is there a way to obtain two functions parse_implem (as
>    > above) that mix X with different syntaxes (eg Ocaml + X and
>    > Revised + X)? For that, I believe, it would be necessary to
>    > extract parse_implem from a syntax module such that it
>    > survives clearing that Syntax.
> 
>    Yes but your extension X has to be a functor, then you can get
>    X(OCamlOriginal), X(OCamlRevised).
> 
> But this only works if I can statically link against X. There is
> no way to access the functor X through the Dynlink interface.

Dynlink cannot access the X functor, but it can register itself.

> With the standard interface of Camlp4 syntax extensions the
> loaded module applies X only to PreCast.Syntax. This could be
> repeated but one would have to clear PreCast.Syntax in between.

You  can  either  reuse  the  current setup, by clearing the PreCast.Syntax in
between but also setup another registration functor.

> Changing the Camlp4 interface one could chieve that X is applied
> to two or maybe three times (to PreCast.Syntax1 ... Syntax3). But
> how about the general case, where X is applied to n Syntax
> modules and n depends on the input? I believe this is impossible.

I think that's doable (not tested), basically you need three primitive functors:

Register : (Syntax -> Syntax) -> Unit
Clear    : Unit -> Unit
Apply    : Syntax -> Syntax

-- 
Nicolas Pouillard aka Ertai

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 194 bytes --]

Re: [Caml-list] Camlp4: example/parse_files.ml

[Hendrik Tews]

"Nicolas Pouillard" <nicolas.pouillard@gmail.com> writes:

   I think that's doable (not tested), basically you need three
   primitive functors:

I don't see how. Let me clarify again what I mean. And let me
clarify that I only pursue this discussion out of curiosity, to
see, what can be done with the functorial Camlp4 interface. 

Assume we have a table like this:

    s1.cmo s2.cmo ... : f1.ml f2.ml ...
    s3.cmo s4.cmo ... : f3.ml f4.ml ...

The sx.cmo files are Camlp4 syntax extentions and the fx.ml are
files to be parsed using those syntax extentions. There are a
number of possible solutions:
- starting camlp4 for each file or each row
- clearing PreCast.Syntax before each row and then calling the
  callbacks that the syntax extensions registered

But I would like to know if the functorial Camlp4 design is
strong enough to solve this exercise within one program (without
fork/exec) and without reusing/clearing one syntax module.

It is clear that the registration in Register.SyntaxExtension is
not good enough, because it always applies the syntax extension
to PreCast.Syntax.

However, even with changing Register.SyntaxExtension or how
syntax extensions register themselves, I can't see a solution.

Bye,

Hendrik

Re: [Caml-list] Camlp4: example/parse_files.ml

[Nicolas Pouillard]

[-- Attachment #1: Type: text/plain, Size: 1760 bytes --]

Excerpts from Hendrik Tews's message of Fri Apr 04 23:30:58 +0200 2008:
> "Nicolas Pouillard" <nicolas.pouillard@gmail.com> writes:
> 
>    I think that's doable (not tested), basically you need three
>    primitive functors:
> 
> I don't see how. Let me clarify again what I mean. And let me
> clarify that I only pursue this discussion out of curiosity, to
> see, what can be done with the functorial Camlp4 interface. 
> 
> Assume we have a table like this:
> 
>     s1.cmo s2.cmo ... : f1.ml f2.ml ...
>     s3.cmo s4.cmo ... : f3.ml f4.ml ...
> 
> The sx.cmo files are Camlp4 syntax extentions and the fx.ml are
> files to be parsed using those syntax extentions. There are a
> number of possible solutions:
> - starting camlp4 for each file or each row
> - clearing PreCast.Syntax before each row and then calling the
>   callbacks that the syntax extensions registered
> 
> But I would like to know if the functorial Camlp4 design is
> strong enough to solve this exercise within one program (without
> fork/exec) and without reusing/clearing one syntax module.
> 
> It is clear that the registration in Register.SyntaxExtension is
> not good enough, because it always applies the syntax extension
> to PreCast.Syntax.
> 
> However, even with changing Register.SyntaxExtension or how
> syntax extensions register themselves, I can't see a solution.

That's  some  corner  case where first-class modules would help a lot, however
one  can  use some black magic under the hood to get the same effect. One need
to  store  extensions once and for all, in a map from names to "modules". Then
one  can  have  some function (functor) that takes a list modules names and an
initial module, and will apply these extensions to it.

-- 
Nicolas Pouillard aka Ertai

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 194 bytes --]

Re: [Caml-list] Camlp4: example/parse_files.ml

[Hendrik Tews]

"Nicolas Pouillard" <nicolas.pouillard@gmail.com> writes:

   That's some corner case where first-class modules would help a
   lot, however one can use some black magic under the hood to
   get the same effect. One need to store extensions once and for
   all, in a map from names to "modules". 

Sure, if you can store modules in an association list, you are
done. But how to store modules in an association list?

Bye,

Hendrik

Re: [Caml-list] Camlp4: example/parse_files.ml

[Nicolas Pouillard]

[-- Attachment #1: Type: text/plain, Size: 693 bytes --]

Excerpts from Hendrik Tews's message of Mon Apr 07 09:31:09 +0200 2008:
> "Nicolas Pouillard" <nicolas.pouillard@gmail.com> writes:
> 
>    That's some corner case where first-class modules would help a
>    lot, however one can use some black magic under the hood to
>    get the same effect. One need to store extensions once and for
>    all, in a map from names to "modules". 
> 
> Sure, if you can store modules in an association list, you are
> done. But how to store modules in an association list?

When  know  which  fields  makes  interest,  one can build a record with those
particular fields namely the two parsing functions and printing functions.

-- 
Nicolas Pouillard aka Ertai

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 194 bytes --]