camlp4: question about functor-style syntax extensions

camlp4: question about functor-style syntax extensions

[Bruno De Fraine]

Hello,

Here's a minimal syntax extension in the functor-style, which seems  
to be preferred style with the new camlp4. It defines a PI constant  
which can be used inside expressions:

   module Id = struct let name = "pi" and version = "3.14" end

   open Camlp4.Sig

   module Pi(Syntax : Camlp4Syntax) = struct
     include Syntax
     EXTEND Gram
       expr: LEVEL "simple"
       [[ "PI" -> <:expr< $flo:"3.14159265358979312"$ >> ]];
     END
   end

   let module M = Camlp4.Register.OCamlSyntaxExtension(Id)(Pi) in ()

As mentioned in the documentation, an extension (such as Pi) must be  
functor: Camlp4Syntax -> Camlp4Syntax, and  
Register.OCamlSyntaxExtension requires this type. This is why I have  
to "include" (instead of "open") the original syntax: to produce a  
valid output syntax. My question is: how is this output syntax ever  
used? (Note that the EXTEND-statement does not make any structural  
changes to the syntax module, just dynamic changes as a side-effect  
upon functor application.)

To put all my cards on the table: I believe the output syntax is  
never used. For example, you can sabotage one of the main grammar  
entries by finishing the definition of Pi with:

   let top_phrase : Ast.str_item option Gram.Entry.t = Obj.magic 0

And the extension keeps working all the same from the toplevel. In  
fact, a look at the code of OCamlSyntaxExtension in Register.ml  
confirms it is never used:

   module OCamlSyntaxExtension
     (Id : Sig.Id) (Maker : functor (Syn : Sig.Camlp4Syntax) ->  
Sig.Camlp4Syntax) =
   struct
     declare_dyn_module Id.name (fun _ -> let module M = Maker Syntax  
in ());
   end;

The output syntax M is thrown away, i.e. the syntax extension relies  
entirely on side-effects of the functor application. I think the type  
required for a syntax extension could just as well have been a  
functor that return an empty module: Camlp4Syntax -> sig end

Why bother making this remark if you can just include the original  
syntax at the beginning and it works? I believe there are two  
important reasons. The first is didactical: the signature  
Camlp4Syntax -> Camlp4Syntax suggests that the syntax extension works  
by structurally transforming one syntax into another, while this is  
not what is going on. This situation makes the workings of camlp4 all  
the more difficult to understand for novice (and perhaps seasoned)  
camlp4 developers. The second reason is practical: you can easily  
define something in your extension that clashes with a name from  
Camlp4Syntax (e.g. "expr"), and then the compiler will complain if  
the types do not agree. You can assure you export the exact original  
definitions by putting "include Syntax" at the end of the extension  
instead of the beginning, but then you still need an "open Syntax" at  
the beginning to have the useful modules (like Ast, Gram, etc.)  
available. All of this is an annoying redundant idiom given that the  
output Syntax is not used.

Regards,
Bruno


--
Bruno De Fraine
Vrije Universiteit Brussel
Faculty of Applied Sciences, DINF - SSEL
Room 4K208, Pleinlaan 2, B-1050 Brussels
tel: +32 (0)2 629 29 75
fax: +32 (0)2 629 28 70
e-mail: Bruno.De.Fraine@vub.ac.be

Re: [Caml-list] camlp4: question about functor-style syntax extensions

[Christian Stork]

On Tue, Jul 17, 2007 at 11:08:09AM +0200, Bruno De Fraine wrote:
...
> As mentioned in the documentation, an extension (such as Pi) must be  
> functor: Camlp4Syntax -> Camlp4Syntax, and  
> Register.OCamlSyntaxExtension requires this type. This is why I have  
> to "include" (instead of "open") the original syntax: to produce a  
> valid output syntax. My question is: how is this output syntax ever  
> used? (Note that the EXTEND-statement does not make any structural  
> changes to the syntax module, just dynamic changes as a side-effect  
> upon functor application.)
> 
> To put all my cards on the table: I believe the output syntax is  
> never used. For example, you can sabotage one of the main grammar  
> entries by finishing the definition of Pi with:
> 
>   let top_phrase : Ast.str_item option Gram.Entry.t = Obj.magic 0
> 
> And the extension keeps working all the same from the toplevel. In  
> fact, a look at the code of OCamlSyntaxExtension in Register.ml  
> confirms it is never used:
> 
>   module OCamlSyntaxExtension
>     (Id : Sig.Id) (Maker : functor (Syn : Sig.Camlp4Syntax) ->  
> Sig.Camlp4Syntax) =
>   struct
>     declare_dyn_module Id.name (fun _ -> let module M = Maker Syntax  
> in ());
>   end;
> 
> The output syntax M is thrown away, i.e. the syntax extension relies  
> entirely on side-effects of the functor application. I think the type  
> required for a syntax extension could just as well have been a  
> functor that return an empty module: Camlp4Syntax -> sig end
> 
> Why bother making this remark if you can just include the original  
> syntax at the beginning and it works? I believe there are two  
> important reasons. The first is didactical: the signature  
> Camlp4Syntax -> Camlp4Syntax suggests that the syntax extension works  
> by structurally transforming one syntax into another, while this is  
> not what is going on. This situation makes the workings of camlp4 all  
> the more difficult to understand for novice (and perhaps seasoned)  
> camlp4 developers. The second reason is practical: you can easily  
> define something in your extension that clashes with a name from  
> Camlp4Syntax (e.g. "expr"), and then the compiler will complain if  
> the types do not agree. You can assure you export the exact original  
> definitions by putting "include Syntax" at the end of the extension  
> instead of the beginning, but then you still need an "open Syntax" at  
> the beginning to have the useful modules (like Ast, Gram, etc.)  
> available. All of this is an annoying redundant idiom given that the  
> output Syntax is not used.

Bruno, I had exactly the same thoughts when I originally studied the new
camlp4 sources and I definitely think that these observations should be
part of the camlp4 documentation.  Maybe you could update the Camlp4
Wiki with it? :-)  I intended to do so already but didn't find the time
yet.

Anyway, when I asked Nicolas about it he hinted at the possibility that
one day the current grammar update via module initialization side effect
would be replaced by "true" functor applications.  Would be interesting
to know how realistic such a possibility is.

-- 
Chris Stork   <>  Support eff.org!  <>   http://www.ics.uci.edu/~cstork/
OpenPGP fingerprint:  B08B 602C C806 C492 D069  021E 41F3 8C8D 50F9 CA2F

Re: [Caml-list] camlp4: question about functor-style syntax extensions

[Nicolas Pouillard]

In short...

As you've seen it the output of the functor is not used by the
Register module. However that's not the only possible usage of an
extension functor.

If you look at Camlp4Parsers/Camlp4OCamlOriginalQuotationExpander.ml

open PreCast;
let module Gram = MakeGram Lexer in
let module M1 = OCamlInitSyntax.Make Ast Gram Quotation in
let module M2 = Camlp4OCamlRevisedParser.Make M1 in
let module M3 = Camlp4OCamlParser.Make M2 in
let module M3 = Camlp4QuotationCommon.Make M3 Syntax.AntiquotSyntax in ();

Here syntax extensions are plugged together to make a new grammar. If
your extension is functorized you can extend that language easily.

However one can argue that a signature like <<Syntax -> sig end>> would suffice.

Indeed I wanted to make it more functional, but I failed it seems that
we need first class modules.

Finally it will be perhaps simpler to switch to a signature like
<<Syntax -> sig end>>

On 7/17/07, Bruno De Fraine <Bruno.De.Fraine@vub.ac.be> wrote:
> Hello,
>
> Here's a minimal syntax extension in the functor-style, which seems
> to be preferred style with the new camlp4. It defines a PI constant
> which can be used inside expressions:
>
>    module Id = struct let name = "pi" and version = "3.14" end
>
>    open Camlp4.Sig
>
>    module Pi(Syntax : Camlp4Syntax) = struct
>      include Syntax
>      EXTEND Gram
>        expr: LEVEL "simple"
>        [[ "PI" -> <:expr< $flo:"3.14159265358979312"$ >> ]];
>      END
>    end
>
>    let module M = Camlp4.Register.OCamlSyntaxExtension(Id)(Pi) in ()
>
> As mentioned in the documentation, an extension (such as Pi) must be
> functor: Camlp4Syntax -> Camlp4Syntax, and
> Register.OCamlSyntaxExtension requires this type. This is why I have
> to "include" (instead of "open") the original syntax: to produce a
> valid output syntax. My question is: how is this output syntax ever
> used? (Note that the EXTEND-statement does not make any structural
> changes to the syntax module, just dynamic changes as a side-effect
> upon functor application.)
>
> To put all my cards on the table: I believe the output syntax is
> never used. For example, you can sabotage one of the main grammar
> entries by finishing the definition of Pi with:
>
>    let top_phrase : Ast.str_item option Gram.Entry.t = Obj.magic 0
>
> And the extension keeps working all the same from the toplevel. In
> fact, a look at the code of OCamlSyntaxExtension in Register.ml
> confirms it is never used:
>
>    module OCamlSyntaxExtension
>      (Id : Sig.Id) (Maker : functor (Syn : Sig.Camlp4Syntax) ->
> Sig.Camlp4Syntax) =
>    struct
>      declare_dyn_module Id.name (fun _ -> let module M = Maker Syntax
> in ());
>    end;
>
> The output syntax M is thrown away, i.e. the syntax extension relies
> entirely on side-effects of the functor application. I think the type
> required for a syntax extension could just as well have been a
> functor that return an empty module: Camlp4Syntax -> sig end
>
> Why bother making this remark if you can just include the original
> syntax at the beginning and it works? I believe there are two
> important reasons. The first is didactical: the signature
> Camlp4Syntax -> Camlp4Syntax suggests that the syntax extension works
> by structurally transforming one syntax into another, while this is
> not what is going on. This situation makes the workings of camlp4 all
> the more difficult to understand for novice (and perhaps seasoned)
> camlp4 developers. The second reason is practical: you can easily
> define something in your extension that clashes with a name from
> Camlp4Syntax (e.g. "expr"), and then the compiler will complain if
> the types do not agree. You can assure you export the exact original
> definitions by putting "include Syntax" at the end of the extension
> instead of the beginning, but then you still need an "open Syntax" at
> the beginning to have the useful modules (like Ast, Gram, etc.)
> available. All of this is an annoying redundant idiom given that the
> output Syntax is not used.
>
> Regards,
> Bruno
>
>
> --
> Bruno De Fraine
> Vrije Universiteit Brussel
> Faculty of Applied Sciences, DINF - SSEL
> Room 4K208, Pleinlaan 2, B-1050 Brussels
> tel: +32 (0)2 629 29 75
> fax: +32 (0)2 629 28 70
> e-mail: Bruno.De.Fraine@vub.ac.be
>
>
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>


-- 
Nicolas Pouillard