Chain Camlp4 syntax extensions?

Chain Camlp4 syntax extensions?

[Dmitry Bely]

I have two similar syntax extension that generate some functions for
each type in the module:

EXTEND Gram
  str_item:
    [ [ "type"; tdl = LIST1 type_declaration SEP "and" ->
        ...
END;

So I use them as usual:

ocamlc -c -pp "camlp4o pa_ext1.cmo" file.ml

or

ocamlc -c -pp "camlp4o pa_ext2.cmo" file.ml

Now I need to apply them both to the same source file in sequence
(without rewriting the extensions if possible). What is the right way
to do that? Of course, If  I write

ocamlc -c -pp "camlp4o pa_ext1.cmo pa_ext2.cmo" file.ml

only pa_ext2 extension is actually applied. Calling camlp4o twice with
an intermediate text representation works but is actually unacceptable
(the error location info is lost etc.). Any advice?

And another question. For camlp5 it was possible to inject some code
into the beginning of the generated file (the code is taken from IoXML
syntax extension):

value first = ref True;

Pcaml.parse_implem.val :=
  fun strm ->
    let (sil, stopped) = Grammar.Entry.parse Pcaml.implem strm in
    let sil =
      if first.val then
        let _ = do { first.val := False } in
        let _loc =
          Stdpp.dummy_loc
        in
        [(<:str_item< open IoXML >>, _loc) :: sil]
      else sil
    in
    (sil, stopped)
;

How to do the same with camlp4?

- Dmitry Bely

Re: [Caml-list] Chain Camlp4 syntax extensions?

[Jake Donham]

On Tue, Sep 28, 2010 at 6:13 AM, Dmitry Bely <dmitry.bely@gmail.com> wrote:
> I have two similar syntax extension that generate some functions for
> each type in the module:
>
> EXTEND Gram
>  str_item:
>    [ [ "type"; tdl = LIST1 type_declaration SEP "and" ->
>        ...
> END;

There is not a good solution without modifying the extensions; looking
at the signature of the Grammar module, you can modify a grammar but
not look up or call existing rules / actions.

This problem of combining extensions for type-derived functions is the
motivation for the type-conv framework:

  http://www.ocaml.info/home/ocaml_sources.html#toc9

It's trivial to convert an extension to use type-conv.

> And another question. For camlp5 it was possible to inject some code
> into the beginning of the generated file

It should be doable with a filter and a bool ref---inject the code
into the first str_item then set the flag so you don't add it to the
others.

Jake

Re: [Caml-list] Chain Camlp4 syntax extensions?

[Dmitry Bely]

On Tue, Sep 28, 2010 at 8:07 PM, Jake Donham <jake@donham.org> wrote:
> On Tue, Sep 28, 2010 at 6:13 AM, Dmitry Bely <dmitry.bely@gmail.com> wrote:
>> I have two similar syntax extension that generate some functions for
>> each type in the module:
>>
>> EXTEND Gram
>>  str_item:
>>    [ [ "type"; tdl = LIST1 type_declaration SEP "and" ->
>>        ...
>> END;
>
> There is not a good solution without modifying the extensions; looking
> at the signature of the Grammar module, you can modify a grammar but
> not look up or call existing rules / actions.
>
> This problem of combining extensions for type-derived functions is the
> motivation for the type-conv framework:
>
>  http://www.ocaml.info/home/ocaml_sources.html#toc9
>
> It's trivial to convert an extension to use type-conv.

Thanks. But I have not found any docs for type-conv package. Is there
any example demonstrating how to make Camlp4 grammar compatible with
type-conv framework?

>> And another question. For camlp5 it was possible to inject some code
>> into the beginning of the generated file
>
> It should be doable with a filter and a bool ref---inject the code
> into the first str_item then set the flag so you don't add it to the
> others.

Thanks again. Indeed this is the way to go.

- Dmitry Bely

Re: [Caml-list] Chain Camlp4 syntax extensions?

[Hendrik Tews]

Dmitry Bely <dmitry.bely@gmail.com> writes:

   I have two similar syntax extension that generate some functions for
   each type in the module:

   EXTEND Gram
     str_item:
       [ [ "type"; tdl = LIST1 type_declaration SEP "and" ->
           ...
   END;

I would suggest to have an additional module TypeDeclMod where
you can register functions to modify type declarations. This
module would also change the str_item parser to call all
registered functions on tdl. 

Your two modules pa_ext1 and pa_ext2 would then only register one
function with TypeDeclMod (without changing the grammar).

This is probably more or less what type-conv does.


For a type-conv example you can look at sexplib.


   And another question. For camlp5 it was possible to inject some code
   into the beginning of the generated file (the code is taken from IoXML
   syntax extension):

You can use Camlp4.Register.register_str_item_parser, to my
surprise it turns out that this requires actually less effort
than in Camlp5. The only problem is that
Register.register_str_item_parser changes a reference cell and
there is no way to get its contents. So the one who registers
last wins. The default value of that reference is
Camlp4.PreCast.Syntax.parse_implem, which gets implicitely
changed by all syntax extensions (because they modify
PreCast.Syntax). Here is an example:

==============================================================================
open Camlp4.PreCast

(* Extend or modify the ast of an implementation.
 *
 * val extend_ast : Ast.str_item -> Ast.str_item
 *)
let extend_ast str_item =
  let _loc = Loc.ghost in
  <:str_item< open Bla ; $str_item$ >>


(* Implementation parser to be used instead of the default one. To 
 * be registered with Register.register_str_item_parser.
 * 
 * my_parse_implem dir_handler start_loc char_stream  parses char_stream 
 * with the default parser from the PreCast.Syntax module. Argument 
 * start_loc is the initial location obtained for example by Loc.mk.
 * Argument dir_handler is called whenever a top-level directive is 
 * encountered in the char_stream.
 * 
 * val my_parse_implem :
 *   ?directive_handler:(Syntax.Ast.str_item -> Syntax.Ast.str_item option) ->
 *   Syntax.Ast.loc -> char Stream.t -> Ast.str_item
 *)
let my_parse_implem ?(directive_handler = fun _ -> None) _loc cs =
  let str_item = Syntax.parse_implem ~directive_handler _loc cs in
  extend_ast str_item


(* Register my_parse_implem as default implementation parser. This
 * will implicitely change Register.CurrentParser.parse_implem, which
 * is called by camlp4 to parse an implementation.
 * 
 * Note that there is currently (as of 3.12.0) no way to retrieve the
 * current contents of the reference cell that is changed by
 * Register.register_str_item_parser. However, currently no standard camlp4 
 * parser calls Register.register_str_item_parser. So our approach 
 * to register my_parse_implem which in turn calls Syntax.parse_implem 
 * works as long we are the only parsing extension which does that. 
 * If there are more parsing extensions that want to modify the ast 
 * after parsing they need to synchronize in some way.
 *)
let _ = Camlp4.Register.register_str_item_parser my_parse_implem
==============================================================================


Compile with ocamlc -I +camlp4 -c -pp camlp4orf example.ml
Use with ocamlc -pp 'camlp4o example.cmo'

Bye,

Hendrik