Generating comments from camlp4

Generating comments from camlp4

[Alexey Rodriguez]

Dear list,

Is there a way to generate comments from camlp4 code?

We have preprocessors that generate the following kind of code in signatures:

>  <:sig_item<
>    $sig_generator the_type_declaration$;
>  >>

However, these extra function signatures show up in the output of
ocamldoc which is very annoying. We thought of using "ocamldoc stop"
tags (** / **), but we do not know how to generate such comments from
camlp4. Do you know of a way to do this:

>  <:sig_item<
>    (** / **)
>    $sig_generator the_type_declaration$;
>    (** / **)
>  >>

Comments are not part of the AST so I suppose we have to do some
hacking at the lexing level.

Note that having a dummy preprocessor that generates empty signatures
is not an option: this causes type checking errors that require a lot
of effort to circumvent.

Cheers,

Alexey

Re: Generating comments from camlp4

[Alexey Rodriguez]

Just an update on the situation.

We are using Jane Street's excellent type_conv library to generate
code for user-defined types when preprocessing. However, these
generated functions appear in ocamldoc documentation and we would like
to avoid that. The easiest solution seemed to be to bracket the
generated code with stop tags: (** / **). In camlp4 comments are
tokens but I do not quite see how to generate tokens from camlp4
quotations. So at the moment I have the impression that I cannot solve
this problem at the camlp4 level.

Cheers,

Alexey

On Fri, Sep 11, 2009 at 3:59 PM, Alexey Rodriguez <mrchebas@gmail.com> wrote:
> Dear list,
>
> Is there a way to generate comments from camlp4 code?
>
> We have preprocessors that generate the following kind of code in signatures:
>
>>  <:sig_item<
>>    $sig_generator the_type_declaration$;
>>  >>
>
> However, these extra function signatures show up in the output of
> ocamldoc which is very annoying. We thought of using "ocamldoc stop"
> tags (** / **), but we do not know how to generate such comments from
> camlp4. Do you know of a way to do this:
>
>>  <:sig_item<
>>    (** / **)
>>    $sig_generator the_type_declaration$;
>>    (** / **)
>>  >>
>
> Comments are not part of the AST so I suppose we have to do some
> hacking at the lexing level.
>
> Note that having a dummy preprocessor that generates empty signatures
> is not an option: this causes type checking errors that require a lot
> of effort to circumvent.
>
> Cheers,
>
> Alexey
>

Re: [Caml-list] Re: Generating comments from camlp4

[Anil Madhavapeddy]

I dont believe you can add comments with camlp4; I'm using type-conv  
to hack on an ORM layer, and it would be lovely to have the generated  
functions also have comments to not dirty up the OCamldoc.  And also,  
to maintain sanity when reading through pages of generated OCaml...

-anil

On 15 Sep 2009, at 13:26, Alexey Rodriguez wrote:

> Just an update on the situation.
>
> We are using Jane Street's excellent type_conv library to generate
> code for user-defined types when preprocessing. However, these
> generated functions appear in ocamldoc documentation and we would like
> to avoid that. The easiest solution seemed to be to bracket the
> generated code with stop tags: (** / **). In camlp4 comments are
> tokens but I do not quite see how to generate tokens from camlp4
> quotations. So at the moment I have the impression that I cannot solve
> this problem at the camlp4 level.
>
> Cheers,
>
> Alexey
>
> On Fri, Sep 11, 2009 at 3:59 PM, Alexey Rodriguez  
> <mrchebas@gmail.com> wrote:
>> Dear list,
>>
>> Is there a way to generate comments from camlp4 code?
>>
>> We have preprocessors that generate the following kind of code in  
>> signatures:
>>
>>>  <:sig_item<
>>>    $sig_generator the_type_declaration$;
>>>  >>
>>
>> However, these extra function signatures show up in the output of
>> ocamldoc which is very annoying. We thought of using "ocamldoc stop"
>> tags (** / **), but we do not know how to generate such comments from
>> camlp4. Do you know of a way to do this:
>>
>>>  <:sig_item<
>>>    (** / **)
>>>    $sig_generator the_type_declaration$;
>>>    (** / **)
>>>  >>
>>
>> Comments are not part of the AST so I suppose we have to do some
>> hacking at the lexing level.
>>
>> Note that having a dummy preprocessor that generates empty signatures
>> is not an option: this causes type checking errors that require a lot
>> of effort to circumvent.
>>
>> Cheers,
>>
>> Alexey
>>
>
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] Re: Generating comments from camlp4

[Jake Donham]

On Tue, Sep 15, 2009 at 5:42 AM, Anil Madhavapeddy <anil@recoil.org> wrote:
> I dont believe you can add comments with camlp4; I'm using type-conv to hack
> on an ORM layer, and it would be lovely to have the generated functions also
> have comments to not dirty up the OCamldoc.  And also, to maintain sanity
> when reading through pages of generated OCaml...

Comments are not passed through the AST, but somehow they make it from
the input file to the output file. It looks like in
Camlp4/Printers/OCaml.ml there is a hook into the tokenizer to keep
track of comments as they are tokenized (search for CommentFilter).
Maybe there is some way to inject comments (at particular locations)?
Possibly by calling Gram.lex_string and/or Gram.filter? A hack, no
doubt.

Jake

Re: [Caml-list] Re: Generating comments from camlp4

[Alexey Rodriguez]

Thanks for the tip Jake. I arrived to a similar conclusion by doing
experiments on source files with comments but I did not pinpoint the
exact location in camlp4's sources. As you say, a solution that
injects comments would probably look very hacky. I decided to go for
Tiphaine's suggestion. After 45 minutes of hacking on the generator, I
had decent looking ocamldoc. Thanks Tiphaine!

Cheers,

Alexey

On Wed, Sep 16, 2009 at 3:03 AM, Jake Donham <jake@donham.org> wrote:
> On Tue, Sep 15, 2009 at 5:42 AM, Anil Madhavapeddy <anil@recoil.org> wrote:
>> I dont believe you can add comments with camlp4; I'm using type-conv to hack
>> on an ORM layer, and it would be lovely to have the generated functions also
>> have comments to not dirty up the OCamldoc.  And also, to maintain sanity
>> when reading through pages of generated OCaml...
>
> Comments are not passed through the AST, but somehow they make it from
> the input file to the output file. It looks like in
> Camlp4/Printers/OCaml.ml there is a hook into the tokenizer to keep
> track of comments as they are tokenized (search for CommentFilter).
> Maybe there is some way to inject comments (at particular locations)?
> Possibly by calling Gram.lex_string and/or Gram.filter? A hack, no
> doubt.
>
> Jake
>

Re: [Caml-list] Re: Generating comments from camlp4

[Tiphaine Turpin]

Well, I think that this is mainly possible thanks to Maxence Guesdon's
work on ocamldoc...

Tiphaine

Alexey Rodriguez a écrit :
> I decided to go for
> Tiphaine's suggestion. After 45 minutes of hacking on the generator, I
> had decent looking ocamldoc. Thanks Tiphaine!
>
> Cheers,
>   

Re: [Caml-list] Re: Generating comments from camlp4

[Tiphaine Turpin]

I just remembered that I once faced a similar problem. I solved it
rather easily using a custom documentation generator wich justs
redefines a few methods to discard the appropriate elements (those
starting with a n underscore in my case). Here is the code :

open Odoc_info
open Value

class my_doc_gen = object (this)
  inherit Odoc_html.html as super

  method synthetic name =
    (Name.simple name).[0] = '_'

  method html_of_attribute b a =
    if not (this#synthetic a.att_value.val_name)
    then super#html_of_attribute b a

  method html_of_method b a =
    if not (this#synthetic a.met_value.val_name)
    then super#html_of_method b a

end

let my_generator = new my_doc_gen
let _ = Odoc_args.set_doc_generator (Some (my_generator :>
Odoc_args.doc_generator))


Alexey Rodriguez a écrit :
> Just an update on the situation.
>
> We are using Jane Street's excellent type_conv library to generate
> code for user-defined types when preprocessing. However, these
> generated functions appear in ocamldoc documentation and we would like
> to avoid that. The easiest solution seemed to be to bracket the
> generated code with stop tags: (** / **). In camlp4 comments are
> tokens but I do not quite see how to generate tokens from camlp4
> quotations. So at the moment I have the impression that I cannot solve
> this problem at the camlp4 level.
>
> Cheers,
>
> Alexey
>
> On Fri, Sep 11, 2009 at 3:59 PM, Alexey Rodriguez <mrchebas@gmail.com> wrote:
>   
>> Dear list,
>>
>> Is there a way to generate comments from camlp4 code?
>>
>> We have preprocessors that generate the following kind of code in signatures:
>>
>>     
>>>  <:sig_item<
>>>    $sig_generator the_type_declaration$;
>>>  >>
>>>       
>> However, these extra function signatures show up in the output of
>> ocamldoc which is very annoying. We thought of using "ocamldoc stop"
>> tags (** / **), but we do not know how to generate such comments from
>> camlp4. Do you know of a way to do this:
>>
>>     
>>>  <:sig_item<
>>>    (** / **)
>>>    $sig_generator the_type_declaration$;
>>>    (** / **)
>>>  >>
>>>       
>> Comments are not part of the AST so I suppose we have to do some
>> hacking at the lexing level.
>>
>> Note that having a dummy preprocessor that generates empty signatures
>> is not an option: this causes type checking errors that require a lot
>> of effort to circumvent.
>>
>> Cheers,
>>
>> Alexey
>>
>>     
>
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>
>