[Caml-list] ppx_deriving question: deferring code generation?

[Caml-list] ppx_deriving question: deferring code generation?

[François Pottier]

Hello all,

I am currently in the process of writing a ppx_deriving plugin, called
"visitors". Overall, this has been a pleasant experience; a few hundred 
lines
of code have been sufficient to obtain nontrivial results.

In normal use, the user writes something like this:

   type foo =
     Bar | Baz
     [@@deriving visitors]

and some generated code is inserted just after the definition of the 
type foo.

However, I have reached a situation where the generated code cannot be 
placed
just after the type definition. That is, I need to allow user-written 
code to
appear after the type definition and before the generated code.

For instance, this user-written code could be a declaration of a global
variable "x", whose type is "foo ref", and which the generated code 
uses. The
declaration of "x" must appear after the definition of the type "foo", 
because
the type of "x" mentions "foo". And the declaration of "x" must appear 
before
the generated code, because the generated code (intentionally) refers to 
"x".

I am imagining that perhaps the user could write something like this:

   type foo =
     Bar | Baz
     [@@deriving visitors { delayed = true }

   let x : foo option ref =
     ref None

   [@@visitors]

The effect of the flag { delayed = true } would be to store the 
generated code
somewhere in memory (instead of emitting right away), and the effect of the
floating attribute [@@visitors] would be to fetch that code from memory and
emit it.

To me, this seems somewhat ugly, but workable. Does ppx_deriving offer a
better approach? Does anyone have a better suggestion? Comments are
appreciated.

Best regards,

--
François Pottier
francois.pottier@inria.fr
http://gallium.inria.fr/~fpottier/

Re: [Caml-list] ppx_deriving question: deferring code generation?

[Gabriel Scherer]

[-- Attachment #1: Type: text/plain, Size: 3201 bytes --]

The semantics you propose is inherently stateful: delays accumulates some
state and [@@visitors]'s meaning (nitpick: I think it should be a floating
attribute, [@@@visitors]) depends on the current state. You could design a
similar facility using names for references instead of implicit state:

  type foo =
    Bar | Baz

  let x : foo option ref =
    ref None

  [@@@deriving.for foo (visitors)]

(If we had access to the type-checking environment, [@@deriving.for p]
could be valid for any qualified identifier p that points to a transparent
definition in the current environment. Given the current ppx pipeline, I
suppose that would have to be restricted to being in the syntactic scope of
an actual declaration.)

Hongbo Zhang introduced a similar "deriving from a distance" feature in his
preprocessor Fan, for the reason you give, and also to allow deriving
boilerplate code of datatypes defined in third-party libraries without
having to modify them directly.


On Wed, Jan 4, 2017 at 8:08 AM, François Pottier <francois.pottier@inria.fr>
wrote:

>
> Hello all,
>
> I am currently in the process of writing a ppx_deriving plugin, called
> "visitors". Overall, this has been a pleasant experience; a few hundred
> lines
> of code have been sufficient to obtain nontrivial results.
>
> In normal use, the user writes something like this:
>
>   type foo =
>     Bar | Baz
>     [@@deriving visitors]
>
> and some generated code is inserted just after the definition of the type
> foo.
>
> However, I have reached a situation where the generated code cannot be
> placed
> just after the type definition. That is, I need to allow user-written code
> to
> appear after the type definition and before the generated code.
>
> For instance, this user-written code could be a declaration of a global
> variable "x", whose type is "foo ref", and which the generated code uses.
> The
> declaration of "x" must appear after the definition of the type "foo",
> because
> the type of "x" mentions "foo". And the declaration of "x" must appear
> before
> the generated code, because the generated code (intentionally) refers to
> "x".
>
> I am imagining that perhaps the user could write something like this:
>
>   type foo =
>     Bar | Baz
>     [@@deriving visitors { delayed = true }
>
>   let x : foo option ref =
>     ref None
>
>   [@@visitors]
>
> The effect of the flag { delayed = true } would be to store the generated
> code
> somewhere in memory (instead of emitting right away), and the effect of the
> floating attribute [@@visitors] would be to fetch that code from memory and
> emit it.
>
> To me, this seems somewhat ugly, but workable. Does ppx_deriving offer a
> better approach? Does anyone have a better suggestion? Comments are
> appreciated.
>
> Best regards,
>
> --
> François Pottier
> francois.pottier@inria.fr
> http://gallium.inria.fr/~fpottier/
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

[-- Attachment #2: Type: text/html, Size: 4301 bytes --]

Re: [Caml-list] ppx_deriving question: deferring code generation?

[François Pottier]

Hello Gabriel,
On 04/01/2017 14:49, Gabriel Scherer wrote:
> (If we had access to the type-checking environment,

Indeed, that is the question. As far as I understand, the whole ppx 
system today is purely syntactic, and there is no way of looking up
the definition of a type. Or is there?

-- 
François Pottier
francois.pottier@inria.fr
http://gallium.inria.fr/~fpottier/

Re: [Caml-list] ppx_deriving question: deferring code generation?

[Christoph Höger]

[-- Attachment #1.1: Type: text/plain, Size: 2484 bytes --]

Just out of curiosity, what does "vistors" do? If it resembles the usual
Visitor Pattern you might find my ppx_deriving_morphism useful:

https://github.com/choeger/ppx_deriving_morphism/

(either as a starting point or inspiration or as something to obsolete).

Am 04.01.2017 um 14:08 schrieb François Pottier:
> 
> Hello all,
> 
> I am currently in the process of writing a ppx_deriving plugin, called
> "visitors". Overall, this has been a pleasant experience; a few hundred
> lines
> of code have been sufficient to obtain nontrivial results.
> 
> In normal use, the user writes something like this:
> 
>   type foo =
>     Bar | Baz
>     [@@deriving visitors]
> 
> and some generated code is inserted just after the definition of the
> type foo.
> 
> However, I have reached a situation where the generated code cannot be
> placed
> just after the type definition. That is, I need to allow user-written
> code to
> appear after the type definition and before the generated code.
> 
> For instance, this user-written code could be a declaration of a global
> variable "x", whose type is "foo ref", and which the generated code
> uses. The
> declaration of "x" must appear after the definition of the type "foo",
> because
> the type of "x" mentions "foo". And the declaration of "x" must appear
> before
> the generated code, because the generated code (intentionally) refers to
> "x".
> 
> I am imagining that perhaps the user could write something like this:
> 
>   type foo =
>     Bar | Baz
>     [@@deriving visitors { delayed = true }
> 
>   let x : foo option ref =
>     ref None
> 
>   [@@visitors]
> 
> The effect of the flag { delayed = true } would be to store the
> generated code
> somewhere in memory (instead of emitting right away), and the effect of the
> floating attribute [@@visitors] would be to fetch that code from memory and
> emit it.
> 
> To me, this seems somewhat ugly, but workable. Does ppx_deriving offer a
> better approach? Does anyone have a better suggestion? Comments are
> appreciated.
> 
> Best regards,
> 
> -- 
> François Pottier
> francois.pottier@inria.fr
> http://gallium.inria.fr/~fpottier/
> 


-- 
Christoph Höger

Technische Universität Berlin
Fakultät IV - Elektrotechnik und Informatik
Übersetzerbau und Programmiersprachen

Sekr. TEL12-2, Ernst-Reuter-Platz 7, 10587 Berlin

Tel.: +49 (30) 314-24890
E-Mail: christoph.hoeger@tu-berlin.de


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [Caml-list] ppx_deriving question: deferring code generation?

[François Pottier]

Dear Christoph,

On 04/01/2017 14:53, Christoph Höger wrote:
> Just out of curiosity, what does "vistors" do? If it resembles the usual
> Visitor Pattern you might find my ppx_deriving_morphism useful:

Interesting -- thanks for your reply. Although I did search for related ppx
plugins, I did not find yours.

At a glance, my plugin seems broadly analogous to yours; it also generates
"iter" (= "fold") and "map" visitors. It differs superficially in that 
it uses
OCaml objects instead of OCaml records. It differs perhaps more deeply 
in that
the generated code can have more varied types, beyond your "map_routine" and
"fold_routine". But I need to experiment more, and write more documentation,
before making a public release.

--
François Pottier
francois.pottier@inria.fr
http://gallium.inria.fr/~fpottier/

Re: [Caml-list] ppx_deriving question: deferring code generation?

[Alain Frisch]

Hello François,

I don't know if this would cover all similar cases, and perhaps it is a 
bit verbose, but what about something like:

     include(struct
      type foo = Bar | Baz

      let x = ...
     end [@deriving visitors])

i.e. use an attribute on the module expression (and interpret it by 
appending more declarations to the structure).


Cheers,

Alain

On 04/01/2017 14:08, François Pottier wrote:
>
> Hello all,
>
> I am currently in the process of writing a ppx_deriving plugin, called
> "visitors". Overall, this has been a pleasant experience; a few hundred
> lines
> of code have been sufficient to obtain nontrivial results.
>
> In normal use, the user writes something like this:
>
>   type foo =
>     Bar | Baz
>     [@@deriving visitors]
>
> and some generated code is inserted just after the definition of the
> type foo.
>
> However, I have reached a situation where the generated code cannot be
> placed
> just after the type definition. That is, I need to allow user-written
> code to
> appear after the type definition and before the generated code.
>
> For instance, this user-written code could be a declaration of a global
> variable "x", whose type is "foo ref", and which the generated code
> uses. The
> declaration of "x" must appear after the definition of the type "foo",
> because
> the type of "x" mentions "foo". And the declaration of "x" must appear
> before
> the generated code, because the generated code (intentionally) refers to
> "x".
>
> I am imagining that perhaps the user could write something like this:
>
>   type foo =
>     Bar | Baz
>     [@@deriving visitors { delayed = true }
>
>   let x : foo option ref =
>     ref None
>
>   [@@visitors]
>
> The effect of the flag { delayed = true } would be to store the
> generated code
> somewhere in memory (instead of emitting right away), and the effect of the
> floating attribute [@@visitors] would be to fetch that code from memory and
> emit it.
>
> To me, this seems somewhat ugly, but workable. Does ppx_deriving offer a
> better approach? Does anyone have a better suggestion? Comments are
> appreciated.
>
> Best regards,
>
> --
> François Pottier
> francois.pottier@inria.fr
> http://gallium.inria.fr/~fpottier/
>

Re: [Caml-list] ppx_deriving question: deferring code generation?

[François Pottier]

Hello Alain,

On 04/01/2017 16:20, Alain Frisch wrote:
> I don't know if this would cover all similar cases, and perhaps it is a
> bit verbose, but what about something like:
>
>     include(struct
>      type foo = Bar | Baz
>
>      let x = ...
>     end [@deriving visitors])

This sounds potentially reasonable. However, I am not sure if ppx_deriving
allows this. If it doesn't, then I would have to package my code 
generator as
a ppx preprocessor, instead of a ppx_deriving plugin. Maybe that would be
easy; I am not sure exactly how much grunt work ppx_deriving is doing 
for me.

--
François Pottier
francois.pottier@inria.fr
http://gallium.inria.fr/~fpottier/

Re: [Caml-list] ppx_deriving question: deferring code generation?

[whitequark]

On 2017-01-04 13:08, François Pottier wrote:
> I am imagining that perhaps the user could write something like this:
> 
>   type foo =
>     Bar | Baz
>     [@@deriving visitors { delayed = true }
> 
>   let x : foo option ref =
>     ref None
> 
>   [@@visitors]
> 
> The effect of the flag { delayed = true } would be to store the 
> generated code
> somewhere in memory (instead of emitting right away), and the effect of 
> the
> floating attribute [@@visitors] would be to fetch that code from memory 
> and
> emit it.
> 
> To me, this seems somewhat ugly, but workable. Does ppx_deriving offer 
> a
> better approach? Does anyone have a better suggestion? Comments are
> appreciated.

ppx_deriving currently does not allow for any (non-horrible) way to do 
this.
I am OK with the approach you propose.

-- 
whitequark