Re: [Caml-list] [ANN] ppx_protobuf

[Malcolm Matalka <mmatalka@gmail.com>]

In my fantasy scenario you could annotate the accessor functions in a
module.

Peter Zotov <whitequark@whitequark.org> writes:

> On 2014-05-04 08:49, Malcolm Matalka wrote:
>> Not exactly. I don't mean I want a functor, I just used that style to
>> express that I think it would be best if these sort of things worked on
>> a module-to-module level rather than type.  That way I can separate out
>> the data type and it's business logic from its encoding/decoding logic.
>> I want to decouple a type definition from all of the transformations
>> that can be done on the type.  Everything an still happen at a
>> preprocessor point, but I just want it to happen on a module level.
>
> Still not a good idea. Consider the annotations like @key and @encoding:
> where would you specify them? If right on the type signature, then what
> is the point of separation?
>
>>
>>
>> Peter Zotov <whitequark@whitequark.org> writes:
>>
>>> On 2014-05-03 22:46, Malcolm Matalka wrote:
>>>> The idea I mean is more to do this at the module level than the type
>>>> level, like a functor.  So rather than defining protobuf for a type
>>>> definition, define it for a module, and have some convention for how to
>>>> pick out setter/getter functions.  Then create a new module from that.
>>>
>>> Oh! You want a functor which would be able to examine the structure
>>> of the module that was passed to it.
>>>
>>> It's probably technically feasible (you need a syntactic extension
>>> which would essentially serialize the module that will be passed), but
>>> it is a really horrible solution:
>>>
>>>   * You won't be able to report some interesting errors (such as
>>>     incorrect annotations... [@key -1] until runtime.
>>>   * It will be really slow, because the implementation of the functor
>>>     will have to traverse the lists of fields dynamically and invoke
>>>     accessors one by one. My current implementation directly pattern
>>>     matches the input.
>>>   * It is just really complicated and does too much at runtime.
>>>
>>>>
>>>> For example of the top of my head:
>>>>
>>>> module Foo = sig
>>>>    type t
>>>>    val set_x : t -> int -> t
>>>>    val get_x : t -> int
>>>> end
>>>>
>>>> Then I can do:
>>>>
>>>> module Foo_protobuf = Protobuf.Make(Foo)
>>>>
>>>> In this case I stole how most people to functors to make it clear the
>>>> translation is actually module to module.
>>>>
>>>> The reason I prefer this is because I can also do:
>>>>
>>>> module Foo_xml = Xml.Make(Foo)
>>>> module Foo_json = Json.Make(Foo)
>>>>
>>>> By separating the mechanism for creating the decoders from the type
>>>> definition, I can add decoders for any type I want without disturbing
>>>> the original definition.  This feels more right to me.  But I have no
>>>> idea how to do it.
>>>>
>>>>
>>>> Peter Zotov <whitequark@whitequark.org> writes:
>>>>
>>>>> On 2014-05-03 20:08, Malcolm Matalka wrote:
>>>>>> Nice, great work!
>>>>>>
>>>>>> I'm not actually a huge fan of mixing type definitions and the protocols
>>>>>> they can be encoded/decoded from.  How hard would it be to take a module
>>>>>> definition accessors on a type and produce a new module with
>>>>>> encode/decode functions?  That way I could create JSON, XML, Protobufs,
>>>>>> etc modules from one module.
>>>>>
>>>>> Do you suggest generating the following signature instead of the current
>>>>> one?
>>>>>
>>>>> type t = ... [@@protobuf]
>>>>> module Protobuf_t : sig
>>>>>   val decode : Protobuf.Decoder.t -> t
>>>>>   val encode : Protobuf.Encoder.t -> t -> unit
>>>>> end
>>>>>
>>>>> This would be similar to what deriving currently does.
>>>>>
>>>>> In principle, this is not a complex change. It would add just a few lines
>>>>> to ppx_protobuf.
>>>>>
>>>>> However, I don't like it conceptually. I think the flat signature is
>>>>> more natural, it mimics what one would usually write by hand without
>>>>> introducing too much deep nesting of modules. You may notice how
>>>>> ppx_protobuf doesn't generate the signature items for you; this is
>>>>> because ppx_protobuf is a mere implementation detail, a convenient
>>>>> way to generate the serializer/deserializer.
>>>>>
>>>>> I'm not going to oppose addition of such a mode for two reasons:
>>>>>   * I don't like fighting over minute details.
>>>>>   * More importantly, deriving, when rewritten with ppx in mind,
>>>>>     will surely contain this mode for compatibility. ppx_protobuf
>>>>>     will be (ideally) rewritten over deriving some day.
>>>>>
>>>>> I will happily merge a PR adding such a mode to ppx_protobuf.
>>>>>
>>>>>>
>>>>>> Just an idea!
>>>>>>
>>>>>> Peter Zotov <whitequark@whitequark.org> writes:
>>>>>>
>>>>>>> Greetings.
>>>>>>>
>>>>>>> I have just released the first version of ppx_protobuf, a complete
>>>>>>> Protocol Buffers implementation. Unlike Google's implementation,
>>>>>>> ppx_protobuf derives the message structure directly from OCaml type
>>>>>>> definitions, which allows a much more seamless integration with
>>>>>>> OCaml's types. In particular, ppx_protobuf natively supports
>>>>>>> sum types, while maintaining full backwards compatibility with
>>>>>>> protoc.
>>>>>>>
>>>>>>> ppx_protobuf uses the extension points API, and thus requires
>>>>>>> a recent (>= 2014-04-29) 4.02 (trunk) compiler. It also requires
>>>>>>> an unreleased version of ppx_tools. It is probably easiest
>>>>>>> to install both from the source repositories[1][2].
>>>>>>>
>>>>>>> The API is extensively documented at [3].
>>>>>>>
>>>>>>> [1]: https://github.com/whitequark/ocaml-ppx_protobuf.git
>>>>>>> [2]: https://github.com/alainfrisch/ppx_tools.git
>>>>>>> [3]: https://github.com/whitequark/ocaml-ppx_protobuf/blob/master/README.md
>>>>>>>
>>>>>>> --
>>>>>>>   WBR, Peter Zotov.