Re: [Caml-list] [ANN] ppx_protobuf

[Peter Zotov <whitequark@whitequark.org>]

On 2014-05-03 22:46, Malcolm Matalka wrote:
> The idea I mean is more to do this at the module level than the type
> level, like a functor.  So rather than defining protobuf for a type
> definition, define it for a module, and have some convention for how to
> pick out setter/getter functions.  Then create a new module from that.

Oh! You want a functor which would be able to examine the structure
of the module that was passed to it.

It's probably technically feasible (you need a syntactic extension
which would essentially serialize the module that will be passed), but
it is a really horrible solution:

   * You won't be able to report some interesting errors (such as
     incorrect annotations... [@key -1] until runtime.
   * It will be really slow, because the implementation of the functor
     will have to traverse the lists of fields dynamically and invoke
     accessors one by one. My current implementation directly pattern
     matches the input.
   * It is just really complicated and does too much at runtime.

> 
> For example of the top of my head:
> 
> module Foo = sig
>    type t
>    val set_x : t -> int -> t
>    val get_x : t -> int
> end
> 
> Then I can do:
> 
> module Foo_protobuf = Protobuf.Make(Foo)
> 
> In this case I stole how most people to functors to make it clear the
> translation is actually module to module.
> 
> The reason I prefer this is because I can also do:
> 
> module Foo_xml = Xml.Make(Foo)
> module Foo_json = Json.Make(Foo)
> 
> By separating the mechanism for creating the decoders from the type
> definition, I can add decoders for any type I want without disturbing
> the original definition.  This feels more right to me.  But I have no
> idea how to do it.
> 
> 
> Peter Zotov <whitequark@whitequark.org> writes:
> 
>> On 2014-05-03 20:08, Malcolm Matalka wrote:
>>> Nice, great work!
>>> 
>>> I'm not actually a huge fan of mixing type definitions and the 
>>> protocols
>>> they can be encoded/decoded from.  How hard would it be to take a 
>>> module
>>> definition accessors on a type and produce a new module with
>>> encode/decode functions?  That way I could create JSON, XML, 
>>> Protobufs,
>>> etc modules from one module.
>> 
>> Do you suggest generating the following signature instead of the 
>> current
>> one?
>> 
>> type t = ... [@@protobuf]
>> module Protobuf_t : sig
>>   val decode : Protobuf.Decoder.t -> t
>>   val encode : Protobuf.Encoder.t -> t -> unit
>> end
>> 
>> This would be similar to what deriving currently does.
>> 
>> In principle, this is not a complex change. It would add just a few 
>> lines
>> to ppx_protobuf.
>> 
>> However, I don't like it conceptually. I think the flat signature is
>> more natural, it mimics what one would usually write by hand without
>> introducing too much deep nesting of modules. You may notice how
>> ppx_protobuf doesn't generate the signature items for you; this is
>> because ppx_protobuf is a mere implementation detail, a convenient
>> way to generate the serializer/deserializer.
>> 
>> I'm not going to oppose addition of such a mode for two reasons:
>>   * I don't like fighting over minute details.
>>   * More importantly, deriving, when rewritten with ppx in mind,
>>     will surely contain this mode for compatibility. ppx_protobuf
>>     will be (ideally) rewritten over deriving some day.
>> 
>> I will happily merge a PR adding such a mode to ppx_protobuf.
>> 
>>> 
>>> Just an idea!
>>> 
>>> Peter Zotov <whitequark@whitequark.org> writes:
>>> 
>>>> Greetings.
>>>> 
>>>> I have just released the first version of ppx_protobuf, a complete
>>>> Protocol Buffers implementation. Unlike Google's implementation,
>>>> ppx_protobuf derives the message structure directly from OCaml type
>>>> definitions, which allows a much more seamless integration with
>>>> OCaml's types. In particular, ppx_protobuf natively supports
>>>> sum types, while maintaining full backwards compatibility with
>>>> protoc.
>>>> 
>>>> ppx_protobuf uses the extension points API, and thus requires
>>>> a recent (>= 2014-04-29) 4.02 (trunk) compiler. It also requires
>>>> an unreleased version of ppx_tools. It is probably easiest
>>>> to install both from the source repositories[1][2].
>>>> 
>>>> The API is extensively documented at [3].
>>>> 
>>>> [1]: https://github.com/whitequark/ocaml-ppx_protobuf.git
>>>> [2]: https://github.com/alainfrisch/ppx_tools.git
>>>> [3]: 
>>>> https://github.com/whitequark/ocaml-ppx_protobuf/blob/master/README.md
>>>> 
>>>> --
>>>>   WBR, Peter Zotov.

-- 
Peter Zotov
sip:whitequark@sipnet.ru