Re: [Caml-list] [ANN] ppx_protobuf

[Malcolm Matalka <mmatalka@gmail.com>]

Not exactly. I don't mean I want a functor, I just used that style to
express that I think it would be best if these sort of things worked on
a module-to-module level rather than type.  That way I can separate out
the data type and it's business logic from its encoding/decoding logic.
I want to decouple a type definition from all of the transformations
that can be done on the type.  Everything an still happen at a
preprocessor point, but I just want it to happen on a module level.


Peter Zotov <whitequark@whitequark.org> writes:

> On 2014-05-03 22:46, Malcolm Matalka wrote:
>> The idea I mean is more to do this at the module level than the type
>> level, like a functor.  So rather than defining protobuf for a type
>> definition, define it for a module, and have some convention for how to
>> pick out setter/getter functions.  Then create a new module from that.
>
> Oh! You want a functor which would be able to examine the structure
> of the module that was passed to it.
>
> It's probably technically feasible (you need a syntactic extension
> which would essentially serialize the module that will be passed), but
> it is a really horrible solution:
>
>   * You won't be able to report some interesting errors (such as
>     incorrect annotations... [@key -1] until runtime.
>   * It will be really slow, because the implementation of the functor
>     will have to traverse the lists of fields dynamically and invoke
>     accessors one by one. My current implementation directly pattern
>     matches the input.
>   * It is just really complicated and does too much at runtime.
>
>>
>> For example of the top of my head:
>>
>> module Foo = sig
>>    type t
>>    val set_x : t -> int -> t
>>    val get_x : t -> int
>> end
>>
>> Then I can do:
>>
>> module Foo_protobuf = Protobuf.Make(Foo)
>>
>> In this case I stole how most people to functors to make it clear the
>> translation is actually module to module.
>>
>> The reason I prefer this is because I can also do:
>>
>> module Foo_xml = Xml.Make(Foo)
>> module Foo_json = Json.Make(Foo)
>>
>> By separating the mechanism for creating the decoders from the type
>> definition, I can add decoders for any type I want without disturbing
>> the original definition.  This feels more right to me.  But I have no
>> idea how to do it.
>>
>>
>> Peter Zotov <whitequark@whitequark.org> writes:
>>
>>> On 2014-05-03 20:08, Malcolm Matalka wrote:
>>>> Nice, great work!
>>>>
>>>> I'm not actually a huge fan of mixing type definitions and the protocols
>>>> they can be encoded/decoded from.  How hard would it be to take a module
>>>> definition accessors on a type and produce a new module with
>>>> encode/decode functions?  That way I could create JSON, XML, Protobufs,
>>>> etc modules from one module.
>>>
>>> Do you suggest generating the following signature instead of the current
>>> one?
>>>
>>> type t = ... [@@protobuf]
>>> module Protobuf_t : sig
>>>   val decode : Protobuf.Decoder.t -> t
>>>   val encode : Protobuf.Encoder.t -> t -> unit
>>> end
>>>
>>> This would be similar to what deriving currently does.
>>>
>>> In principle, this is not a complex change. It would add just a few lines
>>> to ppx_protobuf.
>>>
>>> However, I don't like it conceptually. I think the flat signature is
>>> more natural, it mimics what one would usually write by hand without
>>> introducing too much deep nesting of modules. You may notice how
>>> ppx_protobuf doesn't generate the signature items for you; this is
>>> because ppx_protobuf is a mere implementation detail, a convenient
>>> way to generate the serializer/deserializer.
>>>
>>> I'm not going to oppose addition of such a mode for two reasons:
>>>   * I don't like fighting over minute details.
>>>   * More importantly, deriving, when rewritten with ppx in mind,
>>>     will surely contain this mode for compatibility. ppx_protobuf
>>>     will be (ideally) rewritten over deriving some day.
>>>
>>> I will happily merge a PR adding such a mode to ppx_protobuf.
>>>
>>>>
>>>> Just an idea!
>>>>
>>>> Peter Zotov <whitequark@whitequark.org> writes:
>>>>
>>>>> Greetings.
>>>>>
>>>>> I have just released the first version of ppx_protobuf, a complete
>>>>> Protocol Buffers implementation. Unlike Google's implementation,
>>>>> ppx_protobuf derives the message structure directly from OCaml type
>>>>> definitions, which allows a much more seamless integration with
>>>>> OCaml's types. In particular, ppx_protobuf natively supports
>>>>> sum types, while maintaining full backwards compatibility with
>>>>> protoc.
>>>>>
>>>>> ppx_protobuf uses the extension points API, and thus requires
>>>>> a recent (>= 2014-04-29) 4.02 (trunk) compiler. It also requires
>>>>> an unreleased version of ppx_tools. It is probably easiest
>>>>> to install both from the source repositories[1][2].
>>>>>
>>>>> The API is extensively documented at [3].
>>>>>
>>>>> [1]: https://github.com/whitequark/ocaml-ppx_protobuf.git
>>>>> [2]: https://github.com/alainfrisch/ppx_tools.git
>>>>> [3]: https://github.com/whitequark/ocaml-ppx_protobuf/blob/master/README.md
>>>>>
>>>>> --
>>>>>   WBR, Peter Zotov.