Re: [Caml-list] [ANN] ppx_protobuf

[Peter Zotov <whitequark@whitequark.org>]

On 2014-05-04 08:49, Malcolm Matalka wrote:
> Not exactly. I don't mean I want a functor, I just used that style to
> express that I think it would be best if these sort of things worked on
> a module-to-module level rather than type.  That way I can separate out
> the data type and it's business logic from its encoding/decoding logic.
> I want to decouple a type definition from all of the transformations
> that can be done on the type.  Everything an still happen at a
> preprocessor point, but I just want it to happen on a module level.

Still not a good idea. Consider the annotations like @key and @encoding:
where would you specify them? If right on the type signature, then what
is the point of separation?

> 
> 
> Peter Zotov <whitequark@whitequark.org> writes:
> 
>> On 2014-05-03 22:46, Malcolm Matalka wrote:
>>> The idea I mean is more to do this at the module level than the type
>>> level, like a functor.  So rather than defining protobuf for a type
>>> definition, define it for a module, and have some convention for how 
>>> to
>>> pick out setter/getter functions.  Then create a new module from 
>>> that.
>> 
>> Oh! You want a functor which would be able to examine the structure
>> of the module that was passed to it.
>> 
>> It's probably technically feasible (you need a syntactic extension
>> which would essentially serialize the module that will be passed), but
>> it is a really horrible solution:
>> 
>>   * You won't be able to report some interesting errors (such as
>>     incorrect annotations... [@key -1] until runtime.
>>   * It will be really slow, because the implementation of the functor
>>     will have to traverse the lists of fields dynamically and invoke
>>     accessors one by one. My current implementation directly pattern
>>     matches the input.
>>   * It is just really complicated and does too much at runtime.
>> 
>>> 
>>> For example of the top of my head:
>>> 
>>> module Foo = sig
>>>    type t
>>>    val set_x : t -> int -> t
>>>    val get_x : t -> int
>>> end
>>> 
>>> Then I can do:
>>> 
>>> module Foo_protobuf = Protobuf.Make(Foo)
>>> 
>>> In this case I stole how most people to functors to make it clear the
>>> translation is actually module to module.
>>> 
>>> The reason I prefer this is because I can also do:
>>> 
>>> module Foo_xml = Xml.Make(Foo)
>>> module Foo_json = Json.Make(Foo)
>>> 
>>> By separating the mechanism for creating the decoders from the type
>>> definition, I can add decoders for any type I want without disturbing
>>> the original definition.  This feels more right to me.  But I have no
>>> idea how to do it.
>>> 
>>> 
>>> Peter Zotov <whitequark@whitequark.org> writes:
>>> 
>>>> On 2014-05-03 20:08, Malcolm Matalka wrote:
>>>>> Nice, great work!
>>>>> 
>>>>> I'm not actually a huge fan of mixing type definitions and the 
>>>>> protocols
>>>>> they can be encoded/decoded from.  How hard would it be to take a 
>>>>> module
>>>>> definition accessors on a type and produce a new module with
>>>>> encode/decode functions?  That way I could create JSON, XML, 
>>>>> Protobufs,
>>>>> etc modules from one module.
>>>> 
>>>> Do you suggest generating the following signature instead of the 
>>>> current
>>>> one?
>>>> 
>>>> type t = ... [@@protobuf]
>>>> module Protobuf_t : sig
>>>>   val decode : Protobuf.Decoder.t -> t
>>>>   val encode : Protobuf.Encoder.t -> t -> unit
>>>> end
>>>> 
>>>> This would be similar to what deriving currently does.
>>>> 
>>>> In principle, this is not a complex change. It would add just a few 
>>>> lines
>>>> to ppx_protobuf.
>>>> 
>>>> However, I don't like it conceptually. I think the flat signature is
>>>> more natural, it mimics what one would usually write by hand without
>>>> introducing too much deep nesting of modules. You may notice how
>>>> ppx_protobuf doesn't generate the signature items for you; this is
>>>> because ppx_protobuf is a mere implementation detail, a convenient
>>>> way to generate the serializer/deserializer.
>>>> 
>>>> I'm not going to oppose addition of such a mode for two reasons:
>>>>   * I don't like fighting over minute details.
>>>>   * More importantly, deriving, when rewritten with ppx in mind,
>>>>     will surely contain this mode for compatibility. ppx_protobuf
>>>>     will be (ideally) rewritten over deriving some day.
>>>> 
>>>> I will happily merge a PR adding such a mode to ppx_protobuf.
>>>> 
>>>>> 
>>>>> Just an idea!
>>>>> 
>>>>> Peter Zotov <whitequark@whitequark.org> writes:
>>>>> 
>>>>>> Greetings.
>>>>>> 
>>>>>> I have just released the first version of ppx_protobuf, a complete
>>>>>> Protocol Buffers implementation. Unlike Google's implementation,
>>>>>> ppx_protobuf derives the message structure directly from OCaml 
>>>>>> type
>>>>>> definitions, which allows a much more seamless integration with
>>>>>> OCaml's types. In particular, ppx_protobuf natively supports
>>>>>> sum types, while maintaining full backwards compatibility with
>>>>>> protoc.
>>>>>> 
>>>>>> ppx_protobuf uses the extension points API, and thus requires
>>>>>> a recent (>= 2014-04-29) 4.02 (trunk) compiler. It also requires
>>>>>> an unreleased version of ppx_tools. It is probably easiest
>>>>>> to install both from the source repositories[1][2].
>>>>>> 
>>>>>> The API is extensively documented at [3].
>>>>>> 
>>>>>> [1]: https://github.com/whitequark/ocaml-ppx_protobuf.git
>>>>>> [2]: https://github.com/alainfrisch/ppx_tools.git
>>>>>> [3]: 
>>>>>> https://github.com/whitequark/ocaml-ppx_protobuf/blob/master/README.md
>>>>>> 
>>>>>> --
>>>>>>   WBR, Peter Zotov.

-- 
Peter Zotov
sip:whitequark@sipnet.ru