Re: [Caml-list] [ANN] ppx_protobuf

[Gerd Stolpmann <info@gerd-stolpmann.de>]

Am Sonntag, den 04.05.2014, 12:55 +0400 schrieb Peter Zotov:
> On 2014-05-04 08:49, Malcolm Matalka wrote:
> > Not exactly. I don't mean I want a functor, I just used that style to
> > express that I think it would be best if these sort of things worked on
> > a module-to-module level rather than type.  That way I can separate out
> > the data type and it's business logic from its encoding/decoding logic.
> > I want to decouple a type definition from all of the transformations
> > that can be done on the type.  Everything an still happen at a
> > preprocessor point, but I just want it to happen on a module level.
> 
> Still not a good idea. Consider the annotations like @key and @encoding:
> where would you specify them? If right on the type signature, then what
> is the point of separation?

Which just leads to the question why annotations are no real module
entities. We would need something like

module Foo = sig
  type t
  val set_x : t -> int -> t
  val get_x : t -> int
end

module Foo_protobuf_ann = sig
  import Foo
  annotate t [@@ whatever]
  ...
end

module Foo_protobuf = Protobuf.Make(Foo_protobuf_ann)

The "functor application" would still be fake, of course.

Gerd



> 
> > 
> > 
> > Peter Zotov <whitequark@whitequark.org> writes:
> > 
> >> On 2014-05-03 22:46, Malcolm Matalka wrote:
> >>> The idea I mean is more to do this at the module level than the type
> >>> level, like a functor.  So rather than defining protobuf for a type
> >>> definition, define it for a module, and have some convention for how 
> >>> to
> >>> pick out setter/getter functions.  Then create a new module from 
> >>> that.
> >> 
> >> Oh! You want a functor which would be able to examine the structure
> >> of the module that was passed to it.
> >> 
> >> It's probably technically feasible (you need a syntactic extension
> >> which would essentially serialize the module that will be passed), but
> >> it is a really horrible solution:
> >> 
> >>   * You won't be able to report some interesting errors (such as
> >>     incorrect annotations... [@key -1] until runtime.
> >>   * It will be really slow, because the implementation of the functor
> >>     will have to traverse the lists of fields dynamically and invoke
> >>     accessors one by one. My current implementation directly pattern
> >>     matches the input.
> >>   * It is just really complicated and does too much at runtime.
> >> 
> >>> 
> >>> For example of the top of my head:
> >>> 
> >>> module Foo = sig
> >>>    type t
> >>>    val set_x : t -> int -> t
> >>>    val get_x : t -> int
> >>> end
> >>> 
> >>> Then I can do:
> >>> 
> >>> module Foo_protobuf = Protobuf.Make(Foo)
> >>> 
> >>> In this case I stole how most people to functors to make it clear the
> >>> translation is actually module to module.
> >>> 
> >>> The reason I prefer this is because I can also do:
> >>> 
> >>> module Foo_xml = Xml.Make(Foo)
> >>> module Foo_json = Json.Make(Foo)
> >>> 
> >>> By separating the mechanism for creating the decoders from the type
> >>> definition, I can add decoders for any type I want without disturbing
> >>> the original definition.  This feels more right to me.  But I have no
> >>> idea how to do it.
> >>> 
> >>> 
> >>> Peter Zotov <whitequark@whitequark.org> writes:
> >>> 
> >>>> On 2014-05-03 20:08, Malcolm Matalka wrote:
> >>>>> Nice, great work!
> >>>>> 
> >>>>> I'm not actually a huge fan of mixing type definitions and the 
> >>>>> protocols
> >>>>> they can be encoded/decoded from.  How hard would it be to take a 
> >>>>> module
> >>>>> definition accessors on a type and produce a new module with
> >>>>> encode/decode functions?  That way I could create JSON, XML, 
> >>>>> Protobufs,
> >>>>> etc modules from one module.
> >>>> 
> >>>> Do you suggest generating the following signature instead of the 
> >>>> current
> >>>> one?
> >>>> 
> >>>> type t = ... [@@protobuf]
> >>>> module Protobuf_t : sig
> >>>>   val decode : Protobuf.Decoder.t -> t
> >>>>   val encode : Protobuf.Encoder.t -> t -> unit
> >>>> end
> >>>> 
> >>>> This would be similar to what deriving currently does.
> >>>> 
> >>>> In principle, this is not a complex change. It would add just a few 
> >>>> lines
> >>>> to ppx_protobuf.
> >>>> 
> >>>> However, I don't like it conceptually. I think the flat signature is
> >>>> more natural, it mimics what one would usually write by hand without
> >>>> introducing too much deep nesting of modules. You may notice how
> >>>> ppx_protobuf doesn't generate the signature items for you; this is
> >>>> because ppx_protobuf is a mere implementation detail, a convenient
> >>>> way to generate the serializer/deserializer.
> >>>> 
> >>>> I'm not going to oppose addition of such a mode for two reasons:
> >>>>   * I don't like fighting over minute details.
> >>>>   * More importantly, deriving, when rewritten with ppx in mind,
> >>>>     will surely contain this mode for compatibility. ppx_protobuf
> >>>>     will be (ideally) rewritten over deriving some day.
> >>>> 
> >>>> I will happily merge a PR adding such a mode to ppx_protobuf.
> >>>> 
> >>>>> 
> >>>>> Just an idea!
> >>>>> 
> >>>>> Peter Zotov <whitequark@whitequark.org> writes:
> >>>>> 
> >>>>>> Greetings.
> >>>>>> 
> >>>>>> I have just released the first version of ppx_protobuf, a complete
> >>>>>> Protocol Buffers implementation. Unlike Google's implementation,
> >>>>>> ppx_protobuf derives the message structure directly from OCaml 
> >>>>>> type
> >>>>>> definitions, which allows a much more seamless integration with
> >>>>>> OCaml's types. In particular, ppx_protobuf natively supports
> >>>>>> sum types, while maintaining full backwards compatibility with
> >>>>>> protoc.
> >>>>>> 
> >>>>>> ppx_protobuf uses the extension points API, and thus requires
> >>>>>> a recent (>= 2014-04-29) 4.02 (trunk) compiler. It also requires
> >>>>>> an unreleased version of ppx_tools. It is probably easiest
> >>>>>> to install both from the source repositories[1][2].
> >>>>>> 
> >>>>>> The API is extensively documented at [3].
> >>>>>> 
> >>>>>> [1]: https://github.com/whitequark/ocaml-ppx_protobuf.git
> >>>>>> [2]: https://github.com/alainfrisch/ppx_tools.git
> >>>>>> [3]: 
> >>>>>> https://github.com/whitequark/ocaml-ppx_protobuf/blob/master/README.md
> >>>>>> 
> >>>>>> --
> >>>>>>   WBR, Peter Zotov.
> 
> -- 
> Peter Zotov
> sip:whitequark@sipnet.ru
> 

-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
My OCaml site:          http://www.camlcity.org
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------