From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by sympa.inria.fr (Postfix) with ESMTPS id 27D547F726 for ; Sun, 4 May 2014 10:55:27 +0200 (CEST) Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of whitequark@whitequark.org) identity=pra; client-ip=176.58.103.125; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="whitequark@whitequark.org"; x-sender="whitequark@whitequark.org"; x-conformance=sidf_compatible Received-SPF: Pass (mail2-smtp-roc.national.inria.fr: domain of whitequark@whitequark.org designates 176.58.103.125 as permitted sender) identity=mailfrom; client-ip=176.58.103.125; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="whitequark@whitequark.org"; x-sender="whitequark@whitequark.org"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of postmaster@mail.whitequark.org) identity=helo; client-ip=176.58.103.125; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="whitequark@whitequark.org"; x-sender="postmaster@mail.whitequark.org"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av9UADn/ZVOwOmd9/2dsb2JhbABZg1WCbqg9AQEBBQGBKJhigSZ0giUBAQQBJxECPxAEBw4KLiwrBhMIE4geDAnKBRMEhVaDPh2EFIENB4Q/AQOEWgKFJZoxh3aDOTg X-IPAS-Result: Av9UADn/ZVOwOmd9/2dsb2JhbABZg1WCbqg9AQEBBQGBKJhigSZ0giUBAQQBJxECPxAEBw4KLiwrBhMIE4geDAnKBRMEhVaDPh2EFIENB4Q/AQOEWgKFJZoxh3aDOTg X-Spam-Status: Yes X-IronPort-AV: E=Sophos;i="4.97,980,1389740400"; d="scan'208";a="71987229" Received: from fehu.whitequark.org (HELO mail.whitequark.org) ([176.58.103.125]) by mail2-smtp-roc.national.inria.fr with ESMTP; 04 May 2014 10:55:25 +0200 Received: by mail.whitequark.org (Postfix, from userid 33) id 3BFAE4D743; Sun, 4 May 2014 08:55:24 +0000 (UTC) To: Malcolm Matalka X-PHP-Originating-Script: 1000:rcube.php MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Sun, 04 May 2014 12:55:24 +0400 From: Peter Zotov Cc: caml-list In-Reply-To: <87iopm2lnk.fsf@gmail.com> References: <6232dace806569a16f7fbfaa1689ef42@whitequark.org> <87wqe23kvw.fsf@gmail.com> <0f8bb41e5638586926c6975e1def53e9@whitequark.org> <87ppju3dk6.fsf@gmail.com> <3beba9cb0078930fd8a48c4408fa1773@whitequark.org> <87iopm2lnk.fsf@gmail.com> Message-ID: <3d5dc7368c12c92ab9cb1a45f1210360@whitequark.org> X-Sender: whitequark@whitequark.org User-Agent: Roundcube Webmail/1.0.0 Subject: Re: [Caml-list] [ANN] ppx_protobuf On 2014-05-04 08:49, Malcolm Matalka wrote: > Not exactly. I don't mean I want a functor, I just used that style to > express that I think it would be best if these sort of things worked on > a module-to-module level rather than type. That way I can separate out > the data type and it's business logic from its encoding/decoding logic. > I want to decouple a type definition from all of the transformations > that can be done on the type. Everything an still happen at a > preprocessor point, but I just want it to happen on a module level. Still not a good idea. Consider the annotations like @key and @encoding: where would you specify them? If right on the type signature, then what is the point of separation? > > > Peter Zotov writes: > >> On 2014-05-03 22:46, Malcolm Matalka wrote: >>> The idea I mean is more to do this at the module level than the type >>> level, like a functor. So rather than defining protobuf for a type >>> definition, define it for a module, and have some convention for how >>> to >>> pick out setter/getter functions. Then create a new module from >>> that. >> >> Oh! You want a functor which would be able to examine the structure >> of the module that was passed to it. >> >> It's probably technically feasible (you need a syntactic extension >> which would essentially serialize the module that will be passed), but >> it is a really horrible solution: >> >> * You won't be able to report some interesting errors (such as >> incorrect annotations... [@key -1] until runtime. >> * It will be really slow, because the implementation of the functor >> will have to traverse the lists of fields dynamically and invoke >> accessors one by one. My current implementation directly pattern >> matches the input. >> * It is just really complicated and does too much at runtime. >> >>> >>> For example of the top of my head: >>> >>> module Foo = sig >>> type t >>> val set_x : t -> int -> t >>> val get_x : t -> int >>> end >>> >>> Then I can do: >>> >>> module Foo_protobuf = Protobuf.Make(Foo) >>> >>> In this case I stole how most people to functors to make it clear the >>> translation is actually module to module. >>> >>> The reason I prefer this is because I can also do: >>> >>> module Foo_xml = Xml.Make(Foo) >>> module Foo_json = Json.Make(Foo) >>> >>> By separating the mechanism for creating the decoders from the type >>> definition, I can add decoders for any type I want without disturbing >>> the original definition. This feels more right to me. But I have no >>> idea how to do it. >>> >>> >>> Peter Zotov writes: >>> >>>> On 2014-05-03 20:08, Malcolm Matalka wrote: >>>>> Nice, great work! >>>>> >>>>> I'm not actually a huge fan of mixing type definitions and the >>>>> protocols >>>>> they can be encoded/decoded from. How hard would it be to take a >>>>> module >>>>> definition accessors on a type and produce a new module with >>>>> encode/decode functions? That way I could create JSON, XML, >>>>> Protobufs, >>>>> etc modules from one module. >>>> >>>> Do you suggest generating the following signature instead of the >>>> current >>>> one? >>>> >>>> type t = ... [@@protobuf] >>>> module Protobuf_t : sig >>>> val decode : Protobuf.Decoder.t -> t >>>> val encode : Protobuf.Encoder.t -> t -> unit >>>> end >>>> >>>> This would be similar to what deriving currently does. >>>> >>>> In principle, this is not a complex change. It would add just a few >>>> lines >>>> to ppx_protobuf. >>>> >>>> However, I don't like it conceptually. I think the flat signature is >>>> more natural, it mimics what one would usually write by hand without >>>> introducing too much deep nesting of modules. You may notice how >>>> ppx_protobuf doesn't generate the signature items for you; this is >>>> because ppx_protobuf is a mere implementation detail, a convenient >>>> way to generate the serializer/deserializer. >>>> >>>> I'm not going to oppose addition of such a mode for two reasons: >>>> * I don't like fighting over minute details. >>>> * More importantly, deriving, when rewritten with ppx in mind, >>>> will surely contain this mode for compatibility. ppx_protobuf >>>> will be (ideally) rewritten over deriving some day. >>>> >>>> I will happily merge a PR adding such a mode to ppx_protobuf. >>>> >>>>> >>>>> Just an idea! >>>>> >>>>> Peter Zotov writes: >>>>> >>>>>> Greetings. >>>>>> >>>>>> I have just released the first version of ppx_protobuf, a complete >>>>>> Protocol Buffers implementation. Unlike Google's implementation, >>>>>> ppx_protobuf derives the message structure directly from OCaml >>>>>> type >>>>>> definitions, which allows a much more seamless integration with >>>>>> OCaml's types. In particular, ppx_protobuf natively supports >>>>>> sum types, while maintaining full backwards compatibility with >>>>>> protoc. >>>>>> >>>>>> ppx_protobuf uses the extension points API, and thus requires >>>>>> a recent (>= 2014-04-29) 4.02 (trunk) compiler. It also requires >>>>>> an unreleased version of ppx_tools. It is probably easiest >>>>>> to install both from the source repositories[1][2]. >>>>>> >>>>>> The API is extensively documented at [3]. >>>>>> >>>>>> [1]: https://github.com/whitequark/ocaml-ppx_protobuf.git >>>>>> [2]: https://github.com/alainfrisch/ppx_tools.git >>>>>> [3]: >>>>>> https://github.com/whitequark/ocaml-ppx_protobuf/blob/master/README.md >>>>>> >>>>>> -- >>>>>> WBR, Peter Zotov. -- Peter Zotov sip:whitequark@sipnet.ru