From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by sympa.inria.fr (Postfix) with ESMTPS id A9C527F726 for ; Sun, 4 May 2014 17:18:10 +0200 (CEST) Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of mmatalka@gmail.com) identity=pra; client-ip=74.125.82.174; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="mmatalka@gmail.com"; x-sender="mmatalka@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail2-smtp-roc.national.inria.fr: domain of mmatalka@gmail.com designates 74.125.82.174 as permitted sender) identity=mailfrom; client-ip=74.125.82.174; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="mmatalka@gmail.com"; x-sender="mmatalka@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-we0-f174.google.com) identity=helo; client-ip=74.125.82.174; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="mmatalka@gmail.com"; x-sender="postmaster@mail-we0-f174.google.com"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ap4BAL1ZZlNKfVKulGdsb2JhbABYgmVwxT6BDhYOAQEBAQcLCwkSKoIlAQEEAScZARsdAQMBCwYFCwMKCSUPAQQPEQEFASITG4gRAQMJCAEECJl4jGKDDZkAChknDWSFYBEBAQQMiSWFIQeEPwSEWgKUWIp+hgVBhGQ X-IPAS-Result: Ap4BAL1ZZlNKfVKulGdsb2JhbABYgmVwxT6BDhYOAQEBAQcLCwkSKoIlAQEEAScZARsdAQMBCwYFCwMKCSUPAQQPEQEFASITG4gRAQMJCAEECJl4jGKDDZkAChknDWSFYBEBAQQMiSWFIQeEPwSEWgKUWIp+hgVBhGQ X-IronPort-AV: E=Sophos;i="4.97,982,1389740400"; d="scan'208";a="72020575" Received: from mail-we0-f174.google.com ([74.125.82.174]) by mail2-smtp-roc.national.inria.fr with ESMTP/TLS/RC4-SHA; 04 May 2014 17:18:10 +0200 Received: by mail-we0-f174.google.com with SMTP id k48so6453211wev.19 for ; Sun, 04 May 2014 08:18:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-type; bh=PdxpyeYzukB6DsxVhAzQ950HDnrE5kmcY4GG5oMC5zY=; b=qvc6dvh/mLSrkimSm48siCnBydZE1h3rySYcne0RVotYr5kUYX/U9LqDpdF3jJTV1B hHvLRv6ptuNrHQClKkVjjiHomndEgfvLEjSsoETY/G9/GMMANPr7tF3jmRX2kFrBSUie GBTiVKMium2M+AqiQxuC58PGzuGKWgTeuvqpJWeN9NIoj3a8PA9QZocehyA3fb63jZT5 hM2qvVc3DaHN/QzxqLtgZPSMEuE8xlxtw75DdcbyE0Pw8jRraFbpb8iVFiCRIirksfFb AD2zuUQtjz9hiBp4fMJf5zs8HYsu6TZ/pKbFg0I+iFotX26onChYiemhy94K23n1JV3Y z59Q== X-Received: by 10.180.93.101 with SMTP id ct5mr11883758wib.23.1399216689772; Sun, 04 May 2014 08:18:09 -0700 (PDT) Received: from localhost ([2a01:7e00::f03c:91ff:fe70:2696]) by mx.google.com with ESMTPSA id fi2sm11416685wic.15.2014.05.04.08.18.08 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 04 May 2014 08:18:09 -0700 (PDT) From: Malcolm Matalka To: Peter Zotov Cc: caml-list References: <6232dace806569a16f7fbfaa1689ef42@whitequark.org> <87wqe23kvw.fsf@gmail.com> <0f8bb41e5638586926c6975e1def53e9@whitequark.org> <87ppju3dk6.fsf@gmail.com> <3beba9cb0078930fd8a48c4408fa1773@whitequark.org> <87iopm2lnk.fsf@gmail.com> <3d5dc7368c12c92ab9cb1a45f1210360@whitequark.org> Date: Sun, 04 May 2014 15:18:08 +0000 In-Reply-To: <3d5dc7368c12c92ab9cb1a45f1210360@whitequark.org> (Peter Zotov's message of "Sun, 04 May 2014 12:55:24 +0400") Message-ID: <87bnvd374f.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain Subject: Re: [Caml-list] [ANN] ppx_protobuf In my fantasy scenario you could annotate the accessor functions in a module. Peter Zotov writes: > On 2014-05-04 08:49, Malcolm Matalka wrote: >> Not exactly. I don't mean I want a functor, I just used that style to >> express that I think it would be best if these sort of things worked on >> a module-to-module level rather than type. That way I can separate out >> the data type and it's business logic from its encoding/decoding logic. >> I want to decouple a type definition from all of the transformations >> that can be done on the type. Everything an still happen at a >> preprocessor point, but I just want it to happen on a module level. > > Still not a good idea. Consider the annotations like @key and @encoding: > where would you specify them? If right on the type signature, then what > is the point of separation? > >> >> >> Peter Zotov writes: >> >>> On 2014-05-03 22:46, Malcolm Matalka wrote: >>>> The idea I mean is more to do this at the module level than the type >>>> level, like a functor. So rather than defining protobuf for a type >>>> definition, define it for a module, and have some convention for how to >>>> pick out setter/getter functions. Then create a new module from that. >>> >>> Oh! You want a functor which would be able to examine the structure >>> of the module that was passed to it. >>> >>> It's probably technically feasible (you need a syntactic extension >>> which would essentially serialize the module that will be passed), but >>> it is a really horrible solution: >>> >>> * You won't be able to report some interesting errors (such as >>> incorrect annotations... [@key -1] until runtime. >>> * It will be really slow, because the implementation of the functor >>> will have to traverse the lists of fields dynamically and invoke >>> accessors one by one. My current implementation directly pattern >>> matches the input. >>> * It is just really complicated and does too much at runtime. >>> >>>> >>>> For example of the top of my head: >>>> >>>> module Foo = sig >>>> type t >>>> val set_x : t -> int -> t >>>> val get_x : t -> int >>>> end >>>> >>>> Then I can do: >>>> >>>> module Foo_protobuf = Protobuf.Make(Foo) >>>> >>>> In this case I stole how most people to functors to make it clear the >>>> translation is actually module to module. >>>> >>>> The reason I prefer this is because I can also do: >>>> >>>> module Foo_xml = Xml.Make(Foo) >>>> module Foo_json = Json.Make(Foo) >>>> >>>> By separating the mechanism for creating the decoders from the type >>>> definition, I can add decoders for any type I want without disturbing >>>> the original definition. This feels more right to me. But I have no >>>> idea how to do it. >>>> >>>> >>>> Peter Zotov writes: >>>> >>>>> On 2014-05-03 20:08, Malcolm Matalka wrote: >>>>>> Nice, great work! >>>>>> >>>>>> I'm not actually a huge fan of mixing type definitions and the protocols >>>>>> they can be encoded/decoded from. How hard would it be to take a module >>>>>> definition accessors on a type and produce a new module with >>>>>> encode/decode functions? That way I could create JSON, XML, Protobufs, >>>>>> etc modules from one module. >>>>> >>>>> Do you suggest generating the following signature instead of the current >>>>> one? >>>>> >>>>> type t = ... [@@protobuf] >>>>> module Protobuf_t : sig >>>>> val decode : Protobuf.Decoder.t -> t >>>>> val encode : Protobuf.Encoder.t -> t -> unit >>>>> end >>>>> >>>>> This would be similar to what deriving currently does. >>>>> >>>>> In principle, this is not a complex change. It would add just a few lines >>>>> to ppx_protobuf. >>>>> >>>>> However, I don't like it conceptually. I think the flat signature is >>>>> more natural, it mimics what one would usually write by hand without >>>>> introducing too much deep nesting of modules. You may notice how >>>>> ppx_protobuf doesn't generate the signature items for you; this is >>>>> because ppx_protobuf is a mere implementation detail, a convenient >>>>> way to generate the serializer/deserializer. >>>>> >>>>> I'm not going to oppose addition of such a mode for two reasons: >>>>> * I don't like fighting over minute details. >>>>> * More importantly, deriving, when rewritten with ppx in mind, >>>>> will surely contain this mode for compatibility. ppx_protobuf >>>>> will be (ideally) rewritten over deriving some day. >>>>> >>>>> I will happily merge a PR adding such a mode to ppx_protobuf. >>>>> >>>>>> >>>>>> Just an idea! >>>>>> >>>>>> Peter Zotov writes: >>>>>> >>>>>>> Greetings. >>>>>>> >>>>>>> I have just released the first version of ppx_protobuf, a complete >>>>>>> Protocol Buffers implementation. Unlike Google's implementation, >>>>>>> ppx_protobuf derives the message structure directly from OCaml type >>>>>>> definitions, which allows a much more seamless integration with >>>>>>> OCaml's types. In particular, ppx_protobuf natively supports >>>>>>> sum types, while maintaining full backwards compatibility with >>>>>>> protoc. >>>>>>> >>>>>>> ppx_protobuf uses the extension points API, and thus requires >>>>>>> a recent (>= 2014-04-29) 4.02 (trunk) compiler. It also requires >>>>>>> an unreleased version of ppx_tools. It is probably easiest >>>>>>> to install both from the source repositories[1][2]. >>>>>>> >>>>>>> The API is extensively documented at [3]. >>>>>>> >>>>>>> [1]: https://github.com/whitequark/ocaml-ppx_protobuf.git >>>>>>> [2]: https://github.com/alainfrisch/ppx_tools.git >>>>>>> [3]: https://github.com/whitequark/ocaml-ppx_protobuf/blob/master/README.md >>>>>>> >>>>>>> -- >>>>>>> WBR, Peter Zotov.