From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by sympa.inria.fr (Postfix) with ESMTPS id 154037F726 for ; Sun, 4 May 2014 06:49:39 +0200 (CEST) Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of mmatalka@gmail.com) identity=pra; client-ip=74.125.82.52; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="mmatalka@gmail.com"; x-sender="mmatalka@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail2-smtp-roc.national.inria.fr: domain of mmatalka@gmail.com designates 74.125.82.52 as permitted sender) identity=mailfrom; client-ip=74.125.82.52; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="mmatalka@gmail.com"; x-sender="mmatalka@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-wg0-f52.google.com) identity=helo; client-ip=74.125.82.52; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="mmatalka@gmail.com"; x-sender="postmaster@mail-wg0-f52.google.com"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AlEBAJ3FZVNKfVI0lGdsb2JhbABZgmVwxUCBEBYOAQEBAQcLCwkSKoIlAQEEAScZARsdAQMBCwYFCwMKCSUPAQQPEQEFASITG4gRAQMJCAEECJlejGKDDZkIChknDWSFYBEBAQQMiSWFIQeEPwSEWgKUWIp+hgVBhGQ X-IPAS-Result: AlEBAJ3FZVNKfVI0lGdsb2JhbABZgmVwxUCBEBYOAQEBAQcLCwkSKoIlAQEEAScZARsdAQMBCwYFCwMKCSUPAQQPEQEFASITG4gRAQMJCAEECJlejGKDDZkIChknDWSFYBEBAQQMiSWFIQeEPwSEWgKUWIp+hgVBhGQ X-IronPort-AV: E=Sophos;i="4.97,980,1389740400"; d="scan'208";a="71960626" Received: from mail-wg0-f52.google.com ([74.125.82.52]) by mail2-smtp-roc.national.inria.fr with ESMTP/TLS/RC4-SHA; 04 May 2014 06:49:38 +0200 Received: by mail-wg0-f52.google.com with SMTP id l18so5145765wgh.11 for ; Sat, 03 May 2014 21:49:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-type; bh=e1C473TlFdpu37ge0+X2XmTuaFhmgnc091iqFCzYX7Y=; b=UFoajPRb2q6i9iIykM/HQEBDr7IWaOMiVpc8z76nZh7TyvupzvCg7bD+qDLaApBTm0 zVDJNajegE4K+LPz9P5AP0ZUmQ/UGRjCCxiHt4AzZcbNln/c/w86LxTS+tyTgUPFmogz CpfITzEUPIHXz4ml+Cl04ThYINzkeAPXbViAEBhymkPUa/SyjGbpUELctI51yQmQZLxx PvaPV5usHD/kft2sI/XrkcniOzr/Z2bv69rgorz0aTgAzR+RclutWpTVVfQxF3AFa8IT NYvx/RUvWMl2Gvw+tDkpt0xyt1z1qyN8RwqXZY7bUkhLbUbB0sPH6Gwb0Dr1L4SYID5U 3GWw== X-Received: by 10.180.211.116 with SMTP id nb20mr9988587wic.5.1399178978014; Sat, 03 May 2014 21:49:38 -0700 (PDT) Received: from localhost ([2a01:7e00::f03c:91ff:fe70:2696]) by mx.google.com with ESMTPSA id f7sm10099187wjy.24.2014.05.03.21.49.36 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 03 May 2014 21:49:36 -0700 (PDT) From: Malcolm Matalka To: Peter Zotov Cc: caml-list References: <6232dace806569a16f7fbfaa1689ef42@whitequark.org> <87wqe23kvw.fsf@gmail.com> <0f8bb41e5638586926c6975e1def53e9@whitequark.org> <87ppju3dk6.fsf@gmail.com> <3beba9cb0078930fd8a48c4408fa1773@whitequark.org> Date: Sun, 04 May 2014 04:49:35 +0000 In-Reply-To: <3beba9cb0078930fd8a48c4408fa1773@whitequark.org> (Peter Zotov's message of "Sat, 03 May 2014 22:52:53 +0400") Message-ID: <87iopm2lnk.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain Subject: Re: [Caml-list] [ANN] ppx_protobuf Not exactly. I don't mean I want a functor, I just used that style to express that I think it would be best if these sort of things worked on a module-to-module level rather than type. That way I can separate out the data type and it's business logic from its encoding/decoding logic. I want to decouple a type definition from all of the transformations that can be done on the type. Everything an still happen at a preprocessor point, but I just want it to happen on a module level. Peter Zotov writes: > On 2014-05-03 22:46, Malcolm Matalka wrote: >> The idea I mean is more to do this at the module level than the type >> level, like a functor. So rather than defining protobuf for a type >> definition, define it for a module, and have some convention for how to >> pick out setter/getter functions. Then create a new module from that. > > Oh! You want a functor which would be able to examine the structure > of the module that was passed to it. > > It's probably technically feasible (you need a syntactic extension > which would essentially serialize the module that will be passed), but > it is a really horrible solution: > > * You won't be able to report some interesting errors (such as > incorrect annotations... [@key -1] until runtime. > * It will be really slow, because the implementation of the functor > will have to traverse the lists of fields dynamically and invoke > accessors one by one. My current implementation directly pattern > matches the input. > * It is just really complicated and does too much at runtime. > >> >> For example of the top of my head: >> >> module Foo = sig >> type t >> val set_x : t -> int -> t >> val get_x : t -> int >> end >> >> Then I can do: >> >> module Foo_protobuf = Protobuf.Make(Foo) >> >> In this case I stole how most people to functors to make it clear the >> translation is actually module to module. >> >> The reason I prefer this is because I can also do: >> >> module Foo_xml = Xml.Make(Foo) >> module Foo_json = Json.Make(Foo) >> >> By separating the mechanism for creating the decoders from the type >> definition, I can add decoders for any type I want without disturbing >> the original definition. This feels more right to me. But I have no >> idea how to do it. >> >> >> Peter Zotov writes: >> >>> On 2014-05-03 20:08, Malcolm Matalka wrote: >>>> Nice, great work! >>>> >>>> I'm not actually a huge fan of mixing type definitions and the protocols >>>> they can be encoded/decoded from. How hard would it be to take a module >>>> definition accessors on a type and produce a new module with >>>> encode/decode functions? That way I could create JSON, XML, Protobufs, >>>> etc modules from one module. >>> >>> Do you suggest generating the following signature instead of the current >>> one? >>> >>> type t = ... [@@protobuf] >>> module Protobuf_t : sig >>> val decode : Protobuf.Decoder.t -> t >>> val encode : Protobuf.Encoder.t -> t -> unit >>> end >>> >>> This would be similar to what deriving currently does. >>> >>> In principle, this is not a complex change. It would add just a few lines >>> to ppx_protobuf. >>> >>> However, I don't like it conceptually. I think the flat signature is >>> more natural, it mimics what one would usually write by hand without >>> introducing too much deep nesting of modules. You may notice how >>> ppx_protobuf doesn't generate the signature items for you; this is >>> because ppx_protobuf is a mere implementation detail, a convenient >>> way to generate the serializer/deserializer. >>> >>> I'm not going to oppose addition of such a mode for two reasons: >>> * I don't like fighting over minute details. >>> * More importantly, deriving, when rewritten with ppx in mind, >>> will surely contain this mode for compatibility. ppx_protobuf >>> will be (ideally) rewritten over deriving some day. >>> >>> I will happily merge a PR adding such a mode to ppx_protobuf. >>> >>>> >>>> Just an idea! >>>> >>>> Peter Zotov writes: >>>> >>>>> Greetings. >>>>> >>>>> I have just released the first version of ppx_protobuf, a complete >>>>> Protocol Buffers implementation. Unlike Google's implementation, >>>>> ppx_protobuf derives the message structure directly from OCaml type >>>>> definitions, which allows a much more seamless integration with >>>>> OCaml's types. In particular, ppx_protobuf natively supports >>>>> sum types, while maintaining full backwards compatibility with >>>>> protoc. >>>>> >>>>> ppx_protobuf uses the extension points API, and thus requires >>>>> a recent (>= 2014-04-29) 4.02 (trunk) compiler. It also requires >>>>> an unreleased version of ppx_tools. It is probably easiest >>>>> to install both from the source repositories[1][2]. >>>>> >>>>> The API is extensively documented at [3]. >>>>> >>>>> [1]: https://github.com/whitequark/ocaml-ppx_protobuf.git >>>>> [2]: https://github.com/alainfrisch/ppx_tools.git >>>>> [3]: https://github.com/whitequark/ocaml-ppx_protobuf/blob/master/README.md >>>>> >>>>> -- >>>>> WBR, Peter Zotov.