Re: [Caml-list] Re: Serialisation of PXP DTDs

["Mikkel Fahnøe Jørgensen" <mikkel@dvide.com>]

I guess this discussion is an overkill for the problem at hand, but
speaking of binary extensible protocols, have you looked at ASN.1? It
is an abstraction over any number of encodings. At least one binary
encoding has extension bits to allow future growth of object
collections and similar.

Mikkel

2008/10/24 Mauricio Fernandez <mfp@acm.org>:
> On Thu, Oct 23, 2008 at 03:21:01PM -0700, Dario Teixeira wrote:
>> Hi,
>>
>> > This protocol definition is fed to the compiler, which
>> > generates the OCaml type definitions, as well as the
>> > encoders/decoders and pretty-printers (as you can see,
>> > the specification uses a mix of OCaml, Haskell and C++
>> > syntax, but it's pretty clear IMO)
>>
>> Basically the XDR approach, but with a syntax inspired
>> by more modern, functional languages, right?
>
> Yes, something like XDR (and Google's Protocol Buffers, and Facebook's Thrift,
> and and :) with richer data types (algebraic and polymorphic types, etc.) and a
> self-describing encoding that allows you to extend the type definitions while
> ensuring interoperability.
>
>> > It's not a drop-in solution like sexplib's "with sexp",
>> > by design (since it is meant to allow interoperability between
>> > different languages), but it's still fairly easy to use.
>>
>> Personally, I think that a sexplib-like syntax extension is the killer
>> feature for serialisation libraries, and the reason why I was immediately
>> swayed by sexplib.  However, writing a sexplib-like syntax extension for
>> your serialisation library would entail solving the reverse problem now
>> handled by your compiler.  This might not always be possible because some
>> features of Ocaml's type system might not map neatly into your format.
>> Nevertheless, the sheer convenience of the syntax extension approach makes
>> it worth while having, even if on occasion the preprocessor were to produce
>> an error message stating that it could not convert a certain structure.  For
>> reference purposes, you could even have the syntax extension output to an
>> external file the inferred structure definition in your language format!  (I
>> know this would be a very complex project, but it does illustrate the power
>> of Camlp4).
>
> In fact, the wire format easily supports all of OCaml's type system (bin-prot
> does, after all, and this is essentially a self-describing, extensible
> bin-prot). I introduced limitations in the data schema to ensure extensibility
> and portability. Any OCaml type can be encoded easily, but not all possible
> changes to an OCaml type are safe with regard to protocol compatibility. Using
> a separate language makes it easier to prevent altogether (by making them
> impossible to express) or catch such errors.
>
> Leaving unsafe protocol modifications aside (which just means that you have to
> be careful when you change a type), the approach you suggest (supporting only
> a subset of OCaml's type system in a "with protocol"-style syntax extension)
> seems very doable. However, sexplib seems to be the safest option for
> convenient, more or less future-proof serialization in OCaml, for the time
> being.
>
> Cheers,
> --
> Mauricio Fernandez  -   http://eigenclass.org
>
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>