Re: [Caml-list] extending a type with Marshal

[William Chesters <williamc@paneris.org>]

Christian Rinderknecht writes:
 > Hello Ker,
 > 
 > On Wed, 2003-10-08 at 23:22, Ker Lutyn wrote:
 > > [...]
 > > In other words, Foo sends message A to Bar. We'd like to be able to
 > > handle two possibilities:
 > > 
 > > (1) Foo sends message A' to Bar, who interprets it as A.
 > > (2) Foo sends message A to Bar, who interprets it as A'.
 > > 
 > > ...where A' is an extension of A.
 > > 
 > > To do this it would be necessary to 'extend' a type at one end [...]
 > 
 > This kind of concept is already present (subtyping, versioning,
 > relaying) in a specification language called Abstract Syntax Notation
 > One (ASN.1), and, at least, easily supported by one of its encoding rule
 > called Basic Encoding Rule (BER).
 > 
 > The point of ASN.1 is sharing the same ASN.1 module between the two
 > possibly hetereogeneous peers, also agreeing on an encoding/decoding
 > scheme (e.g. BER) and then each peer compile the ASN.1 module into type
 > definitions and codecs in his application language (obviously, you seems
 > interested in O'Caml). Then compile and link.

   I think it's only fair to point out that many people dislike ASN.1,
and that it may not be a perfect match for what you're trying to
achieve.

   It's true that protocol forward compatibility and negotiation is
supported, but to get it you have to use ASN.1, plus the encoding
rules (nearly always BER, will be handled by the ASN.1 library if you
find one), plus the OSI-inspired TCAP meta-protocol.  All of these are
surprisingly complicated, obscure, and heavyweight given what they
actually do; and of course it's ultimately left up to the client app
to decide what to do when talking to an older server, so they don't
magically solve all your problems.

   And the much trumpted forward compatibility in ASN.1 basically
boils down to the following facts:

   -- Sum types (in ocaml terms) can be encoded in ASN.1 using
so-called "explicit tags": a number is prefixed saying which variant
is present, just like in ocaml's marshal representation.  Because you
are required to give your own numeric values for these tags, you can
achieve forward compatibility by adding new tag-values rather than
re-using old ones when you extend the datatype.

   -- Product types (records and tuples) can be encoded using
"sequences"; again, the elements of the sequence are tagged, and
forward compatibility comes from inventing new tags for new members.

   That's all.  As I said TCAP adds a simple kind of "protocol
negotiation protocol".

   Why go to the trouble of constructing a mapping between the ocaml
values you want to transmit and ASN.1, when you could just encode them
directly in a homegrown encoding with less fuss?  In fact I think with
care you could get upwards compatibility using marshal directly.
Why not just write

      (* version 2 *)

      type foo = { a: string; b: int; extensions: unit option }

      type bar = First of int
               | Second
               | Extended of unit

      (* version 3 *)

      type foo = { a: string; b: int; extensions: foo_v3 option }
      and foo_v3 = { c: float }

      type bar = First of int
               | Second
               | Extended of bar_v3
      and bar_v3 = Third of string
                 | Fourth

This will work because of the way ocaml in practice lays things out,
and automatically induces the right protocol-fallback structure in
your code:

     let doit (foo: foo) =
       printf "a %s b %d " foo.a foo.b;
       match foo.extensions with
         | Some foo_v3 -> printf "c %f" foo_v3.c
         | _ -> printf "(no v3 extensions")



   What would be much more interesting than ASN.1 would be something
that ensured upward compatibility but was (a) more tightly integrated
with ocaml datatype declarations, perhaps via camlp4, so that it
was closer to Marshal's trasparency, and (b) capable of preserving
value identity/pointer aliasing relationships, which is a
completely unknown concept to ASN.1, but a very useful feature of
Marshal.

   In summary my feeling is that unless ASN.1 is mandated for a
certiain application it's best avoided because it creates problems
of complexity and over-engineering without actually offering
very much.  (It would be nice to think that at least the
interoperability nightmares that used to arise because of everyone's
differently broken implementations of the "standard" are a thing of
the past nowadays, but I suspect they're not.)

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners