From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on yquem.inria.fr X-Spam-Level: * X-Spam-Status: No, score=1.5 required=5.0 tests=AWL,HTML_10_20,HTML_MESSAGE, SPF_NEUTRAL autolearn=disabled version=3.1.3 X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from mail4-relais-sop.national.inria.fr (mail4-relais-sop.national.inria.fr [192.134.164.105]) by yquem.inria.fr (Postfix) with ESMTP id 42CEDBBCA for ; Fri, 29 Feb 2008 20:09:17 +0100 (CET) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgAAAJLnx0dC+VLvg2dsb2JhbACCNzeOBAEBAQgCBgoJGIENk3GHaQ X-IronPort-AV: E=Sophos;i="4.25,428,1199660400"; d="scan'208";a="23218012" Received: from wx-out-0506.google.com ([66.249.82.239]) by mail4-smtp-sop.national.inria.fr with ESMTP; 29 Feb 2008 20:09:16 +0100 Received: by wx-out-0506.google.com with SMTP id h27so4144662wxd.15 for ; Fri, 29 Feb 2008 11:09:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender:to:subject:in-reply-to:mime-version:content-type:references:x-google-sender-auth; bh=4Lzz6AISKuCgrj4ro8SEeJ450AOjUBT3Wb30JoGCSkw=; b=EqUbAJJPUrkU9eK2dNuvPvxlgEAfrlWhPJmMUNMYiossOhIYeS5SHMCwO323vz3XCR/M9HXLOHPiNCC/YwtuFVIN+pw457mX5nqkCq2za51n+anML6EoT1ue3p74cLHPj73FRlEJzGn9BTn5EX4Taw3gH2Yk1TNpxYdJ89CPhXc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:sender:to:subject:in-reply-to:mime-version:content-type:references:x-google-sender-auth; b=cu3kDF/DeJ1PXEpLNM/uj12Tr0M00iqVjL2aPVLPc9AcPyabxiUCmMlg5QJ9zCvc+FMQEr7xuYZF/0Ge2qi3F7yvbyDik3c6elGsQqclLe6a9IuiuZGDwXI5yqeQv+FpNunbuHl8546A6cQSVgs3SSMtxR/p30b6qqGQI2oQvHk= Received: by 10.70.56.10 with SMTP id e10mr7699541wxa.83.1204312155450; Fri, 29 Feb 2008 11:09:15 -0800 (PST) Received: by 10.70.109.18 with HTTP; Fri, 29 Feb 2008 11:09:15 -0800 (PST) Message-ID: Date: Fri, 29 Feb 2008 11:09:15 -0800 From: "Jake Donham" Sender: jake.donham@gmail.com To: caml-list@yquem.inria.fr Subject: Re: [Caml-list] Long-term storage of values In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_12251_31991784.1204312155460" References: <191751.36007.qm@web54607.mail.re2.yahoo.com> <47C73121.2060809@starynkevitch.net> X-Google-Sender-Auth: 50c0040bc5e0a438 X-Spam: no; 0.00; ens-lyon:01 subset:01 camlp:01 arbitrarily:01 foo:01 foo:01 nodes:01 ocaml:01 externally:01 ens-lyon:01 subset:01 camlp:01 arbitrarily:01 nodes:01 ocaml:01 ------=_Part_12251_31991784.1204312155460 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline On Fri, Feb 29, 2008 at 6:45 AM, Martin Jambon wrote: > if we restrict the transition to a certain subset of operations, it can be > possible to define a mapping using some Camlp4 tool such as Deriving > (well, that's what I was told, assuming I interpreted correctly). > > For instance: > - adding a record field: a default value is injected > - removing a record field: just remove it > - adding a variant: do nothing since it doesn't exist in the old data > - changing a type arbitrarily (such as changing a type foo1 into foo2 > everywhere): provide a map function that would override the default map > function for such nodes. > - functional and abstract values: left as-is > We do this at Skydeck. We modified Deriving's Typeable module to expose the structure of types, and we marshal values along with a version number (for upgrades) and a type description from Typeable (so you get an error if you change the type and forget to change the version number). Our modified Typeable also has reflection functions so if you have a dynamic (a value along with a Typeable.TypeRep) you can examine its parts dynamically (implemented with Obj.magic). On top of that we have a generic function for changing a value of one type into a value of another, as Martin describes--we try to do the translation automatically as far as we can, and provide a way to pass in mapping functions for particular types. For the most part this works well. Compared to a SQL database it is very nice to have the OCaml type system for data representation. The upgrade mechanism is much safer than hand-coding SQL schema upgrades. On the other hand, there are many things you get for free with a SQL db that you have to do yourself: e.g. putting IDs on objects so you can refer to them externally, easy hand-inspection of the data (it's annoying navigating big structures in the top-level), to name a couple of small ones. Jake ------=_Part_12251_31991784.1204312155460 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline On Fri, Feb 29, 2008 at 6:45 AM, Martin Jambon <martin.jambon@ens-lyon.org> wrote:
if we restrict the transition to a certain subset of operations, it can be
possible to define a mapping using some Camlp4 tool such as Deriving
(well, that's what I was told, assuming I interpreted correctly).

For instance:
- adding a record field: a default value is injected
- removing a record field: just remove it
- adding a variant: do nothing since it doesn't exist in the old data
- changing a type arbitrarily (such as changing a type foo1 into foo2
  everywhere): provide a map function that would override the default map
  function for such nodes.
- functional and abstract values: left as-is

We do this at Skydeck. We modified Deriving's Typeable module to expose the structure of types, and we marshal values along with a version number (for upgrades) and a type description from Typeable (so you get an error if you change the type and forget to change the version number).

Our modified Typeable also has reflection functions so if you have a dynamic (a value along with a Typeable.TypeRep) you can examine its parts dynamically (implemented with Obj.magic). On top of that we have a generic function for changing a value of one type into a value of another, as Martin describes--we try to do the translation automatically as far as we can, and provide a way to pass in mapping functions for particular types.

For the most part this works well. Compared to a SQL database it is very nice to have the OCaml type system for data representation. The upgrade mechanism is much safer than hand-coding SQL schema upgrades. On the other hand, there are many things you get for free with a SQL db that you have to do yourself: e.g. putting IDs on objects so you can refer to them externally, easy hand-inspection of the data (it's annoying navigating big structures in the top-level), to name a couple of small ones.

Jake
------=_Part_12251_31991784.1204312155460--