Re: [Caml-list] Long-term storage of values

[Berke Durak <berke.durak@exalead.com>]

Dario Teixeira a écrit :
> Hi,
> 
> Some weeks ago I asked in the list about solutions for the long-term
> storage of values in Ocaml.  I was looking for a solution that was
> not quite as brittle as the Marshal module, while still maintaining
> reasonable performance and being able to deal with Ocaml types in
> all their glory (recursive structures, etc).
> 
> Thanks for all your replies -- I got plenty of ideas and found out
> about new libraries I didn't know existed.  After some experimenting,
> I settled for Sexplib; the main reason being the syntax extension that
> automatically takes care of writing the (de)serialising functions.
> 
> However, if your requirements are different, you may find that another
> solution might be more suitable.  Here's a brief summary of the main
> options I looked into:  (let me know if there are others!)
> 
> 
> - S-expressions using Sexplib.  It has the advantage of being
>   quite fast, compact, and still human-readable-ish (if you like
>   parentheses).  Moreover, the Camlp4 syntax extension makes the
>   creation of (de)serialising functions as straightforward as
>   appending "with sexp" to type definitions.
> 
> - Config_file.  A strong contender if proper human-readability
>   is a requirement (it is far more readable than Sexplib, for
>   example).  As far as I can tell, you still need to manually
>   create the (de)serialising functions for complex types, though.
>   Also, my guess is that the slightly more complex syntax will
>   make it slower than Sexplib.
> 
> - JSON.  Possibly close to Config_file in terms of effort and
>   speed.  Has the same advantage of being very human-friendly,
>   and the additional advantage of having quickly become a popular
>   interchange format (you'll find libraries for almost any language
>   out there).
> 
> - XML.  Though Ocamlduce makes XML easier to deal with, the
>   only advantage I see in XML is its ubiquity.  The format is
>   verbose, not all that human-friendly, and you still need to
>   manually create the (de)serialising functions.
> 
> - XDR.  Though I have fond memories of using XDR with C (in a
>   different life), I have no experience with it in Ocaml.
>   Though it should be very fast, you also need to manually
>   create the (de)serialising functions.
> 
> - The I/O combinator library by Berke Durak.  Seems similar
>   to XDR in terms of effort required.  Does it still not
>   support recursive structures?  That is a common requirement.
> 
> 

Hello,

I have been testing Sexplib for the last few weeks and I am very
satisfied with it.  The code is very high quality and the automatic
generation of converters (to and from S-expressions) is extremely
pleasant to work with.  It works very well with modules, functors
and polymorphic types.  Congratulations to Markus Mottl.
I certainly do not miss writing I/O combinators by hand for record
or sum types.

The use of uniform S-expressions has the considerable advantage of
simplicity.  You will never spend more than half an hour
writing a parser/printer for S-expressions in any language.

Simplicity and uniform representation are also very important for data
perennity.  The type is just

    type t = Atom of string | List of t list

Values of this type can very easily be processed in Ocaml.  This way,
you can migrate your old data quite easily by loading it, writing a few
recursive functions or using the Path module of Sexp, and then saving
it again.  For simple cases, you could even use sed, sgrep or even Scheme.

For instance, I have migrated twice a (small) user base for a personal
web site when I had to add some record fields or change one key from
ints to int lists to strings, and it works well.

We are also using it now for debugging output of complex types (ASTs,
class hierarchies, etc.) and the output is much more readable than what
you would usually crank by hand for debugging purposes.

Note however that Sexplib doesn't handle recursive data by default, but as
you can override printers/parsers for any type, you could easily use special
"references" thru which recursion could go and handle it manually.
As you all know, there is no good way in Ocaml for maintaining a physical
equality-based set.  (The IO combinator library doesn't either).

So I heavily recommend the use of Sexplib and its integration with
standardized Ocaml distributions.
-- 
Berke DURAK