Re: [Caml-list] adding data persistency in Ocaml...

[John Max Skaller <skaller@ozemail.com.au>]

Basile STARYNKEVITCH wrote:


> If you have wishes or ideas on how to do it, potential applications in
> need of it (CGIs are obvious candidates, but there are other
> interested applications) or pitfalls to avoid, or relevant literature,
> please feel free to discuss them on the list (or to send it to me if
> you feel it is not of general interest).


I am using a basic kind of persistency in my Ocaml
program Felix which is a C++ code generator.
I do it by Marshalling the data (which is all
simple term structure) along with a time-stamp/version info.

------------------------------------

Issue #1: At present, the format is abstract/opaque. This means

it cannot be read by an external program.

--------------------------------------

Issue #2: my test for version is overly aggressive.

I'd like to be able to be able to read the data
even if it's in an old format, provided the newer
data structures used are compatible in some way.

Note I'm not talking about the version of "Marshal"
used in Ocaml, but my own data structure.
For example some polymorphic variants might be
extended to add new terms, without invalidating
the old data.

> Obviously the main problem I identified today is to be able to persist
> (and share) data with a slight change in the program using it -


That's my issue #2. I extend it from 'new program with
same old data structures' to 'new data structures which
are still compatible with old ones'.

You might even consider that one may wish to *convert*
the old data structures with a user defined conversion
routine in case the old data is compatible in-the-abstract,
but not compatible concretely.

--------------------------------------

Issue #3: persistent functions.


This is a nasty one. In theory, some bytecodes can be
written out and reloaded. For native code compilation,
we could write out binary and dynamically link it --
this would require re-entrancy and some other constraints
(such as having a restricted environment such as 'Pervasives'
and nothing else).


Clearly, a programmer can invent an encoding of an
algorithm, write an interpreter, then load the bytecodes
dynamically.

It would be useful perhaps if a native code program
could be linked with the core bytecode interpreter,
which could run bytecodes and/or saved binaries.
I think this is both a nasty problem and also messy
due to the bytecode/nativecode mixed implementation.

------------------------------------
Issue #4: views
This is a kind of extension where you don't load the
saved data, but create a view of it.

The main reason not to do this 'in memory' is a security thing,
but other reasons such as 'the SQL data base is too big'
come to mind :-)

For Felix, I may want to link parts of a library,
without loading the whole library, which implies
some kind of searching of the persistent data.

-- 
John Max Skaller, mailto:skaller@ozemail.com.au
snail:10/1 Toxteth Rd, Glebe, NSW 2037, Australia.
voice:61-2-9660-0850


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners