[Caml-list] adding data persistency in Ocaml...

[Caml-list] adding data persistency in Ocaml...

[Basile STARYNKEVITCH]

Dear All,

I should work (from mid-september 2003) one year at INRIA on adding
data persistency (as much as orthogonal as possible) to
Ocaml. Persistency would use existing support like filesystems or
RDBMS (like MySQL or PostgreSQL etc) or others.

If you have wishes or ideas on how to do it, potential applications in
need of it (CGIs are obvious candidates, but there are other
interested applications) or pitfalls to avoid, or relevant literature,
please feel free to discuss them on the list (or to send it to me if
you feel it is not of general interest).

Obviously the main problem I identified today is to be able to persist
(and share) data with a slight change in the program using it -
persistency for exactly the same program is partly achievable thru the
existing marshalling machinery. I mean that a program persist its
data, then the developer fix a bug and want the modified program to be
able to use the previously persisted data. Perhaps it might require
adding some reflective abilities inside the language, because the
persistence mechanism need to know about the types.

Note that I am not (yet) a persistency or Ocaml internals expert, and
that I am more a coder than a language theorist.

So comments & suggestions are welcome.

Regards.

NB: opinions here are mine only 

-- 

Basile STARYNKEVITCH         http://starynkevitch.net/Basile/ 
email: basile<at>starynkevitch<dot>net 
aliases: basile<at>tunes<dot>org = bstarynk<at>nerim<dot>net
8, rue de la Faïencerie, 92340 Bourg La Reine, France

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] adding data persistency in Ocaml...

[Erik Arneson]

[-- Attachment #1: Type: text/plain, Size: 925 bytes --]

On Thu, Jul 10, 2003 at 01:33:03PM +0200, Basile STARYNKEVITCH wrote:
> I should work (from mid-september 2003) one year at INRIA on adding
> data persistency (as much as orthogonal as possible) to
> Ocaml. Persistency would use existing support like filesystems or
> RDBMS (like MySQL or PostgreSQL etc) or others.

That sounds like a really, really neat idea.  Have you looked at CommonSQL from Xanalsys?
It is a really handy and quite powerful system that binds Common Lisp to database tables
in a fashion that makes CLOS objects persistant.  It might be worth looking at for ideas.

You should be able to find more here:
    <http://www.xanalsys.com/software_tools/reference/lwl41/lwuser/LWUG_127.HTM>

-- 
;; Erik Arneson <erik@aarg.net>      SD, Ashland Lodge No. 23 ;;
;; GPG Key ID: 2048R/8B4CBC9C   CoTH, Siskiyou Chapter No. 21 ;;
;; <http://erik.arneson.org/>     <http://www.aarg.net/mason> ;;


[-- Attachment #2: Type: application/pgp-signature, Size: 481 bytes --]

Re: [Caml-list] adding data persistency in Ocaml...

[Pierre Weis]

> If you have wishes or ideas on how to do it, potential applications in
> need of it (CGIs are obvious candidates, but there are other
> interested applications) or pitfalls to avoid, or relevant literature,
> please feel free to discuss them on the list (or to send it to me if
> you feel it is not of general interest).
[...]
> -- 
> 
> Basile STARYNKEVITCH         http://starynkevitch.net/Basile/ 
> email: basile<at>starynkevitch<dot>net 
> aliases: basile<at>tunes<dot>org = bstarynk<at>nerim<dot>net
> 8, rue de la Faïencerie, 92340 Bourg La Reine, France
> 
> -------------------

Articles have been written on the subject by members of the Cristal
group at INRIA (they called that feature safe value I/Os); they
discussed in long the difference between writing then reading back
values from a single executable and transferring data among unrelated
programs + they explain the tension between being able to read back
stuff from a modified program wrt supporting type abstraction (either
types abstracted in Caml or type abstraction due to external
definitions).  They also study the type systems needed to support this
feature directly from within the language, and the interaction with
dynamic values. Sound relevant to your study.

http://pauillac.inria.fr/jfla/2000/furuse.ps.gz
http://pauillac.inria.fr/jfla/2001/actes/07-furuse.ps

Pierre Weis

INRIA, Projet Cristal, Pierre.Weis@inria.fr, http://pauillac.inria.fr/~weis/


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] adding data persistency in Ocaml...

[John Max Skaller]

Basile STARYNKEVITCH wrote:


> If you have wishes or ideas on how to do it, potential applications in
> need of it (CGIs are obvious candidates, but there are other
> interested applications) or pitfalls to avoid, or relevant literature,
> please feel free to discuss them on the list (or to send it to me if
> you feel it is not of general interest).


I am using a basic kind of persistency in my Ocaml
program Felix which is a C++ code generator.
I do it by Marshalling the data (which is all
simple term structure) along with a time-stamp/version info.

------------------------------------

Issue #1: At present, the format is abstract/opaque. This means

it cannot be read by an external program.

--------------------------------------

Issue #2: my test for version is overly aggressive.

I'd like to be able to be able to read the data
even if it's in an old format, provided the newer
data structures used are compatible in some way.

Note I'm not talking about the version of "Marshal"
used in Ocaml, but my own data structure.
For example some polymorphic variants might be
extended to add new terms, without invalidating
the old data.

> Obviously the main problem I identified today is to be able to persist
> (and share) data with a slight change in the program using it -


That's my issue #2. I extend it from 'new program with
same old data structures' to 'new data structures which
are still compatible with old ones'.

You might even consider that one may wish to *convert*
the old data structures with a user defined conversion
routine in case the old data is compatible in-the-abstract,
but not compatible concretely.

--------------------------------------

Issue #3: persistent functions.


This is a nasty one. In theory, some bytecodes can be
written out and reloaded. For native code compilation,
we could write out binary and dynamically link it --
this would require re-entrancy and some other constraints
(such as having a restricted environment such as 'Pervasives'
and nothing else).


Clearly, a programmer can invent an encoding of an
algorithm, write an interpreter, then load the bytecodes
dynamically.

It would be useful perhaps if a native code program
could be linked with the core bytecode interpreter,
which could run bytecodes and/or saved binaries.
I think this is both a nasty problem and also messy
due to the bytecode/nativecode mixed implementation.

------------------------------------
Issue #4: views
This is a kind of extension where you don't load the
saved data, but create a view of it.

The main reason not to do this 'in memory' is a security thing,
but other reasons such as 'the SQL data base is too big'
come to mind :-)

For Felix, I may want to link parts of a library,
without loading the whole library, which implies
some kind of searching of the persistent data.

-- 
John Max Skaller, mailto:skaller@ozemail.com.au
snail:10/1 Toxteth Rd, Glebe, NSW 2037, Australia.
voice:61-2-9660-0850


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] adding data persistency in Ocaml...

[James Leifer]

Hello Basile,

Pierre Weis <pierre.weis@inria.fr> writes:
...
> discussed in long the difference between writing then reading back
> values from a single executable and transferring data among unrelated
> programs
...

We (J.J. Leifer, G. Peskine, P. Sewell, K. Wansbrough) have worked on
a theory of type-safe marshalling for values of abstract types.  The
theory ensures type and abstraction safety when communicating between
separately built programs.  The idea is to calculate hashes
(fingerprints) of modules to generate type identifiers for abstract
types that are globally meaningful.  These hashes are then compared at
run-time in a dynamic check when unmarshalling.

The ICFP paper about this is ``Global abstraction-safe marshalling
with hash types'':

   http://pauillac.inria.fr/~leifer/research.html

There are limitations to the current theory (no polymorphism, no
functors, no reference cells, no versioning, no dynamic binding, ...)
that we are working to overcome.  Peskine is interested in the
polymorphism question and may try to do an implementation of safe
marshalling for Ocaml.  Some of Sewell's interns are also working on
implementations of various aspects of this in a toy ``mini'' Caml
language.

Basile, come visit us in MOSCOVA (Batiment 8) if you'll be in Rocq!

Best regards,
James
--
James LEIFER, MOSCOVA Project, INRIA Rocquencourt

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners