Saving the OCaml interpreter state

Saving the OCaml interpreter state

[Harrison, John R]

[-- Attachment #1: Type: text/plain, Size: 569 bytes --]

I think I asked roughly this question several years ago, but I would
appreciate

a reminder.

 

In several read-eval-print loops for functional languages (e.g. Poly,
SML/NJ,

various LISPs), it's possible to save and restore the current state so
you can

start up again in an environment with all the same objects and name
bindings

present. OCaml does not currently allow this. How hard would it be to
modify

OCaml to support save/restore of sessions? At the moment I'm forced into

OS-specific checkpointing of processes.

 

John.

 


[-- Attachment #2: Type: text/html, Size: 3036 bytes --]

Re: [Caml-list] Saving the OCaml interpreter state

[Michael Nedzelsky]

On Wed, 11 Apr 2007 02:08 am, Harrison, John R wrote:
> I think I asked roughly this question several years ago, but I would
> appreciate
>
> a reminder.
>
>
>
> In several read-eval-print loops for functional languages (e.g. Poly,
> SML/NJ,
>
> various LISPs), it's possible to save and restore the current state so
> you can
>
> start up again in an environment with all the same objects and name
> bindings
>
> present. OCaml does not currently allow this. How hard would it be to
> modify
>
> OCaml to support save/restore of sessions? At the moment I'm forced into
>
> OS-specific checkpointing of processes.

A new version of Poly ML also doesn't have the persistent storage system.

<quote>
The persistent storage system that has been a feature of Poly/ML almost since 
the beginning has finally reached its sell-by date and has been removed. In 
its place there is the facility to export ML functions as object files and 
link them to produce stand-alone executables.
</quote>

Michael

RE: [Caml-list] Saving the OCaml interpreter state

[Harrison, John R]

| A new version of Poly ML also doesn't have the persistent storage
system.

Thanks; I didn't know that, and it comes as quite a surprise given
Poly's history.

Still, my question about OCaml stands. More specifically, I want to
know whether the facility to save and restore state doesn't exist
because
 
 * None of the main OCaml developers particularly care about it

or

 * There are non-trivial technical problems implementing it.

John.

Re: [Caml-list] Saving the OCaml interpreter state

[Jon Harrop]

On Thursday 12 April 2007 16:53, Harrison, John R wrote:
> | A new version of Poly ML also doesn't have the persistent storage
>
> system.
>
> Thanks; I didn't know that, and it comes as quite a surprise given
> Poly's history.
>
> Still, my question about OCaml stands. More specifically, I want to
> know whether the facility to save and restore state doesn't exist
> because
>
>  * None of the main OCaml developers particularly care about it
>
> or
>
>  * There are non-trivial technical problems implementing it.

Like Michael, I am also not going to answer your question (sorry!) but can I 
just say that, as a commercial developer, there would be significant 
incentive to write a killer IDE for OCaml if the current top-level was free 
for commercial use, e.g. part of the stdlib.

Having been playing with F# recently, I'm starting to appreciate some of the 
features afforded by a decent IDE. However, both OCaml and F# lack features 
found in the other and, more importantly, lack many features that could be 
hugely beneficial, particularly to users of the interactive systems.

Marshalling top-level state is one such feature.

-- 
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
OCaml for Scientists
http://www.ffconsultancy.com/products/ocaml_for_scientists

Re: [Caml-list] Saving the OCaml interpreter state

[Yaron Minsky]

[-- Attachment #1: Type: text/plain, Size: 1896 bytes --]

You should check with someone who knows better, but I suspect that if you
become a member of the OCaml consortium (which is fairly cheap) you would
have the rights to do what you propose.

y

On 4/12/07, Jon Harrop <jon@ffconsultancy.com> wrote:
>
> On Thursday 12 April 2007 16:53, Harrison, John R wrote:
> > | A new version of Poly ML also doesn't have the persistent storage
> >
> > system.
> >
> > Thanks; I didn't know that, and it comes as quite a surprise given
> > Poly's history.
> >
> > Still, my question about OCaml stands. More specifically, I want to
> > know whether the facility to save and restore state doesn't exist
> > because
> >
> >  * None of the main OCaml developers particularly care about it
> >
> > or
> >
> >  * There are non-trivial technical problems implementing it.
>
> Like Michael, I am also not going to answer your question (sorry!) but can
> I
> just say that, as a commercial developer, there would be significant
> incentive to write a killer IDE for OCaml if the current top-level was
> free
> for commercial use, e.g. part of the stdlib.
>
> Having been playing with F# recently, I'm starting to appreciate some of
> the
> features afforded by a decent IDE. However, both OCaml and F# lack
> features
> found in the other and, more importantly, lack many features that could be
> hugely beneficial, particularly to users of the interactive systems.
>
> Marshalling top-level state is one such feature.
>
> --
> Dr Jon D Harrop, Flying Frog Consultancy Ltd.
> OCaml for Scientists
> http://www.ffconsultancy.com/products/ocaml_for_scientists
>
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

[-- Attachment #2: Type: text/html, Size: 2603 bytes --]

RE: [Caml-list] Saving the OCaml interpreter state

[Harrison, John R]

[-- Attachment #1: Type: text/plain, Size: 2749 bytes --]

By the way, I checked with someone who knows Poly/ML, and it turns out
that

the new mechanism can still be used to get save/restore of sessions just
by nominating

the Poly/ML toplevel as the function to export as a binary. So although
the mechanism

has changed, the basic support for saving and restoring sessions has not
vanished.

 

I would very much like to see some such facility in OCaml. How hard is
it? I vaguely

remember some apparent problem with closures being allocated on the
stack, but I

don't recall the details.

 

John.

 

________________________________

From: caml-list-bounces@yquem.inria.fr
[mailto:caml-list-bounces@yquem.inria.fr] On Behalf Of Yaron Minsky
Sent: Thursday, April 12, 2007 4:52 PM
To: Jon Harrop
Cc: caml-list@yquem.inria.fr
Subject: Re: [Caml-list] Saving the OCaml interpreter state

 

You should check with someone who knows better, but I suspect that if
you become a member of the OCaml consortium (which is fairly cheap) you
would have the rights to do what you propose.

y

On 4/12/07, Jon Harrop <jon@ffconsultancy.com> wrote:

On Thursday 12 April 2007 16:53, Harrison, John R wrote:
> | A new version of Poly ML also doesn't have the persistent storage
>
> system.
>
> Thanks; I didn't know that, and it comes as quite a surprise given 
> Poly's history.
>
> Still, my question about OCaml stands. More specifically, I want to
> know whether the facility to save and restore state doesn't exist
> because
>
>  * None of the main OCaml developers particularly care about it 
>
> or
>
>  * There are non-trivial technical problems implementing it.

Like Michael, I am also not going to answer your question (sorry!) but
can I
just say that, as a commercial developer, there would be significant 
incentive to write a killer IDE for OCaml if the current top-level was
free
for commercial use, e.g. part of the stdlib.

Having been playing with F# recently, I'm starting to appreciate some of
the
features afforded by a decent IDE. However, both OCaml and F# lack
features 
found in the other and, more importantly, lack many features that could
be
hugely beneficial, particularly to users of the interactive systems.

Marshalling top-level state is one such feature.

--
Dr Jon D Harrop, Flying Frog Consultancy Ltd. 
OCaml for Scientists
http://www.ffconsultancy.com/products/ocaml_for_scientists

_______________________________________________
Caml-list mailing list. Subscription management: 
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs

 


[-- Attachment #2: Type: text/html, Size: 7876 bytes --]

RE: [Caml-list] Saving the OCaml interpreter state

[Gerd Stolpmann]

Am Freitag, den 13.04.2007, 12:07 -0700 schrieb Harrison, John R:
> By the way, I checked with someone who knows Poly/ML, and it turns out
> that
> 
> the new mechanism can still be used to get save/restore of sessions
> just by nominating
> 
> the Poly/ML toplevel as the function to export as a binary. So
> although the mechanism
> 
> has changed, the basic support for saving and restoring sessions has
> not vanished.
> 
>  
> 
> I would very much like to see some such facility in OCaml. How hard is
> it? I vaguely
> 
> remember some apparent problem with closures being allocated on the
> stack, but I
> 
> don’t recall the details.

My guess is that it is not so hard for core O'Caml: dump the bytecode,
the toploop bindings (using Marshal), the bytecode stack and everything
the interpreter needs.

You'll run into some trouble when you have values in your bindings that
cannot be marshalled, e.g. open files, mutexes, database handles, etc.

You'll run into big trouble, however, once you want to use external (C)
libraries. There is no interface for reinitializing them after being
loaded from a dumped session. This is probably the hardest part. Library
initialization is usually done by the caml module that accompanies the
library. This code wouldn't be executed if you just load the bytecode
from a file. You'd need a second initialization path for the reload
case.

My thinking about this feature at all is: It is reminiscent of the time
when there wasn't a real compiler, but it was practise to create a
session by loading file by file into the interpreter. That was slow, and
dumping the session sped that up. Note that you can do something similar
with the Caml toploop by creating your own one (ocamlmktop), or by
loading cmas into the default one. With some tricks it is even possible
to open modules. What we don't have is a convenient way to create
sessions. Maybe we can add that by recording the interactively submitted
definitions, and replaying them later.

Gerd

>                                    
> ______________________________________________________________________
> From:caml-list-bounces@yquem.inria.fr
> [mailto:caml-list-bounces@yquem.inria.fr] On Behalf Of Yaron Minsky
> Sent: Thursday, April 12, 2007 4:52 PM
> To: Jon Harrop
> Cc: caml-list@yquem.inria.fr
> Subject: Re: [Caml-list] Saving the OCaml interpreter state
> 
> 
>  
> 
> You should check with someone who knows better, but I suspect that if
> you become a member of the OCaml consortium (which is fairly cheap)
> you would have the rights to do what you propose.
> 
> y
> 
> On 4/12/07, Jon Harrop <jon@ffconsultancy.com> wrote:
> 
> On Thursday 12 April 2007 16:53, Harrison, John R wrote:
> > | A new version of Poly ML also doesn't have the persistent storage
> >
> > system.
> >
> > Thanks; I didn't know that, and it comes as quite a surprise given 
> > Poly's history.
> >
> > Still, my question about OCaml stands. More specifically, I want to
> > know whether the facility to save and restore state doesn't exist
> > because
> >
> >  * None of the main OCaml developers particularly care about it 
> >
> > or
> >
> >  * There are non-trivial technical problems implementing it.
> 
> Like Michael, I am also not going to answer your question (sorry!) but
> can I
> just say that, as a commercial developer, there would be significant 
> incentive to write a killer IDE for OCaml if the current top-level was
> free
> for commercial use, e.g. part of the stdlib.
> 
> Having been playing with F# recently, I'm starting to appreciate some
> of the
> features afforded by a decent IDE. However, both OCaml and F# lack
> features 
> found in the other and, more importantly, lack many features that
> could be
> hugely beneficial, particularly to users of the interactive systems.
> 
> Marshalling top-level state is one such feature.
> 
> --
> Dr Jon D Harrop, Flying Frog Consultancy Ltd. 
> OCaml for Scientists
> http://www.ffconsultancy.com/products/ocaml_for_scientists
> 
> _______________________________________________
> Caml-list mailing list. Subscription management: 
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
> 
> 
>  
> 
> 
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
Phone: +49-6151-153855                  Fax: +49-6151-997714
------------------------------------------------------------

Re: [Caml-list] Saving the OCaml interpreter state

[Daniel Bünzli]

Le 13 avr. 07 à 21:38, Gerd Stolpmann a écrit :

> What we don't have is a convenient way to create
> sessions. Maybe we can add that by recording the interactively  
> submitted
> definitions, and replaying them later.

You are right, record expressions that were _correctly_ parsed. ledit  
falls short of this goal :

> > ledit -h session1 ocaml
>         Objective Caml version 3.09.3
>
> # let x = 3;;
> val x : int = 3
> #^D
> > ledit -h session2 ocaml
>         Objective Caml version 3.09.3
>
> # #use "session1";;
> val x : int = 3

The problem is when you type malformed expressions, they are also  
recorded (ledit cannot know they are malformed) and #using the  
session breaks, you need to clean it before.

Your proposal seems a good and reasonable feature wish.

Best,

Daniel

Re: [Caml-list] Saving the OCaml interpreter state

[Michael Nedzelsky]

On Fri, 13 Apr 2007 11:07 pm, Harrison, John R wrote:
> By the way, I checked with someone who knows Poly/ML, and it turns out
> that
>
> the new mechanism can still be used to get save/restore of sessions just
> by nominating
>
> the Poly/ML toplevel as the function to export as a binary. So although
> the mechanism
>
> has changed, the basic support for saving and restoring sessions has not
> vanished.
>

You are correct, it seems that it was an overlook on my part, sorry for this.

Michael

Re: [Caml-list] Saving the OCaml interpreter state

[Xavier Leroy]

Sorry for the late answer, I'm catching up with this list.

> In several read-eval-print loops for functional languages
> (e.g. Poly, SML/NJ, various LISPs), it\x19s possible to save and
> restore the current state so you can start up again in an
> environment with all the same objects and name bindings
> present. OCaml does not currently allow this. How hard would it be
> to modify OCaml to support save/restore of sessions?

Definitely not trivial.  In addition to the general problem with
external libraries mentioned by Gerd Stolpmann, there is a more
Caml-specific problem: code fragments generated by the toplevel reside
in many different blocks, which is something that the Caml marshaller
doesn't support.  There would be a lot of non-trivial code relocation
work to perform.

> More specifically, I want to
> know whether the facility to save and restore state doesn't exist
> because
> * None of the main OCaml developers particularly care about it
> or
> * There are non-trivial technical problems implementing it.

Both :-)  There are technical problems, and you are the only user who
expressed a strong desire for this feature.  Personally, it brings
back bad memories of early Lisp and ML implementations where saving
the state of the toplevel loop was the only way to build executable
programs.

Regards,

- Xavier Leroy