Morphed at little into Grid Computing Re: [Caml-list] Memory Mapped Files and OCaml

[Vasili Galchin <vasiliocaml@yahoo.com>]

Hello,

    Richard question is IMO a very good one. i want to
go a step further however. I remember two research
efforts in the past:

1) Shiva project - in Scheme I believe done at Bell
Labs (I almost 100% sure there was an ACM TOPLAS paper
on this project)

2) The Tube - a PhD dissertation project done at
Cambridge U.  .... IMO ... very, very nice  (caveat it
is a prototype proof-of-concept), but IMO very cool
ideas ....
http://citeseer.ist.psu.edu/cache/papers/cs/5552/http:zSzzSzwww.cl.cam.ac.ukzSzuserszSzdah28zSzdah28ths.pdf/halls97applying.pdf/.


Shiva would call call-cc to capture a computation as a
continuation, serialize the continuation and then this
serialized continuation could be "faulted" across to
another computation node for execution. This allows
"mobile" computations (The Tube is very, very
similar!), i.e. that move to other computation nodes
say for fault tolerant reasons, load balancing, etc.
Of course, the fact that we can serialize implies (at
least to me) that we can have notion of persistence
which Richard seems to be alluding to. yes?

    Frankly in the O'Reilly book when I read that an
entire "closure" could not be faulted across to
another computation node, I was very disappointed. 

    - I realized that this faulting doesn't come for
free in terms om network traffick.

    - on the other hand in grid computing as it now
stands, there is (IMO) is a serious lack of "mobile"
computations (http://www.globus.org)!

Regards, vasili 


ote:
> Hi,
> 
> I wonder if anyone can give me some pointers. I'm
> interested in having 
> all memory used by my ocaml program memory mapped so
> that calculations 
> can be preserved from one run of an ocaml program to
> the next. I'm 
> thinking of something like:
> 
> let empty_list : (string, string) list = [] in
> *let user_list = store.get_value "user_list"
> empty_list in*
> let user : string = Cgi.get_value "user" in
> let password : string = Cgi.get_value "password" in
>   if exists user user_list then
>     if auth user password user_list then
>       print_account_details user       
>     else
>       print_account_details user
>   else
>     begin
>       *store.put_value "user_list" (add_user user
> password user_list);*
>       print_account_details user
>     end 
> 
> With the idea being that all values are stored in
> the memory mapped 
> files so put_value and get_value are very fast.
> Serialization is ok for 
> small data structures, but for 50M data structures,
> for which only a 
> small part of the data structure is accesed, it is a
> pain.
> 
> The idea is that put_value and get_value cause
> structures to survive to 
> the uppermost scope level and so these are not
> garbage collected before 
> program termination since they are still referenced.
> Everything else 
> gets garbage collected and so doesn't clog up the
> persistent store.
> 
> Of course program termination can take a long time
> if there are many 
> dirty pages that need to be synchronised to disk.
> There may be some way 
> to tell unix to sync dirty pages while the program
> is running but 
> without thrashing (i.e. using all system resources).
> 
> Such a persistent store does suffer from a lack of
> safety. i.e. killing 
> the process or the machine going down could leave
> the store in an 
> inconsistent state. If safety is required there must
> be algorithms 
> around to provide it in conjuction with a memory
> mapped file, perhaps 
> via checkpointing. Does referential transparency
> help us here?
> 
> One final question: Are most people using database
> backends for 
> persistence? Is it the case that most data
> structures that one would 
> want to create in Ocaml programs map fairly easily
> into B-tree 
> structures, i.e. are maps or multimaps from a keyed
> domain into some 
> structured domain.
> 
> best regards,
> 
> Richard.
> 
> 


__________________________________
Do you Yahoo!?
Yahoo! Finance Tax Center - File online. File on time.
http://taxes.yahoo.com/filing.html

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners