From mboxrd@z Thu Jan 1 00:00:00 1970 Received: (from majordomo@localhost) by pauillac.inria.fr (8.7.6/8.7.3) id IAA27837; Tue, 23 Mar 2004 08:21:31 +0100 (MET) X-Authentication-Warning: pauillac.inria.fr: majordomo set sender to owner-caml-list@pauillac.inria.fr using -f Received: from nez-perce.inria.fr (nez-perce.inria.fr [192.93.2.78]) by pauillac.inria.fr (8.7.6/8.7.3) with ESMTP id IAA28583 for ; Tue, 23 Mar 2004 08:21:29 +0100 (MET) X-SPAM-Warning: Sending machine is listed in blackholes.five-ten-sg.com Received: from web41213.mail.yahoo.com (web41213.mail.yahoo.com [66.218.93.46]) by nez-perce.inria.fr (8.12.10/8.12.10) with SMTP id i2N7M2KW008587 for ; Tue, 23 Mar 2004 08:22:03 +0100 Message-ID: <20040323072127.53013.qmail@web41213.mail.yahoo.com> Received: from [67.241.218.130] by web41213.mail.yahoo.com via HTTP; Mon, 22 Mar 2004 23:21:27 PST Date: Mon, 22 Mar 2004 23:21:27 -0800 (PST) From: Vasili Galchin Subject: Morphed at little into Grid Computing Re: [Caml-list] Memory Mapped Files and OCaml To: Richard Cole , caml-list@inria.fr Cc: vasiliocaml@yahoo.com In-Reply-To: <405E7404.3050306@itee.uq.edu.au> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Miltered: at nez-perce by Joe's j-chkmail ("http://j-chkmail.ensmp.fr")! X-Loop: caml-list@inria.fr X-Spam: no; 0.00; morphed:01 caml-list:01 caveat:01 citeseer:01 serialize:01 serialized:01 serialize:01 ote:99 pointers:01 thrashing:01 conjuction:01 referential:01 backends:01 inconsistent:01 ocaml:01 Sender: owner-caml-list@pauillac.inria.fr Precedence: bulk X-Status: X-Keywords: X-UID: 319 Hello, Richard question is IMO a very good one. i want to go a step further however. I remember two research efforts in the past: 1) Shiva project - in Scheme I believe done at Bell Labs (I almost 100% sure there was an ACM TOPLAS paper on this project) 2) The Tube - a PhD dissertation project done at Cambridge U. .... IMO ... very, very nice (caveat it is a prototype proof-of-concept), but IMO very cool ideas .... http://citeseer.ist.psu.edu/cache/papers/cs/5552/http:zSzzSzwww.cl.cam.ac.ukzSzuserszSzdah28zSzdah28ths.pdf/halls97applying.pdf/. Shiva would call call-cc to capture a computation as a continuation, serialize the continuation and then this serialized continuation could be "faulted" across to another computation node for execution. This allows "mobile" computations (The Tube is very, very similar!), i.e. that move to other computation nodes say for fault tolerant reasons, load balancing, etc. Of course, the fact that we can serialize implies (at least to me) that we can have notion of persistence which Richard seems to be alluding to. yes? Frankly in the O'Reilly book when I read that an entire "closure" could not be faulted across to another computation node, I was very disappointed. - I realized that this faulting doesn't come for free in terms om network traffick. - on the other hand in grid computing as it now stands, there is (IMO) is a serious lack of "mobile" computations (http://www.globus.org)! Regards, vasili ote: > Hi, > > I wonder if anyone can give me some pointers. I'm > interested in having > all memory used by my ocaml program memory mapped so > that calculations > can be preserved from one run of an ocaml program to > the next. I'm > thinking of something like: > > let empty_list : (string, string) list = [] in > *let user_list = store.get_value "user_list" > empty_list in* > let user : string = Cgi.get_value "user" in > let password : string = Cgi.get_value "password" in > if exists user user_list then > if auth user password user_list then > print_account_details user > else > print_account_details user > else > begin > *store.put_value "user_list" (add_user user > password user_list);* > print_account_details user > end > > With the idea being that all values are stored in > the memory mapped > files so put_value and get_value are very fast. > Serialization is ok for > small data structures, but for 50M data structures, > for which only a > small part of the data structure is accesed, it is a > pain. > > The idea is that put_value and get_value cause > structures to survive to > the uppermost scope level and so these are not > garbage collected before > program termination since they are still referenced. > Everything else > gets garbage collected and so doesn't clog up the > persistent store. > > Of course program termination can take a long time > if there are many > dirty pages that need to be synchronised to disk. > There may be some way > to tell unix to sync dirty pages while the program > is running but > without thrashing (i.e. using all system resources). > > Such a persistent store does suffer from a lack of > safety. i.e. killing > the process or the machine going down could leave > the store in an > inconsistent state. If safety is required there must > be algorithms > around to provide it in conjuction with a memory > mapped file, perhaps > via checkpointing. Does referential transparency > help us here? > > One final question: Are most people using database > backends for > persistence? Is it the case that most data > structures that one would > want to create in Ocaml programs map fairly easily > into B-tree > structures, i.e. are maps or multimaps from a keyed > domain into some > structured domain. > > best regards, > > Richard. > > __________________________________ Do you Yahoo!? Yahoo! Finance Tax Center - File online. File on time. http://taxes.yahoo.com/filing.html ------------------- To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/ Beginner's list: http://groups.yahoo.com/group/ocaml_beginners