Hi,
I wonder if anyone can give me some pointers. I'm interested in having
all memory used by my ocaml program memory mapped so that calculations
can be preserved from one run of an ocaml program to the next. I'm
thinking of something like:
let empty_list : (string, string) list = [] in
let user_list = store.get_value "user_list" empty_list in
let user : string = Cgi.get_value "user" in
let password : string = Cgi.get_value "password" in
if exists user user_list then
if auth user password user_list then
print_account_details user
else
print_account_details user
else
begin
store.put_value "user_list" (add_user user password
user_list);
print_account_details user
end
With the idea being that all values are stored in the memory mapped
files so put_value and get_value are very fast. Serialization is ok for
small data structures, but for 50M data structures, for which only a
small part of the data structure is accesed, it is a pain.
The idea is that put_value and get_value cause structures to survive to
the uppermost scope level and so these are not garbage collected before
program termination since they are still referenced. Everything else
gets garbage collected and so doesn't clog up the persistent store.
Of course program termination can take a long time if there are many
dirty pages that need to be synchronised to disk. There may be some way
to tell unix to sync dirty pages while the program is running but
without thrashing (i.e. using all system resources).
Such a persistent store does suffer from a lack of safety. i.e. killing
the process or the machine going down could leave the store in an
inconsistent state. If safety is required there must be algorithms
around to provide it in conjuction with a memory mapped file, perhaps
via checkpointing. Does referential transparency help us here?
One final question: Are most people using database backends for
persistence? Is it the case that most data structures that one would
want to create in Ocaml programs map fairly easily into B-tree
structures, i.e. are maps or multimaps from a keyed domain into some
structured domain.
best regards,
Richard.