Re: [Caml-list] Long-term storage of values

[Richard Jones <rich@annexia.org>]

On Fri, Feb 29, 2008 at 08:40:00AM +0100, Gabriel Kerneis wrote:
> Hello,
> 
> Le Thu, 28 Feb 2008 20:14:27 -0500 (EST), Brian Hurt <bhurt@spnz.org> a
> écrit :
> > So, mistake number one: either use the data, and structure your data
> > (at that layer) to take advantage of it, or don't use a database.
> > [...]
> > So that's mistake number two: you're communicating between different 
> > versions of the program with an ill-defined (at best) and not 
> > generic protocol/file format.
> 
> Right. But imagine you're communicating with yourself (saving and
> restoring data). And you need to retrieve the data *efficiently*.
> Converting from a generic file format is not efficient - not if you have
> to retrieve the data 100 times per second (imagine a CMS on a popular
> website). It would be faster to use an internal file format. And, of
> course, keeping a backup in a generic file format.

But there are plenty of database-backed websites out there which
perform well[1].

Of course you have to understand databases, optimization of databases,
connection pooling, appropriate use of stored procedures,
denormalizing and occasionally even caching data outside the database.
So as with many things in life you need to know what you're doing.

A good first step for PostgreSQL is to use and understand the "VACUUM"
and "EXPLAIN" commands, and to turn on statement logging (with
timestamps) so you can see which queries are consuming the most time.

OCaml even has type-safe bindings to PostgreSQL ...

Rich.

[1] Some good books on the general issues involved:
http://www.amazon.co.uk/Building-Scalable-Web-Sites-Henderson/dp/0596102356
http://www.amazon.co.uk/Speed-Up-Your-Site-Optimization/dp/0735713243

-- 
Richard Jones
Red Hat