From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on yquem.inria.fr X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=AWL autolearn=disabled version=3.1.3 X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by yquem.inria.fr (Postfix) with ESMTP id F40E8BBCA for ; Fri, 29 Feb 2008 12:44:58 +0100 (CET) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgAAAJB/x0dQRFuwh2dsb2JhbACQbwEBAQgKKYENmyQ X-IronPort-AV: E=Sophos;i="4.25,426,1199660400"; d="scan'208";a="9744753" Received: from furbychan.cocan.org ([80.68.91.176]) by mail3-smtp-sop.national.inria.fr with ESMTP; 29 Feb 2008 12:44:58 +0100 Received: from rich by furbychan.cocan.org with local (Exim 4.63) (envelope-from ) id 1JV3fZ-0005EL-G9; Fri, 29 Feb 2008 11:44:53 +0000 Date: Fri, 29 Feb 2008 11:44:53 +0000 To: Gabriel Kerneis Cc: caml-list@yquem.inria.fr Subject: Re: [Caml-list] Long-term storage of values Message-ID: <20080229114453.GA19532@annexia.org> References: <191751.36007.qm@web54607.mail.re2.yahoo.com> <20080229084000.2d3a95fe@tatanka.kerneis.info> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20080229084000.2d3a95fe@tatanka.kerneis.info> User-Agent: Mutt/1.5.13 (2006-08-11) From: Richard Jones X-Spam: no; 0.00; 0100,:01 caching:01 ocaml:01 type-safe:01 bindings:01 vacuum:98 wrote:01 caml-list:01 data:02 data:02 optimization:03 fri:05 thu:05 converting:05 brian:05 On Fri, Feb 29, 2008 at 08:40:00AM +0100, Gabriel Kerneis wrote: > Hello, > > Le Thu, 28 Feb 2008 20:14:27 -0500 (EST), Brian Hurt a > écrit : > > So, mistake number one: either use the data, and structure your data > > (at that layer) to take advantage of it, or don't use a database. > > [...] > > So that's mistake number two: you're communicating between different > > versions of the program with an ill-defined (at best) and not > > generic protocol/file format. > > Right. But imagine you're communicating with yourself (saving and > restoring data). And you need to retrieve the data *efficiently*. > Converting from a generic file format is not efficient - not if you have > to retrieve the data 100 times per second (imagine a CMS on a popular > website). It would be faster to use an internal file format. And, of > course, keeping a backup in a generic file format. But there are plenty of database-backed websites out there which perform well[1]. Of course you have to understand databases, optimization of databases, connection pooling, appropriate use of stored procedures, denormalizing and occasionally even caching data outside the database. So as with many things in life you need to know what you're doing. A good first step for PostgreSQL is to use and understand the "VACUUM" and "EXPLAIN" commands, and to turn on statement logging (with timestamps) so you can see which queries are consuming the most time. OCaml even has type-safe bindings to PostgreSQL ... Rich. [1] Some good books on the general issues involved: http://www.amazon.co.uk/Building-Scalable-Web-Sites-Henderson/dp/0596102356 http://www.amazon.co.uk/Speed-Up-Your-Site-Optimization/dp/0735713243 -- Richard Jones Red Hat