From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from nez-perce.inria.fr (nez-perce.inria.fr [192.93.2.78]) by yquem.inria.fr (Postfix) with ESMTP id 0A04ABB84 for ; Sat, 20 May 2006 20:41:31 +0200 (CEST) Received: from pauillac.inria.fr (pauillac.inria.fr [128.93.11.35]) by nez-perce.inria.fr (8.13.0/8.13.0) with ESMTP id k4KIfUag023241 for ; Sat, 20 May 2006 20:41:30 +0200 Received: from nez-perce.inria.fr (nez-perce.inria.fr [192.93.2.78]) by pauillac.inria.fr (8.7.6/8.7.3) with ESMTP id UAA11461 for ; Sat, 20 May 2006 20:41:29 +0200 (MET DST) Received: from mta.conjury.org (grymling.conjury.org [69.12.155.90]) by nez-perce.inria.fr (8.13.0/8.13.0) with ESMTP id k4KIfSwY023238 for ; Sat, 20 May 2006 20:41:28 +0200 Received: from localhost (localhost [127.0.0.1]) by mta.conjury.org (Postfix) with ESMTP id 9FB74D5CA8 for ; Sat, 20 May 2006 11:41:25 -0700 (PDT) Received: from mta.conjury.org ([127.0.0.1]) by localhost (localhost [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 10842-02 for ; Sat, 20 May 2006 11:41:18 -0700 (PDT) Received: from [10.0.1.3] (station.conjury.org [69.12.155.91]) by mta.conjury.org (Postfix) with ESMTP id 09A36D5C8D for ; Sat, 20 May 2006 11:41:18 -0700 (PDT) Mime-Version: 1.0 (Apple Message framework v750) In-Reply-To: References: <20060515141230.ajyupn2z28k0484s@horde.akalin.cx> <446986DF.1070308@inria.fr> <446D5E4A.8060005@akalin.cx> <20060519162844.GA32550@osiris.uid0.sk> <20060520105108.GC32550@osiris.uid0.sk> Content-Type: text/plain; charset=WINDOWS-1252; delsp=yes; format=flowed Message-Id: Content-Transfer-Encoding: quoted-printable From: j h woodyatt Subject: Re: [Caml-list] Array 4 MB size limit Date: Sat, 20 May 2006 11:41:14 -0700 To: The Caml Trade X-Mailer: Apple Mail (2.750) X-Virus-Scanned: by amavisd-new at conjury.org X-Miltered: at nez-perce with ID 446F62DA.000 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! X-Miltered: at nez-perce with ID 446F62D8.000 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! X-Spam: no; 0.00; woodyatt:01 jhw:01 ocaml:01 bigarray:01 ocaml:01 woodyatt:01 jhw:01 20,:98 rumors:98 wrote:01 caml-list:01 workaround:01 strings:01 spaces:02 caml:02 X-Spam-Checker-Version: SpamAssassin 3.0.3 (2005-04-27) on yquem.inria.fr X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=disabled version=3.0.3 On May 20, 2006, at 7:22 AM, Brian Hurt wrote: > > Consider the classic programming pattern of reading the entire file =20= > into a list (or array) of strings. [...] boom- welcome to the 4G =20 > limit. Heck, you have to do special incantations just to open and =20 > read a file that big, let only trying to store the whole thing in =20 > memory. This is why I claim that once hard disks start getting =20 > larger than the address space, the address space needs to increase. Sounds like it's an *anti*-pattern to me. The offset parameter to =20 mmap(2) is there for a reason. There is, however, another interesting point I hope the Caml team at =20 INRIA keeps in mind. I hear rumors that 64-bit processes on =20 [redacted] may not have access to all the application support =20 libraries. Some of them, e.g. [redacted] and [redacted], may only be =20= available to 32-bit processes-- for technical reasons, not business =20 reasons. The engineering resources to make the libraries available =20 to 64-bit processes are there, but the technical case for making 64-=20 bit versions of the libraries is apparently losing. At least for the =20= time being. I would be unsurprised if technical decisions like this =20 are happening all over the application services programming world. What this tells me is that 64-bit hardware and 64-bit operating =20 systems will very likely continue to have 32-bit processes running on =20= them for some time to come. We will be living with 32-bit address =20 spaces for many years after everyone gets 64-bit operating systems =20 and 64-bit desktop machines. All that said, I've got little sympathy for the critics on this =20 issue. The 4 MB array size limit is a fact of OCaml life, but =20 there's a reasonable workaround that will get most application =20 programmers around it, i.e. careful use of Bigarray. The 4G size =20 limit is another story altogether, and it will plague everyone =20 equally-- not just the OCaml world. =97 j h woodyatt