From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on yquem.inria.fr X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=AWL autolearn=disabled version=3.1.3 X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from concorde.inria.fr (concorde.inria.fr [192.93.2.39]) by yquem.inria.fr (Postfix) with ESMTP id 76061BC69 for ; Thu, 9 Nov 2006 19:48:14 +0100 (CET) Received: from furbychan.cocan.org (furbychan.cocan.org [80.68.91.176]) by concorde.inria.fr (8.13.6/8.13.6) with ESMTP id kA9Im9aY030204 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO) for ; Thu, 9 Nov 2006 19:48:14 +0100 Received: from rich by furbychan.cocan.org with local (Exim 3.35 #1 (Debian)) id 1GiEwG-0007yR-00; Thu, 09 Nov 2006 18:47:48 +0000 Date: Thu, 9 Nov 2006 18:47:48 +0000 To: =?iso-8859-1?Q?D=E1rio?= Abdulrehman Cc: caml-list@yquem.inria.fr Subject: Re: [Caml-list] using BigArray to mmap huge binary file Message-ID: <20061109184748.GA7016@furbychan.cocan.org> References: <6aeedf580611091030n30ddd9ddjdce831076761bc3@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <6aeedf580611091030n30ddd9ddjdce831076761bc3@mail.gmail.com> User-Agent: Mutt/1.5.9i From: Richard Jones X-Miltered: at concorde with ID 455377E9.000 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! X-Spam: no; 0.00; bigarray:01 bigarrays:01 ocaml:01 compares:01 ocaml:01 bigarray:01 indirections:01 pointer:01 rio:98 blog:98 blog:98 wrote:01 caml-list:01 ints:01 lisp:01 On Thu, Nov 09, 2006 at 06:30:50PM +0000, Dário Abdulrehman wrote: > I have a huge binary data file that I would like to memory map for intensive > reading (no writing) with the best possible performance. > I read some discussions about using BigArrays to achieve this in Ocaml. I > would like to know how this solution compares with the C language mmap. > Please bear in mind I am a total newbie to Ocaml but I have good Common Lisp > experience. We use Bigarray to map geographical data. It works. As far as I'm aware ArrayX.map_file is really just a wrapper around mmap, and read accesses are just indirections on a pointer. Depends a lot on what format your data is in. Ours is stored as a huge array of 16-bit signed ints and Bigarray supports that naturally. If your data was in an unsupported format, or if it wasn't strictly an array, you might end up doing contortions. Rich. -- Richard Jones, CTO Merjis Ltd. Merjis - web marketing and technology - http://merjis.com Internet Marketing and AdWords courses - http://merjis.com/courses - NEW! Merjis blog - http://blog.merjis.com - NEW!