From mboxrd@z Thu Jan 1 00:00:00 1970 Received: (from majordomo@localhost) by pauillac.inria.fr (8.7.6/8.7.3) id FAA06002; Sun, 1 Aug 2004 05:57:33 +0200 (MET DST) X-Authentication-Warning: pauillac.inria.fr: majordomo set sender to owner-caml-list@pauillac.inria.fr using -f Received: from concorde.inria.fr (concorde.inria.fr [192.93.2.39]) by pauillac.inria.fr (8.7.6/8.7.3) with ESMTP id FAA05085 for ; Sun, 1 Aug 2004 05:57:32 +0200 (MET DST) Received: from outbound28-2.lax.untd.com (outbound28-2.lax.untd.com [64.136.28.160]) by concorde.inria.fr (8.12.10/8.12.10) with SMTP id i713vUSH002407 for ; Sun, 1 Aug 2004 05:57:30 +0200 Received: from outbound28-2.lax.untd.com (smtp02.lax.untd.com [10.130.24.122]) by smtpout05.lax.untd.com with SMTP id AABAS24SAAYVEJKA for (sender ); Sat, 31 Jul 2004 20:56:48 -0700 (PDT) Received: (qmail 28136 invoked from network); 1 Aug 2004 03:56:00 -0000 Received: from unknown (HELO vangogh) (66.52.246.104) by smtp02.lax.untd.com with SMTP; 1 Aug 2004 03:56:00 -0000 From: "Brandon J. Van Every" To: "caml" Subject: RE: [Caml-list] Wish List for Large Mutable Objects Date: Sat, 31 Jul 2004 21:06:30 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.6604 (9.0.2911.0) In-Reply-To: <003301c4772c$5435b9d0$0201a8c0@dylan> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441 Importance: Normal X-ContentStamp: 13:6:546510443 X-UNTD-OriginStamp: CI84cOLHFqh7Zd2QWkwvEFvwyO3T/pIsFsCrOjjLH86tFje1v+PXf3lCneA/u6CI X-Miltered: at concorde with ID 410C6A2A.001 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! X-Loop: caml-list@inria.fr X-Spam: no; 0.00; brandon:99 caml-list:01 mcclain:01 bigarrays:01 posts:01 permutation:01 unboxed:01 priorities:01 arbitrarily:01 hash:01 behave:01 tiling:01 homogeneous:01 interop:01 ocamlopt:01 Sender: owner-caml-list@pauillac.inria.fr Precedence: bulk David McClain wrote: > > Something I would like to see appear in the OCaml libraries, > and I don't have it yet myself, is the use of Copy-on-Write > and Scatter-Gather applied to large mutable objects such as > BigArrays. > > [and some other things in other posts] OCaml is not a lazy language. Is it reasonable to expect Bigarray to perform lazy copies, under any permutation of complexity you have in mind? It seems like what you really want is to design the memory management of an Operating System. If your files are so huge that they don't fit in main memory, why aren't you willing to use virtual memory? Scatter-Gather DMA is a device driver level capability that's hardware dependent. I don't see why a high level language like OCaml should be exposing that kind of functionality, and I'm not entirely sure if it should be doing it under the hood either. If your OS doesn't have the kind of memory management you want, maybe you should modify an open source OS, like Linux or BSD Unix, to do what you want? You ask why Array1, Array2, Array3 should be special cases. Well, clearly because they're the most common, and you can perform access optimizations for each of these common cases. It seems that you are only thinking of ***BIG*** arrays, i.e. your problems and nobody else's. Lotsa people don't have your notion of 'big'. Indeed, I don't personally care about arrays being particularly big. 100MB would be pretty darn big for what I do in game development right now. I do care about their contents being unboxed. If someone wanted to rename Bigarray to UnboxedArray, that would suit my own priorities just fine. I don't understand the "starting from zero" complaint, with respect to arbitrary file formats. If the file format is arbitrarily structured, it is not an array. You will have to read it some other way. Arrays are, generally speaking, composed of uniform elements. At least, that's how all of us pedal-to-the-metal guys view them. I suppose high level language guys often define the word 'array' to mean anything they want, like a list or a hash table or a map or whatever, but I don't think they should. I don't see why your Scientific notion of an 'infinite array' should be a basic language interface. What would be so difficult about building your favorite array windowing scheme on top of the basic fixed length components, and calling that a library? Like 'InfiniteArray' or something. Then you'd write some access functions in some syntax you like, it would behave the way you like, and for your problems you'd be good. I've done similar things to perform addressing on icosahedrons, to try to regularize the mathematics of a tiling of it. I don't bother the user about it, my functions just do some computing to make it all work under the hood. I do wish Bigarray handled heterogeneous C structures. Homogeneous arrays impose some design and interop constraints. Finally, I'm told that the "%" in the names of called functions in the sources means that ocamlopt generates different, better code. The C routines are ignored, they're only used for ocamlc. Cheers, www.indiegamedesign.com Brand*n Van Every S*attle, WA Praise Be to the caml-list Bayesian filter! It blesseth my postings, it is evil crap! evil crap! Bigarray! Unboxed overhead group! Wondering! chant chant chant... ------------------- To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/ Beginner's list: http://groups.yahoo.com/group/ocaml_beginners