From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from nez-perce.inria.fr (nez-perce.inria.fr [192.93.2.78]) by yquem.inria.fr (Postfix) with ESMTP id 6D311BBAF for ; Sat, 20 May 2006 22:46:06 +0200 (CEST) Received: from pauillac.inria.fr (pauillac.inria.fr [128.93.11.35]) by nez-perce.inria.fr (8.13.0/8.13.0) with ESMTP id k4KKk55u000591 for ; Sat, 20 May 2006 22:46:05 +0200 Received: from nez-perce.inria.fr (nez-perce.inria.fr [192.93.2.78]) by pauillac.inria.fr (8.7.6/8.7.3) with ESMTP id WAA13176 for ; Sat, 20 May 2006 22:46:05 +0200 (MET DST) Received: from osiris.uid0.sk (osiris.uid0.sk [62.168.97.100]) by nez-perce.inria.fr (8.13.0/8.13.0) with ESMTP id k4KKk4mh000583 for ; Sat, 20 May 2006 22:46:04 +0200 Received: from osiris.uid0.sk (localhost [127.0.0.1]) by nod32.uid0.sk (Postfix) with ESMTP id 61C1787C16F; Sat, 20 May 2006 22:47:29 +0200 (CEST) X-Virus-Scanner: This message was checked by NOD32 Antivirus system NOD32 for Linux Mail Server. For more information on NOD32 Antivirus System, please, visit our website: http://www.nod32.com/ Received: by osiris.uid0.sk (Postfix, from userid 1002) id 5373F87C0C3; Sat, 20 May 2006 22:47:29 +0200 (CEST) Date: Sat, 20 May 2006 22:47:29 +0200 From: Jozef Kosoru To: Brian Hurt Cc: caml-list Subject: Re: [Caml-list] Array 4 MB size limit Message-ID: <20060520204729.GF32550@osiris.uid0.sk> References: <20060515141230.ajyupn2z28k0484s@horde.akalin.cx> <446986DF.1070308@inria.fr> <446D5E4A.8060005@akalin.cx> <20060519162844.GA32550@osiris.uid0.sk> <20060520105108.GC32550@osiris.uid0.sk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.1i X-Miltered: at nez-perce with ID 446F800D.001 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! X-Miltered: at nez-perce with ID 446F800C.001 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! X-Spam: no; 0.00; ocaml:01 on-topic:01 ocaml:01 integers:01 integers:01 pointers:01 arrays:01 patching:01 arrays:01 vectors:01 longs:01 20,:98 2006:98 overnight:98 wrote:01 X-Spam-Checker-Version: SpamAssassin 3.0.3 (2005-04-27) on yquem.inria.fr X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=disabled version=3.0.3 On Sat, May 20, 2006 at 10:22:42 -0400, Brian Hurt wrote: > >4G address space limit is off-topic here. Please note that the OCaml > >array size limit is 4M not 4G. > > It is entirely on-topic. How much headroom do you think you have? > > Let's perform an experiment here. Let's assume that Xavier decides you're > right, and releases a new version of Ocaml with removes the 4M array > limit. How big of an array will you be able to create then? More than > 4M, yes- but how much more? > > The hardware imposes a hard and fast 4G limit. [...] To make life > simple, let's say that everything except the huge array will fit in > the upper 2G, leaving 2G for the huge array. > > Now if it's a huge array of integers, each element only takes up 4 > bytes, which means we can fit 512M integers in this array. [...] Wow, > we could make an array 128 times as big as we can now! Except the > proper way to look at this is that it's a mean 7 bits larger. And > that's best case. > > If we changed it to an array of floats, each float now takes 8 bytes, and > that 2G can only hold 256M floats. Our 128x improvement has now dropped > to a 64x improvement- and we're starting to get a bad feeling about where > this is going. > > If we start trying to hold 3-dimensional pointers, that's 3 floats- > and our 64x improvement has now dropped to about a 21x improvement. > [...] making our 21x improvement (42x improvement) a mere 16x (25x) > improvement. > > Now, what happens if we try to hold a triangle of three points (i.e. > for 3D polygon mapping)? [...] 2G can now hold about 19M unique > polygons- a mere 5x advantage. Thank you for elaborating on this issue and providing us some interesting numbers. I think your results are valid. So it is perfectly possible to find examples in a practice where there is a need to create arrays from 5x to 128x bigger than OCaml allows. And that is the problem. > [...] > > >I disagree. Actually I think an opposite. A suggestion to move to > >64-bit is just patching the symptoms of the real problem - a design > >flaw in the OCaml programming language. And this issue will not go > >away on 64-bits. Just like 22 bits for the array size is a > >completely artificial limit on 32-bit platforms, (N - 10) imposes > >once again such an unlogical 54 bits limit on 64-bit computers. And I > >don't think "it makes complete sense on a 64-bit architecture". > > By this logic, Ocaml should be able to support array sizes of 10^100th > on 16-bit platforms. Why is Ocaml imposing stupid artificial limits? No. An above mentioned logic doesn't imply that. It should be 2^16 on 16-bits of course. > I note with humor that the Java programming language a) defined ints > to be 32 bits, and b) made the index type for arrays, vectors, and > many other data structures ints and not longs. When you move to 64 > bits, it's Java, much more so than Ocaml, that hits artificial and low > limits on array size. Ocaml will be able to have arrays millions of > times larger than the largest possible array in Java. Yeah, but we don't expect a lot of Java fans on this mailing list anyhow ;) > >If you create OCaml programs for yourself then maybe you can switch > >to 64-bits overnight (and perpahs you already did so). But if you > >need to distribute your application to customers/users then it's much > >more complicated. And explaining them that they should upgrade all > >workstations because your software has various 4M limitations here > >and there sounds like a bad joke, doesn't it? > > You call them 32-bit limits, which they are. I admit that this is a > problem- but it's not one created by Ocaml. It's been created by OCaml. CPU architecture is not obliged to have word wide enough to allow programming languages to store some meta data along with array indexes. That was OCaml "invention". An argument that 32-bit is a crappy platform doesn't really hold. As Xavier acknowledged - OCaml was not designed with 32-bit CPUs in mind; and that's the thing. > >32-bit x86 platform is still good enough for many purposes and its > >2^32 limit is acceptable in most cases while 2^22 is not. > > Except that you're here comparing apples to oranges again. The 2^22 > limit is in *words*, while the 2^32 limit is in *bytes*. So it's > really more of a 2^24 vr.s 2^31 comparison. My point was that 2^32 (or 2^31 if you want) is usually enough to count both apples and oranges whereas 2^22 is barely enough to count apple trees. But let's close this thread - I think we have discussed the problem into great details and the final conclusion might be that consensus is not possible. :) Jozef -- jozef kosoru http://zyzstar.kosoru.com