Re: [Caml-list] Array 4 MB size limit

[Jon Harrop <jon@ffconsultancy.com>]

On Friday 19 May 2006 06:57, Frederick Akalin wrote:
> I see.  This is a perfectly reasonable explanation.  I ended up just
> using Bigarrays of floats for now and converting from those to 3-d
> vectors on demand (what I need), but it would be nice to have a type
> that wraps around Array that can get around the 4M limit (using an array
> of arrays like someone suggested earlier).  It's sad, but I think 32-bit
> is going to be around for a while, and surely I can't be the only person
> to run up against this. :) Not that I mind writing such a library and
> releasing it.  I wonder if the Extlib guys would be interested...

If you want your code to be memory efficient then using 32-bit floats via big 
arrays would seem like a good idea. If you want your code to be fast then 
you'll want a monomorphic data structure to avoid the overhead of polymorphic 
function calls (which is often very significant on simple, numerical 
container types).

> > A better idea would be to determine exactly what data structure you need:
> > which abstract operations, what performance requirements, etc.  C++
> > and Lisp programmers tend to encode everything as arrays or lists,
> > respectively, but quite often these are not the best data structure
> > for the application of interest.
>
> I find your assertion surprising.  C++ and Common LISP are by no means
> lacking in standard data structures (and using bare arrays is
> discouraged in C++, as far as I know) and in my experience I haven't
> much seen C++ code that used arrays/vectors when not appropriate.

Yes, I would say that C and Fortran programmers overuse arrays because other 
data structures are prohibitively difficult to implement and reuse.

> In any case, in my application (a raytracer) I am reading in lists of
> numbers from a file representing the points of a mesh and the triangles
> that make up the mesh (represented by a list of indices into the mesh
> list).  A dynamically resizable, reasonable scalable array seems like
> the best data structure to use.

Why not use a list and then apply Array.of_list?

-- 
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
Objective CAML for Scientists
http://www.ffconsultancy.com/products/ocaml_for_scientists