[Caml-list] Sys.max_array_length

[Caml-list] Sys.max_array_length

[Sarah DJEBALI]

Hello,


I have installed OCAML 3.06 and when executing a program that reads a 
very long sequence (49 million characters) and puts it into an array, I 
have got the message :
Fatal error: exception Invalid_argument("Array.make")
 

I understand this is due to a too small value for variable 
Sys.max_array_length. Does anyone know how to increase it?


Thank you.
Sarah Djebali



-- 
******************************
Sarah Djebali
Doctorante LDOG - Lami 
ENS 46 rue d'Ulm 75005 Paris
djebali@ens.fr
01 44 32 23 74


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Sys.max_array_length

[Basile Starynkevitch]

On Tue, Feb 03, 2004 at 01:13:42PM +0100, Sarah DJEBALI wrote:

> I have installed OCAML 3.06 and when executing a program that reads a 
> very long sequence (49 million characters) and puts it into an array, I 
> have got the message :
> Fatal error: exception Invalid_argument("Array.make")

Perhaps you should switch to ocaml-3.07, which is the current release,
which have new interesting features IMHO (but won't solve this
particular problem).


> I understand this is due to a too small value for variable 
> Sys.max_array_length. Does anyone know how to increase it?
> 

The only reasonable way to run your code without changes is to change
your machine to a 64 bit architecture, like AMD64 (aka x86_64) ie
Athlon64 processors or Alpha (or Sparc64).

The maximal array length is wired in a lot of places, and changing it
is not easy, because it is related to the layout of the header word
inside the heap (see the comments inside <caml/mlvalues.h> for
details).


I tend to believe that dealing with very big arrays or strings may
indicate that some parts of your algorithms should be rethought. But I
may be really wrong....

Perhaps you should consider using caml bigarrays (with the ability to
map a file - see documentation of function Bigarray.Array1.map_file at
http://caml.inria.fr/ocaml/htmlman/libref/Bigarray.Array1.html
etc....), or representing your huge data otherwise, eg as lists (or
tuples or arrays) of smaller arrays.

Consider changing some of your datastructures to deal with smaller
arrays! Tell us about your experiments!

Regards.


-- 
Basile STARYNKEVITCH -- basile dot starynkevitch at inria dot fr
Project cristal.inria.fr - INRIA Rocquencourt bat 5
http://cristal.inria.fr/~starynke --- all opinions are only mine 

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners