Re: [Caml-list] Re: Stack_overflow

[Ingo Bormuth <ibormuth@efil.de>]

Hi, 

I recently had a very similar problem, trying to build a simple 
in-binary database holding 80.000 records (12 MB).

In my case even ocamlc.opt couldn't cope. Maybe there are non tail 
recursive calls in the compiler. Eventually recompiling ocaml
itself with greater stack size could do the job.

Cutting the list into pieces and merging at runtime worked fine.

What I did, was using a CSV file to auto generate ocaml code similar to:

  let table = Hashtbl.create 80000 ;;

  Hashtbl.add table "Key" [ Value1 , Value2, Value3 ] ;; (* 80.000 times *)

  Iterate over table and build some index hash tables ValueX -> Key
 
Compilation takes a while and uses plenty of RAM, but building the table
and looking up 80000 values takes just a second on my old laptop.


Ingo



On 2006-03-31 18:23, mulhern wrote:
> To clarify things:
> I'm trying to _compile_ a list definition.
> So, my .ml file looks like this:
> 
> ---
> let myList = [("first", ["some"; more]);
>                    ("second", ["more"; "still"])]
> ---
> except that there are 12,000 elements in the list instead of two as in
> the example.
> _ocamlc_ throws a Stack_overflow error while compiling this list. So,
> I want to know how to influence _ocamlc_ to be able to compile this or
> larger lists.
> The suggestion of chunkifying the list into smaller lists is a
> practical one; I may be forced to try it.
> 
> ocamlc.opt compiles the list fine but that reduces portability.
> 
> Odn 3/31/06, mulhern <mulhern@gmail.com> wrote:
> >
> > I'm trying to compile an automatically generated list definition with
> > approximately 12,000 elements. The list has type (record_type *
> > record_type list) list where record_type is a simple record with 4
> > fields.
> > I'm doing this because I'm trying to achieve a poor man's version of
> > staged compilation.
> > Unfortunately, I get a Stack_overflow exception. If I roughly half the
> > number of elements in the list the Stack_overflow exception goes away.
> > Does anybody have an suggestions for me to get around the stack
> > overflow problem? What if I want to make my data an order of magnitude
> > larger?

-- 
Ingo Bormuth, voicebox & telefax: +49-12125-10226517       '(~o-o~)'
public key 86326EC9, http://ibormuth.efil.de/contact   --ooO--(.)--Ooo--