From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from concorde.inria.fr (concorde.inria.fr [192.93.2.39]) by yquem.inria.fr (Postfix) with ESMTP id 7B1E3BBE4 for ; Sat, 1 Apr 2006 05:35:33 +0200 (CEST) Received: from efil.de (efil.de [81.3.25.37]) by concorde.inria.fr (8.13.0/8.13.0) with ESMTP id k313ZXM5026161 for ; Sat, 1 Apr 2006 05:35:33 +0200 Received: by efil.de (Postfix, from userid 1000) id 0CA36E3E49B; Sat, 1 Apr 2006 05:33:04 +0200 (CEST) Date: Sat, 1 Apr 2006 05:31:27 +0200 From: Ingo Bormuth To: caml-list@yquem.inria.fr Subject: Re: [Caml-list] Re: Stack_overflow Message-ID: <20060401033127.GA7324@kruemel> Reply-To: Ingo Bormuth References: <54f15b6e0603311244u4677c290s481c4aaa859eae35@mail.gmail.com> <54f15b6e0603311623y517b605elc52d6938edea3037@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54f15b6e0603311623y517b605elc52d6938edea3037@mail.gmail.com> X-Attribution: Ingo Bormuth X-Url: http://ibormuth.efil.de/contact User-Agent: Mutt/1.5.11 X-Miltered: at concorde with ID 442DF505.000 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! X-Spam: no; 0.00; stack:01 ocamlc:01 recursive:01 compiler:01 ocaml:01 stack:01 merging:01 runtime:01 ocaml:01 hashtbl:01 hashtbl:01 hash:01 compilation:01 mylist:01 ocamlc:01 X-Spam-Checker-Version: SpamAssassin 3.0.3 (2005-04-27) on yquem.inria.fr X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=disabled version=3.0.3 Hi, I recently had a very similar problem, trying to build a simple in-binary database holding 80.000 records (12 MB). In my case even ocamlc.opt couldn't cope. Maybe there are non tail recursive calls in the compiler. Eventually recompiling ocaml itself with greater stack size could do the job. Cutting the list into pieces and merging at runtime worked fine. What I did, was using a CSV file to auto generate ocaml code similar to: let table = Hashtbl.create 80000 ;; Hashtbl.add table "Key" [ Value1 , Value2, Value3 ] ;; (* 80.000 times *) Iterate over table and build some index hash tables ValueX -> Key Compilation takes a while and uses plenty of RAM, but building the table and looking up 80000 values takes just a second on my old laptop. Ingo On 2006-03-31 18:23, mulhern wrote: > To clarify things: > I'm trying to _compile_ a list definition. > So, my .ml file looks like this: > > --- > let myList = [("first", ["some"; more]); > ("second", ["more"; "still"])] > --- > except that there are 12,000 elements in the list instead of two as in > the example. > _ocamlc_ throws a Stack_overflow error while compiling this list. So, > I want to know how to influence _ocamlc_ to be able to compile this or > larger lists. > The suggestion of chunkifying the list into smaller lists is a > practical one; I may be forced to try it. > > ocamlc.opt compiles the list fine but that reduces portability. > > Odn 3/31/06, mulhern wrote: > > > > I'm trying to compile an automatically generated list definition with > > approximately 12,000 elements. The list has type (record_type * > > record_type list) list where record_type is a simple record with 4 > > fields. > > I'm doing this because I'm trying to achieve a poor man's version of > > staged compilation. > > Unfortunately, I get a Stack_overflow exception. If I roughly half the > > number of elements in the list the Stack_overflow exception goes away. > > Does anybody have an suggestions for me to get around the stack > > overflow problem? What if I want to make my data an order of magnitude > > larger? -- Ingo Bormuth, voicebox & telefax: +49-12125-10226517 '(~o-o~)' public key 86326EC9, http://ibormuth.efil.de/contact --ooO--(.)--Ooo--