From mboxrd@z Thu Jan 1 00:00:00 1970 Received: (from majordomo@localhost) by pauillac.inria.fr (8.7.6/8.7.3) id VAA09089; Fri, 28 May 2004 21:44:53 +0200 (MET DST) X-Authentication-Warning: pauillac.inria.fr: majordomo set sender to owner-caml-list@pauillac.inria.fr using -f Received: from nez-perce.inria.fr (nez-perce.inria.fr [192.93.2.78]) by pauillac.inria.fr (8.7.6/8.7.3) with ESMTP id VAA11281 for ; Fri, 28 May 2004 21:44:52 +0200 (MET DST) Received: from smtp1.adl2.internode.on.net (smtp1.adl2.internode.on.net [203.16.214.181]) by nez-perce.inria.fr (8.12.10/8.12.10) with ESMTP id i4SJimEV005025 for ; Fri, 28 May 2004 21:44:50 +0200 Received: from [192.168.1.200] (ppp114-11.lns1.syd3.internode.on.net [150.101.114.11]) by smtp1.adl2.internode.on.net (8.12.9/8.12.9) with ESMTP id i4SJihZq075090; Sat, 29 May 2004 05:14:44 +0930 (CST) Subject: RE: [Caml-list] Out_of_memory exception in output_value From: skaller Reply-To: skaller@users.sourceforge.net To: carette@mcmaster.ca Cc: "'Richard Jones'" , caml-list In-Reply-To: <003701c444d3$7163a340$1b447182@cas.mcmaster.ca> References: <003701c444d3$7163a340$1b447182@cas.mcmaster.ca> Content-Type: text/plain Message-Id: <1085773482.3036.81.camel@pelican.wigram> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 (1.2.2-4) Date: 29 May 2004 05:44:42 +1000 Content-Transfer-Encoding: 7bit X-Miltered: at nez-perce with ID 40B796B0.001 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! X-Loop: caml-list@inria.fr X-Spam: no; 0.00; caml-list:01 sourceforge:01 2004:99 jacques:01 allocates:01 marshalled:01 doubles:01 9660:01 glebe:01 arrays:01 ocaml:01 handles:01 nsw:01 polymorphic:01 snail:02 Sender: owner-caml-list@pauillac.inria.fr Precedence: bulk On Sat, 2004-05-29 at 02:47, Jacques Carette wrote: > > It turns out that during output_value, the OCaml code allocates a > > block of memory to contain the marshalled representation of the data. > > Each time it runs out of room, it doubles the size of that block using > > realloc(). > > Wouldn't it be more system-friendly to try successively factors *2, *1.5, > *1.1, and *1.05 before actually failing? I have a published book chapter part of which deals with this issue in some detail, including some performance measurements. The general solution is much cleaner -- to use a user supplied allocation function something like: new_size = f old_size requested_size Doubling the size, or indeed even using a pure multiplier, is only one possible option: simply adding a fixed amout is another option, and an arbitrary function handles both cases and many more. So a general solution is to make the operation polymorphic by making the calculator function an argument (in practice with a sensible default). My experiments were with variable length arrays used to hold big integers, so some operations produced small increases (like addition) whereas other produced increases much faster (like multiplication). A quick and easy fix would be to use a global variable containing the function which the client can reset. Yeah, I hate global state but here the actual function chosen has no semantic impact, it only affects performance (unless you run out of memory .. of course). So this time a global might be OK :) -- John Skaller, mailto:skaller@users.sf.net voice: 061-2-9660-0850, snail: PO BOX 401 Glebe NSW 2037 Australia Checkout the Felix programming language http://felix.sf.net ------------------- To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/ Beginner's list: http://groups.yahoo.com/group/ocaml_beginners