From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from concorde.inria.fr (concorde.inria.fr [192.93.2.39]) by yquem.inria.fr (Postfix) with ESMTP id E71E0BB9B for ; Fri, 14 Oct 2005 12:07:12 +0200 (CEST) Received: from pauillac.inria.fr (pauillac.inria.fr [128.93.11.35]) by concorde.inria.fr (8.13.0/8.13.0) with ESMTP id j9EA7CTg023485 for ; Fri, 14 Oct 2005 12:07:12 +0200 Received: from nez-perce.inria.fr (nez-perce.inria.fr [192.93.2.78]) by pauillac.inria.fr (8.7.6/8.7.3) with ESMTP id MAA13190 for ; Fri, 14 Oct 2005 12:07:11 +0200 (MET DST) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.173]) by nez-perce.inria.fr (8.13.0/8.13.0) with ESMTP id j9EA7BlB009381 for ; Fri, 14 Oct 2005 12:07:11 +0200 Received: from [212.227.126.160] (helo=mrelayng.kundenserver.de) by moutng.kundenserver.de with esmtp (Exim 3.35 #1) id 1EQMSx-0002On-00; Fri, 14 Oct 2005 12:07:07 +0200 Received: from [84.58.139.208] (helo=gate.lan.gerd-stolpmann.de) by mrelayng.kundenserver.de with asmtp (Exim 3.35 #1) id 1EQMSx-0007A8-00; Fri, 14 Oct 2005 12:07:07 +0200 Received: from flakew.lan.gerd-stolpmann.de (flakew.lan.gerd-stolpmann.de [192.168.0.32]) by gate.lan.gerd-stolpmann.de (Postfix) with ESMTP id A1706C074; Fri, 14 Oct 2005 12:07:08 +0200 (CEST) Subject: Re: [Caml-list] Memory usage/ garbage collection question From: Gerd Stolpmann To: Richard Jones Cc: yoann padioleau , caml-list@inria.fr In-Reply-To: <20051014101018.GA13302@furbychan.cocan.org> References: <5319313.1129282617505.JavaMail.www@wwinf1608> <20051014101018.GA13302@furbychan.cocan.org> Content-Type: text/plain Date: Fri, 14 Oct 2005 12:07:06 +0200 Message-Id: <1129284426.12434.110.camel@localhost.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.2.1.1 Content-Transfer-Encoding: 7bit X-Provags-ID: kundenserver.de abuse@kundenserver.de auth:a6865a839c0178d9aa0ce41878507ea2 X-Miltered: at concorde with ID 434F8350.000 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! X-Miltered: at nez-perce with ID 434F834F.000 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! X-Spam: no; 0.00; caml-list:01 garbage:01 gerd:01 stolpmann:01 iter:01 iter:01 garbage:01 pointer:01 heap:01 lazy:01 lazy:01 pxp:01 pxp:01 parser:01 parser:01 X-Spam-Checker-Version: SpamAssassin 3.0.3 (2005-04-27) on yquem.inria.fr X-Spam-Level: X-Spam-Status: No, score=0.1 required=5.0 tests=FORGED_RCVD_HELO autolearn=disabled version=3.0.3 Am Freitag, den 14.10.2005, 11:10 +0100 schrieb Richard Jones: > On Fri, Oct 14, 2005 at 11:36:57AM +0200, yoann padioleau wrote: > > > List.iter ( > > > fun row -> > > > (* put row into database and forget about it *) > > > ) rows; > > > (* no further references to rows after this *) > > > > Because rows is still accessible after the List.iter so it is normal > > that it is not garbage collected. > > I agree that rows is "accessible", but it's not actually used. My > understanding is that the GC would be prevented from considering the > list for collection if the pointer to the head of the list (ie. rows) > was stored on the heap or in a register somewhere. Would this be the > case here? > > > I had the same kind of problem and to optimize it I choose to > > produce the elements of rows lazily (but then I had another problem > > with the Lazy modudle where elements were not garbage collected so I > > use my own lazy module (simple via closure) and it works perfectly > > well). > > Unfortunately this isn't really an option here. The rows list comes > from a huge XML doc which is parsed by PXP and passed through some > complex post-processing; PXP doesn't support incremental processing of > XML docs, and the post-processing would be tricky to convert too. PXP has a pull parser. You get the XML document as a lazy stream of XML events. I don't know your document format, but if it is something like ... ... ... lots of them ... I would recommend using the pull parser, and then create XML trees for the individual records only (you can mix both styles). Gerd -- ------------------------------------------------------------ Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany gerd@gerd-stolpmann.de http://www.gerd-stolpmann.de Telefon: 06151/153855 Telefax: 06151/997714 ------------------------------------------------------------