From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from nez-perce.inria.fr (nez-perce.inria.fr [192.93.2.78]) by yquem.inria.fr (Postfix) with ESMTP id 5878BBB83 for ; Wed, 16 Aug 2006 03:48:37 +0200 (CEST) Received: from pauillac.inria.fr (pauillac.inria.fr [128.93.11.35]) by nez-perce.inria.fr (8.13.6/8.13.6) with ESMTP id k7G1mYjp027279 for ; Wed, 16 Aug 2006 03:48:34 +0200 Received: from concorde.inria.fr (concorde.inria.fr [192.93.2.39]) by pauillac.inria.fr (8.7.6/8.7.3) with ESMTP id DAA10002 for ; Wed, 16 Aug 2006 03:48:33 +0200 (MET DST) Received: from kurims.kurims.kyoto-u.ac.jp (kurims.kurims.kyoto-u.ac.jp [130.54.16.1]) by concorde.inria.fr (8.13.6/8.13.6) with ESMTP id k7G1mVLR008142 for ; Wed, 16 Aug 2006 03:48:32 +0200 Received: from localhost (suiren [130.54.16.25]) by kurims.kurims.kyoto-u.ac.jp (8.13.7/8.13.7) with ESMTP id k7G0squc002827; Wed, 16 Aug 2006 09:54:52 +0900 (JST) Date: Wed, 16 Aug 2006 09:54:49 +0900 (JST) Message-Id: <20060816.095449.21360713.garrigue@math.nagoya-u.ac.jp> To: matt@gushee.net Cc: caml-list@inria.fr Subject: Re: [Caml-list] Weak hashtables & aggressive caching From: Jacques Garrigue In-Reply-To: <44E10783.5000605@gushee.net> References: <44E08F95.2070105@gushee.net> <20060815.062335.55488138.garrigue@math.nagoya-u.ac.jp> <44E10783.5000605@gushee.net> X-Mailer: Mew version 4.2 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Miltered: at nez-perce with ID 44E27972.000 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! X-Miltered: at concorde with ID 44E2796F.000 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! X-Spam: no; 0.00; hashtables:01 caching:01 garbage:01 allocations:01 lablgtk:01 invokes:01 garbage:01 lablgtk:01 ocaml:01 alloc:01 ocaml:01 allocations:01 increments:98 increments:98 caml-list:01 X-Spam-Checker-Version: SpamAssassin 3.0.3 (2005-04-27) on yquem.inria.fr X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=disabled version=3.0.3 From: Matt Gushee > > I wonder how you trigger the GC, to both keep the cache long enough, > > and to avoid filling the memory too much, and resulting in lots of > > swapping. > > I wasn't planning to trigger the GC explicitly. My thought was simply to > stop preloading before GC begins (or at least *when* GC begins). But, if you wait for the GC to begin this is too late: all your weak references will be collected as garbage, so that your cache will be emptied as soon as you fill it. > > means that the memory set should not increase. But with external data > > structures like pixbufs, the GC is called in a pre-programmed way, > > currently at least after every 10 pixbuf allocations. > > You mean that LablGTK directly invokes the garbage collector after 10 > images. That's not much (unless, of course, they are big images). Sounds > like it's a lot of trouble for a small benefit. Again, the trouble is that there is only one allocation function for pixbufs, and it doesn't look at their size. And it isn't aware of how much memory is available either. So the choice was to be extremely conservative. This is maybe a bad idea, but the intent is to avoid keeping big garbage around, as I have seen really bad situations in the past (programs growing to more than 100MB pretty fast.) Since weak references are counted as garbage, there is clearly a contradiction. I suppose more GC tuning in lablgtk would be a good thing. But I really don't see how to do it easily with the ocaml allocation API. The only way to interface external allocation with the GC is an increment N you pass when calling alloc_custom. It tells ocaml to shorten the time to next GC by N % (actually this is a ratio, so you can provide smaller increments.) The trouble is that the GC is triggered by the sum of all increments for all allocations. So if you want to slow it, you need to reduce all increments everywhere... Jacques Garrigue