Re: [Caml-list] Weak hashtables & aggressive caching

[Jacques Garrigue <garrigue@math.nagoya-u.ac.jp>]

From: Matt Gushee <matt@gushee.net>
> > I wonder how you trigger the GC, to both keep the cache long enough,
> > and to avoid filling the memory too much, and resulting in lots of
> > swapping.
> 
> I wasn't planning to trigger the GC explicitly. My thought was simply to 
> stop preloading before GC begins (or at least *when* GC begins).

But, if you wait for the GC to begin this is too late: all your weak
references will be collected as garbage, so that your cache will be
emptied as soon as you fill it.

> > means that the memory set should not increase. But with external data
> > structures like pixbufs, the GC is called in a pre-programmed way,
> > currently at least after every 10 pixbuf allocations.
> 
> You mean that LablGTK directly invokes the garbage collector after 10 
> images. That's not much (unless, of course, they are big images). Sounds 
> like it's a lot of trouble for a small benefit.

Again, the trouble is that there is only one allocation function for
pixbufs, and it doesn't look at their size. And it isn't aware of how
much memory is available either. So the choice was to be extremely
conservative. This is maybe a bad idea, but the intent is to avoid
keeping big garbage around, as I have seen really bad situations in
the past (programs growing to more than 100MB pretty fast.) Since weak
references are counted as garbage, there is clearly a contradiction.

I suppose more GC tuning in lablgtk would be a good thing. But I
really don't see how to do it easily with the ocaml allocation API.
The only way to interface external allocation with the GC is an
increment N you pass when calling alloc_custom. It tells ocaml to
shorten the time to next GC by N % (actually this is a ratio, so you
can provide smaller increments.) The trouble is that the GC is
triggered by the sum of all increments for all allocations. So if you
want to slow it, you need to reduce all increments everywhere...

Jacques Garrigue