Re: [Caml-list] More re GC hanging

[Adrien <camaradetux@gmail.com>]

On 01/09/2010, Damien Doligez <damien.doligez@inria.fr> wrote:
>
> On 2010-08-15, at 12:45, Adrien wrote:
>
>> First, remove all non-tail-rec functions: no more List.map, @ or
>> List.concat. All lists were pretty short (definitely less than 1000
>> elements) but maybe the amount of calls generated garbage or something
>> like that: I couldn't get much infos about the problem so I tried what
>> I could think of and being tail-rec couldn't be a bad thing anyway.
>> The idea was to create less values since I first suspected a memory
>> fragmentation issue (iirc I had thousands of fragments), so I also
>> memoized some functions.
>
> That has nothing to do with the GC getting huge counts.

I know but I first had crashes which didn't show the huge counts and
did what I had planned to do for some time.
Also, I was actually generating lots of garbage (well, maybe not 10^20 ;-)).

> Also, if you
> have fragmentation problems, you can try changing the allocation
> policy with this statement:
>
>    Gc.set {(Gc.get ()) with Gc.allocation_policy = 1};;
>
> I'm still waiting for feedback on that alternate allocation policy :-)

I had tried that, it didn't change anything.

>> Then, as Basile mentionned, call something like Gc.compact ()
>> regularly. The overhead is actually pretty small, especially if ran
>> regularly.
>
> That's good for tracking down problems, but I wouldn't recommend it
> for production code.
>
>> Finally, C bindings. I created a few while not having access to the
>> internet and they are quite dirty. I highly doubt they play perfectly
>> well with the garbage collector: they seem ok but probably aren't
>> perfect. That's definitely something to check, even if the bindings
>> were written by someone else because working nicely with the GC can be
>> quite hard.
>>
>> Now, the problem seems to be gone but I can't say for sure. One really
>> annoying thing was that adding a line like 'print_endline "pouet";'
>> would make the out-of-memory problem go away! Same when getting stats
>> from the GC.
>
>
> That almost certainly indicates a problem with your C bindings: some
> pointer gets garbled and the GC may or may not stumble upon it.

That's also what I think: calling Gc.compact () doesn't solve the
problem, it only changes the planet alignment and the phase of the
moon.

Sorry for the late reaction, I was pretty short on time during the
past ten days but it's going to be better now. :-)
I took a quick look at the C stubs and noticed a few variables of type
'value' where not introduced with CAMLlocalX(), in particular the
creation of a list.
I don't know if that's enough to fix the problem since it wasn't
happening anymore on my computer and I'm now waiting for someone to be
able to test.

-- 

Adrien Nader