[Caml-list] Str memory leak in 3.06?

[Caml-list] Str memory leak in 3.06?

[Chris Hecker]

This program leaks 10mb/sec on my machine with ocamlopt 3.06 (msvc, xp).

let _ =
   while true do
     let re = Str.regexp "foo" in ()
   done;
   ()

Inserting a call to Gc.compact in the loop doesn't affect it (well, it 
slows the loop down a bit so the leak rate drops :).

 From a brief trip in the debugger and a glance at strstubs.cpp it appears 
the custom finalizer is being called.  I didn't grovel in the actual regex 
code to see where the leak was (assuming it's not my bug and I'm supposed 
to free the regex somehow in caml code).

I also notice that the strstubs.c has the same problem I reported in 
bigarray (and that was fixed, bug #601) about using stat_alloc() to 
allocate but free() to deallocate, so it should probably be fixed here as 
well, assuming Str is going to live much longer.

Chris

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Str memory leak in 3.06?

[Xavier Leroy]

> This program leaks 10mb/sec on my machine with ocamlopt 3.06 (msvc, xp).
> 
> let _ =
>    while true do
>      let re = Str.regexp "foo" in ()
>    done;
>    ()
> 
> Inserting a call to Gc.compact in the loop doesn't affect it (well, it 
> slows the loop down a bit so the leak rate drops :).
> 
>  From a brief trip in the debugger and a glance at strstubs.cpp it appears 
> the custom finalizer is being called.  I didn't grovel in the actual regex 
> code to see where the leak was (assuming it's not my bug and I'm supposed 
> to free the regex somehow in caml code).

No, there is no need for explicit deallocation of regexps.  Actually,
there is no leak either: if you put a call to Gc.major() in the loop,
memory usage remains constant. 

What happens is that the speed of the (incremental) major collector
isn't appropriately adjusted, so it runs too slowly and lets "floating
garbage" accumulate.  This is a common problem with Caml objects that
are just handle on resources allocated outside the Caml heap: precise
determination of the space consumption of the latter is generally
impossible, causing the incremental major GC to run often too slowly,
sometimes too fast...

> I also notice that the strstubs.c has the same problem I reported in 
> bigarray (and that was fixed, bug #601) about using stat_alloc() to 
> allocate but free() to deallocate, so it should probably be fixed here as 
> well, assuming Str is going to live much longer.

The new implementation of Str that will go in release 3.07 allocates
all its data in the Caml heap, so this fixes both the issue you
mention and the "floating garbage" problem described above.

Best wishes,

- Xavier Leroy

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Str memory leak in 3.06?

[Chris Hecker]

>No, there is no need for explicit deallocation of regexps.  Actually,
>there is no leak either: if you put a call to Gc.major() in the loop,
>memory usage remains constant.

Ah, I thought a Gc.compact () did a major collection (the docs say it 
does), and that didn't help.  Maybe I didn't wait long enough?  I just put 
the Gc.major () in there and waited a long time and it seemed to settle out 
eventually, but only after it had allocated 800MB and had paged the rest of 
the world out, which seems big.  Shouldn't the major collection find that 
there is only one live regexp and collect all of the others, and keep the 
heap way smaller than 800MB?

>What happens is that the speed of the (incremental) major collector
>isn't appropriately adjusted, so it runs too slowly and lets "floating
>garbage" accumulate.  This is a common problem with Caml objects that
>are just handle on resources allocated outside the Caml heap: precise
>determination of the space consumption of the latter is generally
>impossible, causing the incremental major GC to run often too slowly,
>sometimes too fast...

Ah, is there something Str could do to avoid this, like setting the 
mysterious alloc_custom parms differently?  Also, how could I have figured 
this out myself and not bothered the list?

Thanks,
Chris


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners