Mastering the garbage collector

Mastering the garbage collector

[Tiphaine Turpin]

Hi,

I have difficulties with the garbage colloector. I have a program that
use a lot of memory to build a small result, before doing something
else. I expected all the memory but the result to be collected when
doing a full_major, but this is not the case. I have tried to
encapsulate all references to the large intermediate data in a function,
compiling with -inline 0, but it does not work. Any idea ? More
generally, is there a specification of the notion of root used by the
garbage collector somewhere ?

Thanks,

Tiphaine

Re: Mastering the garbage collector

[Sylvain Le Gall]

On 25-06-2009, Tiphaine Turpin <Tiphaine.Turpin@irisa.fr> wrote:
> Hi,
>
> I have difficulties with the garbage colloector. I have a program that
> use a lot of memory to build a small result, before doing something
> else. I expected all the memory but the result to be collected when
> doing a full_major, but this is not the case. I have tried to
> encapsulate all references to the large intermediate data in a function,
> compiling with -inline 0, but it does not work. Any idea ? More
> generally, is there a specification of the notion of root used by the
> garbage collector somewhere ?
>

You need to do a "Gc.compact" to size down. A full_major will collect but I
am not sure it will free allocated page.

Regards,
Sylvain Le Gall

Re: [Caml-list] Re: Mastering the garbage collector

[Tiphaine Turpin]

Sylvain Le Gall a écrit :
> You need to do a "Gc.compact" to size down. A full_major will collect but I
> am not sure it will free allocated page.
>   
But if it is just a matter of compaction, then

(Gc.stat ()).Gc.live_words

should give me a correct (small) result, no ?

Tiphaine

Re: [Caml-list] Re: Mastering the garbage collector

[Christophe Raffalli]

[-- Attachment #1.1: Type: text/plain, Size: 1955 bytes --]

Tiphaine Turpin a écrit :
> Sylvain Le Gall a écrit :
>   
>> You need to do a "Gc.compact" to size down. A full_major will collect but I
>> am not sure it will free allocated page.
>>   
>>     
> But if it is just a matter of compaction, then
>
> (Gc.stat ()).Gc.live_words
>
> should give me a correct (small) result, no ?
>
> Tiphaine
>
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>   
You need to make sure that a complete GC cycle has run after producing
your result, before

(Gc.stat ()).Gc.live_words gives you what you want.

The function you need is that one :

val full_major : unit -> unit

Do a minor collection, finish the current major collection cycle, and perform a complete new cycle. This 
will collect all currently unreachable blocks.

You should make sure that variables with values holding large objects
are not syntactically in the scope of any piece of code that can still
be evaluated (I am not sure I am very clear). Here is a small example :


let x = ... in
(* here you are not sure x is collected even if the variable does not
appear after this comment *)
let y = 3 in
...


Cheers,
Christophe

-- 
Christophe Raffalli
Universite de Savoie
Batiment Le Chablais, bureau 21
73376 Le Bourget-du-Lac Cedex

tel: (33) 4 79 75 81 03
fax: (33) 4 79 75 87 42
mail: Christophe.Raffalli@univ-savoie.fr
www: http://www.lama.univ-savoie.fr/~RAFFALLI
---------------------------------------------
IMPORTANT: this mail is signed using PGP/MIME
At least Enigmail/Mozilla, mutt or evolution 
can check this signature. The public key is
stored on www.keyserver.net
---------------------------------------------


[-- Attachment #1.2: Christophe_Raffalli.vcf --]
[-- Type: text/x-vcard, Size: 310 bytes --]

begin:vcard
fn:Christophe Raffalli
n:Raffalli;Christophe
org:LAMA (UMR 5127)
email;internet:christophe.raffalli@univ-savoie.fr
title;quoted-printable:Ma=C3=AEtre de conf=C3=A9rences
tel;work:+33 4 79 75 81 03
note:http://www.lama.univ-savoie.fr/~raffalli
x-mozilla-html:TRUE
version:2.1
end:vcard


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 249 bytes --]

Re: [Caml-list] Re: Mastering the garbage collector

[Tiphaine Turpin]

Christophe Raffalli a écrit :
> You need to make sure that a complete GC cycle has run after producing
> your result, before
>
> (Gc.stat ()).Gc.live_words gives you what you want.
>
> The function you need is that one :
>
> val full_major : unit -> unit
>   
I already do that before querying the live words count.
> let x = ... in
> (* here you are not sure x is collected even if the variable does not
> appear after this comment *)
>   
Thanks for making this clear. I was indeed thinking that collecting x 
here could require some sort of variable liveness analysis, and that the
GC might not be that smart, and this is why I tried to put x in a
function and prevent inlining, but I don't know if it is enough to make
x definitely unreachable.
> let y = 3 in
> ...
>
>
> Cheers,
> Christophe
>
>   
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>   

Re: [Caml-list] Re: Mastering the garbage collector

[Christophe Raffalli]

>> let x = ... in
>> (* here you are not sure x is collected even if the variable does not
>> appear after this comment *)
>>   
>>     
> Thanks for making this clear. I was indeed thinking that collecting x 
> here could require some sort of variable liveness analysis, and that the
> GC might not be that smart, and this is why I tried to put x in a
> function and prevent inlining, but I don't know if it is enough to make
> x definitely unreachable.
>   
If your problem is still not solved, may be you should post your code or 
an example reproducing the problem.

Cheers,
Christophe

Re: [Caml-list] Mastering the garbage collector

[Julien Signoles]

Hello,

More
> generally, is there a specification of the notion of root used by the
> garbage collector somewhere ?

I recommend you the following article. You should find what you want inside.

@inproceedings{DBLP:conf/ml/CuoqD08,
   author    = {Pascal Cuoq and
                Damien Doligez},
   title     = {Hashconsing in an incrementally garbage-collected system:
                a story of weak pointers and hashconsing in ocaml 3.10.2},
   booktitle = {ML},
   year      = {2008},
   pages     = {13-22},
   ee        = {http://doi.acm.org/10.1145/1411304.1411308},
   crossref  = {DBLP:conf/ml/2008},
   bibsource = {DBLP, http://dblp.uni-trier.de}
}

Hope this helps,
Julien Signoles
-- 
Researcher-engineer
CEA LIST, Software Reliability Lab
91191 Gif-Sur-Yvette Cedex
tel:(+33)1.69.08.71.83  fax:(+33)1.69.08.83.95  Julien.Signoles@cea.fr