GC question

GC question

[dmitry grebeniuk]

Hello, caml-list.

  I have some objects (custom blocks with method suite attached)
which can be referenced from caml heap.  How to ensure that
finalisation function is called for all non-referenced objects?
I know, call to caml_gc_full_major(Val_unit) will do this work,
but it can be costly when there are many values on the heap
(or not?).  Are there any quicker solution?  (I can make no
assumptions about "age" (young/old) of allocated custom blocks
and about current GC state)

-- 
WBR,
 dmitry                          mailto:gds-mlsts@moldavcable.com

Re: [Caml-list] GC question

[Jean-Christophe Filliatre]

dmitry grebeniuk writes:
 > 
 >   I have some objects (custom blocks with method suite attached)
 > which can be referenced from caml heap.  How to ensure that
 > finalisation function is called for all non-referenced objects?
 > I know, call to caml_gc_full_major(Val_unit) will do this work,
 > but it can be costly when there are many values on the heap
 > (or not?).  Are there any quicker solution?  (I can make no
 > assumptions about "age" (young/old) of allocated custom blocks
 > and about current GC state)

The two integer parameters "used" and "max" passed to alloc_custom are
controlling how fast the custom blocks are reclaimed by the GC, as
explained in section 18.9.2 of the manual
(see http://caml.inria.fr/pub/docs/manual-ocaml/manual032.html).

I don't think that calling caml_gc_full_major by yourself is a good
idea. You should trust the GC; it will eventually reclaim any block
that is not referenced anymore.

Hope this helps,
-- 
Jean-Christophe

Re: [Caml-list] GC question.

[Damien Doligez]

On Oct 25, 2005, at 21:20, MATTHEW HAMMER wrote:

> I'm not certain what style of garbage collection is done in the ocaml
> runtime environment,

Generational with two generations, and incremental for the old
generation.

> but if its generational for example, I'd like to
> move certain values into the 'oldest' generation, so that the garbage
> collector doesn't waste time on them.

The only way to do that is to call Gc.minor, but it won't save any time.

> The running program in question has a large datastructure that is very
> persistent which takes a lot of GC time if it is transversed too
> often.

If it's large, it's going to be in the major heap anyway.  If you
don't want the GC to traverse it, you'll have to put it outside the
heap, but it might will require a lot of work.  For example, bigarrays
are outside the heap.  Or you could write some C code to build your
structure out of malloc blocks, but I'd count that as a desperate
measure.

> So I suppose what I'm asking is: is there a way to give the GC'er
> 'hints' through the Obj module, or any other method?

You can control some parameters with the Gc module, but there's
no way to give hints about a specific piece of data.

-- Damien

Re: [Caml-list] GC question.

[MATTHEW HAMMER]

I'm not certain what style of garbage collection is done in the ocaml
runtime environment, but if its generational for example, I'd like to
move certain values into the 'oldest' generation, so that the garbage
collector doesn't waste time on them.

The running program in question has a large datastructure that is very
persistent which takes a lot of GC time if it is transversed too
often.

So I suppose what I'm asking is: is there a way to give the GC'er
'hints' through the Obj module, or any other method?

Thanks,
Matt

On Tue, Oct 25, 2005 at 08:41:34PM +0200, Damien Doligez wrote:
> 
> On Oct 24, 2005, at 19:53, MATTHEW HAMMER wrote:
> 
> >Is there more documentation on the stuff in the Obj module?
> >What I see/know-about is here:
> >
> >  http://caml.inria.fr/pub/docs/manual-ocaml/libref/Obj.html
> >
> >I'm very interested in getting (write) access to GC-information on
> >ocaml values.
> 
> What kind of GC information do you want to modify?
> 
> -- Damien
> 
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] GC question.

[Damien Doligez]

On Oct 24, 2005, at 19:53, MATTHEW HAMMER wrote:

> Is there more documentation on the stuff in the Obj module?
> What I see/know-about is here:
>
>   http://caml.inria.fr/pub/docs/manual-ocaml/libref/Obj.html
>
> I'm very interested in getting (write) access to GC-information on
> ocaml values.

What kind of GC information do you want to modify?

-- Damien

Re: [Caml-list] GC Question

[Xavier Leroy]

> in the following kind code :
> 
> let l = ... a function building a long list ... in
> let l' = List.map fn l in (* or fold or anything similar *)
> ... no more reference to l ...
> 
> Once the beginning of l has been read to compute l' (assuming List.map 
> starts from the beginning of l) is the GC able to collect the beginning 
> of l ?

Short answer: with ocamlc, no.  With ocamlopt, yes.

Longer answer: in the bytecoded implementation, every value in the VM
stack is a GC root.  "let x = e in e'" pushes the value of e on the
stack just before evaluating e', and pops it at the end of e'.  So,
the value of e remains a live GC root throughout the evaluation of e'.

In the native-code implementation, not all machine stack entries are
GC roots, but only the stack slots that hold a variable of type
address that is live at the point where the GC is called.  ("Live"
here is in the sense of liveness analysis of local variables.)
In your example, "l" is not live across the call to "List.map fn l".

> If not how to write the code to ensure this behaviour of the GC ?

As other mentioned, removing the outer "let" gives the desired GC
behavior even in bytecode:

        List.map fn l (... list builder ...)

In the OCaml sources, you can find this strange-looking idiom:

      let (++) x f = f x

      Pparse.file ppf inputfile Parse.implementation ast_impl_magic_number
      ++ print_if ppf Clflags.dump_parsetree Printast.implementation
      ++ Typemod.type_implementation sourcefile prefixname modulename env
      ++ Translmod.transl_implementation modulename
      ++ print_if ppf Clflags.dump_rawlambda Printlambda.lambda
      ++ Simplif.simplify_lambda
      ++ print_if ppf Clflags.dump_lambda Printlambda.lambda
      ++ Bytegen.compile_implementation modulename
      ++ print_if ppf Clflags.dump_instr Printinstr.instrlist
      ++ Emitcode.to_file oc modulename;

which is a nicer way of writing

      Emitcode.to_file oc modulename
        (print_if ....
          (Bytegen.compile_implementation ...
             (print_if ...
                (...
                ))))

and gives better GC behavior than the obvious:

      let x1 = Pparse.file ppf inputfile Parse.implementation ast_impl_magic_number in
      let x2 = print_if ppf Clflags.dump_parsetree Printast.implementation p x1 in
      let x3 = Typemod.type_implementation sourcefile prefixname modulename env x2 in
      ...

- Xavier Leroy

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] GC Question

[Christophe Raffalli]

[-- Attachment #1: Type: text/plain, Size: 967 bytes --]

Pierre Weis wrote:
>>in the following kind code :
>>
>>let l = ... a function building a long list ... in
>>let l' = List.map fn l in (* or fold or anything similar *)
>>... no more reference to l ...
>>
>>Once the beginning of l has been read to compute l' (assuming List.map 
>>starts from the beginning of l) is the GC able to collect the beginning 
>>of l ?

And

let l' =
   let l = ...   a function building a long list ... in
   List.map fn l
in

should work then (it is simpler to read than composition) ?


-- 
Christophe Raffalli
Université de Savoie
Batiment Le Chablais, bureau 21
73376 Le Bourget-du-Lac Cedex

tél: (33) 4 79 75 81 03
fax: (33) 4 79 75 87 42
mail: Christophe.Raffalli@univ-savoie.fr
www: http://www.lama.univ-savoie.fr/~RAFFALLI
---------------------------------------------
IMPORTANT: this mail is signed using PGP/MIME
At least Enigmail/Mozilla, mutt or evolution
can check this signature
---------------------------------------------

[-- Attachment #2: Type: application/pgp-signature, Size: 252 bytes --]

Re: [Caml-list] GC Question

[Pierre Weis]

> in the following kind code :
> 
> let l = ... a function building a long list ... in
> let l' = List.map fn l in (* or fold or anything similar *)
> ... no more reference to l ...
> 
> Once the beginning of l has been read to compute l' (assuming List.map 
> starts from the beginning of l) is the GC able to collect the beginning 
> of l ?

No it cannot since, AFAIK, l is not freed (i.e. assigned to 0) into
the stack (or local environment) before you leave the block where your
code occurs (normally the activation block of the function where it
appears in).

> If not how to write the code to ensure this behaviour of the GC ?
>
> Christophe Raffalli

Do not use let bindings, write function applications or function
compositions instead.
 (Instead of writing
  let l1 = f l0 in
  let l2 = g l1 in
  h l2
  write
  let i l = h (g (f l)) in
  i l0)

Alternatively, use explicit references to lists and free them
explicitely in your code ! This is tractable when the flow of control
of the code is simple enough (for instance in the case of a
clear succession of passes where data from previous passes are clearly
unused in the following steps).

Hope this helps,

Pierre Weis

INRIA, Projet Cristal, Pierre.Weis@inria.fr, http://pauillac.inria.fr/~weis/


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

[Caml-list] GC Question

[Christophe Raffalli]

[-- Attachment #1: Type: text/plain, Size: 846 bytes --]


in the following kind code :

let l = ... a function building a long list ... in
let l' = List.map fn l in (* or fold or anything similar *)
... no more reference to l ...

Once the beginning of l has been read to compute l' (assuming List.map 
starts from the beginning of l) is the GC able to collect the beginning 
of l ?

If not how to write the code to ensure this behaviour of the GC ?


-- 
Christophe Raffalli
Université de Savoie
Batiment Le Chablais, bureau 21
73376 Le Bourget-du-Lac Cedex

tél: (33) 4 79 75 81 03
fax: (33) 4 79 75 87 42
mail: Christophe.Raffalli@univ-savoie.fr
www: http://www.lama.univ-savoie.fr/~RAFFALLI
---------------------------------------------
IMPORTANT: this mail is signed using PGP/MIME
At least Enigmail/Mozilla, mutt or evolution
can check this signature
---------------------------------------------

[-- Attachment #2: Type: application/pgp-signature, Size: 252 bytes --]