Understanding GC and Alarms

Understanding GC and Alarms

[Björn Pelzer]

Hello,

I'm new to this list and fairly new to OCaml. First I posted this on the 
beginner's list, but I was recommended to try it here, so here we go. :-)

Right now I'm having some problems with the garbage collector and the 
Gc.alarm - this combination shows some (to me) odd behaviours which are 
not detailed in the reference manual. Maybe someone could shed some 
light on this?

Basically, at the end of a major GC cycle the GC calls the user 
functions defined for any alarms. This I understand, and for the most 
part it works as I expect it.

The first oddity is that I can define an alarm function which starts
another GC cycle (by calling Gc.full_major). This results, quite
understandably, in an infinite loop - GC calls alarm calls GC calls 
alarm....
However, it seems that the GC no longer works quite the same when called 
from within an alarm, as it no longer calls any finalisation functions. 
Normally (outside alarms) the GC will print "Calling finalisation 
functions. Done calling finalisation functions." if I do a Gc.full_major 
and the verbosity is set appropriately. With the alarm loop, the GC will 
only print the first part ("Calling finalisation functions.") once at 
the start of the loop and then begin looping, starting new cycles but no 
new finalisations. If I limit the looping with a counter, the GC will 
behave normally again in future GC cycles outside the alarm. So it seems 
the alarm or the finalising put the GC into a special mode where it no 
longer does everything in a cycle that it would usually do.(?)

The other (and to me more severe) oddity is that if the alarm function 
raises an exception, then the GC seems to remain in its "special mode" 
where it starts no finalisation calling - but now it will also refuse to 
do any alarms.
The latter is a bit of a problem for me right now, as I was using a 
Gc.alarm to implement a check for a memory limit: if the alarm function 
at the end of a GC cycle finds the memory usage to be above the limit, 
an exception is raised which interrupts the normal operation of the 
program, and the program waits for instructions from stdin.

As the first such exception-from-an-alarm effectively breaks future 
alarm handling, this only works once. This was not noticeable in earlier 
versions of the program which were single-use only - the alarm exception 
would halt the program and say sorry, out of memory. The new program 
version is supposed to be user-interactive, though, so it must be able 
to catch multiple memory excesses. Worse, it also uses Unix.ITIMER_REAL 
induced timer exceptions to limit processing time. Potentially such a 
timer exception could also interrupt an alarm check, again breaking the 
alarm handling.

Is there a way to get the GC back to normal after an exception during 
the alarm? Right now it seems I have to drop alarm usage entirely, and 
instead put explicit memory checks into all sorts of strategic places 
throughout the program, which is not all that elegant.

Here is a short program which shows what I mean. I'm probably just 
making some stupid mistake?

exception Ex

let main () =
    let cnt = ref 0
    in
    (Gc.set {(Gc.get()) with Gc.verbose = 0x081});
    let alarm_function () =
       (print_endline "Alarm!";
       cnt := !cnt + 1;
(* do 3 loop iterations *)
       if !cnt < 3 then
          (Gc.full_major ())
(* after the 3rd iteration, exit loop with an exception *)
       else
          (raise Ex))
       in
          ignore (Gc.create_alarm alarm_function);
          try
(* this will call the alarm and start the loop *)
             (Gc.full_major ())
          with
             | Ex ->
                (print_endline "Exception raised!";
(* new GC cycle calls no alarm after the exception :-( *)
                Gc.full_major ())

let () = main ();;


Thank you for your time,
Björn


-- 
Björn Pelzer
AGKI - Artificial Intelligence Research Group
University Koblenz-Landau, B 225
http://www.uni-koblenz.de/~bpelzer

Tel.(office): (+49) 261 287 2776
Tel.(home): (+49) 261 942 3908

Re: [Caml-list] Understanding GC and Alarms

[Damien Doligez]

Hello,

On 2009-08-04, at 15:39, Björn Pelzer wrote:

> With the alarm loop, the GC will only print the first part ("Calling  
> finalisation functions.") once at the start of the loop and then  
> begin looping, starting new cycles but no new finalisations.

Yes, the GC calls the finalisation functions in order, so it waits for  
your
finalisation function to finish before starting the next one.  If your
function doesn't terminate...

> The other (and to me more severe) oddity is that if the alarm  
> function raises an exception, then the GC seems to remain in its  
> "special mode" where it starts no finalisation calling - but now it  
> will also refuse to do any alarms.

Yes, that is the essence of bug report 4742:
< http://caml.inria.fr/mantis/view.php?id=4742 >

> Is there a way to get the GC back to normal after an exception  
> during the alarm?

That is the purpose of the Gc.finalise_release function: tell the GC  
to behave
as if the current finalisation function had finished.  Note that  
finalise_release
is also safe to call from outside a finalisation function, so you can  
call it
from your exception handler.

When bug 4742 is fixed (probably in 3.12.0), you won't need it any more.
Note that I wanted to fix it by ignoring the exception entirely, but
your use case made me change my mind.

-- Damien

Re: [Caml-list] Understanding GC and Alarms

[Björn Pelzer]

Hello Damien!

Damien Doligez wrote:
> Yes, the GC calls the finalisation functions in order, so it waits for your
> finalisation function to finish before starting the next one.  If your
> function doesn't terminate...
> 

Ok, makes sense.

>> Is there a way to get the GC back to normal after an exception during 
>> the alarm?
> 
> That is the purpose of the Gc.finalise_release function: tell the GC to 
> behave
> as if the current finalisation function had finished.  Note that 
> finalise_release
> is also safe to call from outside a finalisation function, so you can 
> call it
> from your exception handler.

Thank you very much! Very helpful, works as you say.
In hindsight I should have figured this out from the manual, but, well, 
I didn't. :-)


> When bug 4742 is fixed (probably in 3.12.0), you won't need it any more.
> Note that I wanted to fix it by ignoring the exception entirely, but
> your use case made me change my mind.

Sounds good, keeping the exception handling would be useful, at least 
for me. Using the release-function as you explained is not much of a 
bother, in particular now that I get how it works, so there is no 
pressing need for a change/bugfix from my side.

Thanks again and best regards,
Björn


-- 
Björn Pelzer
AGKI - Artificial Intelligence Research Group
University Koblenz-Landau, B 225
http://www.uni-koblenz.de/~bpelzer

Tel.(office): (+49) 261 287 2776
Tel.(home): (+49) 261 942 3908