[Caml-list] Will the data allocated by caml_alloc be automatically freed by g arbage collector?

[Caml-list] Will the data allocated by caml_alloc be automatically freed by g arbage collector?

[syshen]

[-- Attachment #1: Type: text/plain, Size: 1393 bytes --]


 Dear All:

I am writing a program that include a main loop written in Ocaml and a sub-module written in C. The main loop called the sub-module a lot, and a huge array is returned from each call.

So I use the standard C-Caml interface to return these huge data as shown below:

 extern "C"   value minisat_save_proof(value unit) {     CAMLparam0();     CAMLlocal1( ml_data );     vec<long>& vi=(solver->proof)->save("minisat_save_proof");   int sz=vi.size();   ml_data = caml_alloc (sz,0);   for (int i=0;i<sz;i++) {    Store_field( ml_data, i, Val_int((int)(vi[i])) );   }     CAMLreturn( ml_data ); } 
 In the main ocaml program loop, there is a call to a ocaml method A, which again call this C method minisat_save_proof.
 
 When ocaml method A got these data returned from minisat_save_proof, it call another method B to clear all data structure in the sub-module written in C, and then exit to the main loop and call Gc.compress to collect all garbage.    So in this case I think the memory usage of my program should be the same like before calling minisat_save_proof , because the returned data should be collected by the garbage collector when exiting the method A to the main loop.
 
 But from the unix "top" command, I find that these huge return data seems to remain in memory and consume all my memory step by step.
 
 So I want to know if there is any method to free these data?

[-- Attachment #2: Type: text/html, Size: 1907 bytes --]

Re: [Caml-list] Will the data allocated by caml_alloc be automatically freed by g arbage collector?

[Romain Bardou]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=x-gbk; format=flowed, Size: 1920 bytes --]

Le 01/01/2012 17:01, syshen a ¨¦crit :
>
> Dear All:
>
> I am writing a program that include a main loop written in Ocaml and a
> sub-module written in C. The main loop called the sub-module a lot, and
> a huge array is returned from each call.
>
> So I use the standard C-Caml interface to return these huge data as
> shown below:
>
> extern "C" value minisat_save_proof(value unit) {
> CAMLparam0();
> CAMLlocal1( ml_data );
> vec<long>& vi=(solver->proof)->save("minisat_save_proof");
> int sz=vi.size();
> ml_data = caml_alloc (sz,0);
> for (int i=0;i<sz;i++) {
> Store_field( ml_data, i, Val_int((int)(vi[i])) );
> }
> CAMLreturn( ml_data );
> }
>
> In the main ocaml program loop, there is a call to a ocaml method A,
> which again call this C method minisat_save_proof.
>
> When ocaml method A got these data returned from minisat_save_proof, it
> call another method B to clear all data structure in the sub-module
> written in C, and then exit to the main loop and call Gc.compress to
> collect all garbage.
> So in this case I think the memory usage of my program should be the
> same like before calling minisat_save_proof , because the returned data
> should be collected by the garbage collector when exiting the method A
> to the main loop.
>
> But from the unix "top" command, I find that these huge return data
> seems to remain in memory and consume all my memory step by step.
>
> So I want to know if there is any method to free these data?

Hello,

As far as I know, yes, caml_alloc() allocates its data in the OCaml heap 
and your big value will thus be garbage collected.

Maybe you could check that your value is indeed garbage collected by using:
   Gc.finalise (fun _ -> print_endline "I am being garbage collected") v
where v is the value returned by minisat_save_proof.

If the value is not garbage collected, then maybe you forgot that you 
put the value in some table or something.

Cheers,

Re: [Caml-list] Will the data allocated by caml_alloc be automatically freed by g arbage collector?

[Xavier Leroy]

On 01/01/2012 05:01 PM, syshen wrote:

> So in this case I think the memory usage of my program should be the same
> like before calling minisat_save_proof , because the returned data should be
> collected by the garbage collector when exiting the method A to the main loop.
> But from the unix "top" command, I find that these huge return data seems to
> remain in memory and consume all my memory step by step.

There are several possible explanations:

1- Your program is actually keeping a reference to this big piece of
data, so it cannot be reclaimed.

2- You're falling into PR#5389, http://caml.inria.fr/mantis/view.php?id=5389
whereas OCaml's compactor can fail to return some memory to the malloc()
subsystem.

3- The malloc() implementation on your system elected not to / was
unable to return memory to the OS, hence the numbers reported by
"top" stay constant.

What happens if you run your test in an infinite loop?  Does memory
usage (as reported by "top") stays constant or increases as a function
of time?  Only the latter case is really problematic.

- Xavier Leroy