[Caml-list] Memory leak in ocaml runtime with backtraces+threads?

[Caml-list] Memory leak in ocaml runtime with backtraces+threads?

[Rob Hoes]

Hello,

I have noticed that an OCaml program with backtrace recording enabled seems to leak memory each time a new thread is created and completes.

For example, the following program, compiled with the `-g` option, will very quickly eat up all your memory:

let _ =
  Printexc.record_backtrace true;
  let rec loop () =
    let t = Thread.create (fun () ->
      try
        raise Not_found
      with Not_found ->
        ()
    ) () in
    Thread.join t;
    loop ()
  in
  loop ()

I’ve tried this on various compilers (4.02.3, 4.01.0 and 3.12.1), on Linux and Mac, native and bytecode, and see the problem on all of them.

Valgrind reports that the allocation of `caml_backtrace_buffer` in the function `caml_stash_backtrace` in asmrun/backtrace_prim.c is the culprit (https://github.com/ocaml/ocaml/blob/trunk/asmrun/backtrace_prim.c#L98):

  if (caml_backtrace_buffer == NULL) {
    Assert(caml_backtrace_pos == 0);
    caml_backtrace_buffer = malloc(BACKTRACE_BUFFER_SIZE
                                   * sizeof(backtrace_slot));
    if (caml_backtrace_buffer == NULL) return;
  }

The `caml_backtrace_buffer` variable is a global variable initialised to NULL in byterun/backtrace.c. However, it seems to be local to a thread (how does it become thread local?), is set to NULL whenever a thread is created (where?), and a new malloc happens when an exception is raised. It is not freed when the thread completes.

Has anyone seen this before, or have suggestions for how to fix it?

Thanks,
Rob

Re: [Caml-list] Memory leak in ocaml runtime with backtraces+threads?

[Gabriel Scherer]

I did not previously know of that problem, sorry.

Could you submit a bugreport on the Mantis bugtracker?
  http://caml.inria.fr/mantis/


On Mon, Apr 11, 2016 at 7:27 AM, Rob Hoes <Rob.Hoes@citrix.com> wrote:
> Hello,
>
> I have noticed that an OCaml program with backtrace recording enabled seems to leak memory each time a new thread is created and completes.
>
> For example, the following program, compiled with the `-g` option, will very quickly eat up all your memory:
>
> let _ =
>   Printexc.record_backtrace true;
>   let rec loop () =
>     let t = Thread.create (fun () ->
>       try
>         raise Not_found
>       with Not_found ->
>         ()
>     ) () in
>     Thread.join t;
>     loop ()
>   in
>   loop ()
>
> I’ve tried this on various compilers (4.02.3, 4.01.0 and 3.12.1), on Linux and Mac, native and bytecode, and see the problem on all of them.
>
> Valgrind reports that the allocation of `caml_backtrace_buffer` in the function `caml_stash_backtrace` in asmrun/backtrace_prim.c is the culprit (https://github.com/ocaml/ocaml/blob/trunk/asmrun/backtrace_prim.c#L98):
>
>   if (caml_backtrace_buffer == NULL) {
>     Assert(caml_backtrace_pos == 0);
>     caml_backtrace_buffer = malloc(BACKTRACE_BUFFER_SIZE
>                                    * sizeof(backtrace_slot));
>     if (caml_backtrace_buffer == NULL) return;
>   }
>
> The `caml_backtrace_buffer` variable is a global variable initialised to NULL in byterun/backtrace.c. However, it seems to be local to a thread (how does it become thread local?), is set to NULL whenever a thread is created (where?), and a new malloc happens when an exception is raised. It is not freed when the thread completes.
>
> Has anyone seen this before, or have suggestions for how to fix it?
>
> Thanks,
> Rob
>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] Memory leak in ocaml runtime with backtraces+threads?

[Rob Hoes]

Sure: http://caml.inria.fr/mantis/view.php?id=7220

Thanks,
Rob

> On 11 Apr 2016, at 13:01, Gabriel Scherer <gabriel.scherer@gmail.com> wrote:
> 
> I did not previously know of that problem, sorry.
> 
> Could you submit a bugreport on the Mantis bugtracker?
>  http://caml.inria.fr/mantis/
> 
> 
> On Mon, Apr 11, 2016 at 7:27 AM, Rob Hoes <Rob.Hoes@citrix.com> wrote:
>> Hello,
>> 
>> I have noticed that an OCaml program with backtrace recording enabled seems to leak memory each time a new thread is created and completes.
>> 
>> For example, the following program, compiled with the `-g` option, will very quickly eat up all your memory:
>> 
>> let _ =
>>  Printexc.record_backtrace true;
>>  let rec loop () =
>>    let t = Thread.create (fun () ->
>>      try
>>        raise Not_found
>>      with Not_found ->
>>        ()
>>    ) () in
>>    Thread.join t;
>>    loop ()
>>  in
>>  loop ()
>> 
>> I’ve tried this on various compilers (4.02.3, 4.01.0 and 3.12.1), on Linux and Mac, native and bytecode, and see the problem on all of them.
>> 
>> Valgrind reports that the allocation of `caml_backtrace_buffer` in the function `caml_stash_backtrace` in asmrun/backtrace_prim.c is the culprit (https://github.com/ocaml/ocaml/blob/trunk/asmrun/backtrace_prim.c#L98):
>> 
>>  if (caml_backtrace_buffer == NULL) {
>>    Assert(caml_backtrace_pos == 0);
>>    caml_backtrace_buffer = malloc(BACKTRACE_BUFFER_SIZE
>>                                   * sizeof(backtrace_slot));
>>    if (caml_backtrace_buffer == NULL) return;
>>  }
>> 
>> The `caml_backtrace_buffer` variable is a global variable initialised to NULL in byterun/backtrace.c. However, it seems to be local to a thread (how does it become thread local?), is set to NULL whenever a thread is created (where?), and a new malloc happens when an exception is raised. It is not freed when the thread completes.
>> 
>> Has anyone seen this before, or have suggestions for how to fix it?
>> 
>> Thanks,
>> Rob
>> 
>> 
>> --
>> Caml-list mailing list.  Subscription management and archives:
>> https://sympa.inria.fr/sympa/arc/caml-list
>> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>> Bug reports: http://caml.inria.fr/bin/caml-bugs