[Caml-list] How is the frametable handled when throwing exceptions?

[Caml-list] How is the frametable handled when throwing exceptions?

[Christoph Höger]

[-- Attachment #1: Type: text/plain, Size: 857 bytes --]

Dear all,

while studying camls native code generation and garbage collection
strategy, I came upon Connor Benner's bachelor thesis from 2012, where he
implemented a llvm backend for ocamlopt. One intriguing remark mentioned
OCamls exception mechanism as basically consisting a pointer to the stack
frame of the last exception handler in a special register (r14, when I
recall correctly). Throwing an exception that becomes a mov/pop/ret
operation. This however, seems to interfere with garbage collection: From
the C-API, it seems that all local roots are stored in the frametable via
some special macros (e.g. CAMLParam, CAMLlocal).

When control just returns from a stack frame, how are the entries removed
from the frametable?

I would be glad, if someone could answer this or point me to the relevant
documentation (if any exists).

thanks,

Christoph

[-- Attachment #2: Type: text/html, Size: 974 bytes --]

Re: [Caml-list] How is the frametable handled when throwing exceptions?

[Nicolás Ojeda Bär]

[-- Attachment #1: Type: text/plain, Size: 1409 bytes --]

Hi Christoph,

Someone who knows more will be able to get a better answer, but I don't
think there is much of an issue here:
when using the C API to register local roots (CAMLparam, CAMLlocal, etc),
these roots are in effect
stored in the C stack.  When raising an exception the stack is _cut down_
and all those roots
are promptly forgotten (as they should be).

Cheers,
Nicolas


On Sat, Feb 11, 2017 at 3:02 PM, Christoph Höger <
christoph.hoeger@celeraone.com> wrote:

> Dear all,
>
> while studying camls native code generation and garbage collection
> strategy, I came upon Connor Benner's bachelor thesis from 2012, where he
> implemented a llvm backend for ocamlopt. One intriguing remark mentioned
> OCamls exception mechanism as basically consisting a pointer to the stack
> frame of the last exception handler in a special register (r14, when I
> recall correctly). Throwing an exception that becomes a mov/pop/ret
> operation. This however, seems to interfere with garbage collection: From
> the C-API, it seems that all local roots are stored in the frametable via
> some special macros (e.g. CAMLParam, CAMLlocal).
>
> When control just returns from a stack frame, how are the entries removed
> from the frametable?
>
> I would be glad, if someone could answer this or point me to the relevant
> documentation (if any exists).
>
> thanks,
>
> Christoph
>

[-- Attachment #2: Type: text/html, Size: 1859 bytes --]

Re: [Caml-list] How is the frametable handled when throwing exceptions?

[Christoph Höger]

[-- Attachment #1: Type: text/plain, Size: 1946 bytes --]

Nicolás,

that is my understanding, too. But what I am missing is the
_de_-registration of the roots. AFAIK, the frametable is a static, global
object and when the stack is cut down, it then contains stale pointers. So
either the frametable is useless (as a global object), because the GC
traverses the call stack or there is some kind of deregstration going on
somewhere.

On Sat, Feb 11, 2017 at 3:10 PM, Nicolás Ojeda Bär <
nicolas.ojeda.bar@lexifi.com> wrote:

> Hi Christoph,
>
> Someone who knows more will be able to get a better answer, but I don't
> think there is much of an issue here:
> when using the C API to register local roots (CAMLparam, CAMLlocal, etc),
> these roots are in effect
> stored in the C stack.  When raising an exception the stack is _cut down_
> and all those roots
> are promptly forgotten (as they should be).
>
> Cheers,
> Nicolas
>
>
> On Sat, Feb 11, 2017 at 3:02 PM, Christoph Höger <
> christoph.hoeger@celeraone.com> wrote:
>
>> Dear all,
>>
>> while studying camls native code generation and garbage collection
>> strategy, I came upon Connor Benner's bachelor thesis from 2012, where he
>> implemented a llvm backend for ocamlopt. One intriguing remark mentioned
>> OCamls exception mechanism as basically consisting a pointer to the stack
>> frame of the last exception handler in a special register (r14, when I
>> recall correctly). Throwing an exception that becomes a mov/pop/ret
>> operation. This however, seems to interfere with garbage collection: From
>> the C-API, it seems that all local roots are stored in the frametable via
>> some special macros (e.g. CAMLParam, CAMLlocal).
>>
>> When control just returns from a stack frame, how are the entries removed
>> from the frametable?
>>
>> I would be glad, if someone could answer this or point me to the relevant
>> documentation (if any exists).
>>
>> thanks,
>>
>> Christoph
>>
>
>

[-- Attachment #2: Type: text/html, Size: 2715 bytes --]

Re: [Caml-list] How is the frametable handled when throwing exceptions?

[Nicolás Ojeda Bär]

[-- Attachment #1: Type: text/plain, Size: 3320 bytes --]

Hi Christoph,

No, the "frametable" is not a single static, global object (it would be
hard to make this work with separate compilation, etc).
Rather each function (both ML and C) is associated to a "frame descriptor"
detailing where to find live values (for ML functions, the frame descriptor
is associated to its function using the return address as a key).  When the
GC is triggered it traverses the stack (both ML and C) and collects the
full list of GC roots.  Of course there is also a list of "global" roots
which *are* registered in a single, global table.  For these roots indeed
one must deregister them explicitly (see caml_{register,remove}_global_root
and its "generational" friends).

Some pointers:

You can read the stack-walking code in

  https://github.com/ocaml/ocaml/blob/trunk/asmrun/roots.c#L438

When the GC is invoked when running ML code, this code depends on some
global pointers which are set up in the asm
glue, e.g.

  https://github.com/ocaml/ocaml/blob/trunk/asmrun/amd64.S

Finally frame descriptors are emitted in the native-code backends, e.g.:

  https://github.com/ocaml/ocaml/blob/trunk/asmcomp/amd64/emit.mlp#L241

Hope this helps,
Nicolas


On Sun, Feb 12, 2017 at 2:25 PM, Christoph Höger <
christoph.hoeger@celeraone.com> wrote:

> Nicolás,
>
> that is my understanding, too. But what I am missing is the
> _de_-registration of the roots. AFAIK, the frametable is a static, global
> object and when the stack is cut down, it then contains stale pointers. So
> either the frametable is useless (as a global object), because the GC
> traverses the call stack or there is some kind of deregstration going on
> somewhere.
>
> On Sat, Feb 11, 2017 at 3:10 PM, Nicolás Ojeda Bär <
> nicolas.ojeda.bar@lexifi.com> wrote:
>
>> Hi Christoph,
>>
>> Someone who knows more will be able to get a better answer, but I don't
>> think there is much of an issue here:
>> when using the C API to register local roots (CAMLparam, CAMLlocal, etc),
>> these roots are in effect
>> stored in the C stack.  When raising an exception the stack is _cut down_
>> and all those roots
>> are promptly forgotten (as they should be).
>>
>> Cheers,
>> Nicolas
>>
>>
>> On Sat, Feb 11, 2017 at 3:02 PM, Christoph Höger <
>> christoph.hoeger@celeraone.com> wrote:
>>
>>> Dear all,
>>>
>>> while studying camls native code generation and garbage collection
>>> strategy, I came upon Connor Benner's bachelor thesis from 2012, where he
>>> implemented a llvm backend for ocamlopt. One intriguing remark mentioned
>>> OCamls exception mechanism as basically consisting a pointer to the stack
>>> frame of the last exception handler in a special register (r14, when I
>>> recall correctly). Throwing an exception that becomes a mov/pop/ret
>>> operation. This however, seems to interfere with garbage collection: From
>>> the C-API, it seems that all local roots are stored in the frametable via
>>> some special macros (e.g. CAMLParam, CAMLlocal).
>>>
>>> When control just returns from a stack frame, how are the entries
>>> removed from the frametable?
>>>
>>> I would be glad, if someone could answer this or point me to the
>>> relevant documentation (if any exists).
>>>
>>> thanks,
>>>
>>> Christoph
>>>
>>
>>
>

[-- Attachment #2: Type: text/html, Size: 5005 bytes --]

Re: [Caml-list] How is the frametable handled when throwing exceptions?

[Stephen Dolan]

[-- Attachment #1: Type: text/plain, Size: 2202 bytes --]

On 11 February 2017 at 14:02, Christoph Höger <
christoph.hoeger@celeraone.com> wrote:

> Dear all,
>
> while studying camls native code generation and garbage collection
> strategy, I came upon Connor Benner's bachelor thesis from 2012, where he
> implemented a llvm backend for ocamlopt. One intriguing remark mentioned
> OCamls exception mechanism as basically consisting a pointer to the stack
> frame of the last exception handler in a special register (r14, when I
> recall correctly). Throwing an exception that becomes a mov/pop/ret
> operation. This however, seems to interfere with garbage collection: From
> the C-API, it seems that all local roots are stored in the frametable via
> some special macros (e.g. CAMLParam, CAMLlocal).
>
> When control just returns from a stack frame, how are the entries removed
> from the frametable?
>

The frametable is a static data structure, which is a hashtable mapping
return addresses in OCaml functions to sets of stack offsets. When the GC
wants to traverse the stack, it walks the stack, picking out the return
address for each stack frame and looking them up in the frametable to find
out which stack slots are live. When OCaml code raises an exception, it
simply discards the stack up to the exception handler.

Code using the C API registers local roots in a different data structure
(caml_local_roots) using the CAMLlocal/param macros. When C code raises an
exception using caml_raise (see asmrun/fail.c), local roots are popped off
caml_local_roots one by one before raising the exception.

The tricky case is when OCaml code raises an exception, but there are C
frames between the top of the stack and the exception handler. To handle
this case, the implementation of caml_callback (calling OCaml from C)
unconditionally adds an exception handler. Should this exception handler
catch anything, it's rethrown with caml_raise.

The effect of this is that if an exception is thrown and the stack has a
mixture of OCaml and C frames before the exception handler, then the
exception will propagate by alternately cutting off OCaml segments of the
stack, and unwinding C segments of the stack.

Stephen

[-- Attachment #2: Type: text/html, Size: 2650 bytes --]