[Caml-list] Scanning objects outside the OCaml heap

[Caml-list] Scanning objects outside the OCaml heap

[Michael Hicks]

[-- Attachment #1: Type: text/plain, Size: 883 bytes --]

From what I have found on-line (Chap 19 of the OCaml manual, and Chap 20 of
Real-World OCaml), it seems that memory outside of the OCaml heap, but
pointed to by it, is not scanned by the GC.

Does anyone know: Is there any way to make outside memory scannable by the
GC?

The reason I ask:

I'm interested in implementing a custom memory management strategy that I
can prove is safe in a language I will compile to OCaml. I'm thinking I'd
like to allocate freeable memory outside the OCaml heap, but as OCaml
values. Since these values can point to OCaml heap-resident data, they need
to be scanned. But I don't want to have to explicitly register/deregister
them as roots to the GC, or track all mutations to the values, if I can
help it. Rather, having a single "persistent root" to my memory area, for
purposes of scanning, would be ideal.

Thanks in advance for any help,

-Mike

[-- Attachment #2: Type: text/html, Size: 1083 bytes --]

Re: [Caml-list] Scanning objects outside the OCaml heap

[Pierre Chambart]

On 07/07/2015 00:21, Michael Hicks wrote:
>
> I'm interested in implementing a custom memory management strategy
> that I can prove is safe in a language I will compile to OCaml. I'm
> thinking I'd like to allocate freeable memory outside the OCaml heap,
> but as OCaml values. Since these values can point to OCaml
> heap-resident data, they need to be scanned. But I don't want to have
> to explicitly register/deregister them as roots to the GC, or track
> all mutations to the values, if I can help it. Rather, having a single
> "persistent root" to my memory area, for purposes of scanning, would
> be ideal.
>
> Thanks in advance for any help,

There is an undocumented and unexported way to do something like that:
the caml_scan_roots_hook.
You can register a callback in this global variable that takes a
scanning action (another callback) to call
on every of your roots.

See https://github.com/ocaml/ocaml/blob/trunk/byterun/caml/roots.h#L34
for the definition
and https://github.com/ocaml/ocaml/blob/trunk/asmrun/roots.c#L270 as an
example of how the scanning
action should be called.

Since those roots are scanned only at the end of a gc major cycle, you
must handle mutation specificaly (like
mutations in the major heap). You will have to track them as you must
not have unnoticed pointers from outside
of the minor heap to the minor heap. You will have to addapt caml_modify
as it suppose that values not in the minor
are in the major heap. Since it is declared using a weak symbol, this
can be redefined without changing the
runtime. See http://caml.inria.fr/mantis/view.php?id=6084 .

Take care, as this is not exported in the official interface, this could
break at any version change of the runtime.
Have fun with your potential hard to debug segfaults ;)
-- 
Pierre

Re: [Caml-list] Scanning objects outside the OCaml heap

[Oleg]

> Does anyone know: Is there any way to make outside memory scannable by the
> GC?

The library of delimited control delimcc

        http://okmij.org/ftp/continuations/caml-shift.tar.gz

in native mode has to perform a custom scan of pieces of memory stored
outside the OCaml heap. Captured delimited continuation is essentially
a portion of stack. Native OCaml uses stack for boxed and unboxed
values. Thus the captured continuation is a curious data structure
that contains both boxed (OCaml) and unboxed (machine) values. Delimcc
stores captured continuation outside the OCaml heap and has to scan
them at GC time.

The file stacks-native.c in caml-shift.tar.gz shows how the
interaction with GC is done. BTW, when installing a new GC call-back,
don't forget the previously installed one.