Re: [Caml-list] out-of-heap data structures [was: Regarding SMP computing]

[Xavier Leroy <Xavier.Leroy@inria.fr>]

> But isn't it true that the GC doesn't follow pointers that point
> outside the OCaml-heap?  In that case it might be conceivable to copy
> OCaml-data that must not be reclaimed into the C-heap.

Yes, this is possible.  For instance, ocamlopt places structured
constants (e.g. constant lists) in the initialized data section
of the executable, outside the Caml heap.

> Of course,
> this would mean that pointers must not point back into the OCaml-heap
> from there, because there is no way the GC could know then that some
> value in the OCaml-heap is still in use, or how to update the pointer
> in the C-heap in case the OCaml-value gets moved around, e.g. during a
> compaction.

... unless you register the locations of back-pointers as global roots.
But be careful that global roots are scanned at every GC (minor as
well as major), therefore a large number of such roots slow down the
GC significantly.

> If the above really works,

There is one caveat: ad-hoc polymorphic primitives (structural
equality and comparisons, marshaling, hashing) will not work on data
structures that reside outside of the Caml heap.  The reason is that
these primitives treat out-of-heap pointers as opaque data.  There is
a special case (the "Is_atom" test) for pointers that correspond to
ocamlopt-generated static data, but extending this special case is
nonobvious.

> I'd be glad to know whether there is
> already functionality to copy OCaml-structures around.

Apparently, Richard Jones is already working on it...  Basically, it's
just like a copying collection with only one root.  You could draw
inspiration from the OCaml minor GC (Cheney-style breadth-first copying)
and from the marshaller (depth-first quasi-copying).

- Xavier Leroy