collecting the collector?

collecting the collector?

[Michael Hicks]

I've addressed this mail both to the caml-list and the gclist as both
communities presumably could shed light on this area.  No french version ---
sorry!

I have some code written in OCaml that I've exported to .a file (using the
standard procedure outlined in the OCaml manual, including using
-output-obj); this .a file will obviously include the OCaml runtime system,
and thus its precise collector.  I'm linking this .a file with a C program
that has its own garbage collector, the Hans-Demers-Weiser conservative
collector.

I'm wondering if anyone has any stories from doing something of this sort
before, or could at least speculate on how the two collectors will interact.
I'm presuming that the OCaml collector will be "contained" within the area
governed by the conservative collector, and so that potential mishaps will
arise from the Caml collector doing something pointer unsafe, but that the
conservative collector will not do anything screw up the OCaml one (which
operates only on the Caml code contained in the .a file, and not the rest fo
the C program).

Thanks in advance,
Mike

-- 
Michael Hicks
Ph.D. Candidate, the University of Pennsylvania
http://www.cis.upenn.edu/~mwh            mailto://mwh@dsl.cis.upenn.edu
"Conversion is something momentary; sanctification is the work of a lifetime"
-- J. Escriva, "Christ Is Passing By"

Re: collecting the collector?

[Xavier Leroy]

> I have some code written in OCaml that I've exported to .a file (using the
> standard procedure outlined in the OCaml manual, including using
> -output-obj); this .a file will obviously include the OCaml runtime system,
> and thus its precise collector.  I'm linking this .a file with a C program
> that has its own garbage collector, the Hans-Demers-Weiser conservative
> collector.

I don't know of any OCaml user that actually tried this, but I see no
reason why it shouldn't work.  The OCaml heap is composed of a number
of relatively large blocks (typically 256K), all allocated with malloc()
and chained together for easy reference.  Other data structures for
the OCaml runtime are either in malloc()-ed blocks or in global variables.

A conservative collector a la Boehm-Weiser will just see a bunch of
malloc()-ed blocks, with many pointers from one inside another, and
sufficient chaining to ensure that they are always live as long as the
OCaml code is active.  I can't see anything that could go wrong.

> I'm wondering if anyone has any stories from doing something of this sort
> before, or could at least speculate on how the two collectors will interact.
> I'm presuming that the OCaml collector will be "contained" within the area
> governed by the conservative collector, and so that potential mishaps will
> arise from the Caml collector doing something pointer unsafe,

I'm not sure what you mean by "pointer unsafe" here.  If you're
thinking about those hacks that break conservative collectors, such as
storing the difference or the XOR between two pointers, OCaml doesn't
do any of these.

- Xavier Leroy

Re: collecting the collector?

[Michael Hicks]

I've had a conversation with Hans Boehm on the GC-list regarding this issue,
and so I'll share (a summary of) his comments for this list:

The BDW collector allocates using its own allocator, GC_malloc.  It does not
share heap-space with malloc, and so in principle the two heaps should be
completely separate.  BDW can be compiled to "intercept" malloc calls, in
which case the two heaps would be shared.  Thus, as long as the C code that
calls the Ocaml collector properly registers its roots with the Ocaml
collector, then things should work out fine.

In the case that BDW DOES intercept malloc calls, then things become more
interesting, but as you say, are probably still OK.  In my case, I am using
things with two separate heaps, so this is just speculation on my part.

> A conservative collector a la Boehm-Weiser will just see a bunch of
> malloc()-ed blocks, with many pointers from one inside another, and
> sufficient chaining to ensure that they are always live as long as the
> OCaml code is active.  I can't see anything that could go wrong.

Good to know.  And I presume that if the Ocaml collector notices a block is
no longer needed, then it calls free() on that block, which should inform
the BDW collector to no longer consider that block.  It seems that the
invariant we are shooting for is that when BDW scans the Ocaml section of
the heap, that it should always find the entire heap as live.  Only the
Ocaml collector should initiate the freeing of one of its blocks.  It sounds
like this is what's happening.

Thanks,
Mike
-- 
Michael Hicks
Ph.D. Candidate, the University of Pennsylvania
http://www.cis.upenn.edu/~mwh            mailto://mwh@dsl.cis.upenn.edu
"Conversion is something momentary; sanctification is the work of a lifetime"
-- J. Escriva, "Christ Is Passing By"