[Caml-list] Problem with GC and custom blocks

[Caml-list] Problem with GC and custom blocks

[Falk Hueffner]

Hi,

I'm having trouble with the GC destroying my custom blocks, although
it looks like I have properly protected them. Unfortunately, I cannot
reproduce it with a small testcase. Here's part of the source and its
output:

static inline value alloc_graph(unsigned n,
				unsigned vertex_bits, unsigned edge_bits)
{
    value r = alloc_custom((struct custom_operations *) &graph_ops,
			   1 << 16, // huge value to trigger bug earlier
			   0, 1);
    long *l = Data_custom_val(r);
    for (int i = 0; i < 32/4; ++i)
	l[i] = 0xefbeadde;
    dumpbytes(r, 32, __PRETTY_FUNCTION__);
    return r;
}
static inline value clone_graph(const struct graph *g)  {
    dumpbytes(((char*) g)  - 4, 32, __PRETTY_FUNCTION__ " in");

    value gv = alloc_graph(g->n, g->vertex_bits, g->edge_bits);
    dumpbytes(((char*) g)  - 4, 32, __PRETTY_FUNCTION__ " post");

    make_copy(Data_custom_val(gv), g);
    dumpbytes(gv, 32, __PRETTY_FUNCTION__ " out");
    return gv;
}

// set_connected: graph -> int -> int -> graph
value set_connected_c(value gv, value iv, value jv) {
    fprintf(stderr, "protecting %p\n", gv);
    CAMLparam3(gv, iv, jv);
    dumpbytes(gv, 32, __PRETTY_FUNCTION__);

    const struct graph *gin = Data_custom_val(gv);
    ulong i = Long_val(iv);
    ulong j = Long_val(jv);
    CAMLlocal1(goutv);
    goutv = clone_graph(gin);
    struct graph *gout = Data_custom_val(goutv);

    set_connected(gout, i, j);
    dumpbytes(goutv, 32, __PRETTY_FUNCTION__ " out");

    CAMLreturn(goutv);
}

[...]
protecting 0x400c8fec
0x400c8fec      set_connected_c 80 e9 07 08 08 00 01 01 00 00 00 00 00 00
00 00
0x400c8fec       clone_graph in 80 e9 07 08 08 00 01 01 00 00 00 00 00 00
00 00
<>allocated_words = 32790
extra_heap_memory = 0u
amount of work to do = 134084u
ordered work = 0 words
computed work = 221332 words
Sweeping 221332 words
$FL size at phase change = 168445
Estimated overhead = 1000000%
Automatic compaction triggered.
Starting new major GC cycle
Marking 2147483647 words
Sweeping 2147483647 words
Measured overhead: 137%
Compacting heap...
Shrinking heap to 2976k bytes
Shrinking heap to 2728k bytes
Shrinking heap to 2480k bytes
done.
0x400ed004          alloc_graph 80 e9 07 08 de ad be ef de ad be ef de ad
be ef
0x400c8fec     clone_graph post 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00

So 0x400c8fec gets mangled, although I protected it. Did I do
something obvious wrong? Or are there any known bugs in the GC? Any
other suggestions how to track this down? I'd be really grateful even for
a workaround...

I'm using ocamlopt 3.06 on i386 Linux, with gcc 3.2.

	Falk

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Problem with GC and custom blocks

[Damien Doligez]

On Tuesday, March 25, 2003, at 11:31 AM, Falk Hueffner wrote:

> I'm having trouble with the GC destroying my custom blocks, although
> it looks like I have properly protected them. Unfortunately, I cannot
> reproduce it with a small testcase.

You seem to misunderstand the purpose of the CAMLparam and CAMLlocal
macros.  The GC is not destroying your custom blocks, it is moving
them.  Because it is moving them, it has to update all pointers
to them.  Hence, it is the *pointers* (in fact, the local variables
that contain these pointers) that need to be "protected".


> static inline value alloc_graph(unsigned n,
> 				unsigned vertex_bits, unsigned edge_bits)
> {
>     value r = alloc_custom((struct custom_operations *) &graph_ops,
> 			   1 << 16, // huge value to trigger bug earlier
> 			   0, 1);
>     long *l = Data_custom_val(r);
>     for (int i = 0; i < 32/4; ++i)
> 	l[i] = 0xefbeadde;
>     dumpbytes(r, 32, __PRETTY_FUNCTION__);
>     return r;
> }

No problem here.  Note that alloc_graph calls an allocation function,
and thus may move blocks around.

> static inline value clone_graph(const struct graph *g)  {
>     dumpbytes(((char*) g)  - 4, 32, __PRETTY_FUNCTION__ " in");
>
>     value gv = alloc_graph(g->n, g->vertex_bits, g->edge_bits);
>     dumpbytes(((char*) g)  - 4, 32, __PRETTY_FUNCTION__ " post");
>
>     make_copy(Data_custom_val(gv), g);
>     dumpbytes(gv, 32, __PRETTY_FUNCTION__ " out");
>     return gv;
> }

Here, you have a problem if g is a pointer into the heap, because you
call alloc_graph, which may move heap blocks.

> // set_connected: graph -> int -> int -> graph
> value set_connected_c(value gv, value iv, value jv) {
>     fprintf(stderr, "protecting %p\n", gv);
>     CAMLparam3(gv, iv, jv);
>     dumpbytes(gv, 32, __PRETTY_FUNCTION__);
>
>     const struct graph *gin = Data_custom_val(gv);
>     ulong i = Long_val(iv);
>     ulong j = Long_val(jv);
>     CAMLlocal1(goutv);
>     goutv = clone_graph(gin);
>     struct graph *gout = Data_custom_val(goutv);
>
>     set_connected(gout, i, j);
>     dumpbytes(goutv, 32, __PRETTY_FUNCTION__ " out");
>
>     CAMLreturn(goutv);
> }

And here you trigger the problem by passing Data_custom_val(gv)
as argument to clone_graph.

What you should do is:
1. modify clone_graph to take a parameter of type "value"
2. protect this parameter with CAMLparam1

Even if another copy of the same address is protected in
"set_connected_c" under the name of "gv", you still need to protect
it under its new name in clone_graph, so the GC will be able to
update both copies when it moves the data.

Also, note that you cannot protect an infix pointer (a pointer
that points inside a block instead of pointing to its head).
That makes Data_custom_val rather error-prone and you should
avoid storing its result in a variable or passing it as argument
to functions that can allocate in the Caml heap.

> So 0x400c8fec gets mangled, although I protected it. Did I do
> something obvious wrong? Or are there any known bugs in the GC? Any
> other suggestions how to track this down? I'd be really grateful even 
> for
> a workaround...

Please let me know if this helps.

-- Damien

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Problem with GC and custom blocks

[Falk Hueffner]

Damien Doligez <Damien.Doligez@inria.fr> writes:

> What you should do is:
> 1. modify clone_graph to take a parameter of type "value"
> 2. protect this parameter with CAMLparam1
> 
> Even if another copy of the same address is protected in
> "set_connected_c" under the name of "gv", you still need to protect
> it under its new name in clone_graph, so the GC will be able to
> update both copies when it moves the data.
> 
> Please let me know if this helps.

It helps. It's actually pretty obvious now that I know it, I was just
somehow fixed to the idea that the GC considered the object dead
erraneously, and did not consider moving... Thanks!

-- 
	Falk

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners