how to enlarge header of blocks?

how to enlarge header of blocks?

[Christian Lindig]

For tracking additional information about heap blocks I'd like to store  
an additional word in each block. Has anybody experience with changing  
the compiler and run-time system accordingly?

The layout of blocks is defined by #defines in mlvalues.h; however, it  
wasn't obvious for me how to enlarge the header. Also, there must be  
corresponding definitions in the compiler source code.

I have asked this question essentially before but did not receive any  
useful pointers at that time.

	http://groups.google.com/group/fa.caml/browse_frm/thread/ 
b7ca503abf729e9b

-- Christian

Re: [Caml-list] how to enlarge header of blocks?

[Mark Shinwell]

On Fri, Aug 26, 2005 at 10:10:04AM +0200, Christian Lindig wrote:
> For tracking additional information about heap blocks I'd like to store  
> an additional word in each block. Has anybody experience with changing  
> the compiler and run-time system accordingly?

I've had reason to consider such things before in the context of Fresh
O'Caml, but thankfully have so far managed to avoid having to do this:
it is probably quite a serious undertaking, although I imagine it mainly
reduces to adjusting mlvalues.h.

What information do you wish to store?  Rather than adjusting the block
headers, it might be possible to simply tack it onto the end as extra
fields, provided those fields obey the GC structure constraints.  I've
wondered about such schemes to enable me to determine at runtime which
fields of a record were marked mutable in the source code, something
which is not possible at the moment.

Mark

Re: [Caml-list] how to enlarge header of blocks?

[Christian Lindig]

Am 26.08.2005 um 13:34 schrieb Mark Shinwell:

> What information do you wish to store?  Rather than adjusting the block
> headers, it might be possible to simply tack it onto the end as extra
> fields, provided those fields obey the GC structure constraints.

Currently I'd just like to store the address of the allocation site. 
I've tried adding an extra field, hoping that the run-time system would 
never access it. But either I did not get it right, or my assumptions 
about the run-time system were too simple. In any case, the programs 
crashed. I believe this had to do with the representation of arrays but 
no longer remember exactly.

As an alternative, one could maintain a table on the side that maps the 
address of the block to whatever information is associated with it. But 
since the garbage collector moves blocks, this isn't simple to 
maintain, either.

-- Christian

Re: [Caml-list] how to enlarge header of blocks?

[Mark Shinwell]

On Sat, Aug 27, 2005 at 01:17:40PM +0200, Christian Lindig wrote:
> >What information do you wish to store?  Rather than adjusting the block
> >headers, it might be possible to simply tack it onto the end as extra
> >fields, provided those fields obey the GC structure constraints.
> 
> Currently I'd just like to store the address of the allocation site. 
> I've tried adding an extra field, hoping that the run-time system would 
> never access it. But either I did not get it right, or my assumptions 
> about the run-time system were too simple.

If the extra field obeys the GC structure constraints (see the Caml
documentation on interfacing to C) then you shouldn't have any
problem...
 
> As an alternative, one could maintain a table on the side that maps the 
> address of the block to whatever information is associated with it. But 
> since the garbage collector moves blocks, this isn't simple to 
> maintain, either.

Even if the GC were to correctly update the value pointers held in such
a table, such an approach would likely fail anyway as any such GC update
would break the invariants of the table (in particular if the value
pointers are acting as keys in a hashtable, where a change might mean
that values have to switch buckets).  There is the further difficulty
that registering many global memory roots should be avoided due to the
cost, which would affect the design of any such table.

Mark

Re: [Caml-list] how to enlarge header of blocks?

[Christian Lindig]

On Aug 30, 2005, at 1:16 PM, Mark Shinwell wrote:

> If the extra field obeys the GC structure constraints (see the Caml
> documentation on interfacing to C) then you shouldn't have any
> problem...

Maybe we have different things in mind, so I'll elaborate my idea 
briefly and why it was too simple: I changed the allocation routine in 
the bytecode runtime to allocate one extra word where I intended to 
store some information as an Ocaml value. However, I overlooked that 
Array.length and String.length use the block size denoted by the header 
to determine the length. Hence, code looping over an array or string 
would run into my value - and crash. For example, here is the code for 
Array.get:

CAMLprim value caml_array_get_addr(value array, value index)
{
   long idx = Long_val(index);
   if (idx < 0 || idx >= Wosize_val(array)) caml_array_bound_error();
   return Field(array, idx);
}


To make this work I either must refrain from tucking an extra word to 
strings and arrays or change the implementations for the length 
functions and code that checks array and string bounds.

-- Christian

Re: [Caml-list] how to enlarge header of blocks?

[Mark Shinwell]

On Thu, Sep 01, 2005 at 10:07:37AM +0200, Christian Lindig wrote:
> On Aug 30, 2005, at 1:16 PM, Mark Shinwell wrote:
> >If the extra field obeys the GC structure constraints (see the Caml
> >documentation on interfacing to C) then you shouldn't have any
> >problem...
> 
> Maybe we have different things in mind, so I'll elaborate my idea 
> briefly and why it was too simple: I changed the allocation routine in 
> the bytecode runtime to allocate one extra word where I intended to 
> store some information as an Ocaml value. However, I overlooked that 
> Array.length and String.length use the block size denoted by the header 
> to determine the length.

Yes, there does seem to be a problem here; it wouldn't surprise me if
there are other similar pieces of code elsewhere in the runtime as well.
I'm not entirely sure what the best solution is, but I'll be visiting
Rocquencourt next week and will try to ask :-)

Mark