Re: fancy GC question

Re: fancy GC question

[Damien Doligez]

>From: Markus Mottl <mottl@miss.wu-wien.ac.at>

>When I allocate an integer array in OCaml, which is always boxed, both
>the pointers to and the elements are obviously contiguous in memory.

There's no pointer in an integer array.


>  int *ar = (int *) &Field(v_ar, 0);

>But is this really always safe if only C writes to the array? What about
>e.g. heap compactions and other GC-actions? Can other effects mess up the
>fact that the pointers map continuously on a contiguous chunk of memory
>(of integers)?

Heap compaction can move the array and break your code (unless you
make sure to reset your ar variable after each compaction).  Future
versions of the GC may move the array under other circumstances.  And
if your array is small enough and was allocated in the minor heap,
then the minor GC will move it too.

>If yes, this would, of course, require the traditional use of the
>"Field"-macro for every access. Otherwise, one could squeeze out a bit
>more performance in some (probably rare) cases.

Make sure you really need that performance and understand the
maintenance cost before using such tricks.

-- Damien

Re: fancy GC question

[Markus Mottl]

On Tue, 12 Dec 2000, Damien Doligez wrote:
> >When I allocate an integer array in OCaml, which is always boxed, both
> >the pointers to and the elements are obviously contiguous in memory.
> 
> There's no pointer in an integer array.

Ok, they are actually not boxed but tagged (= no pointer) - sorry for
the confusion.

> Heap compaction can move the array and break your code (unless you
> make sure to reset your ar variable after each compaction).  Future
> versions of the GC may move the array under other circumstances.  And
> if your array is small enough and was allocated in the minor heap,
> then the minor GC will move it too.

So (as Xavier mentioned) I have to make absolutely sure that no allocation
can happen during manipulation of the array in C.

> Make sure you really need that performance and understand the
> maintenance cost before using such tricks.

In fact, this piece of code had been in use in the PCRE since the very
beginning without ever causing problems. It is only now that I discovered
that I should better ask whether this is really safe: there is indeed
no allocation there so I can leave it as is and safe the allocation of
an extra array for C, which would otherwise add quite a bit of overhead
in the C-interface.

Best regards,
Markus Mottl

-- 
Markus Mottl, mottl@miss.wu-wien.ac.at, http://miss.wu-wien.ac.at/~mottl

fancy GC question

[Markus Mottl]

Hello,

I am not sure whether the following "trick" is really safe, but it seems
to work:

When I allocate an integer array in OCaml, which is always boxed, both
the pointers to and the elements are obviously contiguous in memory.
One could exploit this in C-interfaces under the restriction that the
array is never changed by the OCaml-runtime, e.g.:

  int *ar = (int *) &Field(v_ar, 0);

And then one can read/write directly into the integer array without
having to follow an indirection (an intermediate pointer) by treating
"ar" as a normal C array.

But is this really always safe if only C writes to the array? What about
e.g. heap compactions and other GC-actions? Can other effects mess up the
fact that the pointers map continuously on a contiguous chunk of memory
(of integers)?

If yes, this would, of course, require the traditional use of the
"Field"-macro for every access. Otherwise, one could squeeze out a bit
more performance in some (probably rare) cases.

- Markus Mottl

-- 
Markus Mottl, mottl@miss.wu-wien.ac.at, http://miss.wu-wien.ac.at/~mottl

Re: fancy GC question

[Xavier Leroy]

> When I allocate an integer array in OCaml, which is always boxed, both
> the pointers to and the elements are obviously contiguous in memory.
> One could exploit this in C-interfaces under the restriction that the
> array is never changed by the OCaml-runtime, e.g.:
> 
>   int *ar = (int *) &Field(v_ar, 0);
> 
> And then one can read/write directly into the integer array without
> having to follow an indirection (an intermediate pointer) by treating
> "ar" as a normal C array.
> 
> But is this really always safe if only C writes to the array?

It is safe if the C code does not perform any allocation in the Caml
heap while it is using the "ar" pointer.  An allocation can trigger a
garbage collection, which can move the Caml block denoted by "v_ar";
after this, the "ar" pointer no longer points inside the block!

(If "v_ar" is registered as a local root with the garbage collector,
its value will be updated after the GC to reflect the new address of
the block; however, the GC has no mechanism for updating derived
pointers such as "ar" in your example.)

But, yes, this is a safe trick to use in e.g. a tight loop that does
not allocate in the Caml heap.

> Can other effects mess up the
> fact that the pointers map continuously on a contiguous chunk of memory
> (of integers)?

I'm not sure I understand the question, but the various "fields" of a
Caml block (as accessed with the Field macro) are always contiguous in
memory.

- Xavier Leroy

Re: fancy GC question

[Markus Mottl]

On Tue, 12 Dec 2000, Xavier Leroy wrote:
> It is safe if the C code does not perform any allocation in the Caml
> heap while it is using the "ar" pointer.  An allocation can trigger a
> garbage collection, which can move the Caml block denoted by "v_ar";
> after this, the "ar" pointer no longer points inside the block!

Yes, this should also be avoided. Another problem that comes
to my mind now is that this makes it rather unwise to use the
"enter/leave_blocking_section"-functions when using this trick: another
thread could then trigger a collection while the C-function is still
writing to the array - right?

> > Can other effects mess up the
> > fact that the pointers map continuously on a contiguous chunk of memory
> > (of integers)?
> 
> I'm not sure I understand the question, but the various "fields" of a
> Caml block (as accessed with the Field macro) are always contiguous in
> memory.

Ah, stupid thinking on my side - I get schizophrenic after four o'clock in
the morning and pose several questions in one (everything clear now)... ;)

- Markus

-- 
Markus Mottl, mottl@miss.wu-wien.ac.at, http://miss.wu-wien.ac.at/~mottl