[Caml-list] GC and interoperability with C

[Caml-list] GC and interoperability with C

[Alexander V. Voinov]

Hi All,

>From what I see in the documentation, I see no way to 'lock' an OCaml
value on the heap so that it can be safely accessed from C code, and
then 'unlock' it, say, in the finalization function of some
Custom_block. In contrast to Python, e.g., where you just increment and
decrement the reference count.

Is this right?

If this is right, the interaction of OCaml to C becomes one-way, you
cannot safely access OCaml world from arbitrary C code.

Is this right?

Alexander
-------------------
Bug reports: http://caml.inria.fr/bin/caml-bugs  FAQ: http://caml.inria.fr/FAQ/
To unsubscribe, mail caml-list-request@inria.fr  Archives: http://caml.inria.fr

Re: [Caml-list] GC and interoperability with C

[Jacques Garrigue]

> From what I see in the documentation, I see no way to 'lock' an OCaml
> value on the heap so that it can be safely accessed from C code, and
> then 'unlock' it, say, in the finalization function of some
> Custom_block. In contrast to Python, e.g., where you just increment and
> decrement the reference count.

Yes, Python uses (used?) reference counting. More clever (and faster)
GC's, like that of caml, have to move things around.

> If this is right, the interaction of OCaml to C becomes one-way, you
> cannot safely access OCaml world from arbitrary C code.

Yes, you can. You just have to register a pointer with the GC, and
everytime the target is moved around, the pointer will be updated.
See
    void register_global_root (value *);
    void remove_global_root (value *);
in memory.h.
After that, you must be careful of always accessing values through the
pointer, not directly.

Jacques Garrigue
-------------------
Bug reports: http://caml.inria.fr/bin/caml-bugs  FAQ: http://caml.inria.fr/FAQ/
To unsubscribe, mail caml-list-request@inria.fr  Archives: http://caml.inria.fr

Re: [Caml-list] GC and interoperability with C

[Alexander V. Voinov]

Hi Jacques,

Jacques Garrigue wrote:
>      value *loca = (value*)malloc(sizeof(value));
>      value *locb = (value*)malloc(sizeof(value));
....
> Now you can put loca and locb directly in you C data structure, the
> real data being accessed by *loca, *locb.
> 
> Of course, don't forget that you must unregister your global roots
> before freeing loca  and locb.

Sure. Thank you!

Alexander
-------------------
Bug reports: http://caml.inria.fr/bin/caml-bugs  FAQ: http://caml.inria.fr/FAQ/
To unsubscribe, mail caml-list-request@inria.fr  Archives: http://caml.inria.fr

Re: [Caml-list] GC and interoperability with C

[Alexander V. Voinov]

Hi All,

Jacques Garrigue wrote:
> Either these values are long ints, and you can just copy them, or they
> are pointers, and you just do as I explained.
> 
> value my_func(value a, value b)
> {
>      value *loca = (value*)malloc(sizeof(value));
>      value *locb = (value*)malloc(sizeof(value));
>      *loca = a; *locb = b;
>      register_global_root(loca);
>      register_global_root(locb);
> 
>      ....
> }
> 
> Now you can put loca and locb directly in you C data structure, the
> real data being accessed by *loca, *locb.
> 
> Of course, don't forget that you must unregister your global roots
> before freeing loca  and locb.

I've played a bit with this, and I have an observation that huge amount
of time is spent on remove_global_root(a). 

I will describe the problem, to let you better understand to context. I
have an implementation of B+-tree, with threading, allowing duplicates
and logarithmic access to the item number. Of course it would be fun to
reimplement it in OCaml, but I have enough fun besides this, and enough
serious tasks for which specifically I started to consider OCaml; also,
I'd like to see on this example how much could I get in the interaction
between C and OCaml; next, the module is tested and used for years, and
I see no point to duplicate the work. If successful, I'd like to make it
available for OCaml community. 

I store arbitrary OCaml objects in the nodes of the tree, using the
advice quoted above. And I see that the performance of an application is
very different with and without remove_global_root(a). Am I doing
something wrong (besides the very idea of building a complex C
structure, containing lots of OCaml objects; let's assume that the idea
is justified in this or that case)?

Thank you in advance

Alexander
-------------------
Bug reports: http://caml.inria.fr/bin/caml-bugs  FAQ: http://caml.inria.fr/FAQ/
To unsubscribe, mail caml-list-request@inria.fr  Archives: http://caml.inria.fr

Re: [Caml-list] GC and interoperability with C

[Xavier Leroy]

> I've played a bit with this, and I have an observation that huge amount
> of time is spent on remove_global_root(a). 

I'm not surprised :-)  Currently, remove_global_root is implemented by
linear search in a singly-linked list...

I believe you're the first to really stress-test remove_global_root;
most OCaml applications just register a dozen external roots and never
remove them.

I'll improve the implementation when I find some free time.

- Xavier Leroy
-------------------
Bug reports: http://caml.inria.fr/bin/caml-bugs  FAQ: http://caml.inria.fr/FAQ/
To unsubscribe, mail caml-list-request@inria.fr  Archives: http://caml.inria.fr

Re: [Caml-list] GC and interoperability with C

[Alexander V. Voinov]

Hi Xavier,

Xavier Leroy wrote:
> 
> > I've played a bit with this, and I have an observation that huge amount
> > of time is spent on remove_global_root(a).
> 
> I'm not surprised :-)  Currently, remove_global_root is implemented by
> linear search in a singly-linked list...
> 
> I believe you're the first to really stress-test remove_global_root;
> most OCaml applications just register a dozen external roots and never
> remove them.
> 
> I'll improve the implementation when I find some free time.

Thank you. At least I'm glad to know that you do not dismiss the very
idea as described and the like: there are many various [data structure]
packages in C/C++, floating around, and in a part of cases it would be
better just to interface them than to reimplement.

As an intermediate solution, if you implement a kind of
'remove_many_roots', taking an array of pointers, I will adapt my code
to make use of this serialization. That is I will accumulate this array
until it overflows and then process in one shot.

Alexander
-------------------
Bug reports: http://caml.inria.fr/bin/caml-bugs  FAQ: http://caml.inria.fr/FAQ/
To unsubscribe, mail caml-list-request@inria.fr  Archives: http://caml.inria.fr

Re: [Caml-list] GC and interoperability with C

[Alexander V. Voinov]

Hi Jacques,

Jacques Garrigue wrote:
> 
> > From what I see in the documentation, I see no way to 'lock' an OCaml
> > value on the heap so that it can be safely accessed from C code, and
> > then 'unlock' it, say, in the finalization function of some
> > Custom_block. In contrast to Python, e.g., where you just increment and
> > decrement the reference count.
> 
> Yes, Python uses (used?) reference counting. More clever (and faster)
> GC's, like that of caml, have to move things around.
> 
> > If this is right, the interaction of OCaml to C becomes one-way, you
> > cannot safely access OCaml world from arbitrary C code.
> 
> Yes, you can. You just have to register a pointer with the GC, and
> everytime the target is moved around, the pointer will be updated.
> See
>     void register_global_root (value *);
>     void remove_global_root (value *);

Thank you. But how can get a pointer to a value, which came as a
parameter, say:

value my_func(value a, value b)
{
   ...
   make a sophisticated C structure, referring to these values...
   ...
   return result;
}

because &a and &b point to the stack which will vanish upon exit. Or do
the CAMLparam wrappers do what is needed in this case?

Alexander
-------------------
Bug reports: http://caml.inria.fr/bin/caml-bugs  FAQ: http://caml.inria.fr/FAQ/
To unsubscribe, mail caml-list-request@inria.fr  Archives: http://caml.inria.fr