[Caml-list] Save callbacks from OCaml to C

[Caml-list] Save callbacks from OCaml to C

[Christoph Höger]

Seems this problem bugs me forever...

I have a scenario, where some C-stubs call back into OCaml, so the stack
(growing downwards) looks like this:

caml_foo ...
/* OCaml control ends here */
...
c_stub ...
c_library_routine (user_data; &callback_wrapper, ...)
...
c_callback_wrapper(user_data; ...)
/* OCaml controlled again */
caml_callbackN(user_data->closure, ....)


The pattern should be somewhat familiar to anyone who has ever combined
C and OCaml in a non-trivial way: I store the actual ocaml callbacks in
the heap-allocated user_data structure, which is guaranteed to be passed
to the callbacks by the c-library.

The callback wrapper then wraps all arguments and uses caml_callback on
these closures.

The callback wrapper is a pure C-routine, i.e. no CamlParamN or
CamlLocalN macros are used here. But still, the garbage collector runs
amok. After a while I see the user_data pointer set to 0x0! How can that
be, how can it overwrite a stack value of a calling C-routine?

Here is the real callback:

void g(void* user_data, double t, double const *x, double *g) {
  const struct interface_data* data = ((struct interface_data*)user_data);
  static long count = 0;
  count++;

  /* Wrap the values in fresh big arrays */
  value ml_t = caml_copy_double(t);
  value ml_x = caml_ba_alloc_dims(CAML_BA_FLOAT64 | CAML_BA_C_LAYOUT, 1,
(double*)x, data->qs->n);
  value ml_g = caml_ba_alloc_dims(CAML_BA_FLOAT64 | CAML_BA_C_LAYOUT, 1,
g, data->qs->mc);

  /* call the OCaml callback */
  caml_callback3(data->g, ml_t, ml_x, ml_g);
}

Is there anything obvious, I am doing wrong?

-- 
Christoph Höger

Technische Universität Berlin
Fakultät IV - Elektrotechnik und Informatik
Übersetzerbau und Programmiersprachen

Sekr. TEL12-2, Ernst-Reuter-Platz 7, 10587 Berlin

Tel.: +49 (30) 314-24890
E-Mail: christoph.hoeger@tu-berlin.de

Re: [Caml-list] Save callbacks from OCaml to C

[Jeremie Dimino]

[-- Attachment #1: Type: text/plain, Size: 1202 bytes --]

On Wed, Feb 3, 2016 at 10:54 AM, Christoph Höger <
christoph.hoeger@tu-berlin.de> wrote:

> void g(void* user_data, double t, double const *x, double *g) {
>   const struct interface_data* data = ((struct interface_data*)user_data);
>   static long count = 0;
>   count++;
>
>   /* Wrap the values in fresh big arrays */
>   value ml_t = caml_copy_double(t);
>   value ml_x = caml_ba_alloc_dims(CAML_BA_FLOAT64 | CAML_BA_C_LAYOUT, 1,
> (double*)x, data->qs->n);
>   value ml_g = caml_ba_alloc_dims(CAML_BA_FLOAT64 | CAML_BA_C_LAYOUT, 1,
> g, data->qs->mc);
>
>   /* call the OCaml callback */
>   caml_callback3(data->g, ml_t, ml_x, ml_g);
> }
>
> Is there anything obvious, I am doing wrong?
>

​You need to register [ml_t], [ml_x] and [ml_g​
] as GC roots. Otherwise if the GC runs in caml_ba_alloc for instance,
[ml_t] might ends up containing garbage even before reaching
[caml_callback3]. You can use the normal macros for that:

void g(...) {
  CAMLparam0();
  CAMLlocal3(ml_t, ml_x, ml_g);
  ...
  CAMLreturn0;
}

​Note that &(user_data->g) must be a GC root as well. Are you registering
&(user_data->g) with the GC in any way?
​
​
-- 
Jeremie

[-- Attachment #2: Type: text/html, Size: 2806 bytes --]

Re: [Caml-list] Save callbacks from OCaml to C

[Malcolm Matalka]

Jeremie Dimino <jdimino@janestreet.com> writes:

> On Wed, Feb 3, 2016 at 10:54 AM, Christoph Höger <
> christoph.hoeger@tu-berlin.de> wrote:
>
>> void g(void* user_data, double t, double const *x, double *g) {
>>   const struct interface_data* data = ((struct interface_data*)user_data);
>>   static long count = 0;
>>   count++;
>>
>>   /* Wrap the values in fresh big arrays */
>>   value ml_t = caml_copy_double(t);
>>   value ml_x = caml_ba_alloc_dims(CAML_BA_FLOAT64 | CAML_BA_C_LAYOUT, 1,
>> (double*)x, data->qs->n);
>>   value ml_g = caml_ba_alloc_dims(CAML_BA_FLOAT64 | CAML_BA_C_LAYOUT, 1,
>> g, data->qs->mc);
>>
>>   /* call the OCaml callback */
>>   caml_callback3(data->g, ml_t, ml_x, ml_g);
>> }
>>
>> Is there anything obvious, I am doing wrong?
>>
>
> ​You need to register [ml_t], [ml_x] and [ml_g​
> ] as GC roots. Otherwise if the GC runs in caml_ba_alloc for instance,
> [ml_t] might ends up containing garbage even before reaching
> [caml_callback3]. You can use the normal macros for that:
>
> void g(...) {
>   CAMLparam0();
>   CAMLlocal3(ml_t, ml_x, ml_g);
>   ...
>   CAMLreturn0;
> }
>
> ​Note that &(user_data->g) must be a GC root as well. Are you registering
> &(user_data->g) with the GC in any way?
> ​
> ​
> -- 
> Jeremie

If one is using ctypes, is all of this taken care of?  I have a library
that registers a bunch of Ocaml functions in C code, which the C code
calls.  I haven't experienced anything bad happening yet, but that
doesn't mean much...

Re: [Caml-list] Save callbacks from OCaml to C

[David Sheets]

On Wed, Feb 3, 2016 at 12:26 PM, Malcolm Matalka <mmatalka@gmail.com> wrote:
> Jeremie Dimino <jdimino@janestreet.com> writes:
>
>> On Wed, Feb 3, 2016 at 10:54 AM, Christoph Höger <
>> christoph.hoeger@tu-berlin.de> wrote:
>>
>>> void g(void* user_data, double t, double const *x, double *g) {
>>>   const struct interface_data* data = ((struct interface_data*)user_data);
>>>   static long count = 0;
>>>   count++;
>>>
>>>   /* Wrap the values in fresh big arrays */
>>>   value ml_t = caml_copy_double(t);
>>>   value ml_x = caml_ba_alloc_dims(CAML_BA_FLOAT64 | CAML_BA_C_LAYOUT, 1,
>>> (double*)x, data->qs->n);
>>>   value ml_g = caml_ba_alloc_dims(CAML_BA_FLOAT64 | CAML_BA_C_LAYOUT, 1,
>>> g, data->qs->mc);
>>>
>>>   /* call the OCaml callback */
>>>   caml_callback3(data->g, ml_t, ml_x, ml_g);
>>> }
>>>
>>> Is there anything obvious, I am doing wrong?
>>>
>>
>> You need to register [ml_t], [ml_x] and [ml_g
>> ] as GC roots. Otherwise if the GC runs in caml_ba_alloc for instance,
>> [ml_t] might ends up containing garbage even before reaching
>> [caml_callback3]. You can use the normal macros for that:
>>
>> void g(...) {
>>   CAMLparam0();
>>   CAMLlocal3(ml_t, ml_x, ml_g);
>>   ...
>>   CAMLreturn0;
>> }
>>
>> Note that &(user_data->g) must be a GC root as well. Are you registering
>> &(user_data->g) with the GC in any way?
>>
>>
>> --
>> Jeremie
>
> If one is using ctypes, is all of this taken care of?  I have a library
> that registers a bunch of Ocaml functions in C code, which the C code
> calls.  I haven't experienced anything bad happening yet, but that
> doesn't mean much...

If you use ctypes and pass OCaml closures to C, you *must* retain a
reference to the closure to avoid it being GCed. If you do not, you
may experience the exception CallToExpiredClosure sporadically.

> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] Save callbacks from OCaml to C

[Jeremie Dimino]

[-- Attachment #1: Type: text/plain, Size: 1694 bytes --]

On Wed, Feb 3, 2016 at 12:01 PM, Christoph Höger <
christoph.hoeger@tu-berlin.de> wrote:

> Thanks for your reply,
>
> Am 03.02.2016 um 12:48 schrieb Jeremie Dimino:
> > void g(...) {
> >   CAMLparam0();
> >   CAMLlocal3(ml_t, ml_x, ml_g);
> >   ...
> >   CAMLreturn0;
> > }
>
> I tried this before, but it seemed like the GC would still write into
> the arguments here. Is the semantics of CAMLparam0 that I might have
> additional, unmanaged arguments?


CAMLparam0 is for when you have no [value] arguments but have some local
[value] variables. CAMLlocalN must be preceded by a CAMLparamY, that's why
you need the CAMLparam0.


> ​Note that &(user_data->g) must be a GC root as well. Are you
> > registering &(user_data->g) with the GC in any way?
>
> Good question. It _is_ an argument to a function on the other side of
> the stack, so in principle it is alive, but could the GC move it?


​Yes, it should be alive. However I imagine that in the main C stub, you
have something like this:

value blah(value g, ...)
{
  ...
  user_data->g = g;
  ...
}

So if the GC ends up moving [g], it needs to know that it must update
[user_data->g] to point to the new location.

There are several solutions:

- if [user_data] only exists for the duration of the [blah] function call,
I would suggest to change [user_data->g] to be a [value*] instead:

value blah(value g, ...)
{
​  CAMLparam1(g);​
​  ...
  user_data->g = &g;
  ...
}

- if it lives for longer, you need to register [user_data->g] as a global
variable, as described in the manual (Rule 4 of
http://caml.inria.fr/pub/docs/manual-ocaml/intfc.html).

-- 
Jeremie

[-- Attachment #2: Type: text/html, Size: 5289 bytes --]

Re: [Caml-list] Save callbacks from OCaml to C

[Christoph Höger]

Am 03.02.2016 um 14:49 schrieb Jeremie Dimino:
> CAMLparam0 is for when you have no [value] arguments but have some local
> [value] variables. CAMLlocalN must be preceded by a CAMLparamY, that's
> why you need the CAMLparam0.

Thanks. I wasn't sure about that, since I tried before and my stack
values were overwritten (I assumed it was the GC at that time).

> So if the GC ends up moving [g], it needs to know that it must update
> [user_data->g] to point to the new location.

Even more subtle. user_data was a pointer to the body of a custom
allocated block. My huge mistake was to use the custom_val directly
(i.e. I allocated n bytes and passed around the pointer to these n
bytes). As it turns out, the GC may move a custom block by memcopying
its body. So in that body I needed pointers to allocated data not the
allocated data itself.

Thanks again!

-- 
Christoph Höger

Technische Universität Berlin
Fakultät IV - Elektrotechnik und Informatik
Übersetzerbau und Programmiersprachen

Sekr. TEL12-2, Ernst-Reuter-Platz 7, 10587 Berlin

Tel.: +49 (30) 314-24890
E-Mail: christoph.hoeger@tu-berlin.de

Re: [Caml-list] Save callbacks from OCaml to C

[Jeremy Yallop]

On 3 February 2016 at 05:44, David Sheets <sheets@alum.mit.edu> wrote:
> On Wed, Feb 3, 2016 at 12:26 PM, Malcolm Matalka <mmatalka@gmail.com> wrote:
>> Jeremie Dimino <jdimino@janestreet.com> writes:
>>> You need to register [ml_t], [ml_x] and [ml_g
>>> ] as GC roots. Otherwise if the GC runs in caml_ba_alloc for instance,
>>> [ml_t] might ends up containing garbage even before reaching
>>> [caml_callback3]. You can use the normal macros for that:
>>>
>> If one is using ctypes, is all of this taken care of?  I have a library
>> that registers a bunch of Ocaml functions in C code, which the C code
>> calls.  I haven't experienced anything bad happening yet, but that
>> doesn't mean much...
>
> If you use ctypes and pass OCaml closures to C, you *must* retain a
> reference to the closure to avoid it being GCed. If you do not, you
> may experience the exception CallToExpiredClosure sporadically.

Besides David's caveat, the answer is yes: ctypes will take care of
registering arguments as GC roots as necessary.

Re: [Caml-list] Save callbacks from OCaml to C

[Malcolm Matalka]

Jeremy Yallop <yallop@gmail.com> writes:

> On 3 February 2016 at 05:44, David Sheets <sheets@alum.mit.edu> wrote:
>> On Wed, Feb 3, 2016 at 12:26 PM, Malcolm Matalka <mmatalka@gmail.com> wrote:
>>> Jeremie Dimino <jdimino@janestreet.com> writes:
>>>> You need to register [ml_t], [ml_x] and [ml_g
>>>> ] as GC roots. Otherwise if the GC runs in caml_ba_alloc for instance,
>>>> [ml_t] might ends up containing garbage even before reaching
>>>> [caml_callback3]. You can use the normal macros for that:
>>>>
>>> If one is using ctypes, is all of this taken care of?  I have a library
>>> that registers a bunch of Ocaml functions in C code, which the C code
>>> calls.  I haven't experienced anything bad happening yet, but that
>>> doesn't mean much...
>>
>> If you use ctypes and pass OCaml closures to C, you *must* retain a
>> reference to the closure to avoid it being GCed. If you do not, you
>> may experience the exception CallToExpiredClosure sporadically.
>
> Besides David's caveat, the answer is yes: ctypes will take care of
> registering arguments as GC roots as necessary.

Can you clarify this a bit?  I'm not that familiar with how the C FFI
works.  If I pass in a closure to a C function and it is registered as a
GC root, doesn't that mean it won't be GCd if my Ocaml program forgets
about it or?

Also, David and I were talking about how to solve this on IRC.  In my
specific case, callbacks are one-shot, which means I know they need to
be remembered until they are called then they can (possibly) be freed.
Is there a nice solution here?  I'd prefer not to store them in some
other data structure and remove them later just to keep a reference
alive, if possible.  That is overhead I'd prefer to avoid, if possible.
I plan on having possibly hundreds of thousands of these callbacks alive
at any point in time.

Thanks

Re: [Caml-list] Save callbacks from OCaml to C

[Jeremy Yallop]

On 3 February 2016 at 12:15, Malcolm Matalka <mmatalka@gmail.com> wrote:
> Jeremy Yallop <yallop@gmail.com> writes:
>
>> On 3 February 2016 at 05:44, David Sheets <sheets@alum.mit.edu> wrote:
>>> On Wed, Feb 3, 2016 at 12:26 PM, Malcolm Matalka <mmatalka@gmail.com> wrote:
>>>> Jeremie Dimino <jdimino@janestreet.com> writes:
>>>>> You need to register [ml_t], [ml_x] and [ml_g
>>>>> ] as GC roots. Otherwise if the GC runs in caml_ba_alloc for instance,
>>>>> [ml_t] might ends up containing garbage even before reaching
>>>>> [caml_callback3]. You can use the normal macros for that:
>>>>>
>>>> If one is using ctypes, is all of this taken care of?  I have a library
>>>> that registers a bunch of Ocaml functions in C code, which the C code
>>>> calls.  I haven't experienced anything bad happening yet, but that
>>>> doesn't mean much...
>>>
>>> If you use ctypes and pass OCaml closures to C, you *must* retain a
>>> reference to the closure to avoid it being GCed. If you do not, you
>>> may experience the exception CallToExpiredClosure sporadically.
>>
>> Besides David's caveat, the answer is yes: ctypes will take care of
>> registering arguments as GC roots as necessary.
>
> Can you clarify this a bit?  I'm not that familiar with how the C FFI
> works.  If I pass in a closure to a C function and it is registered as a
> GC root, doesn't that mean it won't be GCd if my Ocaml program forgets
> about it or?

That's how roots behave, yes: while a value is registered as a root,
the value won't be collected.   There are (roughly speaking) two types
of root in OCaml: local roots, which persist for the duration of a
function call, and global roots, which persist until explicitly
released.  A C function binding written by hand must ensure that OCaml
values passed to it as arguments are registered as local roots, so
that if a collection occurs while the function is running the values
won't be prematurely collected.

A C binding written using ctypes can generally ignore the matter of
roots.  That's partly because ctypes takes care of root registration,
but also because most types passed between OCaml and C in a ctypes
binding are C values, not OCaml values.  For example, if you want to
pass a structure with several fields between OCaml and C there are two
approaches.  One approach is to represent the structure as an OCaml
record, which involves accessing the fields of the value in your C
binding using various macros, taking care to register values as roots
to protect them from the GC.  The other approach is to represent the
structure as a C struct, which involves accessing its fields in OCaml
using the functions ctypes provides.  (If you enjoy programming in an
untyped dialect of C with ubiquitous concurrency, you'll probably
favour the first approach.  If you prefer programming in OCaml then
the second approach might have some appeal.)

Using the C value representation for values that cross the C-OCaml
boundary generally works well, but when things become higher-order,
the situation changes a bit.  When a C library expects to be given a
first-order value such as a struct we have to give it a struct with
the appropriate layout, since C functions can directly access the
representation of values.  However, when the library expects a
function pointer we have a bit more freedom, since the representation
of functions isn't accessible -- in fact, the only thing that can be
done with a function pointer, besides passing it from place to place,
is calling it.  This freedom means that we can pass an OCaml function,
suitably packaged up, where a C function pointer is expected.

Passing OCaml functions to C as function pointers raises some
interesting issues relating to object lifetime and the garbage
collector.  The main difficulty arises from the fact that once you
pass a function pointer to a C library there's no way of knowing how
long the library holds on to it: for example, the library might
discard the function pointer when the call into the library returns,
or it might store the function pointer in a global variable to be
retrieved and called later.  In order to prevent the associated
function from being collected prematurely, some kind of action is
needed on the OCaml side, whether registering a global root, or
ensuring that the function is reachable from the OCaml program.

> Also, David and I were talking about how to solve this on IRC.  In my
> specific case, callbacks are one-shot, which means I know they need to
> be remembered until they are called then they can (possibly) be freed.
> Is there a nice solution here?  I'd prefer not to store them in some
> other data structure and remove them later just to keep a reference
> alive, if possible.

Storing some kind of references to the functions in a place that the
collector can see is essential to prevent the functions from being
collected prematurely.  The situation is the same whether you use
ctypes or write bindings by hand.

Storing the functions in a table, and removing them automatically
after they're called is one approach.  An alternative is to use the
new Ctypes.Roots module, which will be available in the next release:

   https://github.com/ocamllabs/ocaml-ctypes/blob/182a9e64src/ctypes/ctypes.mli#L419-L435

> That is overhead I'd prefer to avoid, if possible.
> I plan on having possibly hundreds of thousands of these callbacks alive
> at any point in time.

In that case it sounds like there'll be an overhead of up to a few megabytes.

Re: [Caml-list] Save callbacks from OCaml to C

[Malcolm Matalka]

Jeremy Yallop <yallop@gmail.com> writes:

> On 3 February 2016 at 12:15, Malcolm Matalka <mmatalka@gmail.com> wrote:
>> Jeremy Yallop <yallop@gmail.com> writes:
>>
>>> On 3 February 2016 at 05:44, David Sheets <sheets@alum.mit.edu> wrote:
>>>> On Wed, Feb 3, 2016 at 12:26 PM, Malcolm Matalka <mmatalka@gmail.com> wrote:
>>>>> Jeremie Dimino <jdimino@janestreet.com> writes:
>>>>>> You need to register [ml_t], [ml_x] and [ml_g
>>>>>> ] as GC roots. Otherwise if the GC runs in caml_ba_alloc for instance,
>>>>>> [ml_t] might ends up containing garbage even before reaching
>>>>>> [caml_callback3]. You can use the normal macros for that:
>>>>>>
>>>>> If one is using ctypes, is all of this taken care of?  I have a library
>>>>> that registers a bunch of Ocaml functions in C code, which the C code
>>>>> calls.  I haven't experienced anything bad happening yet, but that
>>>>> doesn't mean much...
>>>>
>>>> If you use ctypes and pass OCaml closures to C, you *must* retain a
>>>> reference to the closure to avoid it being GCed. If you do not, you
>>>> may experience the exception CallToExpiredClosure sporadically.
>>>
>>> Besides David's caveat, the answer is yes: ctypes will take care of
>>> registering arguments as GC roots as necessary.
>>
>> Can you clarify this a bit?  I'm not that familiar with how the C FFI
>> works.  If I pass in a closure to a C function and it is registered as a
>> GC root, doesn't that mean it won't be GCd if my Ocaml program forgets
>> about it or?
>
> That's how roots behave, yes: while a value is registered as a root,
> the value won't be collected.   There are (roughly speaking) two types
> of root in OCaml: local roots, which persist for the duration of a
> function call, and global roots, which persist until explicitly
> released.  A C function binding written by hand must ensure that OCaml
> values passed to it as arguments are registered as local roots, so
> that if a collection occurs while the function is running the values
> won't be prematurely collected.
>
> A C binding written using ctypes can generally ignore the matter of
> roots.  That's partly because ctypes takes care of root registration,
> but also because most types passed between OCaml and C in a ctypes
> binding are C values, not OCaml values.  For example, if you want to
> pass a structure with several fields between OCaml and C there are two
> approaches.  One approach is to represent the structure as an OCaml
> record, which involves accessing the fields of the value in your C
> binding using various macros, taking care to register values as roots
> to protect them from the GC.  The other approach is to represent the
> structure as a C struct, which involves accessing its fields in OCaml
> using the functions ctypes provides.  (If you enjoy programming in an
> untyped dialect of C with ubiquitous concurrency, you'll probably
> favour the first approach.  If you prefer programming in OCaml then
> the second approach might have some appeal.)
>
> Using the C value representation for values that cross the C-OCaml
> boundary generally works well, but when things become higher-order,
> the situation changes a bit.  When a C library expects to be given a
> first-order value such as a struct we have to give it a struct with
> the appropriate layout, since C functions can directly access the
> representation of values.  However, when the library expects a
> function pointer we have a bit more freedom, since the representation
> of functions isn't accessible -- in fact, the only thing that can be
> done with a function pointer, besides passing it from place to place,
> is calling it.  This freedom means that we can pass an OCaml function,
> suitably packaged up, where a C function pointer is expected.
>
> Passing OCaml functions to C as function pointers raises some
> interesting issues relating to object lifetime and the garbage
> collector.  The main difficulty arises from the fact that once you
> pass a function pointer to a C library there's no way of knowing how
> long the library holds on to it: for example, the library might
> discard the function pointer when the call into the library returns,
> or it might store the function pointer in a global variable to be
> retrieved and called later.  In order to prevent the associated
> function from being collected prematurely, some kind of action is
> needed on the OCaml side, whether registering a global root, or
> ensuring that the function is reachable from the OCaml program.
>
>> Also, David and I were talking about how to solve this on IRC.  In my
>> specific case, callbacks are one-shot, which means I know they need to
>> be remembered until they are called then they can (possibly) be freed.
>> Is there a nice solution here?  I'd prefer not to store them in some
>> other data structure and remove them later just to keep a reference
>> alive, if possible.
>
> Storing some kind of references to the functions in a place that the
> collector can see is essential to prevent the functions from being
> collected prematurely.  The situation is the same whether you use
> ctypes or write bindings by hand.
>
> Storing the functions in a table, and removing them automatically
> after they're called is one approach.  An alternative is to use the
> new Ctypes.Roots module, which will be available in the next release:
>
>    https://github.com/ocamllabs/ocaml-ctypes/blob/182a9e64src/ctypes/ctypes.mli#L419-L435

Thank you for the thorough response.  It seems like Ctypes.Roots might
solve my problem, although the URL gives me a 404.  Do you have an
estimation of when this will be released (or anything someone like
myself can do to help?)

>
>> That is overhead I'd prefer to avoid, if possible.
>> I plan on having possibly hundreds of thousands of these callbacks alive
>> at any point in time.
>
> In that case it sounds like there'll be an overhead of up to a few megabytes.

Any suggestions for a datatype to use here?  I do have an object that is
long lived that represents the event loop I'm integrating against, so I
can store anything I want in there.  Last night I was really concerned
about storing this extra information in the loop, just seemed like a
waste, but in the morning light I'm less worried about it.  I could just
use a Hashtbl I guess with some reference to the closure.  My current
idea is to make some integer value and wrap the closure up in something
like:

(fun () -> Hashtbl.remove t id; closure ())

What kind of sucks about that is the wrapper needs to be unique to each
type of closure that gets called, there doesn't seem like a really
generic way to do this wrapping.  Am I on the wrong track?

Thanks again,
/Malcolm

Re: [Caml-list] Save callbacks from OCaml to C

[Jeremy Yallop]

On 3 February 2016 at 23:26, Malcolm Matalka <mmatalka@gmail.com> wrote:
> Jeremy Yallop <yallop@gmail.com> writes:
>> Storing the functions in a table, and removing them automatically
>> after they're called is one approach.  An alternative is to use the
>> new Ctypes.Roots module, which will be available in the next release:
>>
>>    https://github.com/ocamllabs/ocaml-ctypes/blob/182a9e64src/ctypes/ctypes.mli#L419-L435
>
> Thank you for the thorough response.  It seems like Ctypes.Roots might
> solve my problem, although the URL gives me a 404.

Oops.  Here's a working URL:

https://github.com/ocamllabs/ocaml-ctypes/blob/182a9e64/src/ctypes/ctypes.mli#L419-L434

> Do you have an estimation of when this will be released (or anything someone like
> myself can do to help?)

There are a few issues left to address before the next release:

   https://github.com/ocamllabs/ocaml-ctypes/milestones/ctypes%200.5

Thanks for the offer of help!  Feedback on the Ctypes.Roots design
would be appreciated.  More generally, some of the outstanding issues
might be fixable by a motivated beginner, e.g.

   https://github.com/ocamllabs/ocaml-ctypes/issues/316
   https://github.com/ocamllabs/ocaml-ctypes/issues/267
   https://github.com/ocamllabs/ocaml-ctypes/issues/106

> What kind of sucks about that is the wrapper needs to be unique to each
> type of closure that gets called, there doesn't seem like a really
> generic way to do this wrapping.  Am I on the wrong track?

If you're only storing the closures in the table, and don't need to
retrieve and call them, then you can use a type which hides the type
of the closure in some way.  One approach is to use Obj.t, and convert
each closure using Obj.repr as you store it.  If using Obj makes you
uneasy (as it generally ought to, although it's currently safe in this
case) then an alternative is to use an existential type, like this:

   type t = T : _ -> t

which allows you to wrap any type of value, regardless of its type:

  # [T (+); T not; T 3; T ""];;
  - : t list = [T <poly>; T <poly>; T <poly>; T <poly>]