C Callback issues

C Callback issues

[Thomas Fischbacher]

Hello all,

suppose I have a C library which provides functionality via registering C 
callbacks that take a closure parameter. For the sake of this example,
let's say via set_foo_callback(void (*f)(void *),void *closure_param).

Now, if I want to lift this to the OCaml level by writing a C 
callback-wrapper which will use the closure arg to pass the OCaml 
function to be called, I encounter a slight problem:

Callbacks have to be "announced globally" to C by means of 
Callback.register. Now, if I want to use callbacks not only for internal 
purposes in my ML wrapper library, but allow the user of that library to set
callbacks into his own code, the situation is as follows:

Ideally, it would be most convenient if the user could just use the 
set_foo_callback ML variant of the equally named C function and provide a 
ML function as callback. This will then be registered on the fly with
Callback.register and a gensym-ed name if necessary, and then passed on to 
the C side.

The problem with this is the "if necessary" part. I do not really mind the 
issue of callback closures becoming non-GC-able, as we will typically 
only see at most about 100 of them in practically all reasonable programs. 
However, I definitely want to avoid callback-registering the same closure 
more than once, as I can imagine situations where the user wants to 
switch many times between a small set of different callback functions. 
So, what I need for this approach is a way to find out reasonably fast
whether a callback closure has been registered before.

One way to do this would be to keep all registered callbacks in an 
associative list, which is scanned for (==) same-ness whenever a new 
callback has to be registered. A nicer approach would be to use a hash 
table, but here we run into the problem that OCaml does not support proper 
EQ hash tables, and the "workaround" suggested earlier by Xavier does not 
work with functions. Certainly, if on the C side, caml_name_value() is 
implemented in such a way (I don't know) that it has to do an O(N) list 
search instead of a table lookup, there would be no point in trying to 
improve this by using a hash for the Caml-side lookup. Anyway, I think 
this is just another indication that there are situations where EQ hash 
tables really are needed.

Right now, I solved this probem in such a way that I split the 
callback-registering phase off the callback-setting function: 
I introduced a foo_callback type, which is passed into set_foo_callback,
and the user has to use register_foo_callback to map a ML function to a 
foo_callback. Then it's up to him not to re-register the same function 
many times as a callback. But somehow, it just does not feel right having 
to annoy the user of the library with such details, just because of some 
unrelated obscure OCaml limitaion (no EQ hash tables) that requires that 
particular kind of design.

One may indeed argue that the approach I implemented in the end might be 
the cleaner one. I somewhat doubt this, as it should eventually become 
possible to completely shield the end user from low-level details such as 
that somewhere deep in my ocaml library, it has to register C->Caml 
callbacks. And as I said, it just does not feel right to have language 
quirks dictate the design to such an extent.


Another minor issue: the documentation of the callbackN function should 
maybe state more explicitly that the args parameter is of type 
values[] args, and not a ML vector. 

(Yes, one may infer this from the presence of the "number of arguments" 
parameter, which would be spurious if it were a ML vector, but 
nevertheless.)

-- 
regards,               tf@cip.physik.uni-muenchen.de              (o_
 Thomas Fischbacher -  http://www.cip.physik.uni-muenchen.de/~tf  //\
(lambda (n) ((lambda (p q r) (p p q r)) (lambda (g x y)           V_/_
(if (= x 0) y (g g (- x 1) (* x y)))) n 1))                  (Debian GNU)

Re: [Caml-list] C Callback issues

[Olivier Andrieu]

 Thomas Fischbacher [Wednesday 23 November 2005] :
 >
 > 
 > Hello all,
 > 
 > suppose I have a C library which provides functionality via registering C 
 > callbacks that take a closure parameter. For the sake of this example,
 > let's say via set_foo_callback(void (*f)(void *),void *closure_param).
 > 
 > Now, if I want to lift this to the OCaml level by writing a C 
 > callback-wrapper which will use the closure arg to pass the OCaml 
 > function to be called, I encounter a slight problem:
 > 
 > Callbacks have to be "announced globally" to C by means of 
 > Callback.register.

Well, no, not really. Bindings with callbacks usually don't use this
registering mechanism. You can do something like this:

,----
| static void foo_wrapper (void *);
| 
| CAMLprim value
| ml_set_foo_callback (value f)
| {
|   static value closure;
|   if (closure == 0)
|       caml_register_global_root (&closure);
|   closure = f;
|   set_foo_callback (foo_wrapper, &closure);
|   return Val_unit;
| }
| 
| static void
| foo_wrapper (void *data)
| {
|   value *closure = data;
|   value res;
|   res = caml_callback_exn (*closure, Val_unit);
|   /* optionally : */
|    *  if (Is_exception_val (res))
|    *    do_something()
|    */
| }
`----

and on the caml side, it's simply:

  external set_foo_callback : (unit -> unit) -> unit = "ml_set_foo_callback"

You can also malloc the space for the C parameter:

  value *closure;
  closure = caml_stat_alloc (sizeof *closure);
  *closure = f;
  caml_register_global_root (closure);
  ...

which you'd need to free at some point:

  caml_unregister_global_root (closure);
  caml_stat_free (closure);

It depends on how exactly the C API works (e.g. wether specifying NULL
as the function callback is used the unregister the callback).

-- 
   Olivier

Re: [Caml-list] C Callback issues

[Alain Frisch]

Thomas Fischbacher wrote:
> Hello all,
> 
> suppose I have a C library which provides functionality via registering C 
> callbacks that take a closure parameter. For the sake of this example,
> let's say via set_foo_callback(void (*f)(void *),void *closure_param).
> 
> Now, if I want to lift this to the OCaml level by writing a C 
> callback-wrapper which will use the closure arg to pass the OCaml 
> function to be called, I encounter a slight problem:
> 
> Callbacks have to be "announced globally" to C by means of 
> Callback.register. 

I guessed I missed something here. Why do you need to use Callback.register?

The problem I see is that you need to copy the pointer to the OCaml 
closure in a C data structure and register it as a global root. Now, if 
I understand correctly, you'd like to share a global root if the same 
function closure is used several times, and you want to implement this 
sharing efficiently.

1. If you think you'll never have more than a few hundreds closures, you 
can simply do a linear lookup.

2. You can also use the data code pointer as a hash key. It never 
changes, but especially in native code, this might not be a very good 
idea (because of the way curryfication and partial applications are 
treated). You can also hash (with something similar to polymorphic 
hashing) the content of the closure environment.

3. You can use directly the closure pointer as a hash key, but you must
update the hash table when the GC is triggered.

-- Alain

Re: [Caml-list] C Callback issues

[Thomas Fischbacher]

On Wed, 23 Nov 2005, Olivier Andrieu wrote:

> |       caml_register_global_root (&closure);

Ah. Thanks for that clarification!

-- 
regards,               tf@cip.physik.uni-muenchen.de              (o_
 Thomas Fischbacher -  http://www.cip.physik.uni-muenchen.de/~tf  //\
(lambda (n) ((lambda (p q r) (p p q r)) (lambda (g x y)           V_/_
(if (= x 0) y (g g (- x 1) (* x y)))) n 1))                  (Debian GNU)