[Caml-list] What's the best practice for bindings for c++ templates?

[Caml-list] What's the best practice for bindings for c++ templates?

[Goswin von Brederlow]

Hi,

I'm halfway done writing a c++ library for a network protocol and
considering writing some bindings for ocaml. The library uses
templates to make objects customizable and am wondering what the best
practice is there.

For a simple example say I have the following:

----------------------------------------------------------------------
template <typename T>
class Foo {
  public:
    Foo(T t);
    ~Foo();
    T swap(T t);
    T get();
};
----------------------------------------------------------------------
type 'a t

val make : 'a -> 'a t
val swap : 'a t -> 'a -> 'a
val get : 'a t -> 'a
----------------------------------------------------------------------

1) For the stub code I need to instantiate the template with 'a, which
obviously doesn't work since the C++ stub has no idea about what type
ocaml will use in the future.

So do I instantiate it with void *? uintptr_t? value?

2) I'm storing ocaml values in C++ structures and getting them back.
How do I make that play nice with the GC?

3) I have the option of changing the libraries API as it's not set in
stone yet. What would be beneficial to interfacing with ocaml (and
maybe python) without making the c++ API to horrible?

E.g. Foo could have:

    void set_T_destructor(void (*destructor)(T *t));

Then ~Foo() would call 'destructor(&t)' to "free" the stored T, which
would unregister it from the GC.

Does that make sense? Is it a bad idea?

Would it be better to add a second template parameter for an Allocator
class that handles interfacing with the GC in the stubs?

Any other little tricks to add to the c++ library to make writing
bindings and working with GCs easier?


What's your experience there?

MfG
	Goswin

Re: [Caml-list] What's the best practice for bindings for c++ templates?

[Xavier Leroy]

On 28/07/2015 20:40, Goswin von Brederlow wrote:
> Hi,
> 
> I'm halfway done writing a c++ library for a network protocol and
> considering writing some bindings for ocaml. The library uses
> templates to make objects customizable and am wondering what the best
> practice is there. [...]
> 
> 1) For the stub code I need to instantiate the template with 'a, which
> obviously doesn't work since the C++ stub has no idea about what type
> ocaml will use in the future.
> 
> So do I instantiate it with void *? uintptr_t? value?
> 
> 2) I'm storing ocaml values in C++ structures and getting them back.
> How do I make that play nice with the GC?

My C++ is rusty, but perhaps you could use the "smart pointer" pattern
to equip OCaml's "value" type with appropriate constructors,
destructors and assignment operations ("appropriate" meaning that they
register and unregister the address of the value with the GC as
needed).  Then, you would instantiate your template with this
"smart value" type.  Just an idea.

- Xavier Leroy

AW: [Caml-list] What's the best practice for bindings for c++ templates?

[Bauer, Christoph]

Hi,

> val make : 'a -> 'a t
> val swap : 'a t -> 'a -> 'a
> val get : 'a t -> 'a
> 1) For the stub code I need to instantiate the template with 'a, which
> obviously doesn't work since the C++ stub has no idea about what type
> ocaml will use in the future.
>
> So do I instantiate it with void *? uintptr_t? value?

1.)  It's value, but I think you ocaml api would need an additional parameter in make to describe the type 'a  (cmp to  float32 in the bigarray library).

> 2) I'm storing ocaml values in C++ structures and getting them back.
> How do I make that play nice with the GC?

My advice: don't do this. It may work for a while but then the GC moves the values and your
program crashes. It would be better to keep the values in OCaml e.g. in a hashtable and to store the key
(integer or string) in C++. With callbacks to ocaml, you could define some  setters/getters for the content
of the value itself. This approach means some overhead, but it should work quite well.

Sometimes it is also a better idea to make just a copy of OCaml value (std::string(String_val(value))).

> 3) I have the option of changing the libraries API as it's not set in stone yet.
> What would be beneficial to interfacing with ocaml (and maybe python)
> without making the c++ API to horrible?
>
> E.g. Foo could have:
>
>     void set_T_destructor(void (*destructor)(T *t));
>
> Then ~Foo() would call 'destructor(&t)' to "free" the stored T, which would
> unregister it from the GC.

So you need something like this (I omit the template stuff):

class Bar {
   ~Bar() {
// call the custom destructor
 }
}

#define CppBar_val(x) (*((Bar**) Data_custom_val(x)))

static
void finalize_cppbar( value v ) {
   delete CppBar_val(v);
}


static struct custom_operations cpp_bar_ops = {
  "de.web.cpp.bar",
  finalize_cppbar,
  custom_compare_default,
  custom_hash_default,
  custom_serialize_default,
  custom_deserialize_default
};


static value alloc_cppbar(Bar * bar)
{
  value v = alloc_custom(&cpp_bar_ops, sizeof(Bar *), 0, 1);
  CppBar_val(v) = bar;
  return v;
}

As Xavier Leroy suggested, you could also use a shared_ptr<T> instead of Bar* and call reset() on it in the finalize_cppbar.
I didn't tried it.

Regards,
Christoph Bauer



-----------------
Siemens Industry Software GmbH; Anschrift: Franz-Geuer-Str. 10, 50823 Köln; Gesellschaft mit beschränkter Haftung; Geschäftsführer: Urban August, Daniel Trebes; Sitz der Gesellschaft: Köln; Registergericht: Amtsgericht Köln, HRB 84564

Re: [Caml-list] What's the best practice for bindings for c++ templates?

[Goswin von Brederlow]

On Wed, Jul 29, 2015 at 12:36:53PM +0000, Bauer, Christoph wrote:
> Hi,
> 
> > val make : 'a -> 'a t
> > val swap : 'a t -> 'a -> 'a
> > val get : 'a t -> 'a
> > 1) For the stub code I need to instantiate the template with 'a, which
> > obviously doesn't work since the C++ stub has no idea about what type
> > ocaml will use in the future.
> >
> > So do I instantiate it with void *? uintptr_t? value?
> 
> 1.)  It's value, but I think you ocaml api would need an additional parameter in make to describe the type 'a  (cmp to  float32 in the bigarray library).

Since make gets an initial 'a as argument the type of 'a t should be
clear. No extra parameter needed.
 
> > 2) I'm storing ocaml values in C++ structures and getting them back.
> > How do I make that play nice with the GC?
> 
> My advice: don't do this. It may work for a while but then the GC moves the values and your
> program crashes. It would be better to keep the values in OCaml e.g. in a hashtable and to store the key
> (integer or string) in C++. With callbacks to ocaml, you could define some  setters/getters for the content
> of the value itself. This approach means some overhead, but it should work quite well.
> 
> Sometimes it is also a better idea to make just a copy of OCaml value (std::string(String_val(value))).

Yeah, I've considered that the GC moves values around. So the address
where c++ stores the value has to be added to the GC root set or there
has to be an indirection, like putting the value into a hashtbl like
you suggest. The later I find ugly so I was looking for a better solution.
 
> > 3) I have the option of changing the libraries API as it's not set in stone yet.
> > What would be beneficial to interfacing with ocaml (and maybe python)
> > without making the c++ API to horrible?
> >
> > E.g. Foo could have:
> >
> >     void set_T_destructor(void (*destructor)(T *t));
> >
> > Then ~Foo() would call 'destructor(&t)' to "free" the stored T, which would
> > unregister it from the GC.
> 
> So you need something like this (I omit the template stuff):
> 
> class Bar {
>    ~Bar() {
> // call the custom destructor
>  }
> }
> 
> #define CppBar_val(x) (*((Bar**) Data_custom_val(x)))
> 
> static
> void finalize_cppbar( value v ) {
>    delete CppBar_val(v);
> }
> 
> 
> static struct custom_operations cpp_bar_ops = {
>   "de.web.cpp.bar",
>   finalize_cppbar,
>   custom_compare_default,
>   custom_hash_default,
>   custom_serialize_default,
>   custom_deserialize_default
> };
> 
> 
> static value alloc_cppbar(Bar * bar)
> {
>   value v = alloc_custom(&cpp_bar_ops, sizeof(Bar *), 0, 1);
>   CppBar_val(v) = bar;
>   return v;
> }
> 
> As Xavier Leroy suggested, you could also use a shared_ptr<T> instead of Bar* and call reset() on it in the finalize_cppbar.
> I didn't tried it.
> 
> Regards,
> Christoph Bauer

I don't think you are seeing this the right way around. Your code
would work if Bar was a C++ structure that you want to keep a
reference to from ocaml. But here I need a C++ structure that keeps a
reference to an ocaml value alive.

And it must be destroyed by either ocaml or c++, whichever comes last.
Maybe something more like this (pseudocode):

class ValueHolder {
  ValueHolder(value val) : val_(val) { gc_register_root(&val); }
  ~ValueHolder() { gc_unregister_root(&val); }
  value val_;
};

Now I can store ValueHolder in C++ and the GC will update val_ when it
moves the value around. Probably better to store a
std::shared_ptr<ValueHolder> because otherwise the move and copy
constructors and assignment operators would need to aquire the runtime
lock and update the GC roots too.

Overall still not the brilliant and clean solution I'm hoping to exist
somewhere out there. :(

MfG
	Goswin