[Caml-list] using modules to wrap c++ classes

[Caml-list] using modules to wrap c++ classes

[Joel Reymont]

Suppose I have a C++ class with a bunch of methods that act as callbacks.

My callbacks will be hit if I derive the class and implement the
methods I'm interested in.

Now, I would like to wrap this class from OCaml.

This is straightforward from using OCaml classes (section 18.3.5 of
the manual) but is there a way to implement this using modules?

I don't think this is possible but would like to confirm.

    Thanks, Joel

--------------------------------------------------------------------------
Working on AlgoKit, a new algorithmic trading platform using Rithmic R|API
---------------------+------------+---------------------------------------
http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont
---------------------+------------+---------------------------------------

Re: [Caml-list] using modules to wrap c++ classes

[Goswin von Brederlow]

Joel Reymont <joelr1@gmail.com> writes:

> Suppose I have a C++ class with a bunch of methods that act as callbacks.
>
> My callbacks will be hit if I derive the class and implement the
> methods I'm interested in.
>
> Now, I would like to wrap this class from OCaml.
>
> This is straightforward from using OCaml classes (section 18.3.5 of
> the manual) but is there a way to implement this using modules?
>
> I don't think this is possible but would like to confirm.
>
>     Thanks, Joel

From the top of my head:

module M = struct
  type t
  external foo : t -> unit = "caml_mine_M_foo"
end


class M {
  void foo(void) { }
};

value caml_mine_M_foo(value ml_m) {
  CAMLparam1(ml_m);
  M *m = (M*)ml_m;
  m->foo();
  CAMLreturn Val_unit;
}

Wouldn't that do?

It gets harder with inheritance, when you have modules A, B, C for the
different classes and want to be able to pass a C.t to a function
expecting a B.t. You will need to declare a C.as_b function and such.

MfG
        Goswin

Re: [Caml-list] using modules to wrap c++ classes

[Joel Reymont]

Say I have a library class AdmCallbacks

class AdmCallbacks
     {
     public :
     AdmCallbacks();
     virtual ~AdmCallbacks();

     /*   ----------------------------------------------------------------   */

     virtual int Alert(AlertInfo * pInfo,
                       void *      pContext,
                       int *       aiCode) = 0;

     private :
     };

I subclass it and implement the Alert method to receive notifications.

I then create an instance of my derived class and pass it to some
initialization function so that the API knows to call the methods of
my instance.

My derived class may look like this

class MyAdmCallbacks: public AdmCallbacks
     {
     public :
     MyAdmCallbacks()  {};
     ~MyAdmCallbacks() {};

     /*   ----------------------------------------------------------------   */

     virtual int Alert(AlertInfo * pInfo,
                       void *      pContext,
                       int *       aiCode);
     };

int MyAdmCallbacks::Alert(AlertInfo * pInfo,
                          void *      pContext,
                          int *       aiCode)
     {
     int iIgnored;

     /*   ----------------------------------------------------------------   */

     cout << endl << endl;
     if (!pInfo -> dump(&iIgnored))
          {
          cout << "error in pInfo -> dump : " << iIgnored << endl;
          }

     /*   ----------------------------------------------------------------   */

     *aiCode = API_OK;
     return (OK);
     }

Now, suppose I want to implement the derived class on the OCaml side.

What is the best way to do it?

I can derive on the C++ side, keep a pointer to the OCaml instance
(self) in a member variable, instantiate MyAdmCallbacks in the
"MyAdmCallbacks_new" stub function which takes the OCaml self and then
set the member variable.

The Alert method on the C++ side would then dispatch to OCaml using
the stored OCaml self to call the Alert method as per 18.3.5 of the
manual

callback(caml_get_public_method(foo, hash_variant("bar")), foo);

Is there a better way to implement this on the OCaml side?

    Thanks, Joel

--------------------------------------------------------------------------
Working on AlgoKit, a new algorithmic trading platform using Rithmic R|API
---------------------+------------+---------------------------------------
http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont
---------------------+------------+---------------------------------------

Re: [Caml-list] using modules to wrap c++ classes

[Gabriel Scherer]

It looks like this callback pattern does not use recursion through
this/self, but is only one standard way to build closures and
callbacks. The most natural way to expose this to OCaml would be to
allow the OCaml programmer to simply use a first-class function
instead of this callback. I see two possible solutions:

(1) You could define, on the C++ side, a generic GenericAdmCallbacks
class whose constructor would take as parameter a function pointer
implementing Alert (so the function itself only does the plumbing).
Then you can expose this class as an OCaml abstract type, and wrap its
constructor to expose it to OCaml, so that it takes ocaml-land
callbacks and return the abstract type. You can then expose to OCaml
your function manipulating AdmCallbacks (let's call it
"call_me_later"). So you would have an OCaml interface like this

  let callback : abstract_admcallbacks = amdcallback (fun p_info
p_context ai_code -> ...)
  call_me_later callback

(2) You can put that wrapping logic in the wrapper for call_me_later
directly (take an ocaml function, builds the GenericAdmCallbacks, and
pass it to the underlying C++ function)

   call_me_later (fun p_info p_context ai_code -> ....)

If the AmdCallbacks interface really is as simple as you've shown, the
latter technique is the most convenient. If callbacks have a richer
interface (eg. your C++ API expose functions to build callbacks from
other callbacks), you'll need a type to speak about callbacks, so the
first option is better.

(I think I'd go for (1): you can still re-wrap amdcallback into a more
convenient form on the ocaml-land, so that you have as little wrapping
logic as possible in the unsafe C++ wrapping layer.)

On Thu, May 3, 2012 at 3:56 PM, Joel Reymont <joelr1@gmail.com> wrote:
> Say I have a library class AdmCallbacks
>
> class AdmCallbacks
>     {
>     public :
>     AdmCallbacks();
>     virtual ~AdmCallbacks();
>
>     /*   ----------------------------------------------------------------   */
>
>     virtual int Alert(AlertInfo * pInfo,
>                       void *      pContext,
>                       int *       aiCode) = 0;
>
>     private :
>     };
>
> I subclass it and implement the Alert method to receive notifications.
>
> I then create an instance of my derived class and pass it to some
> initialization function so that the API knows to call the methods of
> my instance.
>
> My derived class may look like this
>
> class MyAdmCallbacks: public AdmCallbacks
>     {
>     public :
>     MyAdmCallbacks()  {};
>     ~MyAdmCallbacks() {};
>
>     /*   ----------------------------------------------------------------   */
>
>     virtual int Alert(AlertInfo * pInfo,
>                       void *      pContext,
>                       int *       aiCode);
>     };
>
> int MyAdmCallbacks::Alert(AlertInfo * pInfo,
>                          void *      pContext,
>                          int *       aiCode)
>     {
>     int iIgnored;
>
>     /*   ----------------------------------------------------------------   */
>
>     cout << endl << endl;
>     if (!pInfo -> dump(&iIgnored))
>          {
>          cout << "error in pInfo -> dump : " << iIgnored << endl;
>          }
>
>     /*   ----------------------------------------------------------------   */
>
>     *aiCode = API_OK;
>     return (OK);
>     }
>
> Now, suppose I want to implement the derived class on the OCaml side.
>
> What is the best way to do it?
>
> I can derive on the C++ side, keep a pointer to the OCaml instance
> (self) in a member variable, instantiate MyAdmCallbacks in the
> "MyAdmCallbacks_new" stub function which takes the OCaml self and then
> set the member variable.
>
> The Alert method on the C++ side would then dispatch to OCaml using
> the stored OCaml self to call the Alert method as per 18.3.5 of the
> manual
>
> callback(caml_get_public_method(foo, hash_variant("bar")), foo);
>
> Is there a better way to implement this on the OCaml side?
>
>    Thanks, Joel
>
> --------------------------------------------------------------------------
> Working on AlgoKit, a new algorithmic trading platform using Rithmic R|API
> ---------------------+------------+---------------------------------------
> http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont
> ---------------------+------------+---------------------------------------
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa-roc.inria.fr/wws/info/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

Re: [Caml-list] using modules to wrap c++ classes

[Joel Reymont]

Gabriel,

On Thu, May 3, 2012 at 3:27 PM, Gabriel Scherer
<gabriel.scherer@gmail.com> wrote:

> (1) You could define, on the C++ side, a generic GenericAdmCallbacks
> class whose constructor would take as parameter a function pointer
> implementing Alert (so the function itself only does the plumbing).

There's another class RCallbacks with scores of Alert-like methods, though.

I didn't count all the methods but I'd say about 50.

Then there's another callback class...

> Then you can expose this class as an OCaml abstract type, and wrap its
> constructor to expose it to OCaml, so that it takes ocaml-land
> callbacks and return the abstract type.

I suppose I can have a static table on the C++ side with pointers to
implemented methods.

> You can then expose to OCaml
> your function manipulating AdmCallbacks (let's call it
> "call_me_later").

I guess I can represent the various callbacks for each class as a
variant on the OCaml side that takes a closure per callback.

I think I prefer your approach to my OOP one.

Thanks Gabriel and thank you Goswin!

--------------------------------------------------------------------------
Working on AlgoKit, a new algorithmic trading platform using Rithmic R|API
---------------------+------------+---------------------------------------
http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont
---------------------+------------+---------------------------------------

Re: [Caml-list] using modules to wrap c++ classes

[Joel Reymont]

So I ended up with something like this

--- Callbacks.ml

type t

type 'a callback =
  | Alert of ('a -> Alert.t -> unit)
  | AskQuote of ('a -> Ask.t -> unit)
  | BestAskQuote of ('a -> Ask.t -> unit)
  | BidQuote of ('a -> Bid.t -> unit)
  | BestBidQuote of ('a -> Bid.t -> unit)
  | ClosePrice of ('a -> ClosePrice.t -> unit)
  | ClosingIndicator of ('a -> ClosingIndicator.t -> unit)
  | EndQuote of of ('a -> EndQuote.t -> unit)
  | EquityOptionList of of ('a -> EquityOptionList.t -> unit)
  | EquityOptionStrategyList of of ('a -> EquityOptionStrategyList.t -> unit)
  | HighPrice of ('a -> HighPrice.t -> unit)
  | Indicator of ('a -> Indicator.t -> unit)
  | IndicatorReplay of ('a -> IndicatorReplay.t -> unit)
  | LimitOrderBook of ('a -> LimitOrderBook.t -> unit)
  | LowPrice of ('a -> LowPrice.t -> unit)
  | MarketMode of ('a -> MarketMode.t -> unit)
  | OpenPrice of ('a -> OpenPrice.t -> unit)
  ...

external make : unit -> t = "Callbacks_new"
external set : t -> 'a callback -> unit = "Callbacks_set"

I now want to grab the closure from the constructor on the C side and
stick it into a callback array. Is this sufficient?

extern "C" CAMLprim value
Callbacks_set(value v)
{
  CAMLparam1(v);
  int i = Tag_val(v);
  CAMLlocal1(f);
  f = Field(0, v);
  caml_register_global_root(f);
  callbacks[i] = f;
  CAMLreturn(Val_unit);
}

    Thanks, Joel



--------------------------------------------------------------------------
Working on AlgoKit, a new algorithmic trading platform using Rithmic R|API
---------------------+------------+---------------------------------------
http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont
---------------------+------------+---------------------------------------

Re: [Caml-list] using modules to wrap c++ classes

[Joel Reymont]

I'm doing this to dispatch callbacks on the C side.

Are there any issues with doing it this way?

Is there a better way?

    Thanks, Joel

---

enum
{
  ALERT
};

static CAMLprim value callbacks[] =
  {
    Val_unit // Alert
  };

template<class T>
inline void dispatch(CAMLprim value (*make)(T*), value v, T* p, void* ctx)
{
  if (v != Val_unit)
  {
    caml_c_thread_register();
    CAMLlocal1(o);
    o = make(p);
    callback2(v, (value)ctx, o);
  }
}

class MyAdmCallbacks: public AdmCallbacks
{
 public :

  MyAdmCallbacks()  {};
  ~MyAdmCallbacks() {};

  virtual int Alert(AlertInfo* pInfo, void* pContext, int* aiCode)
  {
    dispatch(Alert_new_from, callbacks[ALERT], pInfo, pContext);
    *aiCode = API_OK;
    return (OK);
  }
};

--------------------------------------------------------------------------
Working on AlgoKit, a new algorithmic trading platform using Rithmic R|API
---------------------+------------+---------------------------------------
http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont
---------------------+------------+---------------------------------------

Re: [Caml-list] using modules to wrap c++ classes

[Goswin von Brederlow]

Joel Reymont <joelr1@gmail.com> writes:

> So I ended up with something like this
>
> --- Callbacks.ml
>
> type t
>
> type 'a callback =
>   | Alert of ('a -> Alert.t -> unit)
>   | AskQuote of ('a -> Ask.t -> unit)
>   | BestAskQuote of ('a -> Ask.t -> unit)
>   | BidQuote of ('a -> Bid.t -> unit)
>   | BestBidQuote of ('a -> Bid.t -> unit)
>   | ClosePrice of ('a -> ClosePrice.t -> unit)
>   | ClosingIndicator of ('a -> ClosingIndicator.t -> unit)
>   | EndQuote of of ('a -> EndQuote.t -> unit)
>   | EquityOptionList of of ('a -> EquityOptionList.t -> unit)
>   | EquityOptionStrategyList of of ('a -> EquityOptionStrategyList.t -> unit)
>   | HighPrice of ('a -> HighPrice.t -> unit)
>   | Indicator of ('a -> Indicator.t -> unit)
>   | IndicatorReplay of ('a -> IndicatorReplay.t -> unit)
>   | LimitOrderBook of ('a -> LimitOrderBook.t -> unit)
>   | LowPrice of ('a -> LowPrice.t -> unit)
>   | MarketMode of ('a -> MarketMode.t -> unit)
>   | OpenPrice of ('a -> OpenPrice.t -> unit)
>   ...
>
> external make : unit -> t = "Callbacks_new"
> external set : t -> 'a callback -> unit = "Callbacks_set"
>
> I now want to grab the closure from the constructor on the C side and
> stick it into a callback array. Is this sufficient?
>
> extern "C" CAMLprim value
> Callbacks_set(value v)
> {
>   CAMLparam1(v);
>   int i = Tag_val(v);
>   CAMLlocal1(f);
>   f = Field(0, v);
>   caml_register_global_root(f);
>   callbacks[i] = f;
>   CAMLreturn(Val_unit);
> }
>
>     Thanks, Joel

This won't work. The value f is a pointer to a closure block butr that
block is under the control of the GC. It will be moved around during
compaction and then your callbacks[i] is invalid.

As discussed on irc you need to create your callbacks[] array as ocaml
block and register that itself as root. That also has the benefit that
you only have to register a single root instead of 50.

For an example you can look at my libfuse bindings:

http://git.ocamlcore.org/cgi-bin/gitweb.cgi?p=libfuse-ocaml/libfuse-ocaml.git;a=tree;f=lib;h=72f44b5090e0d1341d679f52249bbf3974709c1c;hb=HEAD

The idea there is to have a ocaml block with callbacks and a custom
block with finalizer that contains a pointer to the ocaml block mixed
with plain C data. The GC will not inspect the contents of the custom
block so the block of callbacks is registered as global root to keep it
alive and unregistered in the finalizer. Since you are going to use a
static block of calbacks you only need to register the root, it never
dies.

In my case I use a record containing all callbacks to initialize the C
side. With fuse most use cases will set most of the callbacks so having
to set each in turn would be more work. I also use Obj.magic 0 to
initialize callbacks that are unused. An alternative is to use 'a option
types but I decided against that to avoid the extra indirection. Those
callbacks get called a lot and speed is relevant.

Callbacks can be set as needed and the intended use for this is

let my_ops = { default_ops with
    init = my_init;
    destroy = my_destroy;
}

Using the "with" syntx of records one only overrides the callbckas one
wishes to use.

Changing the code to set callbacks individualy is left as an exercise to
the reader. :)

MfG
        Goswin

Re: [Caml-list] using modules to wrap c++ classes

[Joel Reymont]

On Fri, May 4, 2012 at 9:43 AM, Goswin von Brederlow <goswin-v-b@web.de> wrote:

> As discussed on irc you need to create your callbacks[] array as ocaml
> block and register that itself as root. That also has the benefit that
> you only have to register a single root instead of 50.

Assuming that I have a class named MyCallbacks with a public member
callbacks of type value, is this correct?

Also, can I use Is_block to check if there's a closure value stored in
the callbacks block before I dispatch to the closure?

    Thanks, Joel

---

// finalizer, stored in custom_operations

void Callbacks_delete(value v)
{
  CAMLparam1(v);
  MyCallbacks* o = Callbacks_val(v);
  caml_remove_global_root(&o->callbacks);
  delete o;
}

CAMLprim value
Callbacks_new()
{
  CAMLparam0();
  CAMLlocal2(v, cbks);
  v = caml_alloc_custom(&ops, sizeof(MyCallbacks*), 0, 1);
  MyCallbacks* o = new MyCallbacks();
  Callbacks_val(v) = o;
  cbks = caml_alloc(NUM_CBKS, 0);
  o->callbacks = cbks;
  caml_register_global_root(&o->callbacks);
  CAMLreturn(v);
}

extern "C" CAMLprim value
Callbacks_set(value self, value cbk)
{
  CAMLparam2(self, cbk);
  MyCallbacks* o = Callbacks_val(self);
  Store_field(o->callbacks, Tag_val(cbk), Field(0, cbk));
  CAMLreturn(Val_unit);
}

--------------------------------------------------------------------------
Working on AlgoKit, a new algorithmic trading platform using Rithmic R|API
---------------------+------------+---------------------------------------
http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont
---------------------+------------+---------------------------------------

Re: [Caml-list] using modules to wrap c++ classes

[Goswin von Brederlow]

Joel Reymont <joelr1@gmail.com> writes:

> On Fri, May 4, 2012 at 9:43 AM, Goswin von Brederlow <goswin-v-b@web.de> wrote:
>
>> As discussed on irc you need to create your callbacks[] array as ocaml
>> block and register that itself as root. That also has the benefit that
>> you only have to register a single root instead of 50.
>
> Assuming that I have a class named MyCallbacks with a public member
> callbacks of type value, is this correct?

No. The callbacks[] can be public, protect, private or static.  You
could also register the individual callbacks but then you need to
register and deregister all 50 of them seperate. Having just one makes
things easier.

So the "need" above might be to strong. It is just simpler that way. And
I think the GC had some performance issues with too many roots.

> Also, can I use Is_block to check if there's a closure value stored in
> the callbacks block before I dispatch to the closure?

Sure.

>     Thanks, Joel
>
> ---
>
> // finalizer, stored in custom_operations
>
> void Callbacks_delete(value v)
> {
>   CAMLparam1(v);
>   MyCallbacks* o = Callbacks_val(v);
>   caml_remove_global_root(&o->callbacks);
>   delete o;
> }
>
> CAMLprim value
> Callbacks_new()
> {
>   CAMLparam0();
>   CAMLlocal2(v, cbks);
>   v = caml_alloc_custom(&ops, sizeof(MyCallbacks*), 0, 1);
>   MyCallbacks* o = new MyCallbacks();
>   Callbacks_val(v) = o;
>   cbks = caml_alloc(NUM_CBKS, 0);
>   o->callbacks = cbks;
>   caml_register_global_root(&o->callbacks);
>   CAMLreturn(v);
> }
>
> extern "C" CAMLprim value
> Callbacks_set(value self, value cbk)
> {
>   CAMLparam2(self, cbk);
>   MyCallbacks* o = Callbacks_val(self);
>   Store_field(o->callbacks, Tag_val(cbk), Field(0, cbk));
>   CAMLreturn(Val_unit);
> }

Don't you have to use caml_modify() here instead of Store_field()? I
think Store_field is only alowed in freshly allocated blocks.

MfG
        Goswin

Re: [Caml-list] using modules to wrap c++ classes

[Joel Reymont]

On Sat, May 5, 2012 at 1:39 PM, Goswin von Brederlow <goswin-v-b@web.de> wrote:

> Don't you have to use caml_modify() here instead of Store_field()? I
> think Store_field is only alowed in freshly allocated blocks.

Store_field uses caml_modify behind the scenes.

/* convenience macro */
#define Store_field(block, offset, val) do{ \
  mlsize_t caml__temp_offset = (offset); \
  value caml__temp_val = (val); \
  caml_modify (&Field ((block), caml__temp_offset), caml__temp_val); \
}while(0)

--------------------------------------------------------------------------
Working on AlgoKit, a new algorithmic trading platform using Rithmic R|API
---------------------+------------+---------------------------------------
http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont
---------------------+------------+---------------------------------------