CamlIDL - stub code generator and COM binding for OCaml

CamlIDL - stub code generator and COM binding for OCaml

[Xavier Leroy]

Just like everyone else who has read the papers on H/Direct by Meijer,
Peyton-Jones et al, I have been working lately on a COM binding for
Objective Caml.  I have just released one of the preliminary results
of this work: CamlIDL 1.00.  Sources and documentation are available from

  ftp://ftp.inria.fr/lang/caml-light/bazar-ocaml/camlidl-1.00.tar.gz
  ftp://ftp.inria.fr/lang/caml-light/bazar-ocaml/camlidl-1.00.doc.ps.gz
  ftp://ftp.inria.fr/lang/caml-light/bazar-ocaml/camlidl-1.00.doc.html.tar.gz

The documentation can also be browsed at

  http://caml.inria.fr/camlidl/htmlman/

CamlIDL comprises two parts:

- A stub code generator that generates the C stub code required for
  the Caml/C interface, based on an MIDL specification.  (MIDL stands
  for Microsoft's Interface Description Language; it looks like
  C header files with some extra annotations, plus a notion of object
  interfaces that look like C++ classes without inheritance.)

- A (currently small) library of functions and tools to import COM
  components in Caml applications, and export Caml code as COM
  components.

CamlIDL can be used in several ways:

- Under Unix and Windows, as a stub code generator for using C
  libraries from Caml.  It takes care of the gory details of
  the Caml/C interface, such as all those pesky Val_xxx and Xxx_val
  macros, proper registration of GC roots, the different calling
  conventions from bytecode and from native-code, etc.

- Under Unix and Windows, as a simple interface between Caml classes
  and C++ classes.  The C++ fragment supported is that corresponding
  to COM interfaces, i.e. interface polymorphism, but not inheritance.
  Communication in both directions (COM/C++ object to Caml object
  and Caml object to COM/C++ object) is provided.

- Under Windows, as a COM binding for Caml.  There are two applications:

  * Use COM components from Caml applications.  This should eventually
    allow Caml code to exploit many Windows libraries and "script"
    many Windows applications.  See the H/Direct papers for nice
    examples.  This hasn't been tried yet with CamlIDL, because of
    the incredible lack of documentation on Microsoft's components
    and on Microsoft's IDL extensions they use in their interfaces.

  * Encapsulate Caml code in COM components that can be used in
    other applications.  This offers great potential for writing
    the juicy bits of an application in OCaml and use C++ or Visual
    Basic or Internet Explorer or whichever Microsoft behemoth to do
    the GUI and other boring parts.  Currently, CamlIDL can create
    "in-proc servers" for Caml components (i.e. DLLs that are registered
    in the system registry and loaded automatically in applications
    that use the components).  "Out-of-proc" servers and distributed
    objects are not supported yet.  Currently, only native interfaces
    are supported, but not yet dispatch interfaces.

While the stub generation aspects of CamlIDL are already relatively
mature, the COM binding is still very preliminary.  My hope in
releasing it is that others more experienced with COM than I am will
try it on real-world applications and let me know what is wrong or
missing.

- Xavier Leroy

Re: CamlIDL - stub code generator and COM binding for OCaml

[Andrew Martin]

This looks great!.

I have been thinking of writing an interface between Ocaml and the GTK.  This
tool should simplify the task considerably.

I have one concern about the handling of structs. Actually, it's more of a
concern with Ocaml than with idl, but idl makes the problem more apparent.

How can one deal with two struct types whose members have the same names.
In Ocaml, for example:

type foo = {a:int; b:int}
type goo ={a:char; b:char}

How can I now create an object of type foo?  It would be nice if I could write
let x = ({a=4; b=5;}:foo) or
let x = {foo.a=4; foo.b=4} or even
let (x:foo) = {a=4; b=4}

The problem isn't too bad if one is writing native caml code, since one takes
care to avoid re-using field names.  I typically write something like:

module Foo = struct type t = {a:int; b:int} end
module Goo = struct type t = {a:char; b:char} end

However, if one is defining a caml interface for existing C code, the problem is
much worse, since the C code will have been written without regard for the need
to avoid reusing the same field name in different struct types.










Xavier Leroy wrote:

> Just like everyone else who has read the papers on H/Direct by Meijer,
> Peyton-Jones et al, I have been working lately on a COM binding for
> Objective Caml.  I have just released one of the preliminary results
> of this work: CamlIDL 1.00.  Sources and documentation are available from
>
>   ftp://ftp.inria.fr/lang/caml-light/bazar-ocaml/camlidl-1.00.tar.gz
>   ftp://ftp.inria.fr/lang/caml-light/bazar-ocaml/camlidl-1.00.doc.ps.gz
>   ftp://ftp.inria.fr/lang/caml-light/bazar-ocaml/camlidl-1.00.doc.html.tar.gz
>
> The documentation can also be browsed at
>
>   http://caml.inria.fr/camlidl/htmlman/
>
> CamlIDL comprises two parts:
>
> - A stub code generator that generates the C stub code required for
>   the Caml/C interface, based on an MIDL specification.  (MIDL stands
>   for Microsoft's Interface Description Language; it looks like
>   C header files with some extra annotations, plus a notion of object
>   interfaces that look like C++ classes without inheritance.)
>
> - A (currently small) library of functions and tools to import COM
>   components in Caml applications, and export Caml code as COM
>   components.
>
> CamlIDL can be used in several ways:
>
> - Under Unix and Windows, as a stub code generator for using C
>   libraries from Caml.  It takes care of the gory details of
>   the Caml/C interface, such as all those pesky Val_xxx and Xxx_val
>   macros, proper registration of GC roots, the different calling
>   conventions from bytecode and from native-code, etc.
>
> - Under Unix and Windows, as a simple interface between Caml classes
>   and C++ classes.  The C++ fragment supported is that corresponding
>   to COM interfaces, i.e. interface polymorphism, but not inheritance.
>   Communication in both directions (COM/C++ object to Caml object
>   and Caml object to COM/C++ object) is provided.
>
> - Under Windows, as a COM binding for Caml.  There are two applications:
>
>   * Use COM components from Caml applications.  This should eventually
>     allow Caml code to exploit many Windows libraries and "script"
>     many Windows applications.  See the H/Direct papers for nice
>     examples.  This hasn't been tried yet with CamlIDL, because of
>     the incredible lack of documentation on Microsoft's components
>     and on Microsoft's IDL extensions they use in their interfaces.
>
>   * Encapsulate Caml code in COM components that can be used in
>     other applications.  This offers great potential for writing
>     the juicy bits of an application in OCaml and use C++ or Visual
>     Basic or Internet Explorer or whichever Microsoft behemoth to do
>     the GUI and other boring parts.  Currently, CamlIDL can create
>     "in-proc servers" for Caml components (i.e. DLLs that are registered
>     in the system registry and loaded automatically in applications
>     that use the components).  "Out-of-proc" servers and distributed
>     objects are not supported yet.  Currently, only native interfaces
>     are supported, but not yet dispatch interfaces.
>
> While the stub generation aspects of CamlIDL are already relatively
> mature, the COM binding is still very preliminary.  My hope in
> releasing it is that others more experienced with COM than I am will
> try it on real-world applications and let me know what is wrong or
> missing.
>
> - Xavier Leroy



--
Andrew K. Martin, Ph.D.
Motorola Inc., Somerset Design Center
Networking and Computer Systems Group

phone: (512) 424-8325        6200 Bridgepoint Parkway, Building 4,
fax  : (512) 424-8846        Mail Drop OE70
email: amartin@ibmoto.com    Austin, TX 78730

Re: CamlIDL - stub code generator and COM binding for OCaml

[Sven LUTHER]

On Fri, Mar 05, 1999 at 02:46:35PM -0600, Andrew Martin wrote:
> This looks great!.
> 
> I have been thinking of writing an interface between Ocaml and the GTK.  This
> tool should simplify the task considerably.

There is already such an interface being worked on, it is called mlgtk, and
should be released soon ...

Friendly,

Sven LUTHER

Re: CamlIDL - stub code generator and COM binding for OCaml

[Xavier Leroy]

> I have one concern about the handling of structs. Actually, it's more of a
> concern with Ocaml than with idl, but idl makes the problem more apparent.
> 
> How can one deal with two struct types whose members have the same names.
> In Ocaml, for example:
> 
> type foo = {a:int; b:int}
> type goo ={a:char; b:char}
> 
> How can I now create an object of type foo?

Currently, you can't, and I agree this is a serious problem with
CamlIDL-generated code.  (There's also the symmetrical problem of how
do you now access the fields of an object of type foo.)

> It would be nice if I could write
> let x = ({a=4; b=5;}:foo) or
> let x = {foo.a=4; foo.b=4} or even
> let (x:foo) = {a=4; b=4}

Solutions 1 and 3 could be made to work by having a special type
inference rule for record construction when a known record type is
expected by the context.  OCaml already does a similar hack for
typing printf format strings.  However, it's a hack, and it has fairly
bad properties (e.g. the results of type inference become dependent on
the order in which the type-checker works.)

For access, you'd have to say something like (x : foo).a.

The second solution could be made to work for record construction, but is
syntactically ambiguous for field access: does x.foo.a means "field a
of field foo of x", or "field foo.a of x" ?

> However, if one is defining a caml interface for existing C code,
> the problem is much worse, since the C code will have been written
> without regard for the need to avoid reusing the same field name in
> different struct types.

Right.  The problem could also be attacked at the level of CamlIDL by,
for instance,

- adding an attribute "mlname" to struct fields to specify the name of
the label in the Caml code, e.g.

  struct s { [mlname(s_a) int a; [mlname(s_b)] int b; };
  struct u { [mlname(u_a) int a; [mlname(u_b)] int b; };

However, adding all those names becomes tedious.  Maybe CamlIDL could
use some heuristics to find suitably unique Caml label names
(e.g. if all struct field names are unique in the whole interface,
use them, otherwise prefix them by the struct or typedef name).

- putting sub-modules in the generated .ml and .mli files.
For instance, non-object interfaces in the IDL source could be
translated to sub-modules:

        interface I1 { struct s1 { int a; int b; }; }
        interface I2 { struct s2 { int a; int b; }; }

would become

        module I1: sig type struct_s1 = { a :int; b : int } end
        module I2: sig type struct_s2 = { a :int; b : int } end

Any other ideas?

- Xavier Leroy

Re: CamlIDL - stub code generator and COM binding for OCaml

[Andrew Martin]

How about the following change to the Ocaml language with two orthogonal
extensions to the use of the "open" keyword:

1) allow open to be used locally and

2) allow open t where t is a type expression as well as open M where M is a
module
expression.

In detail:

1)  Allow open to be used locally, in a way that parallels the use of "let" --
(in a way, they both introduce bindings into the current environment), so that
in
addition to being able to say "open Foo" in a global context, you would also be

permitted to say "open Foo in <e>" which would limit the scope of the bindings
introduced by open Foo the expression <e>  Thus, if M is a module exporting
symbols a, b, and c, I could pass a, b, and c as arguments to a function f by
writing
(open M in (f a b c))

2) A struct type introduces new bindings into a  name-space of field
identifiers.   For example, the declaration

type t1 = {a:int; b:int}

introduces two new bindings for "a" and "b" respectively into a name-space of
field identifiers.  If I now write

type t2 = {a:char; b:char}

I introduce new bindings for "a" and "b" that eclipse the bindings introduced
by the declaration of t1.

2)  Suppose, I allow one to write

open t1 in <e>

which re-introduces the bindings provided by t1 in the name-space of
field-identifiers.   Thus, I can now write
   let x = (open t1 in {a=4; b=5}) in ....
   or,
   let (aa,bb) = (open t1 in (x.a,x.b))
   or,
   open t1 in (x.a <- 5)

Similarly, any type-constructors associated with t1 would be re-introduced into
the environment so that one could write
type t1 = A | B | C of int
type t2 = A | B | C of char

let x = open t1 in if true then A else C 3

Just a thought,
Andy



Xavier Leroy wrote:

> > I have one concern about the handling of structs. Actually, it's more of a
> > concern with Ocaml than with idl, but idl makes the problem more apparent.
> >
> > How can one deal with two struct types whose members have the same names.
> > In Ocaml, for example:
> >
> > type foo = {a:int; b:int}
> > type goo ={a:char; b:char}
> >
> > How can I now create an object of type foo?
>
> Currently, you can't, and I agree this is a serious problem with
> CamlIDL-generated code.  (There's also the symmetrical problem of how
> do you now access the fields of an object of type foo.)
>
> > It would be nice if I could write
> > let x = ({a=4; b=5;}:foo) or
> > let x = {foo.a=4; foo.b=4} or even
> > let (x:foo) = {a=4; b=4}
>
> Solutions 1 and 3 could be made to work by having a special type
> inference rule for record construction when a known record type is
> expected by the context.  OCaml already does a similar hack for
> typing printf format strings.  However, it's a hack, and it has fairly
> bad properties (e.g. the results of type inference become dependent on
> the order in which the type-checker works.)
>
> For access, you'd have to say something like (x : foo).a.
>
> The second solution could be made to work for record construction, but is
> syntactically ambiguous for field access: does x.foo.a means "field a
> of field foo of x", or "field foo.a of x" ?
>
> > However, if one is defining a caml interface for existing C code,
> > the problem is much worse, since the C code will have been written
> > without regard for the need to avoid reusing the same field name in
> > different struct types.
>
> Right.  The problem could also be attacked at the level of CamlIDL by,
> for instance,
>
> - adding an attribute "mlname" to struct fields to specify the name of
> the label in the Caml code, e.g.
>
>   struct s { [mlname(s_a) int a; [mlname(s_b)] int b; };
>   struct u { [mlname(u_a) int a; [mlname(u_b)] int b; };
>
> However, adding all those names becomes tedious.  Maybe CamlIDL could
> use some heuristics to find suitably unique Caml label names
> (e.g. if all struct field names are unique in the whole interface,
> use them, otherwise prefix them by the struct or typedef name).
>
> - putting sub-modules in the generated .ml and .mli files.
> For instance, non-object interfaces in the IDL source could be
> translated to sub-modules:
>
>         interface I1 { struct s1 { int a; int b; }; }
>         interface I2 { struct s2 { int a; int b; }; }
>
> would become
>
>         module I1: sig type struct_s1 = { a :int; b : int } end
>         module I2: sig type struct_s2 = { a :int; b : int } end
>
> Any other ideas?
>
> - Xavier Leroy



--
Andrew K. Martin, Ph.D.
Motorola Inc., Somerset Design Center
Networking and Computer Systems Group

phone: (512) 424-8325        6200 Bridgepoint Parkway, Building 4,
fax  : (512) 424-8846        Mail Drop OE70
email: amartin@ibmoto.com    Austin, TX 78730

Re: CamlIDL - stub code generator and COM binding for OCaml

[doligez]

>From: Andrew Martin <amartin@ibmoto.com>

>How can one deal with two struct types whose members have the same names.
>In Ocaml, for example:
>
>type foo = {a:int; b:int}
>type goo ={a:char; b:char}
>
>How can I now create an object of type foo?  It would be nice if I could write
>let x = ({a=4; b=5;}:foo) or
>let x = {foo.a=4; foo.b=4} or even
>let (x:foo) = {a=4; b=4}


You can get (almost) your second solution simply by declaring:

  type foo = {foo_a : int; foo_b : int};;
  type goo = {goo_a : char; goo_b : char};;

Then you can write:

  let x = {foo_a = 4; foo_b = 4}


-- Damien

Re: CamlIDL - stub code generator and COM binding for OCaml

[Pierre Weis]

Why not having a way to explicitely designate the label of the type ?
For instance foo`lab would designate the label lab of the type foo.

This is not ambiguous and furthermore it solves the problem of another
useful extension, namely anonymous record arguments of constructors:
C`lab would also designate the label lab of the argument of
constructor C.

All the best,

Pierre Weis

INRIA, Projet Cristal, Pierre.Weis@inria.fr, http://cristal.inria.fr/~weis/

Re: CamlIDL - stub code generator and COM binding for OCaml

[Jacques GARRIGUE]

From: Xavier Leroy <Xavier.Leroy@inria.fr>

> > It would be nice if I could write
> > let x = ({a=4; b=5;}:foo) or
> > let x = {foo.a=4; foo.b=4} or even
> > let (x:foo) = {a=4; b=4}
> 
> Solutions 1 and 3 could be made to work by having a special type
> inference rule for record construction when a known record type is
> expected by the context.  OCaml already does a similar hack for
> typing printf format strings.  However, it's a hack, and it has fairly
> bad properties (e.g. the results of type inference become dependent on
> the order in which the type-checker works.)
> 
> For access, you'd have to say something like (x : foo).a.
> 
> The second solution could be made to work for record construction, but is
> syntactically ambiguous for field access: does x.foo.a means "field a
> of field foo of x", or "field foo.a of x" ?

I am and more thinking that allowing labels in some types might be a
good solution.

Then one just has to write.

type foo = Foo of { a: int; b: int }
type goo = Goo of { a: char; b: char }

let x = Foo {a=1;b=2}
let y = Goo {a='a';b='b'}

With extraction, you just need to have pattern matched on the
record. The two following are ok:

fun (Foo {a=a}) -> a
fun (Foo x) -> x.a

and even, if we defined "a:" to be mutable in foo

fun (Foo x) -> x.a <- 3

The last ones may seem strange, but it is clearly unambiguous,
and if I remember correctly caml-light has already something like that
for mutable sum types.

This is an easy solution. If you are ready for something more
far-fetched, we could go for using type information, and use a
restricting scheme to keep principality, as is done in olabl for class
having polymorphic methods. This is safe and correct, and
syntactically very light. In fact, since exactly the same mechanism
would work, this would be very easy to add it to olabl.

---------------------------------------------------------------------------
Jacques Garrigue      Kyoto University     garrigue at kurims.kyoto-u.ac.jp
		<A HREF=http://wwwfun.kurims.kyoto-u.ac.jp/~garrigue/>JG</A>

Re: CamlIDL - stub code generator and COM binding for OCaml

[Thierry Bravier]

Andrew Martin wrote:

> How about the following change to the Ocaml language with two orthogonal
> extensions to the use of the "open" keyword:
>
> 1) allow open to be used locally (open M in (f a b c)) and
> 2) allow open t where t is a type expression as well as open M where M is a
> module expression (open t in expr)

Both extensions look really neat to me.
They should solve the common record label issue (as explained by A. Martin)
and also another seemingly unrelated problem:
(see Christophe Raffalli's last message)
how to use A.B.(+) in an infix way ?

With 'open in' you could write:

module F = struct let (+) = (+.) and (-) = (-.) and ( * ) = ( *. ) end
let delta a b c = open F in b*b - 4.0 *a*c

without 'open in' you would not be able to 'open F'
because it would hide useful Pervalises.(+) ... till the end of current struct
(most often current file).

Thanks.
--

Thierry Bravier                     Dassault Aviation - DGT / DPR / DESA

78, Quai Marcel Dassault              F-92214 Saint-Cloud Cedex - France

Telephone : (33) 01 47 11 53 07          Telecopie : (33) 01 47 11 52 83

E-Mail :                     mailto:thierry.bravier@dassault-aviation.fr