Easier FFI

Easier FFI

[Jon Harrop]

I'm currently revisiting the topic of OpenGL bindings because I'd like to have 
a play with vertex and fragment shaders. Getting shaders up and running from 
C++ code is very easy: just a few calls and you pass your shader programs in 
as strings. However, I have been unable to get this working from OCaml using 
any of the existing OpenGL bindings (most notably GLCaml). This got me 
thinking about FFIs.

GLCaml currently autogenerates its bindings from a custom annotated C header 
file. There are some aspects that I'd like to change. For example, GLCaml 
currently seems to use only bigarrays when strings and ordinary OCaml arrays 
seem preferable in several circumstances.

Rather than invest time and effort into tweaking only GLCaml, I'm wondering 
what people's thoughts are about autogenerated FFIs for OCaml in general?

For example, why can't we have a generic FFI library that allows us to use any 
external library (unsafely) without leaving OCaml? Then we could write or 
generate our bindings entirely in OCaml and forget about these C stubs.

How does this relate to SML's NLFFI?

-- 
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/products/?e

Re: [Caml-list] Easier FFI

[Dave Benjamin]

Jon Harrop wrote:
> For example, why can't we have a generic FFI library that allows us to use any 
> external library (unsafely) without leaving OCaml? Then we could write or 
> generate our bindings entirely in OCaml and forget about these C stubs.

I would love something like this. I've used "ctypes", a similar library 
for Python, a few times, and it has really come in handy. Not something 
I want to use often (at least not directly), but for those times when 
use of an unsupported C library is necessary, it's so nice to have the 
ability to stay within the language rather than writing a bunch of 
wrapper code. I wonder if something like ctypes would translate well to 
a statically typed language like OCaml.

http://starship.python.net/crew/theller/ctypes/

Re: [Caml-list] Easier FFI

[Jon Harrop]

On Friday 21 December 2007 03:37, Dave Benjamin wrote:
> I would love something like this. I've used "ctypes", a similar library
> for Python, a few times, and it has really come in handy. Not something
> I want to use often (at least not directly), but for those times when
> use of an unsupported C library is necessary, it's so nice to have the
> ability to stay within the language rather than writing a bunch of
> wrapper code. I wonder if something like ctypes would translate well to
> a statically typed language like OCaml.
>
> http://starship.python.net/crew/theller/ctypes/

Absolutely, that's exactly the kind of thing I'm talking about. Basically just 
a glorified C interpreter as a fallback to make unsafe interop as easy as 
possible.

-- 
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/products/?e

Re: [Caml-list] Easier FFI

[Richard Jones]

I've got flu at the moment so apologies about this slightly unfocused
laundry-list ...

Lessons from ocaml-libvirt:

[Line numbers refer to this file:
http://hg.et.redhat.com/virt/applications/virt-top--devel?f=c29881e5aa70;file=libvirt/libvirt_c.c ]

(1) You need to be able to link to multiple versions of a dynamic
library.  Programs written in OCaml, using ocaml-libvirt, can link to
any libvirt.so >= 0.2.1, even to *future* versions of libvirt.so.

At the very least you need to be able to find out if a function exists
in the dynamic library, and turn non-existance into an OCaml exception
(instead of a dynamic linker segfault).  [lines 65-143, example on
lines 560-578]

(2) In this library and in ODE, C objects contain implicit
relationships with each other, and use reference counting to free
themselves.  This had to be modelled by making the implicit references
explicit in OCaml (so the GC knows about them).  [lines 168-208]

(3) Some libvirt API calls are long-running and need
enter_blocking_section / leave_blocking_section around them.  Others
are short-running and so don't.  [lines 47-57]

(4) You can't assume that a library won't add more values to the end
of an enum, particularly if you allow yourself to link to a later
version of the library than you compile against.  [lines 1826-1889]

On the subject of 'ctypes':

perl4caml offers two 'levels' of access to Perl code.  At the low
level you can directly load / instantiate Perl packages and objects.
This low level is unsafe -- it can throw runtime type exceptions --
but requires no explicit binding code at all.  If you see a Perl
library which you like, as long as it is installed on your system an
OCaml program can use it.  There's also a higher level where you can
write OCaml code which uses Perl objects in a type-safe way, but that
requires someone to write bindings for the Perl library.

However does a 'ctypes for OCaml' need to be type unsafe.  PG'OCaml
(my PostgreSQL bindings for OCaml) does this with a camlp4
preprocessor which processes the SQL statements embedded in the OCaml
code, checks the types of the arguments / return values with the
database, and generates inline type-safe code.  If there was a way to
take a block of C code, work out what types it requires (using
something like gcc? CIL?), then you could generate compile-time
type-safe bindings for inline C code.  That would be a marvellous
thing to have ...

Rich.

-- 
Richard Jones
Red Hat

Re: Easier FFI

[Alan Falloon]

Richard Jones wrote:
> If there was a way to
> take a block of C code, work out what types it requires (using
> something like gcc? CIL?), then you could generate compile-time
> type-safe bindings for inline C code.  That would be a marvellous
> thing to have ...

Does anyone know how forklift is coming along? That seemed like a
promising project in this direction.

Is this the authoratative web site? http://code.google.com/p/forklift/