compiling ocaml into a shared object

[oleg@pobox.com]

Hendrik Tews wrote:
> could somebody tell me how to compile ocaml sources into a shared
> object that can be loaded with dlopen?

It is possible: MetaOCaml in the native mode does exactly that. To be
more precise, evaluating `.! .<code here>.', that is, `running' the
code expression, is equivalent to invoking ocamlopt (as a library) to
compile `code here', making a shared object, and loading it into the
running program. 

For more details (the code, a few tests and the Makefile), you might
want to look into the directory `natdyn' of the MetaOCaml
distribution.

Yet it is awfully tricky -- perhaps trickier than one may realize. The
problems may occur long after everything is deemed working. For one
thing, Ocaml native runtime makes an assumption that the data segment
of a program is contiguous. With the dynamically linked code this
assumption is no longer true, and subtle bugs emerge (see below). 

After you have loaded a shared object, chances are you'd like to
invoke an OCaml function from there. That is also tricky: the stack
frame has to be set up appropriately. If not -- everything works,
until the major garbage collection. At which point, you get the
segmentation fault.

Here's an example of a subtle bug (which also may be present in the
`Ancient' extension announced on this list some time ago). 

	let v = Array.create 256 iv in
	Array.length v

where iv is a _float_ datum that is a literal in a loaded
shared object, or had been evicted into the Ancient heap. Now, what is
the reported length of the allocated float array of 256 elements? 128.
Exactly half of what is allocated. Weird, eh?

This problem has actually happened, please see the MetaOCaml list for
discussion. It took a couple of days just to understand the problem.

The natdyn part of MetaOCaml does work, I believe correctly. However,
the implementation takes advantage of the fact that some code just
cannot be part of MetaOCaml quotations. In particular, MetaOcaml
quotations cannot contain any declarations (of data types, of
exceptions) and any module expressions (in particular, modules
requiring initialization). These limitations of natdyn can be remedied
(OTH, since natdyn was needed quite urgently, one may be justified in
taking advantage of the domain knowledge to simplify the problem).