Dynamic linking

Dynamic linking

[Emir Pasalic]

We are writing a program that generates a C file, compiles it to a 
dynamic library, and uses dlopen (and such) to load it, execute it and 
bring its value into ocaml (bytecode) runtime. To do this, we need to 
use some of the functionality of the ocaml runtime (e.g., caml_alloc, 
caml_update) so we can marshall values from the C world into the world 
of ocaml. Our solution works on linux and macos platforms, but we have 
a problem trying to make it run on windows with Cygwin.

So, we're trying to create a shared library on Cygwin that contains 
symbols such as "caml_alloc" and "caml_update".
  We do not know of a way to easily incorporate these symbols in the 
linking process, and so they
remain undefined when we try to create a library, and undefined symbols 
are not allowed in Cygwin shared libraries.

Therefore we tried to resort to another method, where the calls to  
caml_alloc and caml_update are replaced by
calls to dlopen and dlsym functions, i.e., we were trying to do this:

        h = dlopen ("<the library name>", RLTD_NOW);
        /* process error */
        s = dlsym (h, "caml_alloc");
        /* process error */
        my_alloc = /* proper casting */ s;
        result = my_alloc ( /* arguments */ );

Assuming that this is possible, what is the name that should be given 
to the library?
Else, is it possible to build a shared library on Cygwin that contains 
references to these symbols?

Note that all this works perfectly fine on MacOS and linux which allow 
unresolved symbols in dynamic libraries, but Cygwin simply dies. Any 
Windows/Cygwin experts out there who can help us?

Re: [Caml-list] Dynamic linking

[Igor Pechtchanski]

On Tue, 5 Apr 2005, Emir Pasalic wrote:

> We are writing a program that generates a C file, compiles it to a
> dynamic library, and uses dlopen (and such) to load it, execute it and
> bring its value into ocaml (bytecode) runtime. To do this, we need to
> use some of the functionality of the ocaml runtime (e.g., caml_alloc,
> caml_update) so we can marshall values from the C world into the world
> of ocaml. Our solution works on linux and macos platforms, but we have a
> problem trying to make it run on windows with Cygwin.
>
> So, we're trying to create a shared library on Cygwin that contains
> symbols such as "caml_alloc" and "caml_update".
> We do not know of a way to easily incorporate these symbols in the
> linking process, and so they remain undefined when we try to create a
> library, and undefined symbols are not allowed in Cygwin shared
> libraries.

The curent version of O'Caml under Cygwin doesn't support dynamic linking
in any structured way.  I was able to build an ad-hoc set of dynamic
libraries for standard modules, but I'm still in the process of modifying
O'Caml tools to do this seamlessly.

That said, there is still a limitation in Windows that any unresolved
symbols in a DLL have to have a *statically* known target, i.e., the
loader has to know what DLL to load the symbols from.  The two possible
workarounds are to a) extract the unresolved symbols from the executable
into a helper DLL that both the executable and the library are linked
with, or b) use dlopen/dlsym, as you did.

> Therefore we tried to resort to another method, where the calls to
> caml_alloc and caml_update are replaced by calls to dlopen and dlsym
> functions, i.e., we were trying to do this:
>
>        h = dlopen ("<the library name>", RLTD_NOW);
>        /* process error */
>        s = dlsym (h, "caml_alloc");
>        /* process error */
>        my_alloc = /* proper casting */ s;
>        result = my_alloc ( /* arguments */ );
>
> Assuming that this is possible, what is the name that should be given to
> the library?

Any name will do, as long as LD_LIBRARY_PATH contains the directory of
your library (yes, it *is* used on Cygwin for dlopen calls).  It doesn't
even have to end in ".dll".

> Else, is it possible to build a shared library on Cygwin that contains
> references to these symbols?

It is.  You'll need to create a helper DLL and an import library for it.
Then link your executable and the library DLL with the helper.  I would
have referred you to the (experimental) ocaml-3.08.1-2 cygwin package, but
it apparently wasn't uploaded to the main Cygwin package repository.  I
can send you the source/patches if interested.

> Note that all this works perfectly fine on MacOS and linux which allow
> unresolved symbols in dynamic libraries, but Cygwin simply dies. Any
> Windows/Cygwin experts out there who can help us?

I would be willing to help you build a small example app to get you
started.  Let me know where to get the source.

Igor Pechtchanski, the volunteer O'Caml maintainer for Cygwin
-- 
				http://cs.nyu.edu/~pechtcha/
      |\      _,,,---,,_		pechtcha@cs.nyu.edu
ZZZzz /,`.-'`'    -.  ;-;;,_		igor@watson.ibm.com
     |,4-  ) )-,_. ,\ (  `'-'		Igor Pechtchanski, Ph.D.
    '---''(_/--'  `-'\_) fL	a.k.a JaguaR-R-R-r-r-r-.-.-.  Meow!

"The Sun will pass between the Earth and the Moon tonight for a total
Lunar eclipse..." -- WCBS Radio Newsbrief, Oct 27 2004, 12:01 pm EDT

Dynamic linking

[Alain Frisch]

Hello,

I'm pleased to announce the existence of an experimental branch in
OCaml's CVS. Main changes:

* Dynamic loading of OCaml code in native code programs, through
  the Dynlink module.

* Simplified dynamic linking of C code under Windows.


More info: http://alain.frisch.fr/natdynlink.html

Enjoy and send some feedback!

Alain