Native Module Linking

Native Module Linking

[Nicolas Cannasse]

Hi,

I have a project that is structured like this :
- a library with one Entry module and several Plugins
- an application using the library.

In the application I'm doing the following :

open Entry;
open Plugin1;
open Plugin2;
....
(* use "Entry" *)

Plugins are using Entry, and application is only activating plugins with
"open" (in order to ensure they're linked).
On windows everything works well but looks like on Linux some configurations
are not linking the plugins (at least in opt mod : ocamlopt myLib.cmxa
myApp.ml ). And I can't use the -linkall because I'm using other libraries
that I want to partially link . Is this a bug, a feature, or a
misunderstanding ?

Regards,
Nicolas Cannasse

Re: [Caml-list] Native Module Linking

[skaller]

On Wed, 2004-11-10 at 05:38, Nicolas Cannasse wrote:
> Hi,
> 
> I have a project that is structured like this :
> - a library with one Entry module and several Plugins
> - an application using the library.
> 
> In the application I'm doing the following :
> 
> open Entry;
> open Plugin1;
> open Plugin2;
> ....
> (* use "Entry" *)
> 
> Plugins are using Entry, and application is only activating plugins with
> "open" (in order to ensure they're linked).

open has no effect on linking, all it does is allow
you to use unqualified names.

> On windows everything works well but looks like on Linux some configurations
> are not linking the plugins 

If you are not using code in a plugin, it won't be linked
from a library .. and it also won't be a problem, because
you're not using it .. that's the point of a library
after all.

Just place the cmx in the link command line to forcibly
link it, to see the side effects of initialisation.

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net

Re: [Caml-list] Native Module Linking

[Nicolas Cannasse]

> > Hi,
> >
> > I have a project that is structured like this :
> > - a library with one Entry module and several Plugins
> > - an application using the library.
> >
> > In the application I'm doing the following :
> >
> > open Entry;
> > open Plugin1;
> > open Plugin2;
> > ....
> > (* use "Entry" *)
> >
> > Plugins are using Entry, and application is only activating plugins with
> > "open" (in order to ensure they're linked).
>
> open has no effect on linking, all it does is allow
> you to use unqualified names.

It depends exactly on what information basis linking is done
For example, ocamldep works at syntax level and will report a dependency if
an open is done. Ocamlopt could (should ?) enforce such dependency when an
"open" directive is found, since ocamlc and ocamlopt-win32 are working
correctly with it.

Nicolas Cannasse

Re: [Caml-list] Native Module Linking

[skaller]

On Wed, 2004-11-10 at 08:56, Nicolas Cannasse wrote:

> It depends exactly on what information basis linking is done

OK: open isn't relevant to static linking :)

I think its a hack, and probably a bug, that ocamlc
works differently -- same for win32.

However I think the bug is actually in the Ocaml
language itself: there's no way to properly
control order of initialisation.

I would suggest:

import Modname
import rec Modname
open Modname (* implies import *)

Semantics: (1) any side effects of Modname's initialisation
code shall occur prior execution of the client module's
initialisation code. 

(2) If Dynlink is imported, then exposing the symbols
of a module to a Dynlinked module is considered
a side-effect.

(3) The order of initialisation for import rec is unspecified,
however if a module is imported (without rec) and it is mutually
recursive with another, both these modules will be initialised
before the client's initialisation.

The effect of 'import' is principally to guarrantee
side effects won't be dropped, just because a module
happens to be otherwise unused. In particular, these
semantics are made part of the language, rather
than dependent on how the program is linked,
and whether it happens to be bytecode or not...

With one CAVEAT: it isn't specified how the root
or root modules are determined. In C, the root
is the function main(). In Ocaml, the roots would
be any modules listed on the linker command line.

Note that the guarrantee doesn't release the programmer
from the requirement to specify compilation and linkage
order, instead it just forces the actual ordering
constraints to be manifest in the language.

Apart from increasing slightly the determinism
of Ocaml code, this extension would significantly
simplify dependency analysis tools like Ocamldep.

[in fact you could probably use a 3 line shell script
instead .. :]

Any comments appreciated but be warned of the possible
biases of ex-Pythonistas :)

And yes I know this would break a lot of code,
which I think is good. Obviously, the feature should
be introduced with a switch to enable it, then
the default should be changed so you have a switch
to support legacy code.

-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net

Re: [Caml-list] Native Module Linking

[Jacques Garrigue]

From: "Nicolas Cannasse" <warplayer@free.fr>

> I have a project that is structured like this :
> - a library with one Entry module and several Plugins
> - an application using the library.
> 
> In the application I'm doing the following :
> 
> open Entry;
> open Plugin1;
> open Plugin2;
> ....
> (* use "Entry" *)
> 
> Plugins are using Entry, and application is only activating plugins with
> "open" (in order to ensure they're linked).
> On windows everything works well but looks like on Linux some configurations
> are not linking the plugins (at least in opt mod : ocamlopt myLib.cmxa
> myApp.ml ). And I can't use the -linkall because I'm using other libraries
> that I want to partially link . Is this a bug, a feature, or a
> misunderstanding ?

Your first assumption is wrong: open is completely unrelated to what
will be linked or not. Or at least it is not supposed to.
To force linking a module, one of its members should be used.
Fo instance, using "include".
I have no idea why it works on windows...

If you want to force a library to be linked monolythically, an option
is to make it into a package (automatically or by hand).

By the way, initialization order is well-defined: it is the order of
the files given on the command line, and inside archives the order
when the archive was created.

I.e.,
   ocamlc -a -o mylib.cma a.cmo b.cmo c.cmo
   ocamlc -o myprog d.cmo mylib.cma e.cmo
when e.cmo requires A and C will result in the initialization order
   D A C E
Nothing strange there.

Jacques Garrigue

Re: [Caml-list] Native Module Linking

[Nicolas Cannasse]

> Your first assumption is wrong: open is completely unrelated to what
> will be linked or not. Or at least it is not supposed to.
> To force linking a module, one of its members should be used.
> Fo instance, using "include".
> I have no idea why it works on windows...

Yes, that was my point. Such different behaviors are error prone, since it
leads to invalid assumption on 'open' semantics.
So I think I need to add two things to the bug report here :
- fix the open on windows
- feature wish for an explicit import statement (if possible enabling
recursive modules such as proposed by John).

Nicolas Cannasse

Re: [Caml-list] Native Module Linking

[skaller]

On Wed, 2004-11-10 at 16:44, Nicolas Cannasse wrote:

> - feature wish for an explicit import statement (if possible enabling
> recursive modules such as proposed by John).

Well this is only a proposal, it does need to be examined.
It's all too easy to propose something that seems nice
but doesn't pan out in practice .. and very hard
to remove a feature once added.


-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net