[Caml-list] ocamlc linking loads dlls?

[Caml-list] ocamlc linking loads dlls?

[Chris Hecker]

Why does ocamlc load dlls during linking?  From reading 
bytecomp/bytelink.ml it doesn't seem like it does much with the information 
except for getting paths (which you could do without loading the dll), yet 
this is a big problem for a) dlls that aren't in the path during build, and 
b) dlls that do complex stuff in their dllmain.

Am I missing something and this is really necessary?

ocamlopt doesn't do it, for what it's worth.

Chris

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] ocamlc linking loads dlls?

[Xavier Leroy]

> Why does ocamlc load dlls during linking?  

Just to make sure that all external C functions referenced from your
Caml code are indeed defined in the DLLs you provide.  Otherwise, the
error would only be detected at run time, and earlier is better.

> a) dlls that aren't in the path during build, and 
> b) dlls that do complex stuff in their dllmain.

I agree b) is a bit of a problem, but I don't know of any portable way
to test whether DLL x.dll defines symbol "foo" than to link x.dll and
query the address of "foo".

> ocamlopt doesn't do it, for what it's worth.

Sure: ocamlopt links statically the Caml/C stub code.  It doesn't use
the DLLs.

- Xavier
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] ocamlc linking loads dlls?

[Chris Hecker]

> > ocamlopt doesn't do it, for what it's worth.
>Sure: ocamlopt links statically the Caml/C stub code.  It doesn't use
>the DLLs.

Right, but the problem I'm running into is I've got a C interface library 
between caml and another C dll, and that interface is compiled to a DLL on 
bytecode builds (like it's supposed to be since you implemented 
cma+dll).  The interface defines all the caml functions, but it in turn 
loads the C dll when it's loaded during link because it's a 
dependency.  So, the problem dll isn't even a dll that's needed by caml 
directly, but it's getting loaded anyway.  This could get really bad in the 
case of a big set of dependencies.  Or, imagine the case where there was a 
strict ordering of DLLs that needed to be loaded or another process that 
had to be running that the dllmain connected to, and it failed dllmain if 
that process wasn't running (like a license server).

> > a) dlls that aren't in the path during build, and
> > b) dlls that do complex stuff in their dllmain
>I agree b) is a bit of a problem, but I don't know of any portable way
>to test whether DLL x.dll defines symbol "foo" than to link x.dll and
>query the address of "foo".

I actually think a) is important as well, for modularity reasons.  In my 
above case, the interface dll only needed the import library from the C dll 
to link since it was done with the C linker.  This means I don't even need 
access to the real C dll to build, just its import library and a 
header.  In the case of something restrictive with security or licenses, I 
might not even have the C dll around until I move onto a test machine or 
something.

There's also the security problem of a _link_ now can run arbitrary code 
because the dllmain gets called, but I didn't bother mentioning that before 
because it's not like caml is trying to be high-security.  However, it is 
still an issue (imagine an app that needs to be built as root but run as 
another user...the dllmain is now run as root by ocamlrun...very irregular).

I understand your desire to fail early during link instead of at runtime, 
but I think this solution causes more problems, and more serious problems, 
than it solves.  Plus, the app will fail at load time, not in the middle of 
running (usually the cmas are all resolved at load, right?), and developers 
are "used to" dll's failing at load time during test if there's a 
versioning issue or something.

If you really must check this at link time (like I said, it's a worthy 
goal, it's just a question of tradeoffs), the right thing to do is write 
the imagehlp code to check the export without loading the dll.  It's pretty 
short.  I assume there's a similar thing on unix.  Until that code gets 
written, I think the better tradeoff is to disable checking during link.

Chris


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] ocamlc linking loads dlls?

[Nicolas Cannasse]

> > Why does ocamlc load dlls during linking?
>
> Just to make sure that all external C functions referenced from your
> Caml code are indeed defined in the DLLs you provide.  Otherwise, the
> error would only be detected at run time, and earlier is better.
>
> > a) dlls that aren't in the path during build, and
> > b) dlls that do complex stuff in their dllmain.
>
> I agree b) is a bit of a problem, but I don't know of any portable way
> to test whether DLL x.dll defines symbol "foo" than to link x.dll and
> query the address of "foo".

That point can be resolved by calling LoadLibraryEx with the
DONT_RESOLVE_DLL_REFERENCE flag.
According to MS docs, this prevent the call of DllMain.

Nicolas Cannasse

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] ocamlc linking loads dlls?

[Chris Hecker]

>That point can be resolved by calling LoadLibraryEx with the
>DONT_RESOLVE_DLL_REFERENCE flag.
>According to MS docs, this prevent the call of DllMain.

That's better, but it still requires the dll to be available.

Chris


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] ocamlc linking loads dlls?

[Jacques Garrigue]

From: Chris Hecker <checker@d6.com>

> >That point can be resolved by calling LoadLibraryEx with the
> >DONT_RESOLVE_DLL_REFERENCE flag.
> >According to MS docs, this prevent the call of DllMain.
> 
> That's better, but it still requires the dll to be available.

I don't see your point.
All the dlls you mention in ocaml link are supposed to be stub dlls.
As you write them for your caml program, they are going to be
available. If more dlls are needed, they are required by dependencies
in those dlls, and this is what the above flag is supposed to disable.
This looks fine.

However, it might still be simpler to a compiler flag to disable all
dll checking when linking. That's trivial. But if you don't know what
you do, you're going to have runtime errors later, so this is not a
reasonable default.

Jacques Garrigue
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] ocamlc linking loads dlls?

[Gerd Stolpmann]

Am 2002.11.15 01:55 schrieb(en) Jacques Garrigue:
> From: Chris Hecker <checker@d6.com>
> 
> > >That point can be resolved by calling LoadLibraryEx with the
> > >DONT_RESOLVE_DLL_REFERENCE flag.
> > >According to MS docs, this prevent the call of DllMain.
> > 
> > That's better, but it still requires the dll to be available.
> 
> I don't see your point.
> All the dlls you mention in ocaml link are supposed to be stub dlls.
> As you write them for your caml program, they are going to be
> available. If more dlls are needed, they are required by dependencies
> in those dlls, and this is what the above flag is supposed to disable.
> This looks fine.
> 
> However, it might still be simpler to a compiler flag to disable all
> dll checking when linking. That's trivial. But if you don't know what
> you do, you're going to have runtime errors later, so this is not a
> reasonable default.

Some time ago, I ran into problems with dll checking. I tried to
embed the ocaml interpreter into another environment by loading
ocamlrun as dll. Unfortunately, I got a loop:

other_environment
  loads: ocamlrun.so
     loads: other_environment_stubs.so
            (to access functions of other_environment)
         uses symbols of other_environment

>From the dll side, there is absolutely no problem, you have only to
ensure that other_environment exports all its symbols to 
subsequently loaded dlls. However, it is not possible to create
an ocaml bytecode executable for this special construction,
because ocamlc cannot load other_environment_stubs.so which
depends on symbols that are only dynamically available. I solved
that by playing with LD_PRELOAD, but this is really not nice.

A switch that turns dll checking off would really have helped me.
I know that embedding ocamlrun as shown is not officially supported,
but it is absolutely cool to dynamically load the whole interpreter
into any program.

Gerd
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
------------------------------------------------------------
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] ocamlc linking loads dlls?

[Chris Hecker]

>That's trivial. But if you don't know what
>you do, you're going to have runtime errors later, so this is not a
>reasonable default.

Is this any different from a C application with the wrong import 
library?  It will fail on load.  Will the caml program fail at an arbitrary 
point during runtime or at load time?  Ignoring using dynlink, of course, 
which has the equivalent runtime failure in the C case.

I'd be fine with a compiler switch, though.

Chris

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] ocamlc linking loads dlls?

[Jacques Garrigue]

From: Chris Hecker <checker@d6.com>
> >That's trivial. But if you don't know what
> >you do, you're going to have runtime errors later, so this is not a
> >reasonable default.
> 
> Is this any different from a C application with the wrong import 
> library?  It will fail on load.  Will the caml program fail at an arbitrary 
> point during runtime or at load time?  Ignoring using dynlink, of course, 
> which has the equivalent runtime failure in the C case.

It would fail at load time.
As I'm not a windows programmer, I don't know (too much) what is an
import library. On unix, an undefined symbol in a linked dll is a
static linker error. You only may get a load-/runtime error if you
explicitly use dlopen, or if you physically change the dll for an
incompatible one (not supposed to happen!).

---------------------------------------------------------------------------
Jacques Garrigue      Kyoto University     garrigue at kurims.kyoto-u.ac.jp
		<A HREF=http://wwwfun.kurims.kyoto-u.ac.jp/~garrigue/>JG</A>
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] ocamlc linking loads dlls?

[Chris Hecker]

>It would fail at load time.
>As I'm not a windows programmer, I don't know (too much) what is an
>import library. On unix, an undefined symbol in a linked dll is a
>static linker error. You only may get a load-/runtime error if you
>explicitly use dlopen, or if you physically change the dll for an
>incompatible one (not supposed to happen!).

On windows you link with a library called the "import library" for the 
dll.  The linker never touches the dll.  So yes, you can get the wrong 
import library and fail to load (just like changing the dll on unix).

But, I now see where the disconnect is...I didn't know unix touched the 
actual dll to link.  I can see advantages and disadvantages to that.  It 
still doesn't guarantee loading, for the "change the dll" case, as you 
point out, but it does avoid the [very rare "(not supposed to happen!)"] 
error with a dll-import library mismatch.  The big advantage is not needing 
to carry around and distribute an import library, which is a pain on windows.

Chris


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners