ocamlopt and *using* DLLs

ocamlopt and *using* DLLs

[Robert Roessler]

I tried this on the beginners list, but may not have been asking the 
right question(s)... :)

1) Can DLLs appear as input to the ocamlopt compiler like they can 
with ocamlc?  The manual says "yes": "Arguments ending in .o, .a or 
.so (.obj, .lib and .dll under Windows) are assumed to be C object 
files and libraries. They are linked with the program."

I get "ocamlopt.opt: don't know what to do with ml_scintilla.dll."

2) If DLLs can in fact be used with ocamlopt, is the model the same as 
with ocamlc?  In particular,

2a) can I use them *without* stub libraries?

2b) can I in general expect that an app that works well with ocamlc 
and a collection of OCaml cma libs and compiled DLLs (LablGTK and 
associated runtime) will work with ocamlopt - assuming that I generate 
and replace cmo/cma files with cmx/cmxa files?

Robert Roessler
roessler@rftp.com
http://www.rftp.com

Re: [Caml-list] ocamlopt and *using* DLLs

[Jacques Garrigue]

From: Robert Roessler <roessler@rftp.com>

> I tried this on the beginners list, but may not have been asking the 
> right question(s)... :)

Doesn't look like a beginner question...

> 1) Can DLLs appear as input to the ocamlopt compiler like they can 
> with ocamlc?  The manual says "yes": "Arguments ending in .o, .a or 
> .so (.obj, .lib and .dll under Windows) are assumed to be C object 
> files and libraries. They are linked with the program."
> 
> I get "ocamlopt.opt: don't know what to do with ml_scintilla.dll."

Indeed, looking at the sources, .so/.dll is only handled in ocamlc,
not in ocamlopt.
That means that, in ocamlopt, you must either provide a static
library, or use the -cclib option and hope that the system linker does
the right thing. In practice, -cclib ml_scintilla.so should work on
Unix, but the .dll equivalent is not going to work on Windows, as the
linker does not support it.

> 2) If DLLs can in fact be used with ocamlopt, is the model the same as 
> with ocamlc?  In particular,
> 
> 2a) can I use them *without* stub libraries?

If by stub you mean the ml_scintilla.lib that the windows librarian
builds for you, then you will need it with ocamlopt (but not with
ocamlc).
Of course in both cases you also need an ocaml specific stub library
to wrap calls.

> 2b) can I in general expect that an app that works well with ocamlc 
> and a collection of OCaml cma libs and compiled DLLs (LablGTK and 
> associated runtime) will work with ocamlopt - assuming that I generate 
> and replace cmo/cma files with cmx/cmxa files?

Except these linking problems, you can assume identical behaviour.
If there are significant discrepancies, please report them.

Jacques Garrigue

Re: [Caml-list] ocamlopt and *using* DLLs

[Nicolas Cannasse]

> I tried this on the beginners list, but may not have been asking the
> right question(s)... :)
>
> 1) Can DLLs appear as input to the ocamlopt compiler like they can
> with ocamlc?  The manual says "yes": "Arguments ending in .o, .a or y
> .so (.obj, .lib and .dll under Windows) are assumed to be C object
> files and libraries. They are linked with the program."
>
> I get "ocamlopt.opt: don't know what to do with ml_scintilla.dll."
>
> 2) If DLLs can in fact be used with ocamlopt, is the model the same as
> with ocamlc?  In particular,
>
> 2a) can I use them *without* stub libraries?
>
> 2b) can I in general expect that an app that works well with ocamlc
> and a collection of OCaml cma libs and compiled DLLs (LablGTK and
> associated runtime) will work with ocamlopt - assuming that I generate
> and replace cmo/cma files with cmx/cmxa files?
>
> Robert Roessler
> roessler@rftp.com
> http://www.rftp.com

I was not maybe clear enough on the beginner list :)
The main problem is that your DLL need to call Ocaml API, which it is linked
with.

In bytecode there is no problem since the whole API is inside ocamlrun.dll
so your DLL and ocamlrun.exe will simply use this API dll together.
In native compilation however there is no more ocamlrun.dll, since the API
is staticly linked into your executable. That means that your DLL that was
working in bytecode will load ocamlrun.dll and call it while your executable
is calling staticly linked API. This will result quite quickly in a crash
since ocamlrun.dll is not initialized correctly.

The only possible way to get the DLL working is to patch it so it loads its
API inside your.exe and not inside ocamlrun.dll. But then you will need a
different version of the DLL for each application (!). It would be nice if
OCaml could provide the equivalent of ocamlrun.dll for the native
compilation mode.

Nicolas Cannasse

Re: [Caml-list] ocamlopt and *using* DLLs

[Robert Roessler]

Jacques Garrigue wrote:

> From: Robert Roessler <roessler@rftp.com>
> 
>>I tried this on the beginners list, but may not have been asking the 
>>right question(s)... :)
> 
> 
> Doesn't look like a beginner question...

I had been hoping that the ocamlopt manual was not wrong, and that I 
was just missing something... simple. :)

>>2b) can I in general expect that an app that works well with ocamlc 
>>and a collection of OCaml cma libs and compiled DLLs (LablGTK and 
>>associated runtime) will work with ocamlopt - assuming that I generate 
>>and replace cmo/cma files with cmx/cmxa files?
> 
> 
> Except these linking problems, you can assume identical behaviour.
> If there are significant discrepancies, please report them.

Actually, it looks like Nicolas brings up a large conceptual 
difference... I was at first tempted to say that his points about 
ocamlrun.dll did not apply to my case, but then I realized that my 
DLL, while primarily concerned with interfacing between OCaml and the 
C-land Scintilla editing widget, does in fact call on the OCaml 
runtime for things like string and tuple allocation.

Put a different way, ml_scintilla.dll definitely has a dependence on 
ocamlrun.dll - which logical need would not go away even if I 
repackaged the DLL as a static lib.

So, where (in both a conceptual as well as linking-specific terms) do 
I find the hooks (caml_alloc_string, caml_alloc_tuple, 
caml_named_value etc) into the OCaml runtime when it is embedded in my 
ocamlopt-generated executable?

Robert Roessler
roessler@rftp.com
http://www.rftp.com

Re: [Caml-list] ocamlopt and *using* DLLs

[Nicolas Cannasse]

> Put a different way, ml_scintilla.dll definitely has a dependence on
> ocamlrun.dll - which logical need would not go away even if I
> repackaged the DLL as a static lib.

Actually the symbols are only imported from ocamlrun.dll if you compile-with
or define the CAML_DLL preprocessor variable.
If you don't have CAML_DLL defined, symbols will be resolved at linking
time, and will then be found in your executable.
So it's possible to have a staticly linked library in both bytecode
(with -custom) and nativecode.

You can have a look at the OCaml Win32 API library Makefile that have
different ways of building and linking it
(http://ocaml-win32.sourceforge.net/).

Nicolas

Re: [Caml-list] ocamlopt and *using* DLLs

[Robert Roessler]

Nicolas Cannasse wrote:

> The main problem is that your DLL need to call Ocaml API, which it is linked
> with.
> 
> In bytecode there is no problem since the whole API is inside ocamlrun.dll
> so your DLL and ocamlrun.exe will simply use this API dll together.
> In native compilation however there is no more ocamlrun.dll, since the API
> is staticly linked into your executable. That means that your DLL that was
> working in bytecode will load ocamlrun.dll and call it while your executable
> is calling staticly linked API. This will result quite quickly in a crash
> since ocamlrun.dll is not initialized correctly.
> 
> The only possible way to get the DLL working is to patch it so it loads its
> API inside your.exe and not inside ocamlrun.dll. But then you will need a
> different version of the DLL for each application (!). It would be nice if
> OCaml could provide the equivalent of ocamlrun.dll for the native
> compilation mode.

Thanks for the explanation of the key difference between the models,
Nicolas!

I have this running just fine now, and it is not really as grim as you
make it sound above. :)  Since my DLL really only exists to provide an
interface between OCaml and C-land, I just followed the model used by
LablGTK, and build that interface one of two ways: "dllscintilla.dll"
for bytecode, and "libscintilla.lib" for native.

Robert Roessler
roessler@rftp.com
http://www.rftp.com

Re: [Caml-list] ocamlopt and *using* DLLs

[Robert Roessler]

Nicolas Cannasse wrote:

>>Put a different way, ml_scintilla.dll definitely has a dependence on
>>ocamlrun.dll - which logical need would not go away even if I
>>repackaged the DLL as a static lib.
> 
> 
> Actually the symbols are only imported from ocamlrun.dll if you compile-with
> or define the CAML_DLL preprocessor variable.
> If you don't have CAML_DLL defined, symbols will be resolved at linking
> time, and will then be found in your executable.
> So it's possible to have a staticly linked library in both bytecode
> (with -custom) and nativecode.

The proper use of CAML_DLL, along with your exposition on the
differences between the bytecode and native runtime models was key to
making this all work for me... but it is now.

> You can have a look at the OCaml Win32 API library Makefile that have
> different ways of building and linking it
> (http://ocaml-win32.sourceforge.net/).

This is actually funny - about a year and a half ago, when I was first
looking at OCaml, I corresponded with Harry Chomsky and pointed out a
bug in the makefile being distributed with this project - having to do
with [not] defining CAML_DLL! :)

I also contributed a small amount of code to the project, but ended up
not being able to use the "OCaml Win32 API" because of the license. :(

Robert Roessler
roessler@rftp.com
http://www.rftp.com

Re: [Caml-list] ocamlopt and *using* DLLs

[Robert Roessler]

Jacques Garrigue wrote:

> From: Robert Roessler <roessler@rftp.com>
> ...
>>2b) can I in general expect that an app that works well with ocamlc 
>>and a collection of OCaml cma libs and compiled DLLs (LablGTK and 
>>associated runtime) will work with ocamlopt - assuming that I generate 
>>and replace cmo/cma files with cmx/cmxa files?
> 
> 
> Except these linking problems, you can assume identical behaviour.
> If there are significant discrepancies, please report them.

I have a request for an additional [very helpful] export from 
dlllablgtk2 which I will detail on the LablGTK list... but in terms of 
  discrepancies, I only see a minor one:

I notice that the fonts in the editing widget itself as well as in the 
surrounding menu and status bars appear differently [in native code] 
relative to their appearance in bytecode.

It looks like the same font face and size in each case, but where the 
bytecode version seems to be "bold", the native one looks thin and 
"spidery".  I can supply screenshots, run further tests, whatever 
would be helpful if someone wants to look into this.

I have one more issue which I will bring up in this group, since it 
presumably effects anyone using GTK apps on Windows: what can I do 
about the [gratuitous to me] console window opened with my OCaml/GTK app?

Aside from the philosophical debate about having a "stdout" or not, a 
very large number of Windows app certainly manage without one. :)

Did someone on this or the LablGTK list have a piece of code yielding 
a tiny/invisible console window for GTK apps on Windows?  I thought I 
had seen something like that previously, but my searches are coming up 
empty.

Robert Roessler
roessler@rftp.com
http://www.rftp.com

Re: [Caml-list] ocamlopt and *using* DLLs

[Anatoly Zaretsky]

On 5/27/05, Robert Roessler <roessler@rftp.com> wrote:
> ...
> I have one more issue which I will bring up in this group, since it
> presumably effects anyone using GTK apps on Windows: what can I do
> about the [gratuitous to me] console window opened with my OCaml/GTK app?
> 
> Aside from the philosophical debate about having a "stdout" or not, a
> very large number of Windows app certainly manage without one. :)
> 
> Did someone on this or the LablGTK list have a piece of code yielding
> a tiny/invisible console window for GTK apps on Windows?  I thought I
> had seen something like that previously, but my searches are coming up
> empty.

There is a small hack in ocaml-win32 for console window elimination,
mkwinapp (http://cvs.sourceforge.net/viewcvs.py/*checkout*/ocaml-win32/ocaml-win32/mkwinapp.ml)

Re: [Caml-list] ocamlopt and *using* DLLs

[Nicolas Cannasse]

> ...
> I have one more issue which I will bring up in this group, since it
> presumably effects anyone using GTK apps on Windows: what can I do
> about the [gratuitous to me] console window opened with my OCaml/GTK app?
>
> Aside from the philosophical debate about having a "stdout" or not, a
> very large number of Windows app certainly manage without one. :)
>
> Did someone on this or the LablGTK list have a piece of code yielding
> a tiny/invisible console window for GTK apps on Windows?  I thought I
> had seen something like that previously, but my searches are coming up
> empty.

> There is a small hack in ocaml-win32 for console window
> elimination, mkwinapp

The other possibility is to call  Win32.free_console()
Be careful however, if you try to write to stdout that will raise an
exception.

Nicolas

Re: [Caml-list] ocamlopt and *using* DLLs

[Robert Roessler]

Nicolas Cannasse wrote:

>>>You can have a look at the OCaml Win32 API library Makefile that have
>>>different ways of building and linking it
>>>(http://ocaml-win32.sourceforge.net/).
>>
>>This is actually funny - about a year and a half ago, when I was first
>>looking at OCaml, I corresponded with Harry Chomsky and pointed out a
>>bug in the makefile being distributed with this project - having to do
>>with [not] defining CAML_DLL! :)
>>
>>I also contributed a small amount of code to the project, but ended up
>>not being able to use the "OCaml Win32 API" because of the license. :(
> 
> 
> Since then I had talk with Harry and I convince him that using LGPL was the
> way to go. The API now is quite complete and usable.

Thanks for answering this, but I had asked him (Harry Chomsky) on
November 8, 2003 if he would consider the "OCaml special exception for
linking" so that his library's licensing terms would be more in line
with [some] other packages and OCaml itself... to date, my email has
not been answered.

And I just checked the current "OCaml Win32" download, and it is as
you say above - LGPL (with no "OCaml linking exception").

This is unfortunate, as I would like to both use *and* contribute to
this project... but, c'est la licensing disagreement. :)

Robert Roessler
roessler@rftp.com
http://www.rftp.com

Re: [Caml-list] ocamlopt and *using* DLLs

[Robert Roessler]

I have resent the original message. :)

I would be happy to discuss this further with yourself and/or 
Nicolas... although off-list might be a better venue, since licensing 
discussions are not always well-received here.

Harry Chomsky wrote:

> Oh goodness...  I'm afraid your question about the license got lost from 
> my Inbox when I cleaned out all my Caml-related stuff last year.  I'm 
> sorry I never answered you!
> 
> I haven't taken any time to think about or work on OCaml-Win32 since 
> then, but I made Nicolas an administrator of the SourceForge project and 
> hoped that he could keep things moving.  I actually don't have strong 
> opinions about the best license for this project.  Nicolas, do you think 
> there is an advantage (or disadvantage) to changing the license to "LPGL 
> + linking exception", as Robert requests?  If you both agree that that 
> would be better, I would be happy to reissue the code with that change 
> in the license.
> 
> ----- Original Message ----- From: "Robert Roessler" <roessler@rftp.com>
> To: "Caml-list" <caml-list@inria.fr>
> Sent: Wednesday, June 01, 2005 1:32 PM
> Subject: Re: [Caml-list] ocamlopt and *using* DLLs
> 
> 
>> Nicolas Cannasse wrote:
>>
>>>>> You can have a look at the OCaml Win32 API library Makefile that have
>>>>> different ways of building and linking it
>>>>> (http://ocaml-win32.sourceforge.net/).
>>>>
>>>>
>>>> This is actually funny - about a year and a half ago, when I was first
>>>> looking at OCaml, I corresponded with Harry Chomsky and pointed out a
>>>> bug in the makefile being distributed with this project - having to do
>>>> with [not] defining CAML_DLL! :)
>>>>
>>>> I also contributed a small amount of code to the project, but ended up
>>>> not being able to use the "OCaml Win32 API" because of the license. :(
>>>
>>>
>>>
>>> Since then I had talk with Harry and I convince him that using LGPL 
>>> was the
>>> way to go. The API now is quite complete and usable.
>>
>>
>> Thanks for answering this, but I had asked him (Harry Chomsky) on
>> November 8, 2003 if he would consider the "OCaml special exception for
>> linking" so that his library's licensing terms would be more in line
>> with [some] other packages and OCaml itself... to date, my email has
>> not been answered.
>>
>> And I just checked the current "OCaml Win32" download, and it is as
>> you say above - LGPL (with no "OCaml linking exception").
>>
>> This is unfortunate, as I would like to both use *and* contribute to
>> this project... but, c'est la licensing disagreement. :)

Robert Roessler
roessler@rftp.com
http://www.rftp.com