Dependencies and rebuilding

Dependencies and rebuilding

[Jakob Lichtenberg]

If I change the body of functions in a base library, but not the
externally visible signature, I still have to recompile the consumers of
the base library prior to linking the main application.  While this is
not a problem in the trivial case I'll show beneath, it may be a concern
from a componentization and scalability point of view.  Regular C code
does not have this limitation.  This e-mail to request why the design is
as it is?


Example:

===
>type base.ml
let base () = 2+8;;

>ocamlopt -c base.ml

>type consumer.ml
let _ = Printf.printf "Base.base: %d" (Base.base());;

>ocamlopt -c consumer.ml

>ocamlopt base.cmx consumer.cmx -o app.exe

>app.exe
Base.base: 10

>notepad base.ml

>type base.ml
let base () = 2+9;;

>ocamlopt -c base.ml

>ocamlopt base.cmx consumer.cmx -o app.exe
Files consumer.cmx and base.cmx
make inconsistent assumptions over implementation Base

>ocamlopt -c consumer.ml

>ocamlopt base.cmx consumer.cmx -o app.exe

>app.exe
Base.base: 11
===

Thanks,

- Jakob Lichtenberg

Re: [Caml-list] Dependencies and rebuilding

[Olivier Andrieu]

On 3/7/07, Jakob Lichtenberg <jakobl@windows.microsoft.com> wrote:
> If I change the body of functions in a base library, but not the
> externally visible signature, I still have to recompile the consumers of
> the base library prior to linking the main application.  While this is
> not a problem in the trivial case I'll show beneath, it may be a concern
> from a componentization and scalability point of view.  Regular C code
> does not have this limitation.  This e-mail to request why the design is
> as it is?

I'd say cross-module inlining of code ?

This happens because ocamlopt finds the base.cmx file during the
compilation of consumer. If you put your base module in a library and
remove the .cmx file, consumer won't depend on the implementation of
base, only on its interface.

-- 
  Olivier

RE: [Caml-list] Dependencies and rebuilding

[Jakob Lichtenberg]

Olivier,

First of all: Your statement is correct.  The following works great:

===
>type base.ml
let base () = 2+8;;

>ocamlopt -c base.ml

>ocamlopt -a base.cmx -o base.cmxa

>del base.cmx base.obj

>type consumer.ml
let _ = Printf.printf "Base.base: %d" (Base.base());;

>ocamlopt -c consumer.ml

>ocamlopt base.cmxa consumer.cmx -o app.exe

>app.exe
Base.base: 10

>notepad base.ml

>ocamlopt -c base.ml

>ocamlopt -a base.cmx -o base.cmxa

>del base.cmx base.obj

>ocamlopt base.cmxa consumer.cmx -o app.exe

>app.exe
Base.base: 11
===

Now, wrt. to cross-module inlining of code: Ohh, that's a very good
reason, I was not aware of this language feature.  Is there a higher
level of inlining going on if I leave the .cmx files next to the .cmxa
file? Maybe I should not use .cmxa files at all, for maximum inlining?
If .cmx files is present then what is the .cmxa file used for?

Thanks,

- Jakob

-----Original Message-----
From: oandrieu@gmail.com [mailto:oandrieu@gmail.com] On Behalf Of
Olivier Andrieu
Sent: Wednesday, March 07, 2007 1:25 PM
To: Jakob Lichtenberg
Cc: caml-list@inria.fr; Donn Terry
Subject: Re: [Caml-list] Dependencies and rebuilding

On 3/7/07, Jakob Lichtenberg <jakobl@windows.microsoft.com> wrote:
> If I change the body of functions in a base library, but not the
> externally visible signature, I still have to recompile the consumers
of
> the base library prior to linking the main application.  While this is
> not a problem in the trivial case I'll show beneath, it may be a
concern
> from a componentization and scalability point of view.  Regular C code
> does not have this limitation.  This e-mail to request why the design
is
> as it is?

I'd say cross-module inlining of code ?

This happens because ocamlopt finds the base.cmx file during the
compilation of consumer. If you put your base module in a library and
remove the .cmx file, consumer won't depend on the implementation of
base, only on its interface.

-- 
  Olivier

Re: Dependencies and rebuilding

[Zheng Li]

Hi,

Jakob Lichtenberg <jakobl@windows.microsoft.com> writes:
> If I change the body of functions in a base library, but not the
> externally visible signature, I still have to recompile the consumers of
> the base library prior to linking the main application.  While this is
> not a problem in the trivial case I'll show beneath, it may be a concern
> from a componentization and scalability point of view.  Regular C code
> does not have this limitation.  This e-mail to request why the design is
> as it is?

You compile your tests case with ocamlopt which produce cmx file. Unlike cmo
file, a cmx file does depend on the cmxs of the modules it use. The result of
ocamldep will tell you about that. The reason for that is a .cmx files carries
information for cross-module optimization, once recompiled, its visitor modules
should change correspondingly. 

There should be some FAQ somewhere contains more information, you may just look
up.


-li

-- 
Zheng Li
http://www.pps.jussieu.fr/~li

Re: [Caml-list] Dependencies and rebuilding

[Alain Frisch]

Jakob Lichtenberg wrote:
> If I change the body of functions in a base library, but not the
> externally visible signature, I still have to recompile the consumers of
> the base library prior to linking the main application.  While this is
> not a problem in the trivial case I'll show beneath, it may be a concern
> from a componentization and scalability point of view.  Regular C code
> does not have this limitation.  This e-mail to request why the design is
> as it is?

A .cmx file basically contains information:
- for the linker, most importantly dependencies information (md5 of .cmi
and .cmx used to compile them);
- for the compilation of other units which refer to this unit:
cross-module optimization information (linker symbols for direct
function calls, inlining).

A .cmx file also contains the information needed to compile against
*and* link with a unit compiled with -for-pack.

Note that the real object code is not found in the .cmx file, but in the
corresponding .o/.obj/... file.

When ocamlopt compiles a file b.ml which refers to a module A, it must
be able to find a.cmi (as with ocamlc), and it will also look for a.cmx.
If it cannot find a.cmx, then compilation will still succeed (without
cross-module optimization), but linking against A will fail if A had
been compiled with -for-pack. If you don't use -for-pack, you can
achieve the level of separate compilation you want by hiding .cmx files
to the compiler.

Libraries (.cmxa + .a/.lib) provide a convenient way to:
- mention a single file on the linker command line instead of many .cmx
files;
- let the linker figure out which units in the library are really needed;
- add C linker options (e.g. extra C objects/libraries to link with);
- combine many .o/.obj files together (but you still need the .cmi files
for "public" modules).

Libraries files are only used by the linker, and never by the compiler.
You can still provide the .cmx files in addition to the .cmxa file if
you want to enable cross-module optimization, and you have to provide
them if some public modules in the library have been compiled with
-for-pack.


-- Alain

Re: [Caml-list] Dependencies and rebuilding

[malc]

On Thu, 8 Mar 2007, Alain Frisch wrote:

> Jakob Lichtenberg wrote:
>> If I change the body of functions in a base library, but not the
>> externally visible signature, I still have to recompile the consumers of
>> the base library prior to linking the main application.  While this is
>> not a problem in the trivial case I'll show beneath, it may be a concern
>> from a componentization and scalability point of view.  Regular C code
>> does not have this limitation.  This e-mail to request why the design is
>> as it is?
>
> A .cmx file basically contains information:
> - for the linker, most importantly dependencies information (md5 of .cmi
> and .cmx used to compile them);
> - for the compilation of other units which refer to this unit:
> cross-module optimization information (linker symbols for direct
> function calls, inlining).

That's not all, there's also information embedded to make direct calling
possible: http://groups.google.com/group/fa.caml/msg/52264cef8f6da97f

[..snip..]

-- 
vale

Re: [Caml-list] Dependencies and rebuilding

[Alain Frisch]

malc wrote:
>> - for the compilation of other units which refer to this unit:
>> cross-module optimization information (linker symbols for direct
>> function calls, inlining).
> 
> That's not all, there's also information embedded to make direct calling
> possible: http://groups.google.com/group/fa.caml/msg/52264cef8f6da97f

I guess that's exactly what I meant by "linker symbols for direct
function calls".

-- Alain

Re: [Caml-list] Dependencies and rebuilding

[malc]

On Fri, 9 Mar 2007, Alain Frisch wrote:

> malc wrote:
>>> - for the compilation of other units which refer to this unit:
>>> cross-module optimization information (linker symbols for direct
>>> function calls, inlining).
>>
>> That's not all, there's also information embedded to make direct calling
>> possible: http://groups.google.com/group/fa.caml/msg/52264cef8f6da97f
>
> I guess that's exactly what I meant by "linker symbols for direct
> function calls".

Indeed, this portion of the message elluded me, sorry for the noise.

-- 
vale