[Caml-list] swapping variant modules via subdirectories fails at link time

[Caml-list] swapping variant modules via subdirectories fails at link time

[Alexy Khrabrov]

I have a module a.ml, containing some functions I want to alter in a version of my program.  I'd like to do it on the command line at link time, to obtain either the original, or an altered version.  

Here's the original make line, simplified:

%.cmx: %.ml
	ocamlfind ocamlopt $(DEBUG) $(OPTFLAGS) -package $(PACKAGES) -c $^ -o $@

prog.opt: lib.cmxa a.cmx x.cmx prog.ml
	ocamlfind ocamlopt $(DEBUG) $(OPTFLAGS) -package $(PACKAGES) -linkpkg $^ -o $@

I tried to create the variant by placing an altered version of a.ml in a subdirectory, var/, and linking that:

prog.var.opt: lib.cmxa var/a.cmx x.cmx prog.ml
	ocamlfind ocamlopt $(DEBUG) $(OPTFLAGS) -package $(PACKAGES) -linkpkg $^ -o $@

Further, I've created a.mli at the root level, and compiled it with

ocamlfind ocamlc a.mli

-- it needs lib.cmo to work, a byte-code version of lib.cmxa, which I make first.

With that a.cmi at the root level, I can remake prog.opt fine.  If I copy or symlink a.cmi to var/, or symlink a.mli there, and try to make prog.var.opt, I get an error at prog.ml that files var/a.cmx and x.cmx make inconsistent assumptions about the interface A.

The var/a.cmx is made by the same above pattern rule as ./a.cmx.  Even when ./a.cmi and var/a.cmi are the same, the same inconsistent error shows up!  Renaming ./a.ml out of the way and symlinking var/a.ml to ./ remakes prog.opt fine, which I can then rename prog.var.opt manually and hope to do it every time I need that, yet it's tiresome.   What prevents the original scheme from working?

-- Alexy

Re: [Caml-list] swapping variant modules via subdirectories fails at link time

[Goswin von Brederlow]

Alexy Khrabrov <deliverable@gmail.com> writes:

> I have a module a.ml, containing some functions I want to alter in a version of my program.  I'd like to do it on the command line at link time, to obtain either the original, or an altered version.  
>
> Here's the original make line, simplified:
>
> %.cmx: %.ml
> 	ocamlfind ocamlopt $(DEBUG) $(OPTFLAGS) -package $(PACKAGES) -c $^ -o $@
>
> prog.opt: lib.cmxa a.cmx x.cmx prog.ml
> 	ocamlfind ocamlopt $(DEBUG) $(OPTFLAGS) -package $(PACKAGES) -linkpkg $^ -o $@
>
> I tried to create the variant by placing an altered version of a.ml in a subdirectory, var/, and linking that:
>
> prog.var.opt: lib.cmxa var/a.cmx x.cmx prog.ml
> 	ocamlfind ocamlopt $(DEBUG) $(OPTFLAGS) -package $(PACKAGES) -linkpkg $^ -o $@
>
> Further, I've created a.mli at the root level, and compiled it with
>
> ocamlfind ocamlc a.mli
>
> -- it needs lib.cmo to work, a byte-code version of lib.cmxa, which I make first.
>
> With that a.cmi at the root level, I can remake prog.opt fine.  If I copy or symlink a.cmi to var/, or symlink a.mli there, and try to make prog.var.opt, I get an error at prog.ml that files var/a.cmx and x.cmx make inconsistent assumptions about the interface A.
>
> The var/a.cmx is made by the same above pattern rule as ./a.cmx.  Even when ./a.cmi and var/a.cmi are the same, the same inconsistent error shows up!  Renaming ./a.ml out of the way and symlinking var/a.ml to ./ remakes prog.opt fine, which I can then rename prog.var.opt manually and hope to do it every time I need that, yet it's tiresome.   What prevents the original scheme from working?
>
> -- Alexy

You need to compile a.ml inside var/ instead of compiling var/a.ml.

MfG
        Goswin

Re: [Caml-list] swapping variant modules via subdirectories fails at link time

[Alexy Khrabrov]

On Apr 15, 2011, at 4:37 AM, Goswin von Brederlow wrote:

> Alexy Khrabrov <deliverable@gmail.com> writes:
> 
>> I have a module a.ml, containing some functions I want to alter in a version of my program.  I'd like to do it on the command line at link time, to obtain either the original, or an altered version.  
>> 
>> Here's the original make line, simplified:
>> 
>> %.cmx: %.ml
>> 	ocamlfind ocamlopt $(DEBUG) $(OPTFLAGS) -package $(PACKAGES) -c $^ -o $@
>> 
>> prog.opt: lib.cmxa a.cmx x.cmx prog.ml
>> 	ocamlfind ocamlopt $(DEBUG) $(OPTFLAGS) -package $(PACKAGES) -linkpkg $^ -o $@
>> 
>> I tried to create the variant by placing an altered version of a.ml in a subdirectory, var/, and linking that:
>> 
>> prog.var.opt: lib.cmxa var/a.cmx x.cmx prog.ml
>> 	ocamlfind ocamlopt $(DEBUG) $(OPTFLAGS) -package $(PACKAGES) -linkpkg $^ -o $@
>> 
>> Further, I've created a.mli at the root level, and compiled it with
>> 
>> ocamlfind ocamlc a.mli
>> 
>> -- it needs lib.cmo to work, a byte-code version of lib.cmxa, which I make first.
>> 
>> With that a.cmi at the root level, I can remake prog.opt fine.  If I copy or symlink a.cmi to var/, or symlink a.mli there, and try to make prog.var.opt, I get an error at prog.ml that files var/a.cmx and x.cmx make inconsistent assumptions about the interface A.
>> 
>> The var/a.cmx is made by the same above pattern rule as ./a.cmx.  Even when ./a.cmi and var/a.cmi are the same, the same inconsistent error shows up!  Renaming ./a.ml out of the way and symlinking var/a.ml to ./ remakes prog.opt fine, which I can then rename prog.var.opt manually and hope to do it every time I need that, yet it's tiresome.   What prevents the original scheme from working?
>> 
>> -- Alexy
> 
> You need to compile a.ml inside var/ instead of compiling var/a.ml.
> 
> MfG
>        Goswin


I tried to do that, and found that my type definitions have all to be present -- so I had to symlink or copy *.cmi to var/.  Even then, I couldn't get a top-level cmx to accept the equivalence of a.cmx to var/a.cmx.

This is really disappointing, and makes me think about OCaml compilation model.  In order for the type inference to see that your type definitions are equivalent, it uses all the .cmi files in scope without telling you so explicitly.  Moving even one .cmx into a subdirectory is nearly impossible unless you check every type and see where it was defined, and make sure the subdirectory has access to these .cmi files; even then I get an error that two files make inconsistent assumptions about the implementation which is one of them.

Good old C has separate compilation working, which doesn't work for OCaml so easily.  A command line which links fine will fail if you move a file into a subdirectory and the debugging process is utterly non-trivial.  Doesn't anybody work with versions of the same module compiled to be swappable, against the same .mli?!

-- Aexy

Re: [Caml-list] swapping variant modules via subdirectories fails at link time

[Stéphane Glondu]

Le 15/04/2011 01:32, Alexy Khrabrov a écrit :
> With that a.cmi at the root level, I can remake prog.opt fine.  If I copy or symlink a.cmi to var/, or symlink a.mli there, and try to make prog.var.opt, I get an error at prog.ml that files var/a.cmx and x.cmx make inconsistent assumptions about the interface A.

Aren't you facing inconsistent assumptions over *implementations* instead?

You have to compile your two implementations of A in separate
directories, and compile whatever depends on A in your root directory
with only a.cmi (for example), so that no a.cmx is seen. Then you can
link using either .cmx file.

The reason is that the compiler uses information from .cmx files if they
are available to do some optimizations such as inling. These
optimizations are turned off (but compilations still succeeds) if the
.cmx are missing.

Maybe I can make it clearer with an example tree:

.
|-- a.cmi
|-- a.mli
|-- dir1
|   |-- a.cmi -> ../a.cmi
|   |-- a.cmx
|   |-- a.ml
|   |-- a.mli -> ../a.mli
|   `-- a.o
`-- dir2
    |-- a.cmi -> ../a.cmi
    |-- a.cmx
    |-- a.ml
    |-- a.mli -> ../a.mli
    `-- a.o

Of course, dir1 and dir2 must *not* be in the include patch when
compiling stuff depending on module A.

The ocamlobjinfo (in 3.12) or dumpapprox (in < 3.12) can be useful here:
no A must appear in "implementation assumptions" in whatever
reverse-dependencies of A if you intend to provides several
implementations of A.


Hope this helps,

-- 
Stéphane

Re: [Caml-list] swapping variant modules via subdirectories fails at link time

[Alexy Khrabrov]

On Apr 17, 2011, at 4:49 PM, Stéphane Glondu wrote:

> Le 15/04/2011 01:32, Alexy Khrabrov a écrit :
>> With that a.cmi at the root level, I can remake prog.opt fine.  If I copy or symlink a.cmi to var/, or symlink a.mli there, and try to make prog.var.opt, I get an error at prog.ml that files var/a.cmx and x.cmx make inconsistent assumptions about the interface A.
> 
> Aren't you facing inconsistent assumptions over *implementations* instead?
> 
> You have to compile your two implementations of A in separate
> directories, and compile whatever depends on A in your root directory
> with only a.cmi (for example), so that no a.cmx is seen. Then you can
> link using either .cmx file.
> 
> The reason is that the compiler uses information from .cmx files if they
> are available to do some optimizations such as inling. These
> optimizations are turned off (but compilations still succeeds) if the
> .cmx are missing.
> 
> Maybe I can make it clearer with an example tree:
> 
> .
> |-- a.cmi
> |-- a.mli
> |-- dir1
> |   |-- a.cmi -> ../a.cmi
> |   |-- a.cmx
> |   |-- a.ml
> |   |-- a.mli -> ../a.mli
> |   `-- a.o
> `-- dir2
>    |-- a.cmi -> ../a.cmi
>    |-- a.cmx
>    |-- a.ml
>    |-- a.mli -> ../a.mli
>    `-- a.o
> 
> Of course, dir1 and dir2 must *not* be in the include patch when
> compiling stuff depending on module A.
> 
> The ocamlobjinfo (in 3.12) or dumpapprox (in < 3.12) can be useful here:
> no A must appear in "implementation assumptions" in whatever
> reverse-dependencies of A if you intend to provides several
> implementations of A.

Stéphane -- this is great, thanks!  Although in my case, I need highly optimal executable, so I'm willing to symlink files from their subdirectories onto the top level and link everything there.  Yet I'll use the technique when I need to quickly swap implementations without recompiling, and most importantly, without renaming/resymlinking, when I need to compare versions.  This is great as each separate .cmx can be used under its own path in a Makefile, leading to several distinctly named executables.  Embedding a string in each .cmx can also allow status reporting, etc.

-- Alexy