[Caml-list] "Inconsistent assumptions" when moving files across subdirectories

[Caml-list] "Inconsistent assumptions" when moving files across subdirectories

[Josh Watzman]

I've noticed that it's pretty easy to confuse ocamlc/ocamlopt when moving a module across subdirectories. Here's an example, the most minimized repro I could get; it uses ocamlbuild, but a similar problem happens if you use OCamlMakefile and I assume other build systems. https://gist.github.com/jwatzman/9979951afb5b87304c18 -- running that will consistently terminate with the dreaded

> Error: Files main.cmx and a/quux.cmx
>        make inconsistent assumptions over interface Quux

(The script flips the quux module back and forth twice, but that's only to exhibit the problem on both 4.01 and 4.02; you can get the same problem with only one move of quux.ml, but which way you need to move it depends on which version of ocaml you're using.)

A clean build will of course resolve the problem, but that's quite annoying to have to go broadcast to a large team, particularly when the build may take many minutes, and when this problem is specific to the OCaml parts of our system (a humongous C++ codebase never requires a clean rebuild). Renaming a module across subdirectories doesn't seem like that uncommon of an operation.

The root problem seems to be that ocamlc/ocamlopt are picking up build artifacts by directory only, and can't be explicitly told which artifacts to pick up, and so they are picking up the "wrong" quux.cmi/cmx left over in a build directory, which ocamlbuild should be cleaning up. Is that right? Is there any way to tell ocamlc/ocamlopt not to do things by directory, but to be more explicit, for the usages of build systems?

Not working around this limitation of ocamlc/ocamlopt seems like a bug in ocamlbuild, no? I'm a bit surprised by it though, given that I've found the same problem in other build systems -- have other folks not run into this? How do other teams deal with this, trying to avoid clean builds?

Thanks!
Josh Watzman

Re: [Caml-list] "Inconsistent assumptions" when moving files across subdirectories

[Boris Yakobowski]

[-- Attachment #1: Type: text/plain, Size: 2646 bytes --]

Hi,

I can confirm you're not alone, unfortunately. In Frama-C (which uses good
ol' make), we have added a new target, dubbed 'smartclean', that removes
all orphaned .cm*. More precisely, if there exists a .cm* without a
corresponding .ml*, the .cm* is removed.

This is quite hackish, and not entirely satisfactory. Still, it's always
better than performing a full clean/rebuild.

HTH,

On Mon, Jun 1, 2015 at 11:52 PM, Josh Watzman <jwatzman@fb.com> wrote:

> I've noticed that it's pretty easy to confuse ocamlc/ocamlopt when moving
> a module across subdirectories. Here's an example, the most minimized repro
> I could get; it uses ocamlbuild, but a similar problem happens if you use
> OCamlMakefile and I assume other build systems.
> https://gist.github.com/jwatzman/9979951afb5b87304c18 -- running that
> will consistently terminate with the dreaded
>
> > Error: Files main.cmx and a/quux.cmx
> >        make inconsistent assumptions over interface Quux
>
> (The script flips the quux module back and forth twice, but that's only to
> exhibit the problem on both 4.01 and 4.02; you can get the same problem
> with only one move of quux.ml, but which way you need to move it depends
> on which version of ocaml you're using.)
>
> A clean build will of course resolve the problem, but that's quite
> annoying to have to go broadcast to a large team, particularly when the
> build may take many minutes, and when this problem is specific to the OCaml
> parts of our system (a humongous C++ codebase never requires a clean
> rebuild). Renaming a module across subdirectories doesn't seem like that
> uncommon of an operation.
>
> The root problem seems to be that ocamlc/ocamlopt are picking up build
> artifacts by directory only, and can't be explicitly told which artifacts
> to pick up, and so they are picking up the "wrong" quux.cmi/cmx left over
> in a build directory, which ocamlbuild should be cleaning up. Is that
> right? Is there any way to tell ocamlc/ocamlopt not to do things by
> directory, but to be more explicit, for the usages of build systems?
>
> Not working around this limitation of ocamlc/ocamlopt seems like a bug in
> ocamlbuild, no? I'm a bit surprised by it though, given that I've found the
> same problem in other build systems -- have other folks not run into this?
> How do other teams deal with this, trying to avoid clean builds?
>
> Thanks!
> Josh Watzman
>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs




-- 
Boris

[-- Attachment #2: Type: text/html, Size: 3546 bytes --]

Re: [Caml-list] "Inconsistent assumptions" when moving files across subdirectories

[Gerd Stolpmann]

[-- Attachment #1: Type: text/plain, Size: 2788 bytes --]

I would have thought ocamlbuild is immune to this problem because it
puts the translated files into a separate directory tree. But maybe this
problem was overlooked.

In a more traditional "make" setup, you cannot distinguish between a cmi
left over from a move and a "naked" cmi that intentionally is not
accompanied by a ml/mli (i.e. a binary install). But this is exactly
possible when files are copied to a build tree.

Gerd

Am Montag, den 01.06.2015, 21:52 +0000 schrieb Josh Watzman:
> I've noticed that it's pretty easy to confuse ocamlc/ocamlopt when moving a module across subdirectories. Here's an example, the most minimized repro I could get; it uses ocamlbuild, but a similar problem happens if you use OCamlMakefile and I assume other build systems. https://gist.github.com/jwatzman/9979951afb5b87304c18 -- running that will consistently terminate with the dreaded
> 
> > Error: Files main.cmx and a/quux.cmx
> >        make inconsistent assumptions over interface Quux
> 
> (The script flips the quux module back and forth twice, but that's only to exhibit the problem on both 4.01 and 4.02; you can get the same problem with only one move of quux.ml, but which way you need to move it depends on which version of ocaml you're using.)
> 
> A clean build will of course resolve the problem, but that's quite annoying to have to go broadcast to a large team, particularly when the build may take many minutes, and when this problem is specific to the OCaml parts of our system (a humongous C++ codebase never requires a clean rebuild). Renaming a module across subdirectories doesn't seem like that uncommon of an operation.
> 
> The root problem seems to be that ocamlc/ocamlopt are picking up build artifacts by directory only, and can't be explicitly told which artifacts to pick up, and so they are picking up the "wrong" quux.cmi/cmx left over in a build directory, which ocamlbuild should be cleaning up. Is that right? Is there any way to tell ocamlc/ocamlopt not to do things by directory, but to be more explicit, for the usages of build systems?
> 
> Not working around this limitation of ocamlc/ocamlopt seems like a bug in ocamlbuild, no? I'm a bit surprised by it though, given that I've found the same problem in other build systems -- have other folks not run into this? How do other teams deal with this, trying to avoid clean builds?
> 
> Thanks!
> Josh Watzman
> 
> 

-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
My OCaml site:          http://www.camlcity.org
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [Caml-list] "Inconsistent assumptions" when moving files across subdirectories

[Fabrice Le Fessant]

On Tue, Jun 2, 2015 at 10:34 AM, Gerd Stolpmann <info@gerd-stolpmann.de> wrote:
> I would have thought ocamlbuild is immune to this problem because it
> puts the translated files into a separate directory tree. But maybe this
> problem was overlooked.

I used ocamlbuild once a few years ago, and got these
"inconsistent-assumptions" errors many times. Some of them were
related to packing (if A would be packed as B.A, the a.cmi file would
still conflict with another toplevel A module).

Maybe also the sanitization done by ocamlbuild is only done in the
source directory, not in the _build directory ?

--Fabrice