[Caml-list] A way to avoid "WARNING: myocamlbuild.cmi occurs in several directories"?

[Caml-list] A way to avoid "WARNING: myocamlbuild.cmi occurs in several directories"?

[Anthony Tavener]

[-- Attachment #1: Type: text/plain, Size: 2232 bytes --]

(Or a better structure to my build process?)

I've recently switched to OPAM and ocamlfind (from manual management and
makefiles).

I have some libraries which undergo frequent changes -- as frequent as
application code. For these, I don't "install" after every change; instead
I refer to the _build directory.

  ocaml_lib ~extern:true ~dir:"/home/anthony/src/gui/_build" "gui";

Now that I'm using ocamlfind (with ocamlbuild) these _build directories are
included in a more general sense... causing multiple myocamlbuild.cmi's to
be seen -- resulting in "findlib: [WARNING] Interface myocamlbuild.cmi
occurs in several directories"

Does someone know a way to avoid the inclusion of these spurious
myocamlbuild.cmi's, or to suppress the warning, or have another suggestion?

The obvious thing is adding an install step which dumps the interesting
library files in a local lib dir. But then I'd have two kinds of install: a
"package" install (using ocamlfind, and OPAM-friendly), and this
immediate-use local install. Yuck, I say.


Ultimately what I strive for is install-less build, and build dependency on
local library changes. For example:

(unit : dependencies)
 App1 : Lib2 Lib3
 App2 : Lib1 Lib2
 Lib1 : -
 Lib2 : -
 Lib3 : Lib1

<~/Lib1> touch lib1file.ml
<~/App1> make
  Build Lib2
  Build Lib1
  Build Lib3
  Build App1

No "install", as these are all in flux. The libraries are just like the
app-code but shared. Like they used to be before the world of package
management. ;) I'm sure others must still do this for internal development
too? Or does everyone work with libraries as explicitly built and
separately installed units? (Note: I do have some slowly-evolving libraries
which I install as packages via OPAM.)

To rephrase: Am I doing it all wrong? :D

I haven't figured out how to get ocamlbuild to handle library dependency
like this yet. The tools, or at least the examples of how to use them, seem
very geared toward usage of infrequently-changed packages. So any tips on
an example of using ocamlbuild in this manner would be great too!

I'm always hesitant to pester the mailing list, but it generally follows
days of frustration on my part, and I don't know any other OCaml people, so
thank-you!

 -Tony

[-- Attachment #2: Type: text/html, Size: 2931 bytes --]

Re: [Caml-list] A way to avoid "WARNING: myocamlbuild.cmi occurs in several directories"?

[Gabriel Scherer]

[-- Attachment #1: Type: text/plain, Size: 3518 bytes --]

I'm not trying to dismiss your workflow, it's new¹ to me and interesting.
But is there really such a benefit to the "install-less" part of your
setup? On the project I've worked on, installation always took a time
neglectible when compared to compilation or (even more so) unit-testing.
When you say "immediate use", is there any reason other than speed for
avoiding a local install? Did you measure the speed difference?

¹: we've been aware of "huge monolithic builds" used in some OCaml-using
companies, but they generally use a single build system to direct all the
build, instead of several separate-but-connected islands of libraries. I
also use Stog, a static blog engine by Maxence Guesdon, that allows plugin
and has built-in support for finding them through ocamlfind. In my periods
of Stog hacking I've always frequently modified and rebuild the plugins, in
synchronization with evolution Stog's code itself; but then I re-install
each of them separately and never felt that to be a problem.


On Thu, Dec 12, 2013 at 10:47 PM, Anthony Tavener <anthony.tavener@gmail.com
> wrote:

> (Or a better structure to my build process?)
>
> I've recently switched to OPAM and ocamlfind (from manual management and
> makefiles).
>
> I have some libraries which undergo frequent changes -- as frequent as
> application code. For these, I don't "install" after every change; instead
> I refer to the _build directory.
>
>   ocaml_lib ~extern:true ~dir:"/home/anthony/src/gui/_build" "gui";
>
> Now that I'm using ocamlfind (with ocamlbuild) these _build directories
> are included in a more general sense... causing multiple myocamlbuild.cmi's
> to be seen -- resulting in "findlib: [WARNING] Interface myocamlbuild.cmi
> occurs in several directories"
>
> Does someone know a way to avoid the inclusion of these spurious
> myocamlbuild.cmi's, or to suppress the warning, or have another suggestion?
>
> The obvious thing is adding an install step which dumps the interesting
> library files in a local lib dir. But then I'd have two kinds of install: a
> "package" install (using ocamlfind, and OPAM-friendly), and this
> immediate-use local install. Yuck, I say.
>
>
> Ultimately what I strive for is install-less build, and build dependency
> on local library changes. For example:
>
> (unit : dependencies)
>  App1 : Lib2 Lib3
>  App2 : Lib1 Lib2
>  Lib1 : -
>  Lib2 : -
>  Lib3 : Lib1
>
> <~/Lib1> touch lib1file.ml
> <~/App1> make
>   Build Lib2
>   Build Lib1
>   Build Lib3
>   Build App1
>
> No "install", as these are all in flux. The libraries are just like the
> app-code but shared. Like they used to be before the world of package
> management. ;) I'm sure others must still do this for internal development
> too? Or does everyone work with libraries as explicitly built and
> separately installed units? (Note: I do have some slowly-evolving libraries
> which I install as packages via OPAM.)
>
> To rephrase: Am I doing it all wrong? :D
>
> I haven't figured out how to get ocamlbuild to handle library dependency
> like this yet. The tools, or at least the examples of how to use them, seem
> very geared toward usage of infrequently-changed packages. So any tips on
> an example of using ocamlbuild in this manner would be great too!
>
> I'm always hesitant to pester the mailing list, but it generally follows
> days of frustration on my part, and I don't know any other OCaml people, so
> thank-you!
>
>  -Tony
>
>

[-- Attachment #2: Type: text/html, Size: 4434 bytes --]

Re: [Caml-list] A way to avoid "WARNING: myocamlbuild.cmi occurs in several directories"?

[Anthony Tavener]

[-- Attachment #1: Type: text/plain, Size: 6667 bytes --]

The benefit is editing library code as freely as application code. If I
have to do a separate build/install, and then build/install up the
hierarchy, this becomes a significant pain if it's a bit of code which I'm
doing a edit/test cycle with.

For example, say I have some procedural noise function (in noiselib) which
I'm enhancing, with respect to a particular application (app), and we have
this dependency: app > texturelib > noiselib. I'd rather not have this
process to iterate with:

<noiselib> make; make install_local
<noiselib> cd ../texturelib
<texturelib> make clean; make; make install_local
<texturelib> cd ../app
<app> make clean; make

Also note the "install_local" as opposed to "install" -- this would be a
minor ugliness too. I have applications which rely on more stable versions
of the libraries installed system-wide, and infrequently updated. If I'm
needing a special install process anyway, why install at all? Seems better
to use it in-place.

Creating an "app-local" version of the function in question (extending
modules via include) can work, and I do this, but sometimes the changes are
more complex or pervasive than a single function -- and you don't always
realize the changes are going to escape their initial scope.

I regularly develop libraries in parallel with application code. This was
also the case in the large studios I've worked in (games). Typically there
are a few tiers of library code -- where a library is common code, rather
than rigid code. In practice the library code doesn't change much because
other code depends on it and has fixed the interfaces, but when you do work
with library code, it's in the context of application and needs the benefit
of easy edit/test iteration. I think another view of libraries is much more
standalone, with unit-tests to determine correct behavior. Again, this can
apply in some cases, but perhaps it's less common with an interactive artsy
application like games -- where correctness is more a matter of end-result
behavior, rather than something you can build good unit-tests for.

Thank-you for asking, Gabriel... it helped me sort some of this out for
myself too! The stumbling blocks I'm encountering have made me feel a bit
uncertain -- there's an impedance mismatch which might indicate I'm doing
something inappropriate. Now I think I might just be working with a
slightly different bias: library as shared code, moreso than rigid code...
which might be the assumption the OCaml build ecosystem is working with.
I'm not developing/changing library code in a vacuum -- it's with respect
to higher level usage and results.

A reflection of the distinction between library use... the ossified
libraries (rarely changed, system-wide) are used as packages, while
libraries under active development as part of several apps are... not
installed, and are included by "use_xxx" of ocamlbuild.

Does this make sense?


On Thu, Dec 12, 2013 at 3:33 PM, Gabriel Scherer
<gabriel.scherer@gmail.com>wrote:

> I'm not trying to dismiss your workflow, it's new¹ to me and interesting.
> But is there really such a benefit to the "install-less" part of your
> setup? On the project I've worked on, installation always took a time
> neglectible when compared to compilation or (even more so) unit-testing.
> When you say "immediate use", is there any reason other than speed for
> avoiding a local install? Did you measure the speed difference?
>
> ¹: we've been aware of "huge monolithic builds" used in some OCaml-using
> companies, but they generally use a single build system to direct all the
> build, instead of several separate-but-connected islands of libraries. I
> also use Stog, a static blog engine by Maxence Guesdon, that allows plugin
> and has built-in support for finding them through ocamlfind. In my periods
> of Stog hacking I've always frequently modified and rebuild the plugins, in
> synchronization with evolution Stog's code itself; but then I re-install
> each of them separately and never felt that to be a problem.
>
>
> On Thu, Dec 12, 2013 at 10:47 PM, Anthony Tavener <
> anthony.tavener@gmail.com> wrote:
>
>> (Or a better structure to my build process?)
>>
>> I've recently switched to OPAM and ocamlfind (from manual management and
>> makefiles).
>>
>> I have some libraries which undergo frequent changes -- as frequent as
>> application code. For these, I don't "install" after every change; instead
>> I refer to the _build directory.
>>
>>   ocaml_lib ~extern:true ~dir:"/home/anthony/src/gui/_build" "gui";
>>
>> Now that I'm using ocamlfind (with ocamlbuild) these _build directories
>> are included in a more general sense... causing multiple myocamlbuild.cmi's
>> to be seen -- resulting in "findlib: [WARNING] Interface myocamlbuild.cmi
>> occurs in several directories"
>>
>> Does someone know a way to avoid the inclusion of these spurious
>> myocamlbuild.cmi's, or to suppress the warning, or have another suggestion?
>>
>> The obvious thing is adding an install step which dumps the interesting
>> library files in a local lib dir. But then I'd have two kinds of install: a
>> "package" install (using ocamlfind, and OPAM-friendly), and this
>> immediate-use local install. Yuck, I say.
>>
>>
>> Ultimately what I strive for is install-less build, and build dependency
>> on local library changes. For example:
>>
>> (unit : dependencies)
>>  App1 : Lib2 Lib3
>>  App2 : Lib1 Lib2
>>  Lib1 : -
>>  Lib2 : -
>>  Lib3 : Lib1
>>
>> <~/Lib1> touch lib1file.ml
>> <~/App1> make
>>   Build Lib2
>>   Build Lib1
>>   Build Lib3
>>   Build App1
>>
>> No "install", as these are all in flux. The libraries are just like the
>> app-code but shared. Like they used to be before the world of package
>> management. ;) I'm sure others must still do this for internal development
>> too? Or does everyone work with libraries as explicitly built and
>> separately installed units? (Note: I do have some slowly-evolving libraries
>> which I install as packages via OPAM.)
>>
>> To rephrase: Am I doing it all wrong? :D
>>
>> I haven't figured out how to get ocamlbuild to handle library dependency
>> like this yet. The tools, or at least the examples of how to use them, seem
>> very geared toward usage of infrequently-changed packages. So any tips on
>> an example of using ocamlbuild in this manner would be great too!
>>
>> I'm always hesitant to pester the mailing list, but it generally follows
>> days of frustration on my part, and I don't know any other OCaml people, so
>> thank-you!
>>
>>  -Tony
>>
>>
>

[-- Attachment #2: Type: text/html, Size: 8294 bytes --]

Re: [Caml-list] A way to avoid "WARNING: myocamlbuild.cmi occurs in several directories"?

[Daniel Bünzli]

Le vendredi, 13 décembre 2013 à 00:57, Anthony Tavener a écrit :

> I regularly develop libraries in parallel with application code.

Since you seem to be using ocamlbuild, at a certain point I was using (and still use sometimes) a workflow that I described here:  

http://brion.inria.fr/gallium/index.php/Working_on_dependent_projects_with_ocamlbuild

If you work with version control you could also make local git package for the libs. That way you can update your all your libraries after new commits with a single `opam update && opam upgrade`.

Best,

Daniel

Re: [Caml-list] A way to avoid "WARNING: myocamlbuild.cmi occurs in several directories"?

[Stephen Magill]

[-- Attachment #1: Type: text/plain, Size: 2612 bytes --]

I have a similar development setup and have made effective use of ocp-build
to enable a tight development loop while still supporting opam deployment.
 Suppose you have libraries A and B and a program P that uses these, each
in their own directory (or repository).  You set these up with a
per-project .ocp file that describes the library / program sources and a
per-project Makefile.  For each project X, the Makefile (or OPAM build
rules) executes:
  ocp-build init
  ocp-build configure
  ocp-build X

This configures ocp-build to only search in that directory when doing its
build.  When doing development, you skip the per-project makefiles and
instead arrange things as:
  projects/A
  projects/B
  projects/P

And you execute the "ocp-build init; ocp-build configure" portion in the
*projects* directory.  This sets "projects" as the ocp-build root directory
and causes ocp-build to find all the per-project .ocp files.  The
dependencies between the program and libraries will be properly resolved
and even if you have A or B installed with findlib, the local versions of
the libraries will take precedence when building / linking P.  Just compile
P with "ocp-build P" (executed somewhere in the projects/ directory tree).
 You do have to be careful to ensure that during development you never
accidentally create a "projects/X/_obuild" directory (e.g. you never
accidentally invoke "make" in any of the project directories).  ocp-build
will stop at the first _obuild directory it finds, so having _obuild
directories other than projects/_obuild will prevent the full traversal of
the projects/* subdirectories.

--Stephen


On Thu, Dec 12, 2013 at 7:12 PM, Daniel Bünzli
<daniel.buenzli@erratique.ch>wrote:

>
>
> Le vendredi, 13 décembre 2013 à 00:57, Anthony Tavener a écrit :
>
> > I regularly develop libraries in parallel with application code.
>
> Since you seem to be using ocamlbuild, at a certain point I was using (and
> still use sometimes) a workflow that I described here:
>
>
> http://brion.inria.fr/gallium/index.php/Working_on_dependent_projects_with_ocamlbuild
>
> If you work with version control you could also make local git package for
> the libs. That way you can update your all your libraries after new commits
> with a single `opam update && opam upgrade`.
>
> Best,
>
> Daniel
>
>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

[-- Attachment #2: Type: text/html, Size: 3533 bytes --]