Working on dependent projects with ocamlbuild

Working on dependent projects with ocamlbuild

[Daniel Bünzli]

Hello,

Somewhat related to the discussion we had about the (bad) idea of  
embedding dependencies into projects. I'd like to share the following  
setup with ocamlbuild that allows me to work simultaneously on a  
'base' project and two independent projects 'p1' and 'p2' that use  
'base'. Basically my sources are distributed as follows :

base/src
p1/src
p2/src

What I used to do is to build a .cma out of the sources in base/src  
and point the others to that .cma. When I did a change in base, I had  
to build that .cma again, sometimes forgetting and seeking for bugs  
in code while it was just a .cma freshness issue.

What I do now is that I completly forget about .cma's. I just create  
the following links in p1 and p2

ln -s ../base/src p1/base
ln -s ../base/src p2/base

And add the following to their _tags file :

echo "<base>: include" >> p1/_tags
echo "<base>: include" >> p2/_tags

The rest is simply sorted out by ocamlbuild. Whenever I do a change  
in base/src I don't need to recompile anything there, if I rework in  
p1 or p2 things are automatically updated, I always use the latest  
version of base's code. Of course this means longer build time when  
you ocamlbuild -clean in p1 and p2 since they each build their own  
version of base. But on the scale at which I work it is currently not  
an issue.

The only caveat (that may disappear in the future) is that base/src  
should be able to build without a plugin. Otherwise you will
have to integrate base's myocamlbuild.ml instructions into p1 and  
p2's myocamlbuild.ml (btw. couldn't we find a less egoistic name for  
that file). But if you are only working with _tagged caml sources it  
should works perfectly, put your tags for base in base/src/_tags.

Best,

Daniel

Re: [Caml-list] Working on dependent projects with ocamlbuild

[Matthieu Dubuget]

2007/10/24, Daniel Bünzli <daniel.buenzli@erratique.ch>:
>
> ln -s ../base/src p1/base
> ln -s ../base/src p2/base

Interesting.
One problem is that this solution is not usable with all kind of filesystem.

Salutations

Matt

Re: [Caml-list] Working on dependent projects with ocamlbuild

[Nicolas Pouillard]

Excerpts from Daniel Bünzli's message of Wed Oct 24 13:07:27 +0200 2007:
> Hello,
Hello,

> 
> Somewhat related to the discussion we had about the (bad) idea of  
> embedding dependencies into projects. I'd like to share the following  
> setup with ocamlbuild that allows me to work simultaneously on a  
> 'base' project and two independent projects 'p1' and 'p2' that use  
> 'base'. Basically my sources are distributed as follows :
> 
> base/src
> p1/src
> p2/src
> 
> What I used to do is to build a .cma out of the sources in base/src  
> and point the others to that .cma. When I did a change in base, I had  
> to build that .cma again, sometimes forgetting and seeking for bugs  
> in code while it was just a .cma freshness issue.
> 
> What I do now is that I completly forget about .cma's. I just create  
> the following links in p1 and p2
> 
> ln -s ../base/src p1/base
> ln -s ../base/src p2/base
> 
> And add the following to their _tags file :
> 
> echo "<base>: include" >> p1/_tags
> echo "<base>: include" >> p2/_tags
> 
> The rest is simply sorted out by ocamlbuild. Whenever I do a change  
> in base/src I don't need to recompile anything there, if I rework in  
> p1 or p2 things are automatically updated, I always use the latest  
> version of base's code. Of course this means longer build time when  
> you ocamlbuild -clean in p1 and p2 since they each build their own  
> version of base. But on the scale at which I work it is currently not  
> an issue.
> 
> The only caveat (that may disappear in the future) is that base/src  
> should be able to build without a plugin. Otherwise you will
> have to integrate base's myocamlbuild.ml instructions into p1 and  
> p2's myocamlbuild.ml (btw. couldn't we find a less egoistic name for  
> that file). But if you are only working with _tagged caml sources it  
> should works perfectly, put your tags for base in base/src/_tags.

Right. Multiple plugins should be the next new big feature of ocamlbuild.

The  myocamlbuild.ml is egoistic, but is there to insist on the fact that your
are  customizing  the  tool and not giving him full instructions/recipes. What
about ocamlbuildrc.ml?

However  this  setup definitely needs to go on the wiki.

Would you mind adding it?

Cheers,
-- 
Nicolas Pouillard aka Ertai

Re: [Caml-list] Working on dependent projects with ocamlbuild

[Daniel Bünzli]

Hello,

I'm now trying to do something a little bit more complex but fail to  
solve it in the simple way I mentionned (maybe it is not possible). I  
still have the same setup of two independent projects each depending  
on a base project.

base/src
p1/src
p2/src

The base project is structured as follows :

base/src/a.ml
base/src/b.ml
base/src/base.ml
base/src/base.mli

with base.ml as follows

module A = A
module B = B
val ...

The idea is that b.ml can access things in a.ml that users of the  
base project should not access. What I used to do is to  
appropriatlely constrain the signatures of A and B in base.mli and  
then create in a directory :

dest/base.cma (containing a.cmo b.cmo base.cmo)
dest/base.cmi
dest/base.mli

Thus forcing access to A and B via the constrained signature Base.A  
and Base.B.

But if I use the approach mentionned in my initial posting users of  
base will see the unconstrained a.cmi and b.cmi. Is there a way that  
allows me to use this approach while maintaining the abstraction I  
get with cma's ? Even if it means plugins in p1 and p2 ? I have the  
impression that the response is negative because even if I try to  
build a .cma with a mllib the build directory with have all .cmi and  
the compilers will look at them.

Thanks for your answers,

Daniel

Re: [Caml-list] Working on dependent projects with ocamlbuild

[Daniel Bünzli]

Le 1 nov. 07 à 11:53, Daniel Bünzli a écrit :

> The base project is structured as follows :
>
> base/src/a.ml
> base/src/b.ml
> base/src/base.ml
> base/src/base.mli

Actually, if the emacs mode had a code-folding mode (module folding  
to be precise) I'd seriously consider the option of writing all the  
code in base.ml. Which would solve my problem while retaining the  
convenience of the initial symlink solution.

Daniel

Re: [Caml-list] Working on dependent projects with ocamlbuild

[Nicolas Pouillard]

Excerpts from Daniel Bünzli's message of Thu Nov 01 11:53:44 +0100 2007:
> Hello,
Hello,

[...]

> But if I use the approach mentionned in my initial posting users of  
> base will see the unconstrained a.cmi and b.cmi. Is there a way that  
> allows me to use this approach while maintaining the abstraction I  
> get with cma's ? Even if it means plugins in p1 and p2 ? I have the  
> impression that the response is negative because even if I try to  
> build a .cma with a mllib the build directory with have all .cmi and  
> the compilers will look at them.

There  is  a  solution  to specify directory scoping in a fine-grained manner.
This way you can explain precisely which directory is seen by a directory.

This  is  done  in  a  plugin  using the function Pathname.define_context that
takes  a  directory  (an  implicit  pathname  starting  from  the root of your
project) and a list of directories seen by it.

You  can then setup your sources in order to confine some files in a directory
only seen by the modules that wrap them.

p1/src/implem/a.ml
p1/src/implem/b.ml
p1/src/base.ml
p1/myocamlbuild.ml

in p1/myocamlbuild.ml...
open Ocamlbuild_pack;;
dispatch begin function
| After_rules ->
    Pathname.define_context "src" ["src/implem"]
| _ -> ()
end;;

HTH,
-- 
Nicolas Pouillard aka Ertai

Re: [Caml-list] Working on dependent projects with ocamlbuild

[Daniel Bünzli]

Le 2 nov. 07 à 16:32, Nicolas Pouillard a écrit :

> There  is  a  solution  to specify directory scoping in a fine- 
> grained manner.

Thanks, works as advertised. All the information of this thread is  
now on the wiki :

http://brion.inria.fr/gallium/index.php/ 
Working_on_dependent_projects_with_ocamlbuild

I also added classical build errors that are reported differently  
when ocamlbuild is used :

http://brion.inria.fr/gallium/index.php/New_kinds_of_build_errors

Best,

Daniel

Re: [Caml-list] Working on dependent projects with ocamlbuild

[Nicolas Pouillard]

Excerpts from Daniel Bünzli's message of Sun Nov 04 22:08:39 +0100 2007:
> 
> Le 2 nov. 07 à 16:32, Nicolas Pouillard a écrit :
> 
> > There  is  a  solution  to specify directory scoping in a fine- 
> > grained manner.
> 
> Thanks, works as advertised. All the information of this thread is  
> now on the wiki :
> 
> http://brion.inria.fr/gallium/index.php/ 
> Working_on_dependent_projects_with_ocamlbuild
> 
> I also added classical build errors that are reported differently  
> when ocamlbuild is used :
> 
> http://brion.inria.fr/gallium/index.php/New_kinds_of_build_errors

Great, thanks!
-- 
Nicolas Pouillard aka Ertai