Cherry-picking modules (was Re: [Caml-list] [ANN] OCaml Reins 0.1 - Persistent Data Structure Library)

["Daniel Bünzli" <daniel.buenzli@epfl.ch>]

Le 25 sept. 07 à 21:30, Mike Furr a écrit :

> I've deliberately tried to unify many of the conventions that  
> people have used to allow users to easily change between data  
> structure implementations.  A major goal of the project is that you  
> don't have to rely on downloading some random .ml/.mli from  
> someone's webpage and embed it your project.  Instead you just use  
> the reins library and only have to worry about updating one piece  
> of software for updates/ bug fixes.

While I really like having a central repository and unification of  
conventions I do like cherry-picking and embedding my modules.  
Nevertheless I understand and respect your point, maybe it is a  
matter of opinion/taste. But I'll explain why I like picking.

There are libraries that need several tightly dependent modules to  
provide their service and they should be distributed as libraries  
since use of one module implies use of the others -- note however  
that sometimes this is by misdesign e.g. everything can actually be  
easily and conveniently packed in a single module or some parts can  
be functorized away to split the distribution.

But there are also focused services that can be provided as single,  
pure caml, modules and these should IMHO be distributed as single  
modules. I try -- as much as possible -- to embed and statically link  
dependencies in my projects. Of course if there is a bug in one  
dependency it means more work for me. But the thing is that by doing  
so I put the burden and complexity of building my project on me and  
not on my users. If I do not do so, this burden may be aleviated (or  
not ?) by using a package management system but this solution messes  
the environment of my users.

There are a lot of useful reusable modules in many projects like  
extlib, ocamlnae, etc, however they are all tied to their  
"cathedral" [1] even though these ties are sometimes very small and  
could be functorized away (or dropped altogheter). Now if I _just_  
want to use the Unzip module of extlib, the Cf_tai64 module of  
ocamlnae and your Patricia set.  Why do I have to install these three  
whole libraries whereas I could simply integrate these three modules  
in my project ?

Having to install them not only puts a burden on potential users, but  
it also puts a burden on me when I suddenly start to hack my project  
in another environment which has not all the dependencies installed.  
In the end I think module integration makes me loose less time  
because once the build system is setup I have to think about it much  
less at later times. The alternative solution of embedding the whole  
libraries is the start of bloat. It also hides the real dependencies  
which makes it harder for new commers to approach the code base.

I think developers of reusable components should try to strive for  
*focused*, independent modules, even if it means functorization and  
convention. Eventually it makes it easier for developers to integrate  
these modules and exchange specific parts if some module poses a  
problem or better alternatives popup.

And while certainly more could be done to try to simplify this style  
of development (maybe something along the lines of caml-get), I have  
to say that now that we have been provided with ocamlbuild, the task  
of integrating pure caml modules is really a breeze -- even if you  
want to relegate dependencies to a specific directory.

Again I'm not for embedding everything in projects, sometimes it is  
unrealistic. But for exemple I'd systematically do this for data  
structures, unless of course they are in the standard distribution.

Best and thanks again for the work,

Daniel

[1] btw this was my point about the cathedral, many modules that you  
have to take as a one-stop-shop-it-all even if you care only for one,  
not the development style.