Re: Cherry-picking modules (was Re: [Caml-list] [ANN] OCaml Reins 0.1 - Persistent Data Structure Library)

[Michael Furr <furr@cs.umd.edu>]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 3914 bytes --]



On Wed, 26 Sep 2007, Daniel Bünzli wrote:

> There are a lot of useful reusable modules in many projects like extlib, 
> ocamlnae, etc, however they are all tied to their "cathedral" [1] even though 
> these ties are sometimes very small and could be functorized away (or dropped 
> altogheter). Now if I _just_ want to use the Unzip module of extlib, the 
> Cf_tai64 module of ocamlnae and your Patricia set.  Why do I have to install 
> these three whole libraries whereas I could simply integrate these three 
> modules in my project ?

Why should you not?  There is no good reason to not use the entire library 
instead of just pulling out a single funciton.

I think you need to separate who are *developers* and who are *users*. 
Users are those which you should not depend on having an OCaml development 
environment intalled.  This could be users of a stand-alone app, or 
perhaps coders using felix.  When you distribute your software to them, it 
will be in binary form, and you should take care to ensure this is as easy 
as possible (especially if you're on a platform like windows).  However, 
this does not require you to keep them in your own source tree.

For *developers*, you should have every expectation that they can install 
a sufficient environment themselves.  No OCaml build system is that bad 
(especially compared to other languages) and I expect *every* user of my 
library to be able to build its dependences (or, more realistically, 
install a package manager to build the deps for them).

> I think developers of reusable components should try to strive for 
> *focused*, independent modules, even if it means functorization and 
> convention. Eventually it makes it easier for developers to integrate 
> these modules and exchange specific parts if some module poses a problem 
> or better alternatives popup.

But you should start by raising the bar for re-use, not lowering it.  Use 
the libraries like you *should* be able to do.  If developers find this 
too difficult to use, tools will follow.  Rely on the bazaar.

> Again I'm not for embedding everything in projects, sometimes it is 
> unrealistic. But for exemple I'd systematically do this for data structures, 
> unless of course they are in the standard distribution.

There is a significant advantage to combining various data structures into 
a single package.  For one, there is a lot of room for code sharing.  A 
lot of the functionality of data structures can be implemented in terms of 
a few small functions.  (For instance, to_string is easily implemented 
with fold).  Using mixin modules to extend these to complete 
implemetations saves a lot of redundant code.  Also, reins runs over 600 
unit tests, but there certainly aren't 600 separate tests written in the 
source.  It uses a functors to apply to the same tests to different data 
structures.  This ensures that all of the data structures are 
observationally equivalent when they implement the same interface (which 
is especially useful for ensuring they all throw the same exception, for 
instance).

Finally, choosing a single data structure at the beginning of your project 
and embedding it in your source tree forces you to always use that data 
structure.  One of the advantages of Reins is that it includes different 
implementations that can be swapped out.  If you decide you would rather 
using a a list with fast concatenation rather than fast cons, its a one 
line change.  You don't have to go find the software, build it, ensure it 
has the same interface, ensure it behaves the same way, add it to your 
source tree, add it to your build system, and then change your uses of it.

Of course, the software is LGPL for a reason.  If you want to pull out 
parts of it, you are certainly welcome to do so.  I just won't recommend 
that path to anyone.

Cheers,
-mike