Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Romain Bardou <romain.bardou@inria.fr>]

> There remain the issue that having several "base libraries" risks
> fragmenting the community in incompatible islands of software usage.
> It is true that shoving stuff into the compiler distribution is a way
> to resolve this excess of choice by authority, but it is manifest that
> no one currently wants to bear this authority and the responsibilities
> that come with it. (Except possibly on very localized issues, as the
> `result` type that is being integrated in 4.03+dev).
>
> I think the way forward is to have smaller independent packages that
> do not require so large a buy-in and consensus. We could have *one*
> package that provides many useful functions for lists, and one
> separate package with many useful functions for strings. In this style
> Daniel Bünzli beautifully documented small packages, or David Sheets
> doing-one-thing libraries would be role models to follow. This wasn't
> an option when Batteries or Core were started, because the packaging
> story was too confused, but it is now and we should take advantage of
> it. Given time (and help?) I hope to evolve Batteries towards this
> model.

I agree about smaller, independent packages. This is a very general API 
design principle: avoid coupling (the fact that using a module implies 
using another). This may be the main reason I avoid external libraries. 
For instance, Martin Jambon's Yojson depends on biniou, cppo and 
easy-format. I believe Martin is an awesome programmer but this 
particular point just baffles me as there is absolutely no need for 
*any* external dependency to solve such a simple problem (JSON parsing, 
pretty-printing and AST constructors). I understand that Martin wants to 
reuse its own code and be able to integrate Yojson easily with other 
libraries of his, and that is great. For him and users of his other 
libraries. Not for those who just want a JSON parser and have had to 
install all dependencies manually on Windows.

Lwt is another example. While it is obviously an awesome library, using 
it is not a decision to take lightly.

There is something that I love about Yojson though: the main type (the 
JSON AST) uses polymorphic variants. This means that it is possible to 
write a library which is capable of integrating with Yojson without 
actually depending on it. You just copy-paste the type definition and 
use it. Then your user can use any JSON parser and printer. He can use 
Yojson, or another library which happens to use a supertype. Worst case 
scenario, he takes 5 minutes to write functions to convert the Yojson 
AST to and from another JSON library's type.

Writing code which can integrate with another library without actually 
depending on it will scare users less and make more people use your 
library. Here are some more examples.

- Use standard library types like option, list, or array instead of 
trying to generalize to some enumeration concept.

- If the previous point is not possible because the enumeration is too 
big to be constructed before use, consider providing an iter function. 
Such a function can be easily used with a library such as Simon Cruanes' 
sequence library, but your library does not need to depend on it.

- Functors allow your code to depend on any implementation. 
Unfortunately functors are a pain both to write and to use. One of the 
promises of the module system is to be able to replace a module with 
another one with the same signature, but how many times have you seen 
that being done in practice (without functors)? The build system and 
namespace issues do not help you to do that. Especially if you want to 
replace the dependency of a third-party library. Say, use a library 
which depends on Lwt but make it use your implementation of Lwt instead, 
without having to manually reconfigure and recompile the library, just 
by using the compiled version from opam. This is even harder if your Lwt 
implementation does not provide the same interface as Lwt except for the 
part which is used by the library.

Coupling is not bad just between libraries. For instance, you may be 
tempted to write a function "split_list" to split a string into a list, 
by using another function "split_pair" which splits a string into a pair 
(at the first occurrence only). But an independent implementation of 
split_list is much more efficient (if split_pair copies the two 
substrings), can more easily be sent to the toplevel for testing, and 
can be copy-pasted easily between programs (until you decide to put it 
into a library). But more importantly, if you need to change 
"split_pair" for some reason, you will have to check all functions which 
use it, including "split_list". Yes, sharing code allows to share bug 
fixes. But it also makes it easier to break stuff accidentally.

-- 
Romain Bardou