Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Gabriel Scherer <gabriel.scherer@gmail.com>]

Hongbo, you are saying at the same time that:
- you want a *rich* (in particular, large) standard library
- existing third-party base libraries are too large for your taste

If we magically imported one of those libraries in the OCaml
distribution, it would still be large, and it would still have
inter-dependencies between its modules. Why are you unwilling to
depend on a library provided outside the distribution?

Depending on external libraries used to be a problem when there was no
consensus on OCaml packaging (no longer than a few years ago people
where reluctant to even use ocamlfind). We now have a reasonable
consensus on a packaging story, and installing third-party libraries
has never been as easy as it is today -- except on Windows, meh.
I think you should embrace the idea of depending on software outside
the OCaml distribution.

(I understand there are legal issues that make it difficult for some
companies to use third-party software. It is not our job to work in
stead of their lawyers.)

Many of the issues people have with the library distributed with in
the compiler distribution come from the fact that it is included in
the compiler distribution -- and thus imposes a maintenance burden on
the same group, same backward-compatibility constraints, etc. The
general movements in the current years is to make the compiler
distribution *smaller*, not *larger*, to avoid these problems.


I think the criticism you make of existing standard library
alternatives is fair (at least the Batteries part corresponds to
something that I think is consensual¹). My preferred solution to help
solve these issues is to contribute work to improve these libraries.

¹: Simon Cruanes has started good work to split Batteries in smaller
modules that do not depend on each other. This effort is currently
stalled, in large part by my personal fault (I was too
perfectionist in hoping for doing this and at the same time preserving
backward-functionality). Simon may have made the good call in starting
his own parallel library effort suiting his relatively opinionated
view of how things should be. I hope to get more time to devote to
Batteries at some point, and resolve that tension by proposing a good
compromise (integrating most of Simon's work) for a v3 release.

There remain the issue that having several "base libraries" risks
fragmenting the community in incompatible islands of software usage.
It is true that shoving stuff into the compiler distribution is a way
to resolve this excess of choice by authority, but it is manifest that
no one currently wants to bear this authority and the responsibilities
that come with it. (Except possibly on very localized issues, as the
`result` type that is being integrated in 4.03+dev).

I think the way forward is to have smaller independent packages that
do not require so large a buy-in and consensus. We could have *one*
package that provides many useful functions for lists, and one
separate package with many useful functions for strings. In this style
Daniel Bünzli beautifully documented small packages, or David Sheets
doing-one-thing libraries would be role models to follow. This wasn't
an option when Batteries or Core were started, because the packaging
story was too confused, but it is now and we should take advantage of
it. Given time (and help?) I hope to evolve Batteries towards this
model.

(Small independent packages have their own issues ("cabal hell"), so
maybe it's a case of seeing the greener pasture on the other side of
the fence. I think that's at least worth trying.)

On Thu, Aug 27, 2015 at 9:18 AM, Anthony Tavener
<anthony.tavener@gmail.com> wrote:
> I tend to live-in and build my own ecosystem anyway, so I haven't used any
> of these libraries. But I've looked at them... and I think "containers" goes
> for a more self-contained-module approach like you were looking for with
> string split... Looking at it now I see the 'sep' function does use a bit
> from within the CCParse module it's defined in -- but that module is built
> on the parser combinators it defines. I think the individual module can be
> taken as-is, or parts of it, without hunting down other module dependencies.
>
> This doesn't address the core problem you raise. Haven't thought much about
> it, as I'm one of the few(?) who is okay with the sparse standard library.
> For games, it's pretty much tradition that you write your own stdlib anyway
> -- things end up being different... stressing particular use-cases or
> performance characteristics. And, overall, not suitable for general
> purposes. DBuenzli has contributed a lot which is actually quite
> game-relevant, and his style is standalone modules. You could consider his
> collection of "libraries" (modules really) as a library in it's own right.
> :) Maybe that style is a good model? Come to think of it, the OCaml stdlib
> modules are generally quite standalone, aside from Pervasives, aren't they?
> So maybe ccube-containers and dbuenzli's style are really very OCaml-ish.
>
> My own stuff, by comparison, is horribly layered with dependencies. I
> generated a dot-graph showing dependencies and nearly everything depends on
> another module, often which depend on others. I don't like redundant code,
> and this is the result... but it makes for modules which cannot be easily
> teased apart. Probably similar to Batteries-style.
>
> -Tony
>
>
> On Wed, Aug 26, 2015 at 8:52 PM, Hongbo Zhang <bobzhang1988@gmail.com>
> wrote:
>>
>> Dear OCaml developers,
>>     I would like to spend one hour in writing down my experience that why
>> I had to write some small utilities again and again, since this happened so
>> many times that I believe you might come across such issues before.
>>     I am doing some compiler hacking tonight, I needed a utility function
>> “String.split” which split a string into a list of strings by whitespace, it
>> is just one liner if you use str library. However, since I am doing some low
>> level stuff, I would try to avoid such big dependency, and I am pretty sure
>> that I have ever written it  for at least three times, I just hoped that I
>> could get it done quickly, so I am looking into batteries that if I can
>> steal some code, I have to copy some code instead of depend on batteries,
>> batteries is too big for my projects. `BatString.nsplit` seems to fit for
>> what I needed, I copied the definition of `BatString.nsplit` into REPL, no
>> luck, it depends on some previously defined functions, then I copied the
>> whole module `BatString` into REPL, still no luck, it depended on another
>> module `BatReturn`, then I stopped here, it’s time to write my own ad-hoc
>> thrown-away `String.split` function again.
>>    OCaml is my favorite programming language, and I am very productive at
>> it, however, I was annoyed by such things from time to time. We do have four
>> *standard libraries* alternatives: batteries, core, extlib and
>> ocaml-containers. In my opinion, none of them matches the beauty of the
>> OCaml language itself and probably will never catch up if we don’t do
>> anything.
>>     Note that I don’t want to be offensive to any of these libraries, just
>> my personal feedback that why I think it is not a good standard library, I
>> appreciated a lot to people who contribute their precious time in
>> maintaining these libraries, better than nothing : )
>>     - Batteries(extlib)
>>       It’s big with dependencies between different modules (OCaml does not
>> have a good story in dead code elimination), some of its modules are of low
>> quality, for example, batEnum is used everywhere while its implementation is
>> buggy. batIO makes things even worse since it is not compatible with
>> standard library, some type signatures mismatched IIRC.
>>     - ocaml-containers
>>       Mostly one man’s project
>>     - core
>>       I believe core has high quality, however, it suffers the same
>> problem as batteries, a big dependency. There is a blocking issue, its
>> development process is opaque, for an open source community, I would prefer
>> a standard library developed in an open environment.
>>     I am not expecting that we could have a  standard library as rich as
>> python, but for some utilities, I do believe that shipped with standard
>> library or officially supported is the best solution.
>>    Thanks — Hongbo
>
>