Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Hongbo Zhang <bobzhang1988@gmail.com>]

Hi Gabriel,
   To answer your questions in particular, standard library should not be considered a dependency, since for people who use your library, there is no cost. Yes, I agree that it would be even better when enriching the standard library, we would try to make it self contained.
   When I was at school, I can play with OPAM, and install any dependency if I like(thanks to OPAM), however, in the industry setting, esepcially a big company which takes IP in a serious way, it is damn hard to introduce a third party library, the time spent in convincing your legal department is more than you roll your own version.
   Thanks -- Hongbo
> On Aug 27, 2015, at 4:17 AM, Gabriel Scherer <gabriel.scherer@gmail.com> wrote:
> 
> Hongbo, you are saying at the same time that:
> - you want a *rich* (in particular, large) standard library
> - existing third-party base libraries are too large for your taste
> 
> If we magically imported one of those libraries in the OCaml
> distribution, it would still be large, and it would still have
> inter-dependencies between its modules. Why are you unwilling to
> depend on a library provided outside the distribution?
> 
> Depending on external libraries used to be a problem when there was no
> consensus on OCaml packaging (no longer than a few years ago people
> where reluctant to even use ocamlfind). We now have a reasonable
> consensus on a packaging story, and installing third-party libraries
> has never been as easy as it is today -- except on Windows, meh.
> I think you should embrace the idea of depending on software outside
> the OCaml distribution.
> 
> (I understand there are legal issues that make it difficult for some
> companies to use third-party software. It is not our job to work in
> stead of their lawyers.)
> 
> Many of the issues people have with the library distributed with in
> the compiler distribution come from the fact that it is included in
> the compiler distribution -- and thus imposes a maintenance burden on
> the same group, same backward-compatibility constraints, etc. The
> general movements in the current years is to make the compiler
> distribution *smaller*, not *larger*, to avoid these problems.
> 
> 
> I think the criticism you make of existing standard library
> alternatives is fair (at least the Batteries part corresponds to
> something that I think is consensual¹). My preferred solution to help
> solve these issues is to contribute work to improve these libraries.
> 
> ¹: Simon Cruanes has started good work to split Batteries in smaller
> modules that do not depend on each other. This effort is currently
> stalled, in large part by my personal fault (I was too
> perfectionist in hoping for doing this and at the same time preserving
> backward-functionality). Simon may have made the good call in starting
> his own parallel library effort suiting his relatively opinionated
> view of how things should be. I hope to get more time to devote to
> Batteries at some point, and resolve that tension by proposing a good
> compromise (integrating most of Simon's work) for a v3 release.
> 
> There remain the issue that having several "base libraries" risks
> fragmenting the community in incompatible islands of software usage.
> It is true that shoving stuff into the compiler distribution is a way
> to resolve this excess of choice by authority, but it is manifest that
> no one currently wants to bear this authority and the responsibilities
> that come with it. (Except possibly on very localized issues, as the
> `result` type that is being integrated in 4.03+dev).
> 
> I think the way forward is to have smaller independent packages that
> do not require so large a buy-in and consensus. We could have *one*
> package that provides many useful functions for lists, and one
> separate package with many useful functions for strings. In this style
> Daniel Bünzli beautifully documented small packages, or David Sheets
> doing-one-thing libraries would be role models to follow. This wasn't
> an option when Batteries or Core were started, because the packaging
> story was too confused, but it is now and we should take advantage of
> it. Given time (and help?) I hope to evolve Batteries towards this
> model.
> 
> (Small independent packages have their own issues ("cabal hell"), so
> maybe it's a case of seeing the greener pasture on the other side of
> the fence. I think that's at least worth trying.)
> 
> On Thu, Aug 27, 2015 at 9:18 AM, Anthony Tavener
> <anthony.tavener@gmail.com> wrote:
>> I tend to live-in and build my own ecosystem anyway, so I haven't used any
>> of these libraries. But I've looked at them... and I think "containers" goes
>> for a more self-contained-module approach like you were looking for with
>> string split... Looking at it now I see the 'sep' function does use a bit
>> from within the CCParse module it's defined in -- but that module is built
>> on the parser combinators it defines. I think the individual module can be
>> taken as-is, or parts of it, without hunting down other module dependencies.
>> 
>> This doesn't address the core problem you raise. Haven't thought much about
>> it, as I'm one of the few(?) who is okay with the sparse standard library.
>> For games, it's pretty much tradition that you write your own stdlib anyway
>> -- things end up being different... stressing particular use-cases or
>> performance characteristics. And, overall, not suitable for general
>> purposes. DBuenzli has contributed a lot which is actually quite
>> game-relevant, and his style is standalone modules. You could consider his
>> collection of "libraries" (modules really) as a library in it's own right.
>> :) Maybe that style is a good model? Come to think of it, the OCaml stdlib
>> modules are generally quite standalone, aside from Pervasives, aren't they?
>> So maybe ccube-containers and dbuenzli's style are really very OCaml-ish.
>> 
>> My own stuff, by comparison, is horribly layered with dependencies. I
>> generated a dot-graph showing dependencies and nearly everything depends on
>> another module, often which depend on others. I don't like redundant code,
>> and this is the result... but it makes for modules which cannot be easily
>> teased apart. Probably similar to Batteries-style.
>> 
>> -Tony
>> 
>> 
>> On Wed, Aug 26, 2015 at 8:52 PM, Hongbo Zhang <bobzhang1988@gmail.com>
>> wrote:
>>> 
>>> Dear OCaml developers,
>>>    I would like to spend one hour in writing down my experience that why
>>> I had to write some small utilities again and again, since this happened so
>>> many times that I believe you might come across such issues before.
>>>    I am doing some compiler hacking tonight, I needed a utility function
>>> “String.split” which split a string into a list of strings by whitespace, it
>>> is just one liner if you use str library. However, since I am doing some low
>>> level stuff, I would try to avoid such big dependency, and I am pretty sure
>>> that I have ever written it  for at least three times, I just hoped that I
>>> could get it done quickly, so I am looking into batteries that if I can
>>> steal some code, I have to copy some code instead of depend on batteries,
>>> batteries is too big for my projects. `BatString.nsplit` seems to fit for
>>> what I needed, I copied the definition of `BatString.nsplit` into REPL, no
>>> luck, it depends on some previously defined functions, then I copied the
>>> whole module `BatString` into REPL, still no luck, it depended on another
>>> module `BatReturn`, then I stopped here, it’s time to write my own ad-hoc
>>> thrown-away `String.split` function again.
>>>   OCaml is my favorite programming language, and I am very productive at
>>> it, however, I was annoyed by such things from time to time. We do have four
>>> *standard libraries* alternatives: batteries, core, extlib and
>>> ocaml-containers. In my opinion, none of them matches the beauty of the
>>> OCaml language itself and probably will never catch up if we don’t do
>>> anything.
>>>    Note that I don’t want to be offensive to any of these libraries, just
>>> my personal feedback that why I think it is not a good standard library, I
>>> appreciated a lot to people who contribute their precious time in
>>> maintaining these libraries, better than nothing : )
>>>    - Batteries(extlib)
>>>      It’s big with dependencies between different modules (OCaml does not
>>> have a good story in dead code elimination), some of its modules are of low
>>> quality, for example, batEnum is used everywhere while its implementation is
>>> buggy. batIO makes things even worse since it is not compatible with
>>> standard library, some type signatures mismatched IIRC.
>>>    - ocaml-containers
>>>      Mostly one man’s project
>>>    - core
>>>      I believe core has high quality, however, it suffers the same
>>> problem as batteries, a big dependency. There is a blocking issue, its
>>> development process is opaque, for an open source community, I would prefer
>>> a standard library developed in an open environment.
>>>    I am not expecting that we could have a  standard library as rich as
>>> python, but for some utilities, I do believe that shipped with standard
>>> library or officially supported is the best solution.
>>>   Thanks — Hongbo
>> 
>>