Re: [Caml-list] Re: The OCaml Community (aka back from the Developer Days)

["Markus Mottl" <markus.mottl@gmail.com>]

Since there were quite a lot of positive comments about Godi lately, I
think it is also necessary to point out some of its significant
drawbacks.  We (Jane Street Capital) have been using Godi internally
for quite a while, and I have to admit that we are less than thrilled
by it and are planning to phase it out from our development
environment.  Following is a short description of what seem to be the
major problems.

Users of package management systems usually fall into one (or more) of
these roles:

  * Software developers
  * Package maintainers
  * Installation administrators
  * Package users

In our experience the only role that is sufficiently well-supported by
Godi is the one of the package user.  It essentially boils down to
using ocamlfind, which works fine for that purpose.  The majority of
OCaml users belongs to this group most of the time, which probably
explains why Godi caught on so well.

However, the other roles are much worse off.  It seems one design
decision of Godi was to separate software developers and package
maintainers.  In practice, however, package maintainers are usually
also the developers, and they are most often also package users.  If I
want to roll out and work with an updated Godi-package, I have to jump
through several hoops before I can do so: upload my updated tarball to
my webserver, update the package metadata in the Godi SVN-repository,
point my browser to the Godi website to update the Godi distribution,
go to my local Godi-installation, update and recompile the package,
and finally, when I try to use it, I may find out that I had made a
mistake somewhere and have to start all over.  The build system used
by Godi, which is based on NetBSD's package management, is arcane to
say the least.  I still don't quite understand how to e.g. correctly
make packages check for C-library dependencies, etc.

Administrating a Godi installation is no easy task either.  The user
interface seems quite cumbersome and hard to use.  Furthermore, it is
not easy to integrate libraries that are not in Godi (as packages).
They need to be rebuilt, too, when their dependencies get updated, but
automating this task is essentially only possible by jumping through
the hoops of making packages and becoming their maintainer.  A task
that many would rather not take over and hardly makes sense if there
is no intention to release such packages to the general public in the
first place.

The question now is how to solve these issues, and it's clear that
this would require a fairly significant development effort.

As a software developer and package maintainer, I'd ideally like to be
able to work on a source tree containing the complete code of all
packages (and their dependencies), make changes wherever I want (fix
bugs, add features, add new libraries, etc.), share patches or full
versions with other maintainers, and all of this with a minimum amount
of overhead.  Since Godi only tracks dependencies between packages, it
is not possible to just update code and let the build system figure
out what needs to be recompiled.  One needs to build new packages
instead, which is way too much effort.  As installation administrator
I'd like to be able to use a straightforward user interface and easily
add 3rd party libraries without having to manually make sure that
dependencies are not violated.

It is still a point of discussion at our company how to replace Godi,
and also how we could find a solution that would integrate well with
the OCaml-community.  We have developed a fair amount of
infrastructure to improve our team productivity (we have around 20
full time developers now working in three different locations) by
lowering turnaround times associated with code changes.  The use of
distributed version control, compile daemons and omake has made it
very easy for us to share code that is guaranteed to compile and
allows making modifications quickly with a minimum amount of time
required for recompilations.

It seems likely that focusing on a high degree of standardization
around the usage of software development tools (which version control
to use, how to guarantee compilability, what build tools to use,
standards enforcing easy combination and modularity of build
processes) would lower development barriers and thus boost the
productivity of the OCaml community.  But it seems rather unlikely to
most of us that Godi will be a suitable foundation for moving forward.
 We greatly appreciate Gerd's tireless efforts to contribute tools
like Godi, which is a lot of hard work that nobody else wanted to do
before.  We have certainly benefited from it in the past and hope that
a new approach will alleviate the problems that Godi-developers often
run into.

Best regards,
Markus

-- 
Markus Mottl        http://www.ocaml.info        markus.mottl@gmail.com