Re: [Caml-list] GODI (was: CTAN/CPAN for Caml (COCAN ...?))

[Gerd Stolpmann <info@gerd-stolpmann.de>]

Am Fre, 2003-07-18 um 16.29 schrieb Shawn Wagner:
> On Fri, Jul 18, 2003 at 06:14:13PM +1000, John Max Skaller wrote:
> > Richard Jones wrote:
> > 
> > > I wonder if anyone is interested (or opposed) to creating a true
> > > parallel with CPAN for OCaml code?
> > 
> > It is pointless to consider this UNTIL there Ocaml team
> > themselves decide on a packaging model.
> 
> Among library writers, findlib's pretty ubiquitous. Almost everything I've
> looked at either requires it or at lesat supports it via providing a META
> file.
> 
> A findlib-based setup that downloads packages from a mirror, unpacks,
> installs dependencies if needed, builds, and installs the package wouldn't
> be too difficult to write. It'll just take someone to sit down and /do/ it.

Recently I did an experiment called GODI ("Gerd's Ocaml Distribution").
The idea was to implement this kind of extended package management with
minimum effort. I call it an experiment, because the aim was not to
create a working system, but simply to find out what works and what does
not work.

At the beginning there was an analysis of the system environments. The
point is crucial for the success:

      * We have the Unix world, and we have the Windows port, both with
        totally different toolchains
      * We have Open Source systems (Linux distros, BSD distros) that
        have good library support, and we have proprietary systems that
        lack libraries (think of gtk).

Any package management system should integrate well with the operating
system it runs on. It should be possible to use the components of the
system as far as possible, and to create components of the system.
Anything else is impractical. For GODI, I have ignored the Windows
problem, but there could be a solution based on mingw (as good
compromise between "Unixfication" and system integration). So I
concentrated on the integration into Unix-style systems.

The library problem: Many Unixes do not have the C libraries we need, or
have only ancient versions. Example: Solaris has ndbm, but you usually
do not want it for new programs because of its limitations, better you
install gdbm or Berkeley DB. Other required libraries (for a useful
system) are completely missing, e.g. gtk. Further problems arise with
missing tools, e.g. gmake. Of course, this is not only an O'Caml
problem, but it is a problem of the users, and we have certainly to deal
with it.

The next point is that findlib is not enough. It cannot manage arbitrary
files, but only the linkable parts of libraries. (Don't forget that
libraries sometimes need tools (e.g. rpcgen), or data files (e.g.
message catalogs). findlib cannot manage these files.)

So we definetely need a database of installed files. Many operating
systems already have such databases (e.g. rpm is such a database), and
the question arises whether to use it for O'Caml packages or not. Pro:
better system integration, especially with the required libraries if the
system provides them. Con: The various package databases are very
different, and not every system has even one. (Note: Perl has its own
very simple file database, the .packlist files.)

For GODI, I have used one of the simplest available package databases,
the NetBSD package system. It is binary-only, records dependencies and
file checksums, and this is more or less already the end of the feature
list. It fulfils an absolute requirement: It runs everywhere because of
its simplicity.

For the finally used system, I would suggest that (1) we use our own
file database, and (2) allow it to record dependencies on the native
file database of the operating system ("foreign dependencies").

When we have a real file database, we can also provide C libraries that
are missing especially in proprietary operating systems.

This would be the "binary picture" of the package system: The packages
are the units of the file database. They can have dependencies on each
other, and they can have foreign dependencies on the system packages.
Every package that is an O'Caml library would follow findlib's
conventions.

Now the more difficult question: The build environment. For GODI, I have
used the NetBSD pkgsrc system as framework. It has the following
features:

      * The functions of the build environment are implemented with
        Makefiles (of course, the BSD "make" is used), and by including
        a Makefile of the framework into your own Makefile you can
        "load" a certain set of functions
      * Every binary package is created by the Makefile in a source
        directory. This Makefile is a "driver Makefile" that downloads
        the real source, builds it, installs it, and creates the binary
        package. The driver Makefile does usually not include the rules
        to compile the sources, it rather calls the Makefile contained
        in the sources for this purpose. 
      * The framework implements build-time dependencies and the
        automatic download of the source tar.gz.
      * There are much more functions, most of them related to building
        of C programs and libraries

The use of Makefiles for the build framework has one clear advantage:
its flexibility. More or less, the Makefiles are "action objects", and
including another Makefile is "inheritance", i.e. it is nothing but an
object-oriented build system.

The problem of the NetBSD approach is that the driver Makefiles to build
the packages are simple, unpackaged files. There are no "source
packages". If you want to update the driver Makefiles, you usually do
that by "cvs update". This is a simple solution, but there is no
guarantee that the current CVS state of the Makefiles works, especially
regarding the dependencies. One needs a release management, i.e. from
time to time the CVS depot must be stabilized.

I am not sure whether this is the right model for O'Caml packages.
First, you need people that integrate packages into the system (you need
a CVS account). Second, you need users with CVS experience. Third, it is
difficult to deviate from CVS, e.g. by installing different versions.

So far my GODI experiment. It is not yet available for download, but if
somebody is interested in it I can make a web page for it.

> Given a choice between having quick-and-dirty working code and code that
> works the 'right' way, the latter's better, of course. But when the choice
> is between working code and seeing 20 people never ever get past arguing
> what the right way it should be done is (Which seems to be what happens
> every time someone mentions an ocaml version of CPAN), I'll take something
> that works.

This is one of the reasons I actually did GODI. It is working code, and
it is now much harder to argue against its principles.

Gerd
-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
------------------------------------------------------------

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners