Re: [OSR] Ports-like package management system

[Sylvain Le Gall <sylvain@le-gall.net>]

On 29-01-2008, Berke Durak <berke.durak@exalead.com> wrote:
> Hello,
>
> Following Markus's message, discussions with Nicolas Pouillard and
> Sylvain Le Gall and others, and of course my previous work in the EDOS
> project, here are some thoughts about package management systems
> (PMSs) for Ocaml.
>
> First, the PMSs of Linux distributions are perfectly adequate for end
> users and administrators having to deploy and manage identical
> packages accross multiple machines.  These are very complex systems
> including a substantial social part.
>
> But they are not very suitable for developers.
>
> As Markus pointed out, developers cannot go thru a packaging step to
> test the result of a change, much less wait for the fine Debian team.
>
> One reason is that most developers, including myself, cannot be
> bothered to package software for a Linux distribution, because correct
> packaging is complex and requires adherence to a set of rules must be
> remembered and which cannot all be checked by software.  As packaging
> is not done very often, you tend to forget those rules, and that is
> why we have people who package often to not forget the rules:
> maintainers.  Without maintainers, we would be in a world of pain to
> install any piece of non-trivial software and we are thankful to them.
>
> But developers absolutely need to be able to work on multiple versions
> of the same software component at once, patch those versions or
> compile them with unusual flags, and often use the absolutely latest
> unpackaged version.  And that is the second reason why the Debian or
> Red Hat PMSs are not adequate.  They have a single global state per
> system, which includes the installed files, and the package database,
> and cannot handle multiple versions of the same package, nor multiple
> compilations of the same version.  (This also applies to Gentoo.)
>
> We thus need versions, and lots of them!  We need to base our
> developer packages on a version control system, in the style of BSD
> ports.  BSD ports are usually based on CVS, sometimes on Subversion.
> As we are looking to increase collaboration, having a single point of
> contention is a serious limitations of these centralized systems;
> we'll prefer more recent "distributed" version control system.
>
> Of available distributed VCSs with a serious user base, we have Darcs,
> Mercurial and Git.
>

[...]

>
> So I am calling for a solution based on a ports-like system but based
> on a distributed VCS and on an improved ocamlbuild.

Please don't go into this. If you want to talk about PM, don't talk
about VCS. My point is that if you want to build something that last you
should keep focus on PM, which is not really bound to VCS (like content
of files is not bound to filesystem). There is no best VCS for doing PM.
We just should handle a way to :
* let anyone use a different VCS
* be able to download at least a version of each different packages

The most simple way to handle this, to my mind:
* distribute METADATA for packaging into ftp/http
* put a link to the VCS inside METADATA, to tell where you can find most
 recent one

This way, you will be able to download the repository of METADATA using
a simple curl/wget/any other client that allow to download things. You
will also be able to modify it by either checking it out from the VCS
(can be automated using the VCS metadata) or by producing patches and
sending it to the metadata author.

We, in Debian, are using this way. It works pretty well and don't let
people going into endless discussion about Git being faster than Darcs
-- but less useful than Hg.

> Assume you are writing a program FOO and want to use a package BAR
> available from bar.org.  You tell ocamlbuild by adding some tag such
> as
>
>    <mytarget.native>: require(http://bar.org/repository/)
>
> And when you run ocamlbuild, it automatically checks out a copy of
> BAR, compiles and loads its myocamlbuild.ml module which adds the
> required flags.  Of course it should be possible to specify a
> particular revision...  And if BAR has itself dependencies, those too
> would be checked out.
>

This part is really about PM... and i agree.

> Note that Git has a nice option for cloning checked out repositories
> using hard links; that could be used to maintain a cache of checkouts,
> for instance in the user's ~/.ocamlbuild/checkouts/ directory.
>


This part is going back to classical VCS flameware. Please avoid it.

> So basically I propose that we improve ocamlbuild to allow for
> multiple plugin files (using dynamic loading) and use that to define a
> BSD ports-like system targeted at developers.
>

Without the VCS part, i agree.

Regards,
Sylvain Le Gall