Re: [Caml-list] Re: [OSR] Ports-like package management system

[Gerd Stolpmann <info@gerd-stolpmann.de>]

Hi,

first of all sorry for my last message where I lost my temper. This is
not the tone I like to read either. Let's better stick to the real
facts, and ignore messages where we have the impression that it is only
complained without giving any insight.

Am Mittwoch, den 30.01.2008, 23:37 +1100 schrieb Pietro Abate:
> Hi all
> 
> On Wed, Jan 30, 2008 at 09:06:29AM +0000, Sylvain Le Gall wrote:
> > On 29-01-2008, Berke Durak <berke.durak@exalead.com> wrote:
> > > Assume you are writing a program FOO and want to use a package BAR
> > > available from bar.org.  You tell ocamlbuild by adding some tag such
> > > as
> > >
> > >    <mytarget.native>: require(http://bar.org/repository/)
> > >
> > > And when you run ocamlbuild, it automatically checks out a copy of
> > > BAR, compiles and loads its myocamlbuild.ml module which adds the
> > > required flags.  Of course it should be possible to specify a
> > > particular revision...  And if BAR has itself dependencies, those too
> > > would be checked out.
> 
> My two cents here:
> 
> I'd like to follow the lead from the idea above. I've the impression we
> are not converging. I think if we want to succeed in this packaging
> revolution we need to proceed is small steps. 

I agree that less is more in this area. Software distribution is a quite
complex problem, and it is unlikely that we can find a consensus in the
community unless we reduce it to the minimum where cooperation makes
sense. My advice is to see this not mainly as a technical but as a
political challenge.

I've followed the VCS-based versus central debate now for a while, and
my impression is that it is too theoretical. I would like to recall that
GODI implements a mixed system - it is a central system by default, but
you can overlay the centrally distributed packages with your own (I
showed it on a slide in Paris, and note that GODI differs in this
respect from BSD ports), so it is already possible to distribute patched
versions. Of course, it is a bit more work to publish patched versions
than in a VCS-based system - they have to be tared up, and put onto a
web server, but I'm sure this additional overhead can be technically
addressed. But anyway, whoever wants to build a VCS-based distribution
system, go ahead and make your own experience.

The more interesting thing is the question: what is the unit of
distribution? GODI and Debian have the concept of "packages" that reside
in a dependency tree, and the packages have a certain interface for
interaction. I think now that as a common unit for upstream authors such
a concept goes too far. We need something simpler - a descriptive
distribution format so that the distributed units can be plugged in to
existing systems like GODI or Debian, but that the units can also be
built manually for the users who need more control.
 
> The basic and foremost important requirement we have is to try to
> standardize the way ocaml libraries/applications are built. One of the
> main problem package maintainers have is that everybody seems to write
> their own build system and to follow different rules in their projects.
> We have a plethora of building system, but in the end what we really
> need is an interface to use them. Using a simple makefile for example
> with make {opt,byte,install,clean} and to agree to what these rules do
> would be a huge step forward. I'm not saying everybody must use make,
> but that all projects should ship a makefile with these 4 rules. What is
> then effectively used to build the package is not my problem (except for
> building dependencies with exotic tools). In the future we might think
> of proposing a standard building tool for everybody as ocamlbuild... but
> I think this is premature now.

I don't think we will reach a consensus about the build tools.
Currently, the most widely used is make+Unix utilities. I've not yet
seen much omake, and ocamlbuild neither. Suggestions to move away from
make are often driven by better Windows support.

My suggestion is this: A driver file "Obuild" that descriptively says
which build tools (out of some generally accepted options) are supported
by this piece of software. Example for Obuild:

configure_script: configure
build_tool: GNU make
make_bytecode_target: all
make_native_target: opt
make_install_target: install
supports_prefix: true

Obuild could also list several build tools, so the user has choice.

Obuild could also describe build options (with/without a certain
component), and how to pass in paths for C libraries etc.

What's left out are dependencies. They make only sense as a system, and
managing a system is a different story. That should be left to packagers
like GODI or Debian.

> Ok, at this point we have the package, and we know how to build it. Next
> step is to have one place to upload all tarballs. Godi has its own
> repository, debian has its own repository on alioth. Here I'm thinking
> something like CPAN, with a light weight registration and upload
> mechanism based on pgp (Register, sign the package, upload the package).
> The infrastructure to build this kind of service is not rocket science.
> We could even imagine some king of validation process. This way all
> developer could just work as they do, using whatever vcs they like,
> adding just one simple step whan they want to release a new package into
> the wild. Dependencies could be addressed at this level or at the next.
> Anyway a complex project with many dependencies must find a way to
> resolve these dependencies somehow: either in their makefile (see
> require(http://bar.org/repository/) as above) or just by assuming that all
> dependencies are already satisfied and leave the burden to Package
> maintainers (as it is at the moment).

Well, as said, I suggest to leave out dependencies. A dependency error
in a single uploaded package can make the whole tree unbuildable. Deps
are outside the scope of loose cooperation.

> >From this point on the sky. Debian, godi, fedora,... can just keep going
> as they do with their system but using a single upstream repository with
> standardized build system. People with other needs could use a system as
> describe above with ocamlbuild. In the future we can even think to some
> kind of convergence of different distributions to handle dependencies,
> versioning, etc ...
> 
> In this way I tried to chance as small as possible of the current
> infrastructure, still trying to give people the possibility of easily
> recompiling their projects and all dependencies.
> 
> Maybe this view it too naive, but it might be a good way to start.

I think this is more realistic than all the other approaches to start
another complex distribution system from scratch, without seeing that
this is a big undertaking.

Gerd
-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
Phone: +49-6151-153855                  Fax: +49-6151-997714
------------------------------------------------------------