Re: [Caml-list] Alternative proposal: COAN

[Sven Luther <luther@dpt-info.u-strasbg.fr>]

On Sun, Mar 02, 2003 at 07:21:26PM +0100, Xavier Leroy wrote:
> > [Package management in OCaml]
> 
> I'm catching up on this interesting discussion, and find myself in
> violent agreement with Jacques: something like the BSD "ports" system,
> concentrating on (re-)compilation from sources and dependency
> management, sounds like a strong starting point.  
> 
> I tried to read the BSD "ports" manual once, and my head exploded
> midway :-)  I hope we can simplify things a bit, though.
> 
> > * For me the central repository should not contain the source themselves.
> 
> I agree it should be sufficient to give a URL to the sources in the
> metadata describing the package.  In some cases, library authors
> cannot provide a really stable URL, hence some kind of mirrorring of
> the sources might be necessary.  (And is a real nightmare to do: INRIA
> can easily provide lots of disk space and bandwith, but making sure
> that no-one uses the INRIA mirror to trade warez is the hard part :-)

Well, you only have to trust the people who will be uploading stuff. You
give each of them, either some kind of password, or you use an upload
queue or something such. You ask for the gpg/pgp key of the wouldbe
uploaders, and they send to an anonymous writable (but not readable)
directory the files they are uploading, together with some kind of
control file, containing a list of the uploaded files with their md5sum,
and maybe a small description of changes or whatever (and naturally the
package name). Then, a script is started from the cron table, looks the
upload directory for control files, if it finds one, checks it is
signed by someone you know, verify that the package he uploads is one
you have authorized, and verify the md5sum of the uploaded files. If all
this is ok, then the script just move the stuff to the download directory (in a
subdirectory corresponding to the package) and that is it. If all the
rest is ok, but the package is a new one, then you have a manual look at
it, and add it to the list of known packages. This can be done once a
week or whatever is best.

This is how the debian archives are managed, and it has worked well.
Sure, we have the added security that we don't allow anyone to enter the
keyring without having personnaly met another debian developper already
in the web of trust and that is identity has been acertained, but i
believe this is not really an issue here, and that you may also try to
meet other ocaml writing authors at conferences and other such, and thus
have a chance of adding them to your keyring.

> But, yes, let's desing the system around the idea that sources are to
> be downloaded from arbitrary URLs, like BSD ports do.
> 
> > * Dependency resolution and automatic recompilation/reinstallation is
> >   the core of the problem.
> 
> Agreed.  What do you envision for this?  Is it enough for each package
> to list the names and version number ranges for all the packages it
> needs?  Or shall we try to exploit additionally the dependency
> information (MD5 checksums of imported modules) already present in
> compiled OCaml files?  

I don't really like the idea of the range of version numbers. There is
no way you can know when a package will change in the future, and then
there is the issue of the version of ocaml it was built for (err, you
are speaking sources, so this is not as important, but still, there may
be source changes as well).

Anyway, i would much prefer that we begin to look at a formal versioning
system, for example a 3 level version system, where the first will be
for api changes, the second for backward compatible api changes (adding
stuff and the like), and the last will be for bug fixes and other non
api changing changes. This is something that is used for C so numbers,
and could work out well here. Basically it guarantees that if the api
number of a dependency did not change, then you can use it.

Sure, it needs more cooperation between the different module authors,
but i think that this will be accepted by them.

Also, this is for ocaml only, problems may happen with stublibs and
other non ocaml stuff, but in these cases, mostly libraries, i believe
depending on the right level of so number would be enough.

As for the build system, i will say here what i already told the small
group apointed by benjamin, i think it would be nice if we could just
have some kind of wrapper for the most common
configuration/build/installation options, and then have each package
provide a small ocaml module which would comply with the following
module type :

module type Ocaml_Package = sig
  val configure : ... -> string
  val build : ... -> string
  val install : ... -> string
end

where the ... would be replaced with the common option strings.

I think it can be expanded with some kind of listing of additional
option by the packages, and can then be used to build a simple little
installation program, or even dynamically loaded or something such.

This would solve all the request for simple and standard installation
processes, let yet the authors free to use whatever build system they
choose, provided they provide the appropriate wrapper, which would
server as documentation as well.

> > I personally don't think that standardizing the tools to produce
> > individual package is a useful move.
> 
> It's not necessary, but could help.  For the packaging tools to work,
> each port will have to contain a makefile or shell script that
> implements correctly a basic, shared protocol, e.g. "configure,
> compile, and install yourself", or "uninstall yourself".  Providing
> template makefiles that implement this protocol could help library
> authors do the packaging.  

See my above proposal.

> Also, as Nikolaj pointed out, precious few OCaml libraries build on
> Windows.  That's basically because most library authors don't know
> anything beyond Unix.  Again, template makefiles could help overcome
> this issue.

Well, maybe also because most libraries are bindings, which bind to
things not available on Windows, or because the windows install stuff is
much to complex. I had very little luck in making lablgtk work on
windows, it didn't find the right path and so on, very bothersom.

> Speaking of this, I've been considering generating a Makefile.inc file
> as part of the OCaml installation, containing useful information such
> as "where is the OCaml library?", "what version of OCaml is this?",
> and "is the native code compiler supported?".  By just putting
>         include `ocamlc -where`/Makefile.inc
> in your makefiles, most of the need for an autoconfiguration script
> could be avoided.
> 
> Finally, some standardization on where packages install their files
> would help the end user.  Some packages install one module in the
> OCaml stdlib dir, others install in a subdir of the stdlib, others
> install in the "contrib" subdir, etc.  OCamlfind can handle all this,
> but I believe more stringent guidelines on where libraries should go
> would help.

I thought the decision was to install into a subdir of the libdir, and
that the libdir was reserved for ocaml itself ? Did we not agree on that
a year or two ago ?

> > One last comment on Windows. Since most development is taking place on
> > Unix, I think this is reasonnable to make the presence of basic cygwin
> > tools a requirement for compiling packages. The presence of gnu make
> > and some shell commands should be enough for most. Handling of C
> > libraries is more complex, but this must often be handled also at the
> > source level.
> 
> After years of struggling with nmake, I finally decided to rely on GNU
> make and the Cygwin tools for building the windows versions of OCaml.
> So, I agree that assuming Cygwin is present for building packages is a
> reasonable thing to do.  Still, writing a makefile that works both
> under Unix and Windows isn't trivial, due to stupid things like
> different file extensions for C libraries (.a vs. .lib).  At least, I
> haven't yet succeeded in having common makefiles for Unix and Windows
> in the OCaml source tree.

I believe the wya to do is cross compile on a linux system. So nobody
would claim they are not familiar enough with windows and everyone would
have libraries installeable straight from unix sources. I have not done
this personnally, so maybe it is not all that simple, but maybe a
tutorial example of such a build system with one of the simpler
libraries (maybe camlzip) could be nice to have. If i had time, i would
investigate it. We have this under debian :

Description: Minimalist GNU win32 (cross) compiler
 A Linux hosted, win32 target, cross compiler for C/C++
 .
 Freedom through obsolescence.  Those who still really need to
 can now build windows executables from the comfort of Debian.
 .
 This package contains the C and C++ cross compilers.

I don't know if this would be enough to build at least part of ocaml,
and also don't know if it is feasible to use it to build a cross ocaml
compiler, but this would be a neat thing to do. If i had time for it i
would look into this.

Friendly,

Sven Luther

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners