Re: [Caml-list] GODI news

[Gerd Stolpmann <info@gerd-stolpmann.de>]

On Mit, 2003-12-03 at 00:36, Benjamin Geer wrote:
> Alain.Frisch@ens.fr wrote:
> > As a side note, I'd like to encourage OCaml library developers to consider
> > providing GODI build instructions to Gerd. It is quite easy.
> 
> I'd love to, but I can't find a document that explains how to do that. 
> I've read the README.devel, but it's not clear to me which parts of it 
> are talking about the development of GODI itself, and which parts are 
> explaining how to make a GODI package.  There's a section called 'What 
> to put in the Makefile' that looks relevant, but I'm wondering:  Which 
> Makefile?  What do I need besides a Makefile?
> 
> I'd just like something that says, 'A GODI package consists of X [a 
> tar.gz file?]  It must contain Y and Z.  Z must be in this format...' 
> and so on.

Accepted that critique. So far I have found no time to write a tutorial
about package making. Because of this I am thankful that William Lovas
and Ownen Gunden are developing a tool for this purpose. Nevertheless,
here is a small introduction, and I hope it helps understanding
README.devel.

I am assuming you have installed GODI to /opt/godi (because this happens
to be the default), but of course, the same applies to any other prefix.
In /opt/godi there are two special directories:

/opt/godi/build: Here is the place where new sources are downloaded,
where they are installed, and where the compiled archives are stored.

/opt/godi/db: This is the package database that tracks where _binary_
packages are installed, so the files are known that make up a binary
installation of a package.

You can ignore the db directory for most of the time, it is just there,
and stores the state of the current installation.

The other directories below /opt/godi can (must) be used as installation
targets for packages (in the README found in the bootstrap tarball there
is a picture where to install what). More or less this is a standard
Unix tree.

The are some more interesting directories below "build":

/opt/godi/build/distfiles: These are the source tarballs downloaded from
the web or ftp sites of the authors of the packages. ("upstream sources"
in the slang.)

/opt/godi/build/packages: These are the binary packages that are the
result of compilation. If you build a package, the sources in
"distfiles" are taken and one tarball in "packages" is produced.

/opt/godi/build/mk: This directory contains build scripts. Most
important, there are "bsd.pkg.mk" and "bsd.prefs.mk", two Makefile
fragments that can be included by your own Makefiles. Don't change these
scripts.

/opt/godi/build/godi, conf, and base: These directories contain the
build instructions to compile packages. When you develop a package, you
create such build instructions that are put into a subdirectory here.
For example, the package "godi-foo" would reside in
/opt/godi/build/godi/godi-foo (i.e. the first word of the package name
determines the directory).

In this godi-foo directory, there must be four files:

- DESCR: This is just a text file with a description
- Makefile: This is a Makefile containing the entry points for the 
  build  process
- PLIST.godi: This file explains which files belong to the binary
  package (after installation)
- distinfo: This file contains checksums for the source files that
  are compiled

Of course, there can be more files, but these four files are enough for
the beginning. After you have started the build, there will be a
directory "work". At any point, you can remove this directory, it
contains only temporary work files (for example, the unpacked sources).

You can begin by writing DESCR and Makefile, the other two can be added
later. When there is a syntactically correct Makefile, you can create
distinfo by "bmake makesum".

This Makefile is called the driver Makefile. It does not call compilers
etc. directly, but its task is to control the build process. This
Makefile must use the syntax of BSD make. BSD make is installed under
the name godi_make, and usually as bmake, too (for convenience only,
scripts should call make under the name godi_make). There is a man page
for godi_make, or look here:
http://www.freebsd.org/cgi/man.cgi?query=make&apropos=0&sektion=0&manpath=NetBSD+1.6.1&format=html

In README.devel you can find a skeleton for a simple driver Makefile.
More or less, it includes only the already mentioned .mk fragments and
defines lots of variables describing the details of the package. It is
also a good idea to look at already existing packages to see what can be
defined here.

The driver Makefile controls the build process. There are a number of
stages, and a successful build performs all stages in turn:

- fetch: Get the sources from the internet, put them into "distfiles"
- extract: unpack the sources into "work"
- patch: apply patches (beginners should avoid patches!)
- configure: configure the sources
- build: compile the sources
- install: install the sources into /opt/godi
- package: create the tarball for the binary package, and put it into
  the "packages" directory.

You can activate a stage by running "bmake <stage>". For example, "bmake
configure" goes through all stages from the current stage until the
configure stage is done. (Do "ls -a work" to see how GODI remembers what
the current stage is.)

The code for the stages is defined in bsd.pkg.mk. These are internals of
GODI, and there should be no need to look at it or to understand it.

Now assume we have a Makefile for godi-foo. After "bmake fetch", the
source tarball is downloaded into the "distfiles" directory. After
"bmake extract", it is unpacked, e.g. in work/foo-2.3. "bmake configure"
usually changes to work/foo-2.3 and calls the configure script (but only
if you have defined HAS_CONFIGURE in the driver Makefile, otherwise
there is nothing to configure). "bmake build" changes to work/foo-2.3
and does a "make all".

This is an important point: Of course, there is usually another Makefile
in work/foo-2.3 that is part of the source distribution. This is the
Makefile that actually calls compilers etc. Of course, this Makefile can
be written for a different version of "make", e.g. for GNU make (to
enable this, add "USE_GMAKE=yes" to the driver Makefile).

"bmake install" changes again to work/foo-2.3 and does a "make install".
It is now expected that the built software is installed into its final
location. (Currently, "staged installations" into temporary locations
are not yet supported.)

Before you do "bmake install" you must write a PLIST.godi file to
describe which files are installed where. This is because "bmake
install" also registers the package in the package database. Often,
writing PLIST.godi is the most difficult part. There is a trick to find
out which files are installed: Add these lines to the driver Makefile
(in the next release of GODI, they will be part of the framework):

USE_CTIME?=

.PHONY: print-installed
print-installed:
        ${_PKG_SILENT}${_PKG_DEBUG}\
        cd ${PREFIX} && \
        ${FIND} . \! -type d \( -newer ${EXTRACT_COOKIE} ${USE_CTIME:S|^yes$|-o -cnewer ${EXTRACT_COOKIE}|} \) | \
        ${GREP} -v '^./build/' | \
        ${GREP} -v '^${GODI_DBDIR:C|/+|/|g:S|/$||:S|^${PREFIX:C|/+|/|g:S|/$||}/|./|}/' | \
        ${SED} -e 's|^./||'
 
Now do "bmake print-installed USE_CTIME=yes", and a list of all files is
printed that have an mtime or ctime timestamp later than when the source
tarball was unpacked. (Some versions of the "find" command used here do
not support ctime comparisons. In this case, don't pass USE_CTIME=yes,
and check the output more carefully.)

Of course, PLIST.godi should not just list the files, but summarise
them. See README.devel for a list of instructions that can be used here.

Normally, one begins without PLIST.godi, then does "bmake install"
(which will fail at the registration step), then calls "bmake
print-installed". From the output, write a PLIST.godi file. Then one has
to repeat the installation step. To do so, remove the file
work/.install_done, and call "bmake install" again. The point is that
after the first "bmake install" the files are installed but not
registered. So you cannot even remove them by deinstalling the package.
The packaging is only done when this registration step is carried out
correctly, and this is why we call "bmake install" again after we have a
sensible PLIST.godi file.

The last stage is "bmake package". It normally works when "install"
works. After this stage, one can do further tests whether all files are
registered: Delete the package, and install it again from the binary
tarball, e.g.

godi_delete godi-foo
godi_add ../../packages/All/godi-foo-2.3.tar.gz

Of course, one should check whether all files are deleted, and whether
the complete package is restored.

When this all works, you can try to build the package from godi_console.
Especially check whether the dependencies are complete.

And when this works, too, you can send the contents of the godi-foo
directory to me, and I can integrate it into GODI.

I hope this introduction is helpful for you, and maybe I find the time
to expand it to a complete tutorial.

Gerd
-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
------------------------------------------------------------

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners