Re: [Caml-list] Announcing the OMake build system version 0.9.1

[skaller <skaller@users.sourceforge.net>]

On Mon, 2004-09-06 at 01:38, Marcin 'Qrczak' Kowalczyk wrote:
> skaller <skaller@users.sourceforge.net> writes:
> 
> > The issue is how you determine what you can elide
> > from the build.
> >
> > Interscript currently does do just what I said and it works.
> 
> I don't understand. *How* does it know what to rebuild and how?

It executes the build script, which basically is just a Python
program with some library support thrown in.

It is a literate programming tool, which has two commands
of interest here:

	(1) include file
	(2) open output

Function (2) opens output files, and also stores the name
of the file in a persistent repository. I actually
open a temporary file, write it, and the close() function
actually compares the temporary with the pre-existing file,
and overwrites it if they're different.

The include function opens an input, and also keeps track
of the source tree.

together these two functions conspire to determine which
input files generate which outputs.

Now actually, instead of just opening an include file
for input, what really happens is that the system first
checks if the input, or any of its inputs, as determined
by the last run, have changed (by checking time stamps).

If not, the input is simply skipped over.

The whole process is repeated until the output files
are not changed (or a limit is exceeded).

If you try to include a file that is generated,
all is well even if it is generated *after* you
include it. Either the subsequent generation
will change the output or not. If not, there's
no harm including an old copy, and if so,
this will result in our current output being
different and so the whole process will 
repeat.

Whatever you do DO NOT print the time in seconds into
a generated file :)

Now, you may wonder what this has to do with 
building Ocaml programs, C programs, etc.

There are two answers. The first is: the process is
an instance of something more general: as I have
noted interscript sources can contain arbitrary
Python script to generate outputs, so there isn't
anything in the process which is specific 
to a particular compiler or programming language.

What seems to be required is the ability to 
track the output files -- it isn't relevant
how they're constructed.

However to *optimise* the process, you do need to
have more project specific information.

Now I want to give a kind of messy example why
this process is more or less mandatory for bootrapped
systems.

The Felix build first imports configuration information,
and uses it to condition the generated code.
The configuration is defined as an executable Python file.

The configuration file is generated by 'trial and error'
execution of some scripts and C programs, in the autoconf
style. The actual code that runs these programs is a Python
script -- generated from the LP sources of course.

There is also a makefile which hooks a Python script
for doing the main Ocaml building. It is dependent
on the configuration. It is *also* used to 'make'
the configuration.

So, there are quite a few circular dependencies here,
and the client can actually edit the configuration
by hand as well -- and those changes stick, even
across new versions of the system.

The process is bootstrapped with a makefile that
I manually added to the LP sources -- its sure
to be out of date and misconfigured for your system
but that doesn't matter, because it is only used
to launch the bootstrap -- after that a better
one is extracted from the LP sources, and after
making the config an even better one -- eventually
all this stuff converges to a stable configuration.

I don't need a long toolchain to do all this.
One program -- interscript -- does everything
including find your Ocaml installation :)


> > Of course but you're missing the point...
> 
> Indeed I am. I don't see how do you specify how code should be rebuilt.

I specify the rebuild rules (in Python) in a reasonable order
and just execute them.

The rules are no different in principle than make rules,
except that I use a full scale programming language
to write them (and usually just call the system() function).

> What does "fixpoint iteration" mean in this concept? What is iterated?

The build process repeats until the generated outputs
are the same twice in a row. Consider this function:

	build: file_set -> file_set


Which takes all inputs + outputs and generates new outputs.
Then we just apply 'build' repeatedly until it converges
to a fixpoint 'files' specified by the equation:

	files = build (files)

For example you could define 'build' as this:

	for b in *.ml do ocamlc b

and just repeat it until the *.cmo files don't change.
The initial compilation order is probably wrong.
Fixpoint iteration will eventually lead to a correct
ordering -- in fact the algorithm seems equivalent
to a bubble sort: at least on of the ml files
can't depend on anything so it will compile correctly
on the first pass .. next pass things that depend on it
will be ok .. etc.. so eventually all the files
get built -- no matter what the order is.

More generally this will work even with circular
dependencies, provided of course you design the
system so it converges.

As a counter-example: latex doesn't always converge.


-- 
John Skaller, mailto:skaller@users.sf.net
voice: 061-2-9660-0850, 
snail: PO BOX 401 Glebe NSW 2037 Australia
Checkout the Felix programming language http://felix.sf.net



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners