From mboxrd@z Thu Jan 1 00:00:00 1970 Received: (from majordomo@localhost) by pauillac.inria.fr (8.7.6/8.7.3) id TAA32072; Sun, 5 Sep 2004 19:05:03 +0200 (MET DST) X-Authentication-Warning: pauillac.inria.fr: majordomo set sender to owner-caml-list@pauillac.inria.fr using -f Received: from nez-perce.inria.fr (nez-perce.inria.fr [192.93.2.78]) by pauillac.inria.fr (8.7.6/8.7.3) with ESMTP id TAA31983 for ; Sun, 5 Sep 2004 19:05:01 +0200 (MET DST) Received: from smtp3.adl2.internode.on.net (smtp3.adl2.internode.on.net [203.16.214.203]) by nez-perce.inria.fr (8.13.0/8.13.0) with ESMTP id i85H4xY9006198 for ; Sun, 5 Sep 2004 19:05:00 +0200 Received: from [192.168.1.200] (ppp210-32.lns2.syd3.internode.on.net [203.122.210.32]) by smtp3.adl2.internode.on.net (8.12.9/8.12.9) with ESMTP id i85H4fHY023712; Mon, 6 Sep 2004 02:34:55 +0930 (CST) Subject: Re: [Caml-list] Announcing the OMake build system version 0.9.1 From: skaller Reply-To: skaller@users.sourceforge.net To: "Marcin 'Qrczak' Kowalczyk" Cc: caml-list In-Reply-To: <87sm9w4tfk.fsf@qrnik.zagroda> References: <4139ECD3.1050708@cs.caltech.edu> <001e01c492a6$872c7280$19b0e152@warp> <413A0921.7030607@ntlworld.com> <413A1E89.10806@libertysurf.fr> <1094361655.3352.476.camel@pelican.wigram> <87hdqc3kfr.fsf@qrnik.zagroda> <1094396893.3352.635.camel@pelican.wigram> <87sm9w4tfk.fsf@qrnik.zagroda> Content-Type: text/plain Message-Id: <1094403880.3352.751.camel@pelican.wigram> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 (1.2.2-4) Date: 06 Sep 2004 03:04:40 +1000 Content-Transfer-Encoding: 7bit X-Miltered: at nez-perce with ID 413B473B.000 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! X-Loop: caml-list@inria.fr X-Spam: no; 0.00; caml-list:01 sourceforge:01 2004:99 38,:01 marcin:01 'qrczak':01 kowalczyk:01 sourceforge:01 elide:01 interscript:01 python:01 outputs:01 interscript:01 python:01 outputs:01 Sender: owner-caml-list@pauillac.inria.fr Precedence: bulk On Mon, 2004-09-06 at 01:38, Marcin 'Qrczak' Kowalczyk wrote: > skaller writes: > > > The issue is how you determine what you can elide > > from the build. > > > > Interscript currently does do just what I said and it works. > > I don't understand. *How* does it know what to rebuild and how? It executes the build script, which basically is just a Python program with some library support thrown in. It is a literate programming tool, which has two commands of interest here: (1) include file (2) open output Function (2) opens output files, and also stores the name of the file in a persistent repository. I actually open a temporary file, write it, and the close() function actually compares the temporary with the pre-existing file, and overwrites it if they're different. The include function opens an input, and also keeps track of the source tree. together these two functions conspire to determine which input files generate which outputs. Now actually, instead of just opening an include file for input, what really happens is that the system first checks if the input, or any of its inputs, as determined by the last run, have changed (by checking time stamps). If not, the input is simply skipped over. The whole process is repeated until the output files are not changed (or a limit is exceeded). If you try to include a file that is generated, all is well even if it is generated *after* you include it. Either the subsequent generation will change the output or not. If not, there's no harm including an old copy, and if so, this will result in our current output being different and so the whole process will repeat. Whatever you do DO NOT print the time in seconds into a generated file :) Now, you may wonder what this has to do with building Ocaml programs, C programs, etc. There are two answers. The first is: the process is an instance of something more general: as I have noted interscript sources can contain arbitrary Python script to generate outputs, so there isn't anything in the process which is specific to a particular compiler or programming language. What seems to be required is the ability to track the output files -- it isn't relevant how they're constructed. However to *optimise* the process, you do need to have more project specific information. Now I want to give a kind of messy example why this process is more or less mandatory for bootrapped systems. The Felix build first imports configuration information, and uses it to condition the generated code. The configuration is defined as an executable Python file. The configuration file is generated by 'trial and error' execution of some scripts and C programs, in the autoconf style. The actual code that runs these programs is a Python script -- generated from the LP sources of course. There is also a makefile which hooks a Python script for doing the main Ocaml building. It is dependent on the configuration. It is *also* used to 'make' the configuration. So, there are quite a few circular dependencies here, and the client can actually edit the configuration by hand as well -- and those changes stick, even across new versions of the system. The process is bootstrapped with a makefile that I manually added to the LP sources -- its sure to be out of date and misconfigured for your system but that doesn't matter, because it is only used to launch the bootstrap -- after that a better one is extracted from the LP sources, and after making the config an even better one -- eventually all this stuff converges to a stable configuration. I don't need a long toolchain to do all this. One program -- interscript -- does everything including find your Ocaml installation :) > > Of course but you're missing the point... > > Indeed I am. I don't see how do you specify how code should be rebuilt. I specify the rebuild rules (in Python) in a reasonable order and just execute them. The rules are no different in principle than make rules, except that I use a full scale programming language to write them (and usually just call the system() function). > What does "fixpoint iteration" mean in this concept? What is iterated? The build process repeats until the generated outputs are the same twice in a row. Consider this function: build: file_set -> file_set Which takes all inputs + outputs and generates new outputs. Then we just apply 'build' repeatedly until it converges to a fixpoint 'files' specified by the equation: files = build (files) For example you could define 'build' as this: for b in *.ml do ocamlc b and just repeat it until the *.cmo files don't change. The initial compilation order is probably wrong. Fixpoint iteration will eventually lead to a correct ordering -- in fact the algorithm seems equivalent to a bubble sort: at least on of the ml files can't depend on anything so it will compile correctly on the first pass .. next pass things that depend on it will be ok .. etc.. so eventually all the files get built -- no matter what the order is. More generally this will work even with circular dependencies, provided of course you design the system so it converges. As a counter-example: latex doesn't always converge. -- John Skaller, mailto:skaller@users.sf.net voice: 061-2-9660-0850, snail: PO BOX 401 Glebe NSW 2037 Australia Checkout the Felix programming language http://felix.sf.net ------------------- To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/ Beginner's list: http://groups.yahoo.com/group/ocaml_beginners