From mboxrd@z Thu Jan 1 00:00:00 1970 MIME-Version: 1.0 In-Reply-To: References: Date: Mon, 17 Jan 2011 14:51:40 -0300 Message-ID: From: "Federico G. Benavento" To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [9fans] `mk` (from Plan9 ports) efficiency related issue Topicbox-Message-UUID: 9d78d882-ead6-11e9-9d60-3106f5b1d025 about debug, release CONF=3Ddebug DEBUG=3D`{if(~ $CONF release) echo -DNDEBUG} CFLAGS=3D$CFLAGS $DEBUG if you want debug you run mk 'CONF=3Ddebug' wouldn't something like that help? or have 2 files mkdebug and mkrelease so you include them from the other mkfiles as you see fit On Mon, Jan 17, 2011 at 2:46 PM, Federico G. Benavento wrote: > when you have a clean mkfile, doing mk clean; mk install is faster than a= ll the > dependency checking you'd want to do, specially is the project is a big b= loat > > take X11 for instance.... how long does it take to build it? > > On Mon, Jan 17, 2011 at 2:31 PM, Ciprian Dorin Craciun > wrote: >> On Mon, Jan 17, 2011 at 17:53, Robert Raschke = wrote: >>> >>> On Mon, Jan 17, 2011 at 3:33 PM, Ciprian Dorin Craciun >>> wrote: >>>> >>>> On Mon, Jan 17, 2011 at 17:00, Robert Raschke >>>> wrote: >>>> > Your email also doesn't explain why you cannot generate a "normal" >>>> > mk file. >>>> >>>> =C2=A0 =C2=A0I'm afraid I don't understand the question. What do you m= ean by >>>> "generating a normal mk file"? >>>> =C2=A0 =C2=A0A) Do you mean why am I using a generator that writes the= `mk` >>>> script instead of writing the `mk` script myself by hand? The answer >>>> to this is complexity: writing `mk` is Ok when you have a simple >>>> application to build, but as the application grows larger so does the >>>> make script. (And using meta rules is not always possible.) >>>> =C2=A0 =C2=A0B) Why isn't the output script a "normal" `mk` script? Ac= tually is >>>> a very simple script (no meta-rules, no shell expansion, etc.). It's >>>> just big. :) >>>> >>> >>> Sorry, I meant an idiomatic mk file, in the sense as they are used with= in >>> the Plan 9 distribution. Have a look at "Plan 9 Mkfiles" >>> (http://www.cs.bell-labs.com/sys/doc/mkfiles.html) and "Maintaining Fil= es on >>> Plan 9 with Mk" (http://www.cs.bell-labs.com/sys/doc/mk.html), if you >>> haven't already done so. >>> >>> I think by listing all your dependencies one by one, step by step, you = are >>> bypassing a lot of the strengths of a make system. I would expect your >>> generator to produce a mk include file with the meta rules plus the mk = file >>> itself which lists file dependencies in a concise manner. >>> >>> Robby >> >> On Mon, Jan 17, 2011 at 17:51, Federico G. Benavento >> wrote: >>> >>> a normal mkfile does have meta-rules and if you have so many targets >>> wouldn't it make sense to have more mkfiles? >>> >>> -- >>> Federico G. Benavento >> >> >> =C2=A0 =C2=A0I'll respond to both Robert and Federico in the same email,= as >> their observations an suggestions are on the same topic. >> >> =C2=A0 =C2=A0So for starters I've read both mentioned papers "Plan9 Mkfi= les" >> and "Maintaining Files on Plan9 with Mk", and I have the following >> observations: >> =C2=A0 =C2=A0* the second paper "Maintaining Files on ..." is more a gen= eral >> guide that describes the semantic and syntax of `mk` files; >> =C2=A0 =C2=A0* the first paper "Plan9 Mkfiles" focuses entirely and excl= usively >> on writing `mk` files for building Plan9 native applications; that is >> it describes the general rules on how to write short and simple `mk` >> files that take advantage on the existing Plan9 build infrastructure; >> =C2=A0 =C2=A0* as a consequence both of them seem to suggest writing sma= ll `mk` >> scripts that individually build each application or library; >> =C2=A0 =C2=A0* unfortunately what they don't deal with is inter-dependen= cies >> between multiple projects or libraries each with it's own `mk` script; >> =C2=A0 =C2=A0* thus if looking into the `plan9port` source code, inside = the >> `src/mkfile` I see the following snippet (and I count about 50 other >> similar instances): >> ~~~~ >> libs-%:V: >> =C2=A0 =C2=A0 =C2=A0 =C2=A0for i in $LIBDIRS >> =C2=A0 =C2=A0 =C2=A0 =C2=A0do >> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(cd $i; echo cd `= pwd`';' mk $MKFLAGS $stem; mk $MKFLAGS $stem) >> =C2=A0 =C2=A0 =C2=A0 =C2=A0done >> ~~~~ >> >> =C2=A0 =C2=A0But after reading the paper "Recursive Make Considered Harm= ful", I >> see that this approach poses at least the following problems: >> =C2=A0 =C2=A0* when developing and updating a single library, there is n= o way >> in which I can instruct `mk` to rebuild all dependent applications -- >> unless I know about them and enumerate them one by one; as a >> consequence -- unless I know very well the real dependency graph -- >> `mk` doesn't help me at all and I have to `mk clean all`; >> =C2=A0 =C2=A0* second, there is a performance bottleneck as now librarie= s can't >> be built in parallel; (this is fine if a library is quite big, but if >> I have libraries with a number of files on par with the number of >> processors I have wasted time;) >> =C2=A0 =C2=A0* third, it's almost impossible to make a dependency betwee= n one >> library to another; the dependency is encoded in their order in the >> variables like `LIBDIRS`; >> >> =C2=A0 =C2=A0I hope that these observations answer the first question of= why >> don't I just create a number of separate files one for each project or >> library. >> >> =C2=A0 =C2=A0(The cited paper is at http://aegis.sourceforge.net/auug97.= pdf .) >> >> =C2=A0 =C2=A0For the second question about why no "meta-rules", the answ= er is >> somehow trickier, thus I'll give a few problems which arise when using >> meta-rules: >> =C2=A0 =C2=A0* not all files of the same type are built by using the sam= e rule; >> (for example for some files I want to enable debugging, for others >> not); thus I don't see how a meta-rule would solve this problem; >> (unless I create separate `mk` files or I resort to filename tags -- >> for example I would call `*.debug.c` all C files that I want to be >> debugged, and `*.release.c` for those I don't); >> =C2=A0 =C2=A0* second, each file has a unique dependency graph -- for ex= ample >> one `*.c` file might include other headers than another `*.c` file in >> the same project; thus when updating a single header I want to be able >> to build only those `*.c` files that actually depend on it; (this >> observation is less important for C applications, but for other type >> of programs -- like Java -- this fine grained dependency tracking >> means a lot of saved computing power); >> =C2=A0 =C2=A0* third, in my brief experience with make files, meta-rules= are >> quite hard to get right... furthermore it is impossible to have two >> patterns like: `%.%.x`; just imagine I have two types of images: >> background and foreground and I want to superimpose them, then I would >> like to be able to write `%{foreground}.%{background}.jpg : >> %{foreground}.jpg %{background}.jpg ..."; (I know that I can resort >> here to "<| generating command" but it seems just plain wrong...) >> >> =C2=A0 =C2=A0Now I hope nobody was offended by my observations regarding= the >> way `mk` scripts are currently written. I am sure that for the purpose >> of building Plan9 applications this way is the best trade-off. But >> applying the same techniques to more complex programming languages or >> other systems is quite hard... Thus I just wanted to use the good `mk` >> tool as a backend for a build script generator that is more intimately >> acquainted with what it tries to build. (I want to extend my generator >> to Python -- as a replacement for `setup.py` -- and Java -- as a >> replacement for all the awful tools that exist in that ecosystem, >> especially Ant.) >> >> =C2=A0 =C2=A0Ciprian. >> >> > > > > -- > Federico G. Benavento > --=20 Federico G. Benavento