From: "Federico G. Benavento" <benavento@gmail.com>
To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net>
Subject: Re: [9fans] `mk` (from Plan9 ports) efficiency related issue
Date: Mon, 17 Jan 2011 14:51:40 -0300 [thread overview]
Message-ID: <AANLkTinf9M9jfhV4wHJWeWBRdt-d9JSpF=HwVbmrkHNm@mail.gmail.com> (raw)
In-Reply-To: <AANLkTinBwCthyz1H7_-XiCHV3oB_W9XOmJqgngHkE5TJ@mail.gmail.com>
about debug, release
CONF=debug
DEBUG=`{if(~ $CONF release) echo -DNDEBUG}
CFLAGS=$CFLAGS $DEBUG
if you want debug you run
mk 'CONF=debug'
wouldn't something like that help?
or have 2 files mkdebug and mkrelease
so you include them from the other mkfiles as you see fit
On Mon, Jan 17, 2011 at 2:46 PM, Federico G. Benavento
<benavento@gmail.com> wrote:
> when you have a clean mkfile, doing mk clean; mk install is faster than all the
> dependency checking you'd want to do, specially is the project is a big bloat
>
> take X11 for instance.... how long does it take to build it?
>
> On Mon, Jan 17, 2011 at 2:31 PM, Ciprian Dorin Craciun
> <ciprian.craciun@gmail.com> wrote:
>> On Mon, Jan 17, 2011 at 17:53, Robert Raschke <rtrlists@googlemail.com> wrote:
>>>
>>> On Mon, Jan 17, 2011 at 3:33 PM, Ciprian Dorin Craciun
>>> <ciprian.craciun@gmail.com> wrote:
>>>>
>>>> On Mon, Jan 17, 2011 at 17:00, Robert Raschke <rtrlists@googlemail.com>
>>>> wrote:
>>>> > Your email also doesn't explain why you cannot generate a "normal"
>>>> > mk file.
>>>>
>>>> I'm afraid I don't understand the question. What do you mean by
>>>> "generating a normal mk file"?
>>>> A) Do you mean why am I using a generator that writes the `mk`
>>>> script instead of writing the `mk` script myself by hand? The answer
>>>> to this is complexity: writing `mk` is Ok when you have a simple
>>>> application to build, but as the application grows larger so does the
>>>> make script. (And using meta rules is not always possible.)
>>>> B) Why isn't the output script a "normal" `mk` script? Actually is
>>>> a very simple script (no meta-rules, no shell expansion, etc.). It's
>>>> just big. :)
>>>>
>>>
>>> Sorry, I meant an idiomatic mk file, in the sense as they are used within
>>> the Plan 9 distribution. Have a look at "Plan 9 Mkfiles"
>>> (http://www.cs.bell-labs.com/sys/doc/mkfiles.html) and "Maintaining Files on
>>> Plan 9 with Mk" (http://www.cs.bell-labs.com/sys/doc/mk.html), if you
>>> haven't already done so.
>>>
>>> I think by listing all your dependencies one by one, step by step, you are
>>> bypassing a lot of the strengths of a make system. I would expect your
>>> generator to produce a mk include file with the meta rules plus the mk file
>>> itself which lists file dependencies in a concise manner.
>>>
>>> Robby
>>
>> On Mon, Jan 17, 2011 at 17:51, Federico G. Benavento
>> <benavento@gmail.com> wrote:
>>>
>>> a normal mkfile does have meta-rules and if you have so many targets
>>> wouldn't it make sense to have more mkfiles?
>>>
>>> --
>>> Federico G. Benavento
>>
>>
>> I'll respond to both Robert and Federico in the same email, as
>> their observations an suggestions are on the same topic.
>>
>> So for starters I've read both mentioned papers "Plan9 Mkfiles"
>> and "Maintaining Files on Plan9 with Mk", and I have the following
>> observations:
>> * the second paper "Maintaining Files on ..." is more a general
>> guide that describes the semantic and syntax of `mk` files;
>> * the first paper "Plan9 Mkfiles" focuses entirely and exclusively
>> on writing `mk` files for building Plan9 native applications; that is
>> it describes the general rules on how to write short and simple `mk`
>> files that take advantage on the existing Plan9 build infrastructure;
>> * as a consequence both of them seem to suggest writing small `mk`
>> scripts that individually build each application or library;
>> * unfortunately what they don't deal with is inter-dependencies
>> between multiple projects or libraries each with it's own `mk` script;
>> * thus if looking into the `plan9port` source code, inside the
>> `src/mkfile` I see the following snippet (and I count about 50 other
>> similar instances):
>> ~~~~
>> libs-%:V:
>> for i in $LIBDIRS
>> do
>> (cd $i; echo cd `pwd`';' mk $MKFLAGS $stem; mk $MKFLAGS $stem)
>> done
>> ~~~~
>>
>> But after reading the paper "Recursive Make Considered Harmful", I
>> see that this approach poses at least the following problems:
>> * when developing and updating a single library, there is no way
>> in which I can instruct `mk` to rebuild all dependent applications --
>> unless I know about them and enumerate them one by one; as a
>> consequence -- unless I know very well the real dependency graph --
>> `mk` doesn't help me at all and I have to `mk clean all`;
>> * second, there is a performance bottleneck as now libraries can't
>> be built in parallel; (this is fine if a library is quite big, but if
>> I have libraries with a number of files on par with the number of
>> processors I have wasted time;)
>> * third, it's almost impossible to make a dependency between one
>> library to another; the dependency is encoded in their order in the
>> variables like `LIBDIRS`;
>>
>> I hope that these observations answer the first question of why
>> don't I just create a number of separate files one for each project or
>> library.
>>
>> (The cited paper is at http://aegis.sourceforge.net/auug97.pdf .)
>>
>> For the second question about why no "meta-rules", the answer is
>> somehow trickier, thus I'll give a few problems which arise when using
>> meta-rules:
>> * not all files of the same type are built by using the same rule;
>> (for example for some files I want to enable debugging, for others
>> not); thus I don't see how a meta-rule would solve this problem;
>> (unless I create separate `mk` files or I resort to filename tags --
>> for example I would call `*.debug.c` all C files that I want to be
>> debugged, and `*.release.c` for those I don't);
>> * second, each file has a unique dependency graph -- for example
>> one `*.c` file might include other headers than another `*.c` file in
>> the same project; thus when updating a single header I want to be able
>> to build only those `*.c` files that actually depend on it; (this
>> observation is less important for C applications, but for other type
>> of programs -- like Java -- this fine grained dependency tracking
>> means a lot of saved computing power);
>> * third, in my brief experience with make files, meta-rules are
>> quite hard to get right... furthermore it is impossible to have two
>> patterns like: `%.%.x`; just imagine I have two types of images:
>> background and foreground and I want to superimpose them, then I would
>> like to be able to write `%{foreground}.%{background}.jpg :
>> %{foreground}.jpg %{background}.jpg ..."; (I know that I can resort
>> here to "<| generating command" but it seems just plain wrong...)
>>
>> Now I hope nobody was offended by my observations regarding the
>> way `mk` scripts are currently written. I am sure that for the purpose
>> of building Plan9 applications this way is the best trade-off. But
>> applying the same techniques to more complex programming languages or
>> other systems is quite hard... Thus I just wanted to use the good `mk`
>> tool as a backend for a build script generator that is more intimately
>> acquainted with what it tries to build. (I want to extend my generator
>> to Python -- as a replacement for `setup.py` -- and Java -- as a
>> replacement for all the awful tools that exist in that ecosystem,
>> especially Ant.)
>>
>> Ciprian.
>>
>>
>
>
>
> --
> Federico G. Benavento
>
--
Federico G. Benavento
next prev parent reply other threads:[~2011-01-17 17:51 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-01-17 14:47 Ciprian Dorin Craciun
2011-01-17 14:59 ` erik quanstrom
2011-01-17 15:05 ` Ciprian Dorin Craciun
2011-01-17 16:50 ` Bakul Shah
2011-01-17 16:56 ` erik quanstrom
2011-01-17 18:36 ` Bakul Shah
2011-01-17 19:33 ` Ciprian Dorin Craciun
2011-01-17 19:59 ` Ciprian Dorin Craciun
2011-01-17 20:30 ` Bakul Shah
2011-01-17 15:00 ` Robert Raschke
2011-01-17 15:02 ` Robert Raschke
2011-01-17 15:18 ` erik quanstrom
2011-01-17 15:33 ` Ciprian Dorin Craciun
2011-01-17 15:51 ` Federico G. Benavento
2011-01-17 15:53 ` Robert Raschke
2011-01-17 16:02 ` andrey mirtchovski
2011-01-17 16:45 ` Ciprian Dorin Craciun
2011-01-17 17:31 ` Ciprian Dorin Craciun
2011-01-17 17:46 ` Federico G. Benavento
2011-01-17 17:51 ` Federico G. Benavento [this message]
2011-01-18 6:09 ` Andy Spencer
2011-01-18 13:26 ` erik quanstrom
2011-01-17 15:21 ` Ciprian Dorin Craciun
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='AANLkTinf9M9jfhV4wHJWeWBRdt-d9JSpF=HwVbmrkHNm@mail.gmail.com' \
--to=benavento@gmail.com \
--cc=9fans@9fans.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).