* [9front] system mkfile changes @ 2021-03-20 17:18 Amavect 2021-03-24 18:44 ` [9front] " Amavect 0 siblings, 1 reply; 5+ messages in thread From: Amavect @ 2021-03-20 17:18 UTC (permalink / raw) To: 9front All, A while ago, I made some changes to the system mkfiles. I'd like to get them pushed. Git them here: https://git.sr.ht/~amavect/mkfilechanges The tarball is: https://git.sr.ht/~amavect/mkfilechanges/archive/master.tar.gz The DOC file explains everything that is changed, but here's the highlights: /sys/src/mkfile installall works New %.cpus metarule builds for all $CPUS Installs the compiler first Reorganization in a logical order Removing unneeded targets for 100 less lines of code Thanks, Amavect ^ permalink raw reply [flat|nested] 5+ messages in thread
* [9front] Re: system mkfile changes 2021-03-20 17:18 [9front] system mkfile changes Amavect @ 2021-03-24 18:44 ` Amavect 2021-03-27 15:00 ` cinap_lenrek 0 siblings, 1 reply; 5+ messages in thread From: Amavect @ 2021-03-24 18:44 UTC (permalink / raw) To: 9front All, There was some criticism on IRC relayed to me. My /sys/src/cmd/mkfile change would build cp and yacc instead of copying them locally. A philosophical concern was brought up about the new mkfile assuming the build environment was incorrect. That is not a problem the new mkfile intends to solve. It solves the following problems: 1. There's a race between installing cp or yacc and their next invocation. 2. cp and yacc should be built first if there were any changes. 3. Installing cp is a circular dependency. Reason 1 makes no assumption whether writing to the disk file system is atomic. Reason 2 is why it's better to build yacc than copy from /bin. Reason 3 is why it's better to build cp than copy from /bin. Thus, building from scratch is too convenient. Why not just install cp and yacc at /sys/src/mkfile to solve problems 2 and 3? Now cp and yacc need to be excluded from cmd/mkfile's TARG to solve problem 1. But now, cmd/mkfile's mk install no longer builds everything. So why have cp and yacc there in the first place? Alternatively, move cp and yacc to a different directory that represents programs used for the build environment. This avoids the exceptions. I'm not against that. An argument brought up was that LDFLAGS is not always set the same across architectures. This is absurd. No /$objtype/mkfile sets LDFLAGS. Additionally, both the old and the new /sys/src/cmd/mkfile set LDFLAGS to be nothing! For special cases like spim, 0l is an rc script around vl that injects the flag, so its interface is the same. Unless there are concrete plans to make each linker's flag interface different, this argument is wrong. There's an improvement that was not mentioned. $Ocpu can be eliminated. If $objtype to $cputype match, then cp and yacc need to be built from scratch to avoid problems. As a result, the CC and LD rules can be reverted to non-regexp form, solving any supposed LDFLAGS issues. I have updated the repo reflecting these changes. https://git.sr.ht/~amavect/mkfilechanges Thanks, Amavect ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [9front] Re: system mkfile changes 2021-03-24 18:44 ` [9front] " Amavect @ 2021-03-27 15:00 ` cinap_lenrek 2021-03-29 19:44 ` umbraticus 2021-04-02 18:01 ` Amavect 0 siblings, 2 replies; 5+ messages in thread From: cinap_lenrek @ 2021-03-27 15:00 UTC (permalink / raw) To: 9front Ok, heres my review. > diff -r cc26fdf57bb2 sys/src/cmd/mkfile > --- a/sys/src/cmd/mkfile Sat Mar 13 14:57:53 2021 +0100 > +++ b/sys/src/cmd/mkfile Sat Mar 27 14:58:00 2021 +0100 > @@ -1,137 +1,84 @@ > </$objtype/mkfile > -TARG=`{ls *.[cy] | sed '/\.tab\.c$/d;s/..$//'} > +YTARG=`{echo *.y | sed 's/\.y//g'} > +TARG=$YTARG `{echo *.c | sed 's/\.c//g'} this is tricky, i think using ls here is better because then you can actually match the file extension (has to end at a line). otherwise, this can produce false positives. > +# solve race, build new cp and yacc first > +CPDEP=`{if(~ $objtype $cputype) echo $O.cp} > +CP=`{if(~ $objtype $cputype) echo ./$O.cp ; if not echo cp} > +YACCDEP=`{if(~ $objtype $cputype) echo $O.yacc} > +YACC=`{if(~ $objtype $cputype) echo ./$O.yacc ; if not echo yacc} i dont think this should be done here. this is a bootstrapping problem and should be handled by /sys/src/mkfile updating the (cross) compilers and build prerequisites first for the current $cputype, then install them. the rest of the mkfiles should be able to always rely on that the currently installed binaries for compilers, linkers, assemblers, yacc, mk, cp, awk, sed, grep, rc shell are the correct ones. these races should never exist if we have clear separation of build and installation, and the install step makes sure its not relying on the very tools it is currently installing. All these rule changes that use $CP and $CPDEP and $YACC and $YACCDEP have to go. It is just insanity. I'm fine with the cleanup of removing update rules and changing none rule to default. The problem that we want to solve is dealing with toolchain updates after a sysupdate. And this cannot be solved locally as the dependency information is not stated in the mkfiles. To build the toolchain, you also need to build their library dependencies, which is also not stated in the mkfiles. -- cinap ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [9front] Re: system mkfile changes 2021-03-27 15:00 ` cinap_lenrek @ 2021-03-29 19:44 ` umbraticus 2021-04-02 18:01 ` Amavect 1 sibling, 0 replies; 5+ messages in thread From: umbraticus @ 2021-03-29 19:44 UTC (permalink / raw) To: 9front > -TARG=`{ls *.[cy] | sed '/\.tab\.c$/d;s/..$//'} > +YTARG=`{echo *.y | sed 's/\.y//g'} > +TARG=$YTARG `{echo *.c | sed 's/\.c//g'} ls | sed -n 's/\.[cy]$//p' or, to stay true to the original... ls | sed -n '/\.tab\.c$/d;s/\.[cy]$//p' Could use walk instead of ls. I'm definitely not qualified to comment on the rest… umbraticus ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [9front] Re: system mkfile changes 2021-03-27 15:00 ` cinap_lenrek 2021-03-29 19:44 ` umbraticus @ 2021-04-02 18:01 ` Amavect 1 sibling, 0 replies; 5+ messages in thread From: Amavect @ 2021-04-02 18:01 UTC (permalink / raw) To: 9front, cinap_lenrek On March 27, 2021 10:00:27 AM CDT, cinap_lenrek@felloff.net wrote: >this is tricky, i think using ls here is better I believe I was trying to avoid `ls *.[cy]`, which is marginally slower locally, and very slow over a network versus plain `ls`. Umbraticus' suggestion works great. The CPDEP stuff was me thowing my hands in the air. Sorry about that. >the rest of the mkfiles should be able to always rely on that the >currently >installed binaries for compilers, linkers, assemblers, yacc, mk, cp, >awk, >sed, grep, rc shell are the correct ones. > >these races should never exist if we have clear separation of build >and installation, and the install step makes sure its not relying on >the very >tools it is currently installing. I wanted to avoid serializing the install rule, because that screws up mk's ability to read dependencies. Perhaps the best solution is to put cp and yacc in their own directory. Maybe a /sys/src/base directory could hold the build environment sources. Anyways, I gave up. There's now a small chance of the user specifying install targets manually and getting an ioerror as yacc and cp get installed. Here's the latest tarball. https://git.sr.ht/~amavect/mkfilechanges/archive/master.tar.gz >To build the toolchain, you also need to build their library >dependencies, >which is also not stated in the mkfiles. This is a limitation of mk. I would love to "import" the rules to build those dependencies from anywhere in the src tree, and finally solve the issue of recursive mk. I have some ideas brewing there. Thanks, Amavect ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2021-04-03 8:41 UTC | newest] Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-03-20 17:18 [9front] system mkfile changes Amavect 2021-03-24 18:44 ` [9front] " Amavect 2021-03-27 15:00 ` cinap_lenrek 2021-03-29 19:44 ` umbraticus 2021-04-02 18:01 ` Amavect
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).