9front - general discussion about 9front
 help / color / mirror / Atom feed
* [9front] system mkfile changes
@ 2021-03-20 17:18 Amavect
  2021-03-24 18:44 ` [9front] " Amavect
  0 siblings, 1 reply; 5+ messages in thread
From: Amavect @ 2021-03-20 17:18 UTC (permalink / raw)
  To: 9front

All,

A while ago, I made some changes to the system mkfiles. I'd like to get them pushed.
Git them here:
https://git.sr.ht/~amavect/mkfilechanges
The tarball is:
https://git.sr.ht/~amavect/mkfilechanges/archive/master.tar.gz

The DOC file explains everything that is changed, but here's the highlights:
/sys/src/mkfile installall works
New %.cpus metarule builds for all $CPUS
Installs the compiler first
Reorganization in a logical order
Removing unneeded targets for 100 less lines of code

Thanks,
Amavect

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [9front] Re: system mkfile changes
  2021-03-20 17:18 [9front] system mkfile changes Amavect
@ 2021-03-24 18:44 ` Amavect
  2021-03-27 15:00   ` cinap_lenrek
  0 siblings, 1 reply; 5+ messages in thread
From: Amavect @ 2021-03-24 18:44 UTC (permalink / raw)
  To: 9front

All,

There was some criticism on IRC relayed to me.
My /sys/src/cmd/mkfile change would build cp and yacc instead of copying them locally.
A philosophical concern was brought up about the new mkfile assuming the build environment was incorrect.
That is not a problem the new mkfile intends to solve.
It solves the following problems:
1. There's a race between installing cp or yacc and their next invocation.
2. cp and yacc should be built first if there were any changes.
3. Installing cp is a circular dependency.

Reason 1 makes no assumption whether writing to the disk file system is atomic.
Reason 2 is why it's better to build yacc than copy from /bin.
Reason 3 is why it's better to build cp than copy from /bin.

Thus, building from scratch is too convenient.

Why not just install cp and yacc at /sys/src/mkfile to solve problems 2 and 3?
Now cp and yacc need to be excluded from cmd/mkfile's TARG to solve problem 1.
But now, cmd/mkfile's mk install no longer builds everything.
So why have cp and yacc there in the first place?

Alternatively, move cp and yacc to a different directory that represents programs used for the build environment.
This avoids the exceptions.
I'm not against that.

An argument brought up was that LDFLAGS is not always set the same across architectures.
This is absurd. No /$objtype/mkfile sets LDFLAGS.
Additionally, both the old and the new /sys/src/cmd/mkfile set LDFLAGS to be nothing!
For special cases like spim, 0l is an rc script around vl that injects the flag, so its interface is the same.
Unless there are concrete plans to make each linker's flag interface different, this argument is wrong.

There's an improvement that was not mentioned. $Ocpu can be eliminated.
If $objtype to $cputype match, then cp and yacc need to be built from scratch to avoid problems.
As a result, the CC and LD rules can be reverted to non-regexp form, solving any supposed LDFLAGS issues.
I have updated the repo reflecting these changes.
https://git.sr.ht/~amavect/mkfilechanges

Thanks,
Amavect

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [9front] Re: system mkfile changes
  2021-03-24 18:44 ` [9front] " Amavect
@ 2021-03-27 15:00   ` cinap_lenrek
  2021-03-29 19:44     ` umbraticus
  2021-04-02 18:01     ` Amavect
  0 siblings, 2 replies; 5+ messages in thread
From: cinap_lenrek @ 2021-03-27 15:00 UTC (permalink / raw)
  To: 9front

Ok, heres my review.

> diff -r cc26fdf57bb2 sys/src/cmd/mkfile
> --- a/sys/src/cmd/mkfile	Sat Mar 13 14:57:53 2021 +0100
> +++ b/sys/src/cmd/mkfile	Sat Mar 27 14:58:00 2021 +0100
> @@ -1,137 +1,84 @@
>  </$objtype/mkfile


> -TARG=`{ls *.[cy] | sed '/\.tab\.c$/d;s/..$//'}
> +YTARG=`{echo *.y | sed 's/\.y//g'}
> +TARG=$YTARG `{echo *.c | sed 's/\.c//g'}

this is tricky, i think using ls here is better because then you
can actually match the file extension (has to end at a line).
otherwise, this can produce false positives.

> +# solve race, build new cp and yacc first
> +CPDEP=`{if(~ $objtype $cputype) echo $O.cp}
> +CP=`{if(~ $objtype $cputype) echo ./$O.cp ; if not echo cp}
> +YACCDEP=`{if(~ $objtype $cputype) echo $O.yacc}
> +YACC=`{if(~ $objtype $cputype) echo ./$O.yacc ; if not echo yacc}

i dont think this should be done here.

this is a bootstrapping problem and should be handled by /sys/src/mkfile
updating the (cross) compilers and build prerequisites first for the current
$cputype, then install them.

the rest of the mkfiles should be able to always rely on that the currently
installed binaries for compilers, linkers, assemblers, yacc, mk, cp, awk,
sed, grep, rc shell are the correct ones.

these races should never exist if we have clear separation of build
and installation, and the install step makes sure its not relying on the very
tools it is currently installing.

All these rule changes that use $CP and $CPDEP and $YACC and $YACCDEP
have to go. It is just insanity.

I'm fine with the cleanup of removing update rules and changing none
rule to default.

The problem that we want to solve is dealing with toolchain updates
after a sysupdate. And this cannot be solved locally as the dependency
information is not stated in the mkfiles.

To build the toolchain, you also need to build their library dependencies,
which is also not stated in the mkfiles.

--
cinap

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [9front] Re: system mkfile changes
  2021-03-27 15:00   ` cinap_lenrek
@ 2021-03-29 19:44     ` umbraticus
  2021-04-02 18:01     ` Amavect
  1 sibling, 0 replies; 5+ messages in thread
From: umbraticus @ 2021-03-29 19:44 UTC (permalink / raw)
  To: 9front

> -TARG=`{ls *.[cy] | sed '/\.tab\.c$/d;s/..$//'}
> +YTARG=`{echo *.y | sed 's/\.y//g'}
> +TARG=$YTARG `{echo *.c | sed 's/\.c//g'}

ls | sed -n 's/\.[cy]$//p'

or, to stay true to the original...

ls | sed -n '/\.tab\.c$/d;s/\.[cy]$//p'

Could use walk instead of ls.
I'm definitely not qualified to comment on the rest…

umbraticus

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [9front] Re: system mkfile changes
  2021-03-27 15:00   ` cinap_lenrek
  2021-03-29 19:44     ` umbraticus
@ 2021-04-02 18:01     ` Amavect
  1 sibling, 0 replies; 5+ messages in thread
From: Amavect @ 2021-04-02 18:01 UTC (permalink / raw)
  To: 9front, cinap_lenrek

On March 27, 2021 10:00:27 AM CDT, cinap_lenrek@felloff.net wrote:
>this is tricky, i think using ls here is better
I believe I was trying to avoid `ls *.[cy]`, which is marginally slower locally, and very slow over a network versus plain `ls`.
Umbraticus' suggestion works great.

The CPDEP stuff was me thowing my hands in the air. Sorry about that.

>the rest of the mkfiles should be able to always rely on that the
>currently
>installed binaries for compilers, linkers, assemblers, yacc, mk, cp,
>awk,
>sed, grep, rc shell are the correct ones.
>
>these races should never exist if we have clear separation of build
>and installation, and the install step makes sure its not relying on
>the very
>tools it is currently installing.
I wanted to avoid serializing the install rule, because that screws up mk's ability to read dependencies.
Perhaps the best solution is to put cp and yacc in their own directory. Maybe a /sys/src/base directory could hold the build environment sources.

Anyways, I gave up.
There's now a small chance of the user specifying install targets manually and getting an ioerror as yacc and cp get installed.
Here's the latest tarball.
https://git.sr.ht/~amavect/mkfilechanges/archive/master.tar.gz

>To build the toolchain, you also need to build their library
>dependencies,
>which is also not stated in the mkfiles.
This is a limitation of mk.
I would love to "import" the rules to build those dependencies from anywhere in the src tree, and finally solve the issue of recursive mk. I have some ideas brewing there.

Thanks,
Amavect

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2021-04-03  8:41 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-20 17:18 [9front] system mkfile changes Amavect
2021-03-24 18:44 ` [9front] " Amavect
2021-03-27 15:00   ` cinap_lenrek
2021-03-29 19:44     ` umbraticus
2021-04-02 18:01     ` Amavect

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).