[PR PATCH] ./xbps-src bootstrap fixes

[PR PATCH] ./xbps-src bootstrap fixes

[fosslinux]

[-- Attachment #1: Type: text/plain, Size: 548 bytes --]

There is a new pull request by fosslinux against master on the void-packages repository

https://github.com/fosslinux/void-packages bootstrap-fixes
https://github.com/void-linux/void-packages/pull/21226

./xbps-src bootstrap fixes
I found a few missing/broken things while testing `./xbps-src bootstrap`:

1. `bootstrap` requires texinfo on the host.
2. glibc doesn't contain usr/share/info.
3. m4 was broken.

Closes #21163 . Thanks to @rovenmaccorp

A patch file from https://github.com/void-linux/void-packages/pull/21226.patch is attached

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-bootstrap-fixes-21226.patch --]
[-- Type: text/x-diff, Size: 2242 bytes --]

From 17ba74b1a4fde9b1e6068d64a9bfd8e190aa5bfc Mon Sep 17 00:00:00 2001
From: fosslinux <fosslinux@aussies.space>
Date: Tue, 21 Apr 2020 10:56:27 +1000
Subject: [PATCH 1/3] xbps-src: Bootstrap requires makeinfo (texinfo)

---
 xbps-src | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xbps-src b/xbps-src
index 459f27eb7b7..c2d5abb96e5 100755
--- a/xbps-src
+++ b/xbps-src
@@ -394,7 +394,7 @@ umask 022
 #
 # Required utilities in host system for the bootstrap target.
 readonly REQHOST_UTILS_BOOTSTRAP="file objdump find make gawk bash sed gcc g++ gnat \
-    perl bsdtar gzip patch flock pkg-config"
+    perl bsdtar gzip patch flock pkg-config makeinfo"
 
 # Required utilities in host.
 readonly REQHOST_UTILS="xbps-install xbps-query xbps-rindex xbps-uhelper \

From 2374479a8d304c562ffded522ca14b938e37cbc6 Mon Sep 17 00:00:00 2001
From: fosslinux <fosslinux@aussies.space>
Date: Tue, 21 Apr 2020 10:56:51 +1000
Subject: [PATCH 2/3] glibc: usr/share/info is not created

---
 srcpkgs/glibc/template | 1 -
 1 file changed, 1 deletion(-)

diff --git a/srcpkgs/glibc/template b/srcpkgs/glibc/template
index dceecd7e75b..6c382200488 100644
--- a/srcpkgs/glibc/template
+++ b/srcpkgs/glibc/template
@@ -195,7 +195,6 @@ glibc-devel_package() {
 		vmove usr/include
 		vmove "usr/lib/*.a"
 		vmove "usr/lib/*.o"
-		vmove usr/share/info
 	}
 }
 glibc-locales_package() {

From dc1107d9fccc1a3756f9513643595d9f81d75485 Mon Sep 17 00:00:00 2001
From: fosslinux <fosslinux@aussies.space>
Date: Tue, 21 Apr 2020 10:57:26 +1000
Subject: [PATCH 3/3] m4: Fix texinfo locale break

Closes https://github.com/void-linux/void-packages/issues/21163.

Credit to rovenmaccorp.
---
 srcpkgs/m4/template | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/srcpkgs/m4/template b/srcpkgs/m4/template
index 111dc6a6b07..12061cef337 100644
--- a/srcpkgs/m4/template
+++ b/srcpkgs/m4/template
@@ -14,6 +14,10 @@ maintainer="Enno Boland <gottox@voidlinux.org>"
 distfiles="${GNU_SITE}/${pkgname}/${pkgname}-${version}.tar.xz"
 checksum=f2c1e86ca0a404ff281631bdc8377638992744b175afb806e25871a24a934e07
 
+pre_build() {
+    sed -i '1 i @documentencoding ISO-8859-1' doc/m4.texi
+}
+
 pre_check() {
 	case "$XBPS_TARGET_MACHINE" in
 		*-musl* )

Re: ./xbps-src bootstrap fixes

[xtraeme]

[-- Attachment #1: Type: text/plain, Size: 291 bytes --]

New comment by xtraeme on void-packages repository

https://github.com/void-linux/void-packages/pull/21226#issuecomment-617442403

Comment:
Well, in my system this wasn't true with musl.

Anyway I don't agree in depending in texinfo for bootstrapping. Just find a way to not require it.

Re: ./xbps-src bootstrap fixes

[xtraeme]

[-- Attachment #1: Type: text/plain, Size: 333 bytes --]

New comment by xtraeme on void-packages repository

https://github.com/void-linux/void-packages/pull/21226#issuecomment-617443025

Comment:
NOTE texinfo is optional and should not be required for "./xbps-src bootstrap".

This is why I removed it. If any of the `bootstrap` pkgs fails due to missing `texinfo` it needs to be fixed.

Re: ./xbps-src bootstrap fixes

[pullmoll]

[-- Attachment #1: Type: text/plain, Size: 825 bytes --]

New comment by pullmoll on void-packages repository

https://github.com/void-linux/void-packages/pull/21226#issuecomment-617448566

Comment:
After a bootstrap you need to rebuild bot `glibc` and `gcc` anyway, because some of the submodules are not built (e.g. `gcc-fortran`, `gcc-objc`, `gcc-go`, …). The same applies for glibc which in the bootstrapped version does not require any of `bison`, `perl`, `python3`, or `texinfo`, but only when it is built again in a chroot environment with those packages available.

Thus this PR is invalid in that it tries to remove the distinction between bootstrapped and rebuilt core packages. The bootstrap was conceived to be as minimalistic as possible. Building up an entire repository of packages requires some bootstrap packages being rebuilt in a working chroot environment.

Re: ./xbps-src bootstrap fixes

[pullmoll]

[-- Attachment #1: Type: text/plain, Size: 826 bytes --]

New comment by pullmoll on void-packages repository

https://github.com/void-linux/void-packages/pull/21226#issuecomment-617448566

Comment:
After a bootstrap you need to rebuild both `glibc` and `gcc` anyway, because some of the submodules are not built (e.g. `gcc-fortran`, `gcc-objc`, `gcc-go`, …). The same applies for glibc which in the bootstrapped version does not require any of `bison`, `perl`, `python3`, or `texinfo`, but only when it is built again in a chroot environment with those packages available.

Thus this PR is invalid in that it tries to remove the distinction between bootstrapped and rebuilt core packages. The bootstrap was conceived to be as minimalistic as possible. Building up an entire repository of packages requires some bootstrap packages being rebuilt in a working chroot environment.

Re: ./xbps-src bootstrap fixes

[xtraeme]

[-- Attachment #1: Type: text/plain, Size: 319 bytes --]

New comment by xtraeme on void-packages repository

https://github.com/void-linux/void-packages/pull/21226#issuecomment-617449549

Comment:
Exactly what @pullmoll said. I've built past week everything via "./xbps-src bootstrap" and I made sure texinfo wasn't required! it should not be required at all in the host.

Re: ./xbps-src bootstrap fixes

[fosslinux]

[-- Attachment #1: Type: text/plain, Size: 721 bytes --]

New comment by fosslinux on void-packages repository

https://github.com/void-linux/void-packages/pull/21226#issuecomment-617452566

Comment:
Hm, ok. It seems I misunderstood the way the bootstrap is meant to work. I like that it should be as minimalistic as possible.

Let me just clarify; the right way forward with m4 is to remove the need for texinfo when `./xbps-src bootstrap` is run? (ie, remove the doc generation?)

What is the correct way to then make m4 use texinfo inside the chroot? Would it be the creation of a `chroot-m4` package?

@xtraeme You say bootstrap vs chroot case in your review comments - how do you distinguish between the two?

Sorry for the questions, I think I'm missing something.

Re: ./xbps-src bootstrap fixes

[pullmoll]

[-- Attachment #1: Type: text/plain, Size: 445 bytes --]

New comment by pullmoll on void-packages repository

https://github.com/void-linux/void-packages/pull/21226#issuecomment-617453434

Comment:
> @xtraeme You say bootstrap vs chroot case in your review comments - how do you distinguish between the two?

Let me answer this: `if [ -z "$CHROOT_READY" ]; then` you are in a bootstrap.
You see this in e.g. `srcpkgs/glibc/template` where `texinfo` is a host dependency only if `$CHROOT_READY`.

Re: ./xbps-src bootstrap fixes

[fosslinux]

[-- Attachment #1: Type: text/plain, Size: 416 bytes --]

New comment by fosslinux on void-packages repository

https://github.com/void-linux/void-packages/pull/21226#issuecomment-617454032

Comment:
Ahhhh, I see! Right, I will modify this PR to do it the proper way - remove the need for m4 to have texinfo in the bootstrap. And, for glibc to not use `/usr/share/info` during the bootstrap, but use it in the chroot (assuming that as xtrame says it is used in the chroot).

Re: ./xbps-src bootstrap fixes

[xtraeme]

[-- Attachment #1: Type: text/plain, Size: 450 bytes --]

New comment by xtraeme on void-packages repository

https://github.com/void-linux/void-packages/pull/21226#issuecomment-617455626

Comment:
hmm @pullmoll I think we can improve gcc:

 - create a `gcc-stage0` (or whatever) for bootstrap, this should not conflict with the standard `gcc` and should only provide C/C++ support.

- make `gcc` not a bootstrap pkg anymore.

This way we shouldn't need to use `-f` while building `gcc` in the chroot.

Re: ./xbps-src bootstrap fixes

[pullmoll]

[-- Attachment #1: Type: text/plain, Size: 555 bytes --]

New comment by pullmoll on void-packages repository

https://github.com/void-linux/void-packages/pull/21226#issuecomment-617458013

Comment:
@xtraeme LFS even has 3 builds of gcc: [pass1](http://www.linuxfromscratch.org/lfs/view/stable/chapter05/gcc-pass1.html) and [pass2](http://www.linuxfromscratch.org/lfs/view/stable/chapter05/gcc-pass2.html) in the preparation stages, then [another build](http://www.linuxfromscratch.org/lfs/view/stable/chapter06/gcc.html) in the final stage.
I have not taken a closer look as to why they do it that way, though.

Re: ./xbps-src bootstrap fixes

[xtraeme]

[-- Attachment #1: Type: text/plain, Size: 368 bytes --]

New comment by xtraeme on void-packages repository

https://github.com/void-linux/void-packages/pull/21226#issuecomment-617458608

Comment:
@pullmoll I think we don't need this, simply setting `--enable-bootstrap` should be good enough, and I think it's enabled by default. This builds a minimal gcc and builds two more times and then compare them for a final result.

Re: ./xbps-src bootstrap fixes

[fosslinux]

[-- Attachment #1: Type: text/plain, Size: 471 bytes --]

New comment by fosslinux on void-packages repository

https://github.com/void-linux/void-packages/pull/21226#issuecomment-617460329

Comment:
@pullmoll , the first two passes are to completely remove any references to the host system in the initial toolchain. The last build is for the final system. They do the two-pass thing in the initial toolchain as they have speical things that --enable-bootstrap can't do for them. AFAICT, all void needs is `--enable-bootstrap`.

Re: ./xbps-src bootstrap fixes

[pullmoll]

[-- Attachment #1: Type: text/plain, Size: 324 bytes --]

New comment by pullmoll on void-packages repository

https://github.com/void-linux/void-packages/pull/21226#issuecomment-617460406

Comment:
@xtraeme that would make bootstrapping a tad easier to understand and handle for a newbie, sure. Basically you could strip all language options from the gcc template and be done, no?

Re: ./xbps-src bootstrap fixes

[xtraeme]

[-- Attachment #1: Type: text/plain, Size: 244 bytes --]

New comment by xtraeme on void-packages repository

https://github.com/void-linux/void-packages/pull/21226#issuecomment-617461208

Comment:
I'm not sure why they don't use `--enable-bootstrap`, maybe it's just a leftover from old gcc versions.

Re: ./xbps-src bootstrap fixes

[xtraeme]

[-- Attachment #1: Type: text/plain, Size: 225 bytes --]

New comment by xtraeme on void-packages repository

https://github.com/void-linux/void-packages/pull/21226#issuecomment-617461604

Comment:
ah I see, it's because pass1 builds without glibc and then pass2 is built with glibc

Re: ./xbps-src bootstrap fixes

[pullmoll]

[-- Attachment #1: Type: text/plain, Size: 299 bytes --]

New comment by pullmoll on void-packages repository

https://github.com/void-linux/void-packages/pull/21226#issuecomment-617461865

Comment:
Yes, I just read that too. They build stage1 with newlib instead of glibc to also distangle (or dis-entangle?) any possible connection between gcc and glibc.

Re: ./xbps-src bootstrap fixes

[xtraeme]

[-- Attachment #1: Type: text/plain, Size: 205 bytes --]

New comment by xtraeme on void-packages repository

https://github.com/void-linux/void-packages/pull/21226#issuecomment-617462189

Comment:
I haven't seen any artifacts by using our way on these years :-)

Re: ./xbps-src bootstrap fixes

[xtraeme]

[-- Attachment #1: Type: text/plain, Size: 245 bytes --]

New comment by xtraeme on void-packages repository

https://github.com/void-linux/void-packages/pull/21226#issuecomment-617462485

Comment:
to build a compiler you need a compiler and a C compiler needs libc headers... so why complicate things?

Re: ./xbps-src bootstrap fixes

[pullmoll]

[-- Attachment #1: Type: text/plain, Size: 367 bytes --]

New comment by pullmoll on void-packages repository

https://github.com/void-linux/void-packages/pull/21226#issuecomment-617463270

Comment:
LFS is very pedantic about checking their build results, which tended to be a _Good Thing™_ in the past. Also `make check` everything may look like overkill while it could save you from looking way too surprised later on :)

Re: ./xbps-src bootstrap fixes

[xtraeme]

[-- Attachment #1: Type: text/plain, Size: 360 bytes --]

New comment by xtraeme on void-packages repository

https://github.com/void-linux/void-packages/pull/21226#issuecomment-617463645

Comment:
The `--enable-bootstrap` GCC configure option is all what we need. GCC build system is clever enough these days to perform a reliable build that does not depend on the host system once the 3 generated compiler is built.

Re: ./xbps-src bootstrap fixes

[xtraeme]

[-- Attachment #1: Type: text/plain, Size: 204 bytes --]

New comment by xtraeme on void-packages repository

https://github.com/void-linux/void-packages/pull/21226#issuecomment-617464546

Comment:
The issue with the `gcc-stage0` idea is that we need `gnat` :-)

Re: ./xbps-src bootstrap fixes

[pullmoll]

[-- Attachment #1: Type: text/plain, Size: 431 bytes --]

New comment by pullmoll on void-packages repository

https://github.com/void-linux/void-packages/pull/21226#issuecomment-617465446

Comment:
@xtraeme I think Ada can be built with `gcc` later on, using the binary from `adacore.com` for `x86_64` and making all cross compilers depend on `gcc`, not `gcc-stage0`.

Of course this does not allow for a `x86_64-musl` native build of `gcc` but requires a cross build from `x86_64` ;-P

Re: ./xbps-src bootstrap fixes

[pullmoll]

[-- Attachment #1: Type: text/plain, Size: 503 bytes --]

New comment by pullmoll on void-packages repository

https://github.com/void-linux/void-packages/pull/21226#issuecomment-617465446

Comment:
@xtraeme I think Ada can be built with `gcc` later on, using the binary from `adacore.com` for `x86_64` and making all cross compilers depend on `gcc`, not `gcc-stage0`.

Of course this does not allow for an (initial) `x86_64-musl` native build of `gcc` but requires a cross build from `x86_64` ;-P .. or a previously built `gcc-ada` and `libada` in the host.

Re: ./xbps-src bootstrap fixes

[pullmoll]

[-- Attachment #1: Type: text/plain, Size: 710 bytes --]

New comment by pullmoll on void-packages repository

https://github.com/void-linux/void-packages/pull/21226#issuecomment-617465446

Comment:
@xtraeme I think Ada can be built with `gcc` later on, using the binary from `adacore.com` for `x86_64` and making all cross compilers depend on `gcc`, not `gcc-stage0`.

Of course this does not allow for an (initial) `x86_64-musl` native build of `gcc` but requires a cross build from `x86_64` ;-P .. or a previously built `gcc-ada` and `libada` in the host.

Edit: I just thought about: didn't we have a way to pull in a minimal glibc for the `x86_64-musl` environment? Then perhaps the `adacore.com` binary `gnat` could also be run in `x86_64-musl` environment.

Re: ./xbps-src bootstrap fixes

[fosslinux]

[-- Attachment #1: Type: text/plain, Size: 199 bytes --]

New comment by fosslinux on void-packages repository

https://github.com/void-linux/void-packages/pull/21226#issuecomment-617480363

Comment:
@pullmoll @xtraeme Are you good for me to close this PR?

Re: ./xbps-src bootstrap fixes

[pullmoll]

[-- Attachment #1: Type: text/plain, Size: 218 bytes --]

New comment by pullmoll on void-packages repository

https://github.com/void-linux/void-packages/pull/21226#issuecomment-617481123

Comment:
`./xbps-src bootstrap` should succeed for glibc again, thus closing this PR.

Re: [PR PATCH] [Closed]: ./xbps-src bootstrap fixes

[pullmoll]

[-- Attachment #1: Type: text/plain, Size: 385 bytes --]

There's a closed pull request on the void-packages repository

./xbps-src bootstrap fixes
https://github.com/void-linux/void-packages/pull/21226

Description:
I found a few missing/broken things while testing `./xbps-src bootstrap`:

1. `bootstrap` requires texinfo on the host.
2. glibc doesn't contain usr/share/info.
3. m4 was broken.

Closes #21163 . Thanks to @rovenmaccorp