From: classabbyamp <classabbyamp@users.noreply.github.com>
To: ml@inbox.vuxu.org
Subject: Re: [PR PATCH] [Updated] [RFC] xbps-src: add -A flag for host arch, create masterdir per host arch
Date: Wed, 17 Jan 2024 05:50:46 +0100 [thread overview]
Message-ID: <20240117045046.381632877F@inbox.vuxu.org> (raw)
In-Reply-To: <gh-mailinglist-notifications-41a7ca26-5023-4802-975b-f1789d68868e-void-packages-46263@inbox.vuxu.org>
[-- Attachment #1: Type: text/plain, Size: 2373 bytes --]
There is an updated pull request by classabbyamp against master on the void-packages repository
https://github.com/classabbyamp/void-packages host-arch
https://github.com/void-linux/void-packages/pull/46263
[RFC] xbps-src: add -A flag for host arch, create masterdir per host arch
#### Testing the changes
- I tested the changes in this PR: **YES**
### Overview
- default masterdir naming now includes the host `XBPS_ARCH`, e.g. `masterdir-x86_64`
- setting the host `XBPS_ARCH` can be done with `-A <arch>`
- This will create a new masterdir per host arch automatically
- using an arbitrary masterdir is still supported via `-m <masterdir>`, and overrides whatever `-A` would set
- the invocation of `[binary-]bootstrap` no longer (ab)uses `$XBPS_TARGET_PKG` for setting the masterdir's host architecture
#### Before
*default:*
```
$ ./xbps-src binary-bootstrap
$ ls
... masterdir ...
```
*nondefault masterdir:*
```
$ ./xbps-src -m masterdir-idk binary-bootstrap
$ ls
... masterdir-idk ...
```
*specific host arch:*
```
$ ./xbps-src -m masterdir-musl binary-bootstrap x86_64-musl
$ ls
... masterdir-musl ...
```
*cross with specific host arch:*
```
$ ./xbps-src -m masterdir-musl binary-bootstrap x86_64-musl
$ ./xbps-src -m masterdir-musl -a aarch64-musl pkg foo
```
#### After
*default:*
```
$ ./xbps-src binary-bootstrap
$ ls
... masterdir-x86_64 ...
```
*nondefault masterdir:*
```
$ ./xbps-src -m masterdir-idk binary-bootstrap
$ ls
... masterdir-idk ...
```
*specific host arch:*
```
$ ./xbps-src -A x86_64-musl binary-bootstrap
$ ls
... masterdir-x86_64-musl ...
```
*cross with specific host arch:*
```
$ ./xbps-src -A x86_64-musl binary-bootstrap
$ ./xbps-src -A x86_64-musl -a aarch64-musl pkg foo
```
### Thoughts
- By removing the `[binary-]bootstrap [arch]` invocation style, there will be changes needed to buildbot configuration. If left in, should `-A <arch>` take priority over `[arch]` if both are specified?
- This could be an opportunity to change the name of `masterdir` to something else, like `buildroot` (the containers CI uses are already called `void-buildroot-<libc>`)
### TODO
- [ ] various documentation/other xbps-src changes as needed
- [ ] buildbot changes if needed
A patch file from https://github.com/void-linux/void-packages/pull/46263.patch is attached
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-host-arch-46263.patch --]
[-- Type: text/x-diff, Size: 13930 bytes --]
From 18d620e69f34d07e903bedcdf1e7b404ebebc131 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 26 Sep 2023 03:16:39 -0400
Subject: [PATCH 1/5] xbps-src: add -A flag for host arch, create masterdir per
host arch
---
xbps-src | 57 ++++++++++++++++++++++++++++++++------------------------
1 file changed, 33 insertions(+), 24 deletions(-)
diff --git a/xbps-src b/xbps-src
index 712a578c6f91a..dc4635b74925c 100755
--- a/xbps-src
+++ b/xbps-src
@@ -14,14 +14,17 @@ $PROGNAME: [options] <target> [arguments]
Targets: (only one may be specified)
-binary-bootstrap [arch]
+binary-bootstrap
Install bootstrap packages from host repositories into <masterdir>.
- If the optional 'arch' argument is set, it will install bootstrap packages
- from this architecture, and its required xbps utilities. The <masterdir>
- will be initialized for chroot operations.
+ If the optional '-A <arch>' flag is set, it will install bootstrap packages
+ from this architecture, and its required xbps utilities. The <masterdir> will
+ be initialized for chroot operations.
bootstrap
Build and install from source the bootstrap packages into <masterdir>.
+ If the optional '-A <arch>' flag is set, it will build and install bootstrap
+ packages from this architecture, and its required xbps utilities. The <masterdir>
+ will be initialized for chroot operations.
bootstrap-update
Updates bootstrap packages with latest versions available from registered
@@ -151,6 +154,13 @@ Options:
-1 If dependencies of target package are missing, fail instead of building them.
+-A <host>
+ Use this host machine. Automatically creates masterdir-<host> if it doesn't
+ already exist. Some host machines may require qemu-user-static and
+ binfmt-support if not natively supported by the processor. Supported hosts:
+
+$(print_cross_targets)
+
-a <target>
Cross compile packages for this target machine. Supported targets:
@@ -290,19 +300,8 @@ check_native_arch() {
else
export XBPS_ARCH=$(xbps-uhelper arch)
fi
- elif [ "$XBPS_TARGET" = 'binary-bootstrap' ] && [ "$XBPS_TARGET_PKG" ]; then
- # ^ special case for binary-bootstrap for e.g:
- # x86_64 -> x86_64-musl
- # x86_64 -> i686
- export XBPS_ARCH=$XBPS_TARGET_PKG
else
- LDD=$(ldd --version 2>&1|head -1)
- if [[ $LDD == *musl* ]]; then
- export XBPS_ARCH=${XBPS_MACHINE%-musl}-musl
- else
- # XBPS_ARCH == $(uname -m)
- export XBPS_ARCH=$(uname -m)
- fi
+ export XBPS_ARCH="$XBPS_MACHINE"
fi
}
@@ -372,7 +371,7 @@ readonly XBPS_SRC_VERSION="113"
export XBPS_MACHINE=$(xbps-uhelper -C /dev/null arch)
XBPS_OPTIONS=
-XBPS_OPTSTRING="1a:bc:CEfgGhH:iIj:Lm:No:p:qsQKr:tV"
+XBPS_OPTSTRING="1A:a:bc:CEfgGhH:iIj:Lm:No:p:qsQKr:tV"
# Preprocess arguments in order to allow options before and after XBPS_TARGET.
eval set -- $(getopt "$XBPS_OPTSTRING" "$@");
@@ -383,6 +382,7 @@ eval set -- $(getopt "$XBPS_OPTSTRING" "$@");
while getopts "$XBPS_OPTSTRING" opt; do
case $opt in
1) XBPS_ARG_BUILD_ONLY_ONE_PKG=yes; XBPS_OPTIONS+=" -1";;
+ A) XBPS_ARG_MACHINE="$OPTARG"; XBPS_OPTIONS+=" -A $OPTARG";;
a) XBPS_ARG_CROSS_BUILD="$OPTARG"; XBPS_OPTIONS+=" -a $OPTARG";;
b) XBPS_ARG_IGNORE_BROKENNESS=yes; XBPS_OPTIONS+=" -b";;
c) XBPS_ARG_CONFIG="$OPTARG"; XBPS_OPTIONS+=" -c $OPTARG";;
@@ -493,6 +493,7 @@ fi
[ -n "$XBPS_ARG_PRINT_VARIABLES" ] && XBPS_PRINT_VARIABLES="$XBPS_ARG_PRINT_VARIABLES"
[ -n "$XBPS_ARG_ALT_REPOSITORY" ] && XBPS_ALT_REPOSITORY="$XBPS_ARG_ALT_REPOSITORY"
[ -n "$XBPS_ARG_STRICT" ] && XBPS_STRICT="$XBPS_ARG_STRICT"
+[ -n "$XBPS_ARG_MACHINE" ] && XBPS_MACHINE="$XBPS_ARG_MACHINE"
[ -n "$XBPS_ARG_CROSS_BUILD" ] && XBPS_CROSS_BUILD="$XBPS_ARG_CROSS_BUILD"
[ -n "$XBPS_ARG_CHECK_PKGS" ] && XBPS_CHECK_PKGS="$XBPS_ARG_CHECK_PKGS"
[ -n "$XBPS_ARG_MAKEJOBS" ] && XBPS_MAKEJOBS="$XBPS_ARG_MAKEJOBS"
@@ -527,9 +528,9 @@ if [ -z "$IN_CHROOT" -a "$UID" -eq 0 -a -z "$XBPS_ALLOW_CHROOT_BREAKOUT" ]; then
exit 1
fi
-# if XBPS_MASTERDIR unset, defaults to $XBPS_DISTDIR/masterdir.
-: ${XBPS_MASTERDIR:=$XBPS_DISTDIR/masterdir}
-[ ! -d $XBPS_MASTERDIR ] && mkdir -p $XBPS_MASTERDIR
+# if XBPS_MASTERDIR unset, defaults to $XBPS_DISTDIR/masterdir-<host machine>.
+: "${XBPS_MASTERDIR:=$XBPS_DISTDIR/masterdir-$XBPS_MACHINE}"
+[ ! -d "$XBPS_MASTERDIR" ] && mkdir -p "$XBPS_MASTERDIR"
# if XBPS_HOSTDIR unset, defaults to $XBPS_DISTDIR/hostdir.
: ${XBPS_HOSTDIR:=$XBPS_DISTDIR/hostdir}
@@ -617,10 +618,10 @@ if [ -z "$XBPS_REINIT" -a -s $XBPS_MASTERDIR/.xbps_chroot_init ]; then
linux32 xbps-reconfigure -r ${XBPS_MASTERDIR} -a &>/dev/null
# reexec itself via linux32
export XBPS_REINIT=1
- exec linux32 $0 ${XBPS_OPTIONS} $@
+ exec linux32 $0 ${XBPS_OPTIONS} "$@"
fi
fi
-if [ "$XBPS_ARCH" ]; then
+if [ -n "$XBPS_ARCH" ]; then
export XBPS_MACHINE=$XBPS_ARCH
fi
# At this point if XBPS_TARGET_MACHINE isn't defined we assume
@@ -743,13 +744,21 @@ reconfigure_base_chroot
#
case "$XBPS_TARGET" in
binary-bootstrap)
- install_base_chroot ${XBPS_TARGET_PKG:=$XBPS_MACHINE}
+ if [ -n "$XBPS_TARGET_PKG" ]; then
+ msg_error "xbps-src: binary-bootstrap <arch> is no longer supported, use '-A <arch>' instead\n"
+ exit 1
+ fi
+ install_base_chroot "$XBPS_MACHINE"
;;
bootstrap)
if [ -n "$XBPS_CHECK_PKGS" ]; then
msg_error "xbps-src: disable tests for bootstrap\n"
exit 1
fi
+ if [ -n "$XBPS_TARGET_PKG" ]; then
+ msg_error "xbps-src: bootstrap <arch> is no longer supported, use '-A <arch>' instead\n"
+ exit 1
+ fi
# base-chroot building on host
# check for required host utils
check_reqhost_utils bootstrap
@@ -761,7 +770,7 @@ case "$XBPS_TARGET" in
base-chroot base-chroot $XBPS_TARGET || exit 1
) || exit 1
[ -d $XBPS_MASTERDIR ] && rm -rf $XBPS_MASTERDIR
- install_base_chroot ${XBPS_TARGET_PKG:=$XBPS_MACHINE}
+ install_base_chroot "$XBPS_MACHINE"
;;
bootstrap-update)
if [ -n "$CHROOT_READY" -a -z "$IN_CHROOT" ]; then
From d4e81acfea1e0246431b272c5dd5e286cd2e0e2f Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 26 Sep 2023 03:18:01 -0400
Subject: [PATCH 2/5] xbps: patch xbps-src completions for -A
---
srcpkgs/xbps/patches/xbps-src-comp.patch | 21 +++++++++++++++++++++
srcpkgs/xbps/template | 2 +-
2 files changed, 22 insertions(+), 1 deletion(-)
create mode 100644 srcpkgs/xbps/patches/xbps-src-comp.patch
diff --git a/srcpkgs/xbps/patches/xbps-src-comp.patch b/srcpkgs/xbps/patches/xbps-src-comp.patch
new file mode 100644
index 0000000000000..bb0234f38463d
--- /dev/null
+++ b/srcpkgs/xbps/patches/xbps-src-comp.patch
@@ -0,0 +1,21 @@
+--- a/data/_xbps_src
++++ b/data/_xbps_src
+@@ -26,6 +26,7 @@
+ }
+
+ _arguments -s : \
++ '-A[Host architecture]:architecture:($archs)' \
+ '-a[Cross compile packages]:architecture:($archs)' \
+ '-C[Do not remove build directory/autodeps/destdir]' \
+ '-E[Exit immediately when binary package already exists]' \
+@@ -64,9 +65,7 @@
+ case $words[1] in
+ build|check|configure|extract|fetch|install|patch|pkg|show|show-avail|show-build-deps|show-hostmakedepends|show-makedepends|show-options|update-check)
+ _arguments ':package:_xbps_src_all_packages' && ret=0;;
+- binary-bootstrap)
+- _arguments '::architecture:($archs)' && ret=0;;
+- bootstrap|bootstrap-update|chroot|clean-repocache|consistency-check|list|purge-distfiles|remove-autodeps|show-repo-updates|show-sys-updates|update-bulk|update-sys|update-hash-cache)
++ binary-bootstrap|bootstrap|bootstrap-update|chroot|clean-repocache|consistency-check|list|purge-distfiles|remove-autodeps|show-repo-updates|show-sys-updates|update-bulk|update-sys|update-hash-cache)
+ # no further arguments
+ ret=0;;
+ clean)
diff --git a/srcpkgs/xbps/template b/srcpkgs/xbps/template
index 5b34b331d58da..060a70563aeb7 100644
--- a/srcpkgs/xbps/template
+++ b/srcpkgs/xbps/template
@@ -1,7 +1,7 @@
# Template file for 'xbps'
pkgname=xbps
version=0.59.2
-revision=1
+revision=2
bootstrap=yes
build_style=configure
short_desc="XBPS package system utilities"
From 970b66c7f3bf553770d7dcc0cad022e782ad9285 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 16 Jan 2024 23:33:30 -0500
Subject: [PATCH 3/5] README.md: update documentation for new masterdir
creation
---
README.md | 17 +++++++++--------
1 file changed, 9 insertions(+), 8 deletions(-)
diff --git a/README.md b/README.md
index 8710f20ae1916..bd7fed879019f 100644
--- a/README.md
+++ b/README.md
@@ -217,7 +217,7 @@ The following directory hierarchy is used with a default configuration file:
| |- repocache ...
| |- sources ...
|
- |- masterdir
+ |- masterdir-<arch>
| |- builddir -> ...
| |- destdir -> ...
| |- host -> bind mounted from <hostdir>
@@ -226,7 +226,7 @@ The following directory hierarchy is used with a default configuration file:
The description of these directories is as follows:
- - `masterdir`: master directory to be used as rootfs to build/install packages.
+ - `masterdir-<arch>`: master directory to be used as rootfs to build/install packages.
- `builddir`: to unpack package source tarballs and where packages are built.
- `destdir`: to install packages, aka **fake destdir**.
- `hostdir/ccache`: to store ccache data if the `XBPS_CCACHE` option is enabled.
@@ -450,7 +450,7 @@ and `xbps-src` should be fully functional; just start the `bootstrap` process, i
$ ./xbps-src binary-bootstrap
-The default masterdir is created in the current working directory, i.e `void-packages/masterdir`.
+The default masterdir is created in the current working directory, i.e. `void-packages/masterdir-<arch>`, where `<arch>` for the default masterdir is is the native xbps architecture.
<a name="remaking-masterdir"></a>
### Remaking the masterdir
@@ -477,20 +477,21 @@ Two ways are available to build 32bit packages on x86\_64:
The canonical mode (native) needs a new x86 `masterdir`:
- $ ./xbps-src -m masterdir-x86 binary-bootstrap i686
- $ ./xbps-src -m masterdir-x86 ...
+ $ ./xbps-src -A i686 binary-bootstrap
+ $ ./xbps-src -A i686 ...
<a name="building-for-musl"></a>
### Building packages natively for the musl C library
-Canonical way of building packages for same architecture but different C library is through dedicated masterdir.
+The canonical way of building packages for same architecture but different C library is through a dedicated masterdir by using the host architecture flag `-A`.
To build for x86_64-musl on glibc x86_64 system, prepare a new masterdir with the musl packages:
- $ ./xbps-src -m masterdir-x86_64-musl binary-bootstrap x86_64-musl
+ $ ./xbps-src -A x86_64-musl binary-bootstrap
+This will create and bootstrap a new masterdir called `masterdir-x86_64-musl` that will be used when `-A x86_64-musl` is specified.
Your new masterdir is now ready to build packages natively for the musl C library:
- $ ./xbps-src -m masterdir-x86_64-musl pkg ...
+ $ ./xbps-src -A x86_64-musl pkg ...
<a name="building-base-system"></a>
### Building void base-system from scratch
From e5099ae76ceb6a134a3cfa232e336d317b411f1d Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 16 Jan 2024 23:45:24 -0500
Subject: [PATCH 4/5] fixup! xbps-src: add -A flag for host arch, create
masterdir per host arch
---
xbps-src | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/xbps-src b/xbps-src
index dc4635b74925c..024da90ba99a4 100755
--- a/xbps-src
+++ b/xbps-src
@@ -204,7 +204,7 @@ $(print_cross_targets)
-m <masterdir>
Absolute path to a directory to be used as masterdir.
The masterdir is the main directory to build/store/compile packages.
- If unset defaults to void-packages/masterdir.
+ If unset defaults to void-packages/masterdir-<host>.
-N Disable use of remote repositories to resolve dependencies.
@@ -529,6 +529,7 @@ if [ -z "$IN_CHROOT" -a "$UID" -eq 0 -a -z "$XBPS_ALLOW_CHROOT_BREAKOUT" ]; then
fi
# if XBPS_MASTERDIR unset, defaults to $XBPS_DISTDIR/masterdir-<host machine>.
+echo "===============> $XBPS_CHROOT_CMD"
: "${XBPS_MASTERDIR:=$XBPS_DISTDIR/masterdir-$XBPS_MACHINE}"
[ ! -d "$XBPS_MASTERDIR" ] && mkdir -p "$XBPS_MASTERDIR"
From c0d16e22ab57c3d0f361f93b7f720e12cbdb59f4 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Tue, 16 Jan 2024 23:49:33 -0500
Subject: [PATCH 5/5] fixup! xbps-src: add -A flag for host arch, create
masterdir per host arch
---
xbps-src | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/xbps-src b/xbps-src
index 024da90ba99a4..aac7452615596 100755
--- a/xbps-src
+++ b/xbps-src
@@ -528,9 +528,13 @@ if [ -z "$IN_CHROOT" -a "$UID" -eq 0 -a -z "$XBPS_ALLOW_CHROOT_BREAKOUT" ]; then
exit 1
fi
-# if XBPS_MASTERDIR unset, defaults to $XBPS_DISTDIR/masterdir-<host machine>.
-echo "===============> $XBPS_CHROOT_CMD"
-: "${XBPS_MASTERDIR:=$XBPS_DISTDIR/masterdir-$XBPS_MACHINE}"
+# if XBPS_MASTERDIR unset, defaults to $XBPS_DISTDIR/masterdir-<host machine>
+# unless in a container for simplicity of container/CI usage
+if [ "$XBPS_CHROOT_CMD" = ethereal ]; then
+ : "${XBPS_MASTERDIR:=$XBPS_DISTDIR/masterdir}"
+else
+ : "${XBPS_MASTERDIR:=$XBPS_DISTDIR/masterdir-$XBPS_MACHINE}"
+fi
[ ! -d "$XBPS_MASTERDIR" ] && mkdir -p "$XBPS_MASTERDIR"
# if XBPS_HOSTDIR unset, defaults to $XBPS_DISTDIR/hostdir.
next prev parent reply other threads:[~2024-01-17 4:50 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-09-26 7:52 [PR PATCH] " classabbyamp
2023-09-26 8:00 ` [PR PATCH] [Updated] " classabbyamp
2023-09-26 8:10 ` classabbyamp
2023-09-26 8:11 ` classabbyamp
2023-09-26 8:13 ` classabbyamp
2023-09-26 8:17 ` classabbyamp
2023-09-26 11:43 ` lemmi
2023-09-26 18:51 ` SpidFightFR
2023-09-26 18:53 ` [PR REVIEW] " classabbyamp
2023-12-26 1:45 ` github-actions
2023-12-26 17:54 ` [PR PATCH] [Updated] " classabbyamp
2024-01-17 4:34 ` classabbyamp
2024-01-17 4:46 ` classabbyamp
2024-01-17 4:50 ` classabbyamp [this message]
2024-01-17 4:51 ` classabbyamp
2024-01-17 5:02 ` classabbyamp
2024-01-17 5:44 ` classabbyamp
2024-01-17 6:50 ` [PR PATCH] [Updated] [RFC] xbps-src: add -A flag for host arch, create buildroot per host arch, rename masterdir to buildroot classabbyamp
2024-01-22 18:00 ` classabbyamp
2024-01-22 19:58 ` [PR REVIEW] [RFC] xbps-src: add -A flag for host arch, create masterdir per host arch ahesford
2024-01-22 20:15 ` [PR PATCH] [Updated] " classabbyamp
2024-01-23 3:57 ` [PR PATCH] [Merged]: " the-maldridge
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240117045046.381632877F@inbox.vuxu.org \
--to=classabbyamp@users.noreply.github.com \
--cc=ml@inbox.vuxu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).