From: JamiKettunen <JamiKettunen@users.noreply.github.com>
To: ml@inbox.vuxu.org
Subject: [PR PATCH] xbps-src: check for existing packages before syncing chroot repodata
Date: Fri, 17 Jun 2022 12:23:45 +0200 [thread overview]
Message-ID: <gh-mailinglist-notifications-41a7ca26-5023-4802-975b-f1789d68868e-void-packages-37595@inbox.vuxu.org> (raw)
[-- Attachment #1: Type: text/plain, Size: 2731 bytes --]
There is a new pull request by JamiKettunen against master on the void-packages repository
https://github.com/JamiKettunen/void-packages faster-skip-existing-build
https://github.com/void-linux/void-packages/pull/37595
xbps-src: check for existing packages before syncing chroot repodata
#### Testing the changes
- I tested the changes in this PR: **briefly**
This significantly speeds up the package building process for build scripts etc. that run `./xbps-src pkg ...` on a long list of packages that may already have up-to-date local packages built when configured with `XBPS_PRESERVE_PKGS` set.
#### Example with a single package
Before this patch:
```
void-packages $ time ./xbps-src pkg helvum
=> xbps-src: updating repositories for host (x86_64)...
[*] Updating repository `https://repo-default.voidlinux.org/current/x86_64-repodata' ...
[*] Updating repository `https://repo-default.voidlinux.org/current/nonfree/x86_64-repodata' ...
[*] Updating repository `https://repo-default.voidlinux.org/current/debug/x86_64-repodata' ...
[*] Updating repository `https://repo-default.voidlinux.org/current/multilib/x86_64-repodata' ...
[*] Updating repository `https://repo-default.voidlinux.org/current/multilib/nonfree/x86_64-repodata' ...
=> xbps-src: updating software in / masterdir...
=> xbps-src: cleaning up / masterdir...
=> helvum-0.3.4_2: removing autodeps, please wait...
=> WARNING: helvum-0.3.4_2: skipping build due to existing /host/binpkgs//helvum-0.3.4_2.x86_64.xbps
real 0m1.980s
user 0m0.504s
sys 0m0.113s
```
After this patch:
```
void-packages $ time ./xbps-src pkg helvum
=> WARNING: helvum-0.3.4_2: skipping build due to existing /host/binpkgs//helvum-0.3.4_2.x86_64.xbps
real 0m0.062s
user 0m0.041s
sys 0m0.024s
```
Previously this would add up to about a minute of waiting for essentially no reason if 30 packages already have up-to-date local packages, while with this patch it would only take about 2 seconds to come to the same conclusion with the same package list.
I still have one question: what to do with the existing call to `check_existing_pkg` in [`common/xbps-src/libexec/build.sh`](https://github.com/void-linux/void-packages/blob/master/common/xbps-src/libexec/build.sh#L38)? The script appears to be executed from a [few](https://github.com/void-linux/void-packages/tree/master/common/xbps-src/shutils/cross.sh) [other](https://github.com/void-linux/void-packages/tree/master/common/xbps-src/shutils/build_dependencies.sh) [scripts](https://github.com/void-linux/void-packages/tree/master/common/xbps-src/shutils/bulk.sh) as well so I left it alone for now.
A patch file from https://github.com/void-linux/void-packages/pull/37595.patch is attached
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-faster-skip-existing-build-37595.patch --]
[-- Type: text/x-diff, Size: 1006 bytes --]
From f6117695ac78ad078a5c441e329947e1ddd17357 Mon Sep 17 00:00:00 2001
From: Jami Kettunen <jami.kettunen@protonmail.com>
Date: Fri, 17 Jun 2022 13:07:44 +0300
Subject: [PATCH] xbps-src: check for existing packages before syncing chroot
repodata
This significantly speeds up the package building process for build
scripts etc. that run './xbps-src pkg ...' on a long list of packages
that may already have up-to-date local packages built when configured
with XBPS_PRESERVE_PKGS set.
---
xbps-src | 1 +
1 file changed, 1 insertion(+)
diff --git a/xbps-src b/xbps-src
index dfa059927cc2..4e8349091708 100755
--- a/xbps-src
+++ b/xbps-src
@@ -806,6 +806,7 @@ case "$XBPS_TARGET" in
if [ -n "$CHROOT_READY" -a -z "$IN_CHROOT" ]; then
chroot_handler $XBPS_TARGET $XBPS_TARGET_PKG
else
+ check_existing_pkg
chroot_sync_repodata
# prevent update_base_chroot from removing the builddir/destdir
update_base_chroot keep-all-force
next reply other threads:[~2022-06-17 10:23 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-06-17 10:23 JamiKettunen [this message]
2022-06-17 16:53 ` Chocimier
2022-06-17 17:02 ` JamiKettunen
2022-06-17 23:56 ` [PR PATCH] [Merged]: " the-maldridge
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=gh-mailinglist-notifications-41a7ca26-5023-4802-975b-f1789d68868e-void-packages-37595@inbox.vuxu.org \
--to=jamikettunen@users.noreply.github.com \
--cc=ml@inbox.vuxu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).