From: Noah-Huppert <Noah-Huppert@users.noreply.github.com>
To: ml@inbox.vuxu.org
Subject: Re: [PR PATCH] [Updated] xbps-src: fixed dangling binpkg lock on error
Date: Mon, 10 Aug 2020 18:47:10 +0200 [thread overview]
Message-ID: <20200810164710.xX6cfYGlNMlcxT36uy92EgWH6ahAeSNAnVPfHnz4kXA@z> (raw)
In-Reply-To: <gh-mailinglist-notifications-41a7ca26-5023-4802-975b-f1789d68868e-void-packages-24191@inbox.vuxu.org>
[-- Attachment #1: Type: text/plain, Size: 4601 bytes --]
There is an updated pull request by Noah-Huppert against master on the void-packages repository
https://github.com/Noah-Huppert/void-packages binpkgs-lock-fix
https://github.com/void-linux/void-packages/pull/24191
xbps-src: fixed dangling binpkg lock on error
Fixes #21020.
Previously if a binpkg failed to be created due to [`xbps-create` falling over in `common/hooks/do-pkg/00-gen-pkg.sh`](https://github.com/void-linux/void-packages/blob/master/common/hooks/do-pkg/00-gen-pkg.sh#L63) it would leave a `hostdir/binpkgs/*.lock` file around.
This fix traps `ERR` and `EXIT` right after the lock file is created. If an error occurs it deletes the lock file. Right before the lock file is deleted it untraps `ERR` and `EXIT`.
There was an [`xbps-create` return value check in the source](https://github.com/void-linux/void-packages/blob/master/common/hooks/do-pkg/00-gen-pkg.sh#L88-L95). However I think the script is running with `set -e` which results in the failure of `xbps-create` exiting the process. Trapping `ERR` prevents the process from exiting.Trapping `EXIT` prevents the process from exiting if the user hits C-c during the `xbps-create` process.
**Before**
Before these changes when a binpkg build failed due to `xbps-create` falling over it would look like so:
```
=> kernel-libc-headers-4.19.0_2: running do-pkg hook: 00-gen-pkg ...
=> Creating kernel-libc-headers-4.19.0_2.x86_64.xbps for repository /home/noah/documents/void-packages/hostdir/binpkgs ...
xbps-create: ERROR: Failed to open .xbps-pkg-XXX7c01am fd for writing: No such file or directory
[noah@ip-172-31-47-23 void-packages]$ => ERROR: kernel-libc-headers-4.19.0_2: do-pkg_00-gen-pkg: 'xbps-create ${_provides:+--provides "${_provides}"} ${_conflicts:+--conflicts "${_conflicts}"} ${_replaces:+--replaces "${_replaces}"} ${_reverts:+--reverts "${_reverts}"} ${_mutable_files:+--mutable-files "${_mutable_files}"} ${_deps:+--dependencies "${_deps}"} ${_conf_files:+--config-files "${_conf_files}"} ${PKG_BUILD_OPTIONS:+--build-options "${PKG_BUILD_OPTIONS}"} ${_gitrevs:+--source-revisions "${_gitrevs}"} ${_shprovides:+--shlib-provides "${_shprovides}"} ${_shrequires:+--shlib-requires "${_shrequires}"} ${_alternatives:+--alternatives "${_alternatives}"} ${_preserve:+--preserve} ${tags:+--tags "${tags}"} ${_changelog:+--changelog "${_changelog}"} ${XBPS_PKG_COMPTYPE:+--compression $XBPS_PKG_COMPTYPE} --architecture ${arch} --homepage "${homepage}" --license "${license}" --maintainer "${maintainer}" --desc "${desc}" --pkgver "${pkgver}" --quiet ${PKGDESTDIR}' exited with 1
=> ERROR: in genpkg() at common/hooks/do-pkg/00-gen-pkg.sh:64
=> ERROR: in hook() at common/hooks/do-pkg/00-gen-pkg.sh:121
=> ERROR: in run_func() at common/xbps-src/shutils/common.sh:21
=> ERROR: in run_pkg_hooks() at common/xbps-src/shutils/common.sh:232
=> ERROR: in main() at common/xbps-src/libexec/xbps-src-dopkg.sh:44
```
(Note: The `xbps-create: ERROR: Failed to open .xbps-pkg-XXX...` error is not relevant, but was produced by `xbps-create` failing, how `xbps-src` handles this failure is the focus of the PR).
When you tried to re-build any package which previously had a failed binpkg it would [spin forever trying to acquire the lock](https://github.com/void-linux/void-packages/blob/master/common/hooks/do-pkg/00-gen-pkg.sh#L15-L18):
```
./xbps-src pkg kernel-libc-headers
=> kernel-libc-headers-4.19.0_2: running do-pkg hook: 00-gen-pkg ...
=> WARNING: kernel-libc-headers-4.19.0_2: binpkg is being created, waiting for 1s...
=> WARNING: kernel-libc-headers-4.19.0_2: binpkg is being created, waiting for 1s...
=> WARNING: kernel-libc-headers-4.19.0_2: binpkg is being created, waiting for 1s...
=> WARNING: kernel-libc-headers-4.19.0_2: binpkg is being created, waiting for 1s...
...
```
**After**
With the changes a binpkg build failing due to xbps-create appears as:
```
=> Creating kernel-libc-headers-4.19.0_2.x86_64.xbps for repository /home/noah/documents/void-packages/hostdir/binpkgs ...
xbps-create: ERROR: Failed to open .xbps-pkg-XXX316nSv fd for writing: No such file or directory
=> Unlocking kernel-libc-headers-4.19.0_2.x86_64.xbps.lock because an error occurred during kernel-libc-headers-4.19.0_2.x86_64.xbps creation
=> ERROR: Failed to created binary package: kernel-libc-headers-4.19.0_2.x86_64.xbps!
```
When you try to re-build any package with a previously failed binpkg the lock file does not exist. So the build will run normally.
A patch file from https://github.com/void-linux/void-packages/pull/24191.patch is attached
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-binpkgs-lock-fix-24191.patch --]
[-- Type: text/x-diff, Size: 907 bytes --]
From a13d7e64428385e87cd6ed9044a97b326f422d8d Mon Sep 17 00:00:00 2001
From: Noah Huppert <contact@noahh.io>
Date: Mon, 10 Aug 2020 12:46:56 -0400
Subject: [PATCH] [ci skip] xbps-src: fixed dangling binpkg lock on error
---
common/hooks/do-pkg/00-gen-pkg.sh | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/common/hooks/do-pkg/00-gen-pkg.sh b/common/hooks/do-pkg/00-gen-pkg.sh
index e6285b6e4ff..217b913bb89 100644
--- a/common/hooks/do-pkg/00-gen-pkg.sh
+++ b/common/hooks/do-pkg/00-gen-pkg.sh
@@ -23,6 +23,12 @@ genpkg() {
return 0
fi
+ # Lock binpkg
+ unlock_binpkg() {
+ rm -f $pkgdir/${binpkg}.lock
+ }
+
+ trap unlock_binpkg ERR EXIT
touch -f $pkgdir/${binpkg}.lock
if [ ! -d $pkgdir ]; then
@@ -87,7 +93,9 @@ genpkg() {
${PKGDESTDIR}
rval=$?
+ # Unlock binpkg
rm -f $pkgdir/${binpkg}.lock
+ trap - ERR EXIT
if [ $rval -ne 0 ]; then
rm -f $pkgdir/$binpkg
next prev parent reply other threads:[~2020-08-10 16:47 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-08-10 6:01 [PR PATCH] " Noah-Huppert
2020-08-10 6:03 ` [PR PATCH] [Updated] " Noah-Huppert
2020-08-10 6:04 ` Noah-Huppert
2020-08-10 6:20 ` [PR REVIEW] " Noah-Huppert
2020-08-10 6:29 ` [PR PATCH] [Updated] " Noah-Huppert
2020-08-10 6:30 ` Noah-Huppert
2020-08-10 6:33 ` Noah-Huppert
2020-08-10 6:34 ` [PR REVIEW] " Noah-Huppert
2020-08-10 15:28 ` Noah-Huppert
2020-08-10 15:33 ` ahesford
2020-08-10 15:35 ` [PR PATCH] [Updated] " Noah-Huppert
2020-08-10 15:37 ` [PR REVIEW] " Noah-Huppert
2020-08-10 15:51 ` ahesford
2020-08-10 16:47 ` Noah-Huppert [this message]
2020-08-10 16:47 ` Noah-Huppert
2020-08-17 13:42 ` [PR PATCH] [Closed]: " ahesford
2020-08-17 13:45 ` ahesford
2020-08-19 20:33 ` Noah-Huppert
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200810164710.xX6cfYGlNMlcxT36uy92EgWH6ahAeSNAnVPfHnz4kXA@z \
--to=noah-huppert@users.noreply.github.com \
--cc=ml@inbox.vuxu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).