From: classabbyamp <classabbyamp@users.noreply.github.com>
To: ml@inbox.vuxu.org
Subject: Re: [PR PATCH] [Updated] enable running tests in parallel
Date: Tue, 14 Jun 2022 03:00:10 +0200 [thread overview]
Message-ID: <20220614010010.TTfPVZnfUrIzpJPiI-rerS501ufy8BMITOKDQSbcs_Y@z> (raw)
In-Reply-To: <gh-mailinglist-notifications-41a7ca26-5023-4802-975b-f1789d68868e-void-packages-37496@inbox.vuxu.org>
[-- Attachment #1: Type: text/plain, Size: 1281 bytes --]
There is an updated pull request by classabbyamp against master on the void-packages repository
https://github.com/classabbyamp/void-packages common/parallel-check
https://github.com/void-linux/void-packages/pull/37496
enable running tests in parallel
- common: add disable_parallel_check
- common/build-style: make do_check run in parallel unless disabled
- Manual.md: document disable_parallel_check
Redux of #31811, incorporating some of the outstanding feedback when it was stale-closed.
Adds `-j$XBPS_MAKEJOBS` to do_check in several build-styles, with the option to disable it if it causes issues.
I did *not* change the `python3-{modules,pep517}` build-styles to opt-out of parallel checks, because they require a 3rd-party library to do it (and it only works with pytest tests). If a python3 package has tests that require `python3-pytest-xdist` to run, but have issues with parallel tests, parallel behaviour can still be disabled by `disable_parallel_check` (which would set the xdist equivalent of `-j1`).
<!-- Uncomment relevant sections and delete options which are not applicable -->
#### Testing the changes
- I tested the changes in this PR: **YES**
A patch file from https://github.com/void-linux/void-packages/pull/37496.patch is attached
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-common/parallel-check-37496.patch --]
[-- Type: text/x-diff, Size: 5755 bytes --]
From 4b94cdad8395f95bab33627676bf1be41b40557b Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Fri, 10 Jun 2022 14:16:48 -0400
Subject: [PATCH 1/3] common: add disable_parallel_check
---
common/environment/setup/sourcepkg.sh | 3 ++-
common/xbps-src/libexec/xbps-src-docheck.sh | 7 +++++++
2 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/common/environment/setup/sourcepkg.sh b/common/environment/setup/sourcepkg.sh
index 3a9bcdacb872..1191993c9abc 100644
--- a/common/environment/setup/sourcepkg.sh
+++ b/common/environment/setup/sourcepkg.sh
@@ -13,7 +13,8 @@ unset -v cmake_builddir meson_builddir
unset -v meson_crossfile
unset -v gemspec
unset -v go_import_path go_package go_mod_mode
-unset -v patch_args disable_parallel_build keep_libtool_archives make_use_env
+unset -v patch_args disable_parallel_build disable_parallel_check
+unset -v keep_libtool_archives make_use_env
unset -v reverts subpackages makedepends hostmakedepends checkdepends depends restricted
unset -v nopie build_options build_options_default bootstrap repository reverts
unset -v CFLAGS CXXFLAGS FFLAGS CPPFLAGS LDFLAGS LD_LIBRARY_PATH
diff --git a/common/xbps-src/libexec/xbps-src-docheck.sh b/common/xbps-src/libexec/xbps-src-docheck.sh
index 1e82c1a3739f..cac0d48f774e 100755
--- a/common/xbps-src/libexec/xbps-src-docheck.sh
+++ b/common/xbps-src/libexec/xbps-src-docheck.sh
@@ -20,6 +20,13 @@ done
setup_pkg "$PKGNAME" $XBPS_CROSS_BUILD
+if [ -n "$disable_parallel_check" ]; then
+ XBPS_MAKEJOBS=1
+else
+ XBPS_MAKEJOBS="$XBPS_ORIG_MAKEJOBS"
+fi
+makejobs="-j$XBPS_MAKEJOBS"
+
XBPS_CHECK_DONE="${XBPS_STATEDIR}/${sourcepkg}_${XBPS_CROSS_BUILD}_check_done"
if [ -n "$XBPS_CROSS_BUILD" ]; then
From 42fd3e1bb076b6a8158ecad2d169461516250aa4 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Fri, 10 Jun 2022 15:04:57 -0400
Subject: [PATCH 2/3] common/build-style: make do_check run in parallel unless
disabled
---
common/build-style/cmake.sh | 2 +-
common/build-style/configure.sh | 2 +-
common/build-style/gnu-configure.sh | 2 +-
common/build-style/gnu-makefile.sh | 2 +-
common/build-style/perl-module.sh | 2 +-
5 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/common/build-style/cmake.sh b/common/build-style/cmake.sh
index 7d1d675c3aba..711fcea6273d 100644
--- a/common/build-style/cmake.sh
+++ b/common/build-style/cmake.sh
@@ -116,7 +116,7 @@ do_check() {
: ${make_check_target:=test}
- ${make_check_pre} ${make_cmd} ${make_check_args} ${make_check_target}
+ ${make_check_pre} ${make_cmd} ${makejobs} ${make_check_args} ${make_check_target}
}
do_install() {
diff --git a/common/build-style/configure.sh b/common/build-style/configure.sh
index 6f2c94ace694..7a6d136f4cc6 100644
--- a/common/build-style/configure.sh
+++ b/common/build-style/configure.sh
@@ -29,7 +29,7 @@ do_check() {
: ${make_cmd:=make}
: ${make_check_target:=check}
- ${make_check_pre} ${make_cmd} ${make_check_args} ${make_check_target}
+ ${make_check_pre} ${make_cmd} ${makejobs} ${make_check_args} ${make_check_target}
}
do_install() {
diff --git a/common/build-style/gnu-configure.sh b/common/build-style/gnu-configure.sh
index f66b081955e2..4326d07d75cc 100644
--- a/common/build-style/gnu-configure.sh
+++ b/common/build-style/gnu-configure.sh
@@ -30,7 +30,7 @@ do_check() {
: ${make_cmd:=make}
: ${make_check_target:=check}
- ${make_check_pre} ${make_cmd} ${make_check_args} ${make_check_target}
+ ${make_check_pre} ${make_cmd} ${makejobs} ${make_check_args} ${make_check_target}
}
do_install() {
diff --git a/common/build-style/gnu-makefile.sh b/common/build-style/gnu-makefile.sh
index 2492749bd84a..04e407af255c 100644
--- a/common/build-style/gnu-makefile.sh
+++ b/common/build-style/gnu-makefile.sh
@@ -30,7 +30,7 @@ do_check() {
: ${make_cmd:=make}
: ${make_check_target:=check}
- ${make_check_pre} ${make_cmd} ${make_check_args} ${make_check_target}
+ ${make_check_pre} ${make_cmd} ${makejobs} ${make_check_args} ${make_check_target}
}
do_install() {
diff --git a/common/build-style/perl-module.sh b/common/build-style/perl-module.sh
index 9126091cb2cf..2945787ffbe1 100644
--- a/common/build-style/perl-module.sh
+++ b/common/build-style/perl-module.sh
@@ -79,7 +79,7 @@ do_check() {
: ${make_cmd:=make}
: ${make_check_target:=test}
- ${make_check_pre} ${make_cmd} ${make_check_args} ${make_check_target}
+ ${make_check_pre} ${make_cmd} ${makejobs} ${make_check_args} ${make_check_target}
}
do_install() {
From 132a9387b06a8d296f9572d82dbb27cd2c917493 Mon Sep 17 00:00:00 2001
From: classabbyamp <void@placeviolette.net>
Date: Fri, 10 Jun 2022 15:27:34 -0400
Subject: [PATCH 3/3] Manual.md: document disable_parallel_check
---
Manual.md | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/Manual.md b/Manual.md
index 8afe781f0ec1..26e3b9f43413 100644
--- a/Manual.md
+++ b/Manual.md
@@ -613,6 +613,11 @@ and `XBPS_MAKEJOBS` will be set to 1. If a package does not work well with `XBPS
but still has a mechanism to build in parallel, set `disable_parallel_build` and
use `XBPS_ORIG_MAKEJOBS` (which holds the original value of `XBPS_MAKEJOBS`) in the template.
+- `disable_parallel_check` If set tests for the package won't be built and run in parallel
+and `XBPS_MAKEJOBS` will be set to 1. If a package does not work well with `XBPS_MAKEJOBS`
+but still has a mechanism to run checks in parallel, set `disable_parallel_check` and
+use `XBPS_ORIG_MAKEJOBS` (which holds the original value of `XBPS_MAKEJOBS`) in the template.
+
- `make_check` Sets the cases in which the `check` phase is run.
This option has to be accompanied by a comment explaining why the tests fail.
Allowed values:
next prev parent reply other threads:[~2022-06-14 1:00 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-06-10 21:15 [PR PATCH] " classabbyamp
2022-06-14 0:49 ` [PR PATCH] [Updated] " classabbyamp
2022-06-14 1:00 ` classabbyamp [this message]
2022-06-24 4:08 ` [PR PATCH] [Merged]: " the-maldridge
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220614010010.TTfPVZnfUrIzpJPiI-rerS501ufy8BMITOKDQSbcs_Y@z \
--to=classabbyamp@users.noreply.github.com \
--cc=ml@inbox.vuxu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).