* Re: [RFC] common/build-style/: enable running tests in parallel
2021-07-05 1:29 [PR PATCH] common/build-style/: enable running tests in parallel Logarithmus
@ 2021-07-05 23:37 ` Logarithmus
2021-07-05 23:46 ` ericonr
` (20 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Logarithmus @ 2021-07-05 23:37 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 242 bytes --]
New comment by Logarithmus on void-packages repository
https://github.com/void-linux/void-packages/pull/31811#issuecomment-874369466
Comment:
@q66 @Piraty if you approve, then why no merging? I guess you have write permission for the repo.
^ permalink raw reply [flat|nested] 23+ messages in thread* Re: [RFC] common/build-style/: enable running tests in parallel
2021-07-05 1:29 [PR PATCH] common/build-style/: enable running tests in parallel Logarithmus
2021-07-05 23:37 ` [RFC] " Logarithmus
@ 2021-07-05 23:46 ` ericonr
2021-07-06 1:51 ` q66
` (19 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: ericonr @ 2021-07-05 23:46 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 344 bytes --]
New comment by ericonr on void-packages repository
https://github.com/void-linux/void-packages/pull/31811#issuecomment-874371144
Comment:
Should we add `disable_parallel_tests` to override this? Some test suites are explicitly broken when run in parallel (iirc cups, for example).
I assume they left their approval so others could review.
^ permalink raw reply [flat|nested] 23+ messages in thread* Re: [RFC] common/build-style/: enable running tests in parallel
2021-07-05 1:29 [PR PATCH] common/build-style/: enable running tests in parallel Logarithmus
2021-07-05 23:37 ` [RFC] " Logarithmus
2021-07-05 23:46 ` ericonr
@ 2021-07-06 1:51 ` q66
2021-07-14 14:51 ` [PR PATCH] [Updated] " Logarithmus
` (18 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: q66 @ 2021-07-06 1:51 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 360 bytes --]
New comment by q66 on void-packages repository
https://github.com/void-linux/void-packages/pull/31811#issuecomment-874403050
Comment:
yeah i guess we should add that
most suites go through fine with parallel builds (and often it's desirable because a lot of test suites compile things and running it singlethreaded with a large suite can get really slow)
^ permalink raw reply [flat|nested] 23+ messages in thread* Re: [PR PATCH] [Updated] [RFC] common/build-style/: enable running tests in parallel
2021-07-05 1:29 [PR PATCH] common/build-style/: enable running tests in parallel Logarithmus
` (2 preceding siblings ...)
2021-07-06 1:51 ` q66
@ 2021-07-14 14:51 ` Logarithmus
2021-07-14 15:09 ` Logarithmus
` (17 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Logarithmus @ 2021-07-14 14:51 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 413 bytes --]
There is an updated pull request by Logarithmus against master on the void-packages repository
https://github.com/Logarithmus/void-packages common-parallel-tests
https://github.com/void-linux/void-packages/pull/31811
[RFC] common/build-style/: enable running tests in parallel
What do you think? Are there any caveats?
A patch file from https://github.com/void-linux/void-packages/pull/31811.patch is attached
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-common-parallel-tests-31811.patch --]
[-- Type: text/x-diff, Size: 3962 bytes --]
From b2129095645f408a78e159155b51e382846e8366 Mon Sep 17 00:00:00 2001
From: Artur Sinila <freesoftware@logarithmus.dev>
Date: Mon, 5 Jul 2021 03:44:08 +0300
Subject: [PATCH] common/build-style/: enable running tests in parallel
---
common/build-style/cmake.sh | 6 +++++-
common/build-style/configure.sh | 6 +++++-
common/build-style/gnu-configure.sh | 6 +++++-
common/build-style/gnu-makefile.sh | 6 +++++-
common/build-style/perl-module.sh | 6 +++++-
common/environment/setup/sourcepkg.sh | 2 +-
6 files changed, 26 insertions(+), 6 deletions(-)
diff --git a/common/build-style/cmake.sh b/common/build-style/cmake.sh
index a7627e7dcf21..c3f6143d928a 100644
--- a/common/build-style/cmake.sh
+++ b/common/build-style/cmake.sh
@@ -113,7 +113,11 @@ do_check() {
: ${make_check_target:=test}
- ${make_cmd} ${make_check_args} ${make_check_target}
+ if [ -z "$disable_parallel_check" ]; then
+ ${make_cmd} ${makejobs} ${make_check_args} ${make_check_target}
+ else
+ ${make_cmd} ${make_check_args} ${make_check_target}
+ fi
}
do_install() {
diff --git a/common/build-style/configure.sh b/common/build-style/configure.sh
index 0963fc347934..b846a1a77572 100644
--- a/common/build-style/configure.sh
+++ b/common/build-style/configure.sh
@@ -29,7 +29,11 @@ do_check() {
: ${make_cmd:=make}
: ${make_check_target:=check}
- ${make_cmd} ${make_check_args} ${make_check_target}
+ if [ -z "$disable_parallel_check" ]; then
+ ${make_cmd} ${makejobs} ${make_check_args} ${make_check_target}
+ else
+ ${make_cmd} ${make_check_args} ${make_check_target}
+ fi
}
do_install() {
diff --git a/common/build-style/gnu-configure.sh b/common/build-style/gnu-configure.sh
index 8121136cd121..63aee5aebe26 100644
--- a/common/build-style/gnu-configure.sh
+++ b/common/build-style/gnu-configure.sh
@@ -30,7 +30,11 @@ do_check() {
: ${make_cmd:=make}
: ${make_check_target:=check}
- ${make_cmd} ${make_check_args} ${make_check_target}
+ if [ -n "$disable_parallel_check" ]; then
+ ${make_cmd} ${makejobs} ${make_check_args} ${make_check_target}
+ else
+ ${make_cmd} ${make_check_args} ${make_check_target}
+ fi
}
do_install() {
diff --git a/common/build-style/gnu-makefile.sh b/common/build-style/gnu-makefile.sh
index d7ea148fc483..0f7afa7d3695 100644
--- a/common/build-style/gnu-makefile.sh
+++ b/common/build-style/gnu-makefile.sh
@@ -30,7 +30,11 @@ do_check() {
: ${make_cmd:=make}
: ${make_check_target:=check}
- ${make_cmd} ${make_check_args} ${make_check_target}
+ if [ -z "$disable_parallel_check" ]; then
+ ${make_cmd} ${makejobs} ${make_check_args} ${make_check_target}
+ else
+ ${make_cmd} ${make_check_args} ${make_check_target}
+ fi
}
do_install() {
diff --git a/common/build-style/perl-module.sh b/common/build-style/perl-module.sh
index b9a01b13cfa9..d47945b33d36 100644
--- a/common/build-style/perl-module.sh
+++ b/common/build-style/perl-module.sh
@@ -79,7 +79,11 @@ do_check() {
: ${make_cmd:=make}
: ${make_check_target:=test}
- ${make_cmd} ${make_check_args} ${make_check_target}
+ if [ -z "$disable_parallel_check" ]; then
+ ${make_cmd} ${makejobs} ${make_check_args} ${make_check_target}
+ else
+ ${make_cmd} ${make_check_args} ${make_check_target}
+ fi
}
do_install() {
diff --git a/common/environment/setup/sourcepkg.sh b/common/environment/setup/sourcepkg.sh
index bc06f745b78a..b6caa797138f 100644
--- a/common/environment/setup/sourcepkg.sh
+++ b/common/environment/setup/sourcepkg.sh
@@ -3,7 +3,7 @@
## VARIABLES
unset -v pkgname version revision short_desc homepage license maintainer
-unset -v archs distfiles checksum build_style build_helper nocross broken
+unset -v archs distfiles checksum build_style build_helper nocross disable_parallel_check broken
unset -v configure_script configure_args wrksrc build_wrksrc create_wrksrc
unset -v make_build_args make_check_args make_install_args
unset -v make_build_target make_check_target make_install_target
^ permalink raw reply [flat|nested] 23+ messages in thread* Re: [RFC] common/build-style/: enable running tests in parallel
2021-07-05 1:29 [PR PATCH] common/build-style/: enable running tests in parallel Logarithmus
` (3 preceding siblings ...)
2021-07-14 14:51 ` [PR PATCH] [Updated] " Logarithmus
@ 2021-07-14 15:09 ` Logarithmus
2021-07-14 15:09 ` Chocimier
` (16 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Logarithmus @ 2021-07-14 15:09 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 603 bytes --]
New comment by Logarithmus on void-packages repository
https://github.com/void-linux/void-packages/pull/31811#issuecomment-879974474
Comment:
> Should we add `disable_parallel_tests` to override this? Some test suites are explicitly broken when run in parallel (iirc cups, for example).
>
> I assume they left their approval so others could review.
I've added `disable_parallel_check` option and tested it for `mold`, it works as intended.
A little TODO:
- document the new option in `Manual.md`
- try running tests for `cups*` packages and see if it fail without `disable_parallel_check`.
^ permalink raw reply [flat|nested] 23+ messages in thread* Re: [RFC] common/build-style/: enable running tests in parallel
2021-07-05 1:29 [PR PATCH] common/build-style/: enable running tests in parallel Logarithmus
` (4 preceding siblings ...)
2021-07-14 15:09 ` Logarithmus
@ 2021-07-14 15:09 ` Chocimier
2021-07-14 15:09 ` Logarithmus
` (15 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Chocimier @ 2021-07-14 15:09 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 355 bytes --]
New comment by Chocimier on void-packages repository
https://github.com/void-linux/void-packages/pull/31811#issuecomment-879974530
Comment:
Some build styles already run tests in parallel, please add disable_parallel_check there too.
It can be simpler with `if [ -z "$disable_parallel_check" ] local _check_jobs="${makejobs}"; make ... ${_check_jobs}`
^ permalink raw reply [flat|nested] 23+ messages in thread* Re: [RFC] common/build-style/: enable running tests in parallel
2021-07-05 1:29 [PR PATCH] common/build-style/: enable running tests in parallel Logarithmus
` (5 preceding siblings ...)
2021-07-14 15:09 ` Chocimier
@ 2021-07-14 15:09 ` Logarithmus
2021-07-14 15:10 ` [PR PATCH] [Updated] " Logarithmus
` (14 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Logarithmus @ 2021-07-14 15:09 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 604 bytes --]
New comment by Logarithmus on void-packages repository
https://github.com/void-linux/void-packages/pull/31811#issuecomment-879974474
Comment:
> Should we add `disable_parallel_tests` to override this? Some test suites are explicitly broken when run in parallel (iirc cups, for example).
>
> I assume they left their approval so others could review.
I've added `disable_parallel_check` option and tested it for `mold`, it works as intended.
A little TODO:
- document the new option in `Manual.md`
- try running tests for `cups*` packages and see if it fails without `disable_parallel_check`.
^ permalink raw reply [flat|nested] 23+ messages in thread* Re: [PR PATCH] [Updated] [RFC] common/build-style/: enable running tests in parallel
2021-07-05 1:29 [PR PATCH] common/build-style/: enable running tests in parallel Logarithmus
` (6 preceding siblings ...)
2021-07-14 15:09 ` Logarithmus
@ 2021-07-14 15:10 ` Logarithmus
2021-07-14 15:12 ` Logarithmus
` (13 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Logarithmus @ 2021-07-14 15:10 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 413 bytes --]
There is an updated pull request by Logarithmus against master on the void-packages repository
https://github.com/Logarithmus/void-packages common-parallel-tests
https://github.com/void-linux/void-packages/pull/31811
[RFC] common/build-style/: enable running tests in parallel
What do you think? Are there any caveats?
A patch file from https://github.com/void-linux/void-packages/pull/31811.patch is attached
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-common-parallel-tests-31811.patch --]
[-- Type: text/x-diff, Size: 3962 bytes --]
From 09906118bbeb4699637ab689bbf8812cd6a24832 Mon Sep 17 00:00:00 2001
From: Artur Sinila <freesoftware@logarithmus.dev>
Date: Mon, 5 Jul 2021 03:44:08 +0300
Subject: [PATCH] common/build-style/: enable running tests in parallel
---
common/build-style/cmake.sh | 6 +++++-
common/build-style/configure.sh | 6 +++++-
common/build-style/gnu-configure.sh | 6 +++++-
common/build-style/gnu-makefile.sh | 6 +++++-
common/build-style/perl-module.sh | 6 +++++-
common/environment/setup/sourcepkg.sh | 2 +-
6 files changed, 26 insertions(+), 6 deletions(-)
diff --git a/common/build-style/cmake.sh b/common/build-style/cmake.sh
index a7627e7dcf21..c3f6143d928a 100644
--- a/common/build-style/cmake.sh
+++ b/common/build-style/cmake.sh
@@ -113,7 +113,11 @@ do_check() {
: ${make_check_target:=test}
- ${make_cmd} ${make_check_args} ${make_check_target}
+ if [ -z "$disable_parallel_check" ]; then
+ ${make_cmd} ${makejobs} ${make_check_args} ${make_check_target}
+ else
+ ${make_cmd} ${make_check_args} ${make_check_target}
+ fi
}
do_install() {
diff --git a/common/build-style/configure.sh b/common/build-style/configure.sh
index 0963fc347934..b846a1a77572 100644
--- a/common/build-style/configure.sh
+++ b/common/build-style/configure.sh
@@ -29,7 +29,11 @@ do_check() {
: ${make_cmd:=make}
: ${make_check_target:=check}
- ${make_cmd} ${make_check_args} ${make_check_target}
+ if [ -z "$disable_parallel_check" ]; then
+ ${make_cmd} ${makejobs} ${make_check_args} ${make_check_target}
+ else
+ ${make_cmd} ${make_check_args} ${make_check_target}
+ fi
}
do_install() {
diff --git a/common/build-style/gnu-configure.sh b/common/build-style/gnu-configure.sh
index 8121136cd121..b09de442b05b 100644
--- a/common/build-style/gnu-configure.sh
+++ b/common/build-style/gnu-configure.sh
@@ -30,7 +30,11 @@ do_check() {
: ${make_cmd:=make}
: ${make_check_target:=check}
- ${make_cmd} ${make_check_args} ${make_check_target}
+ if [ -z "$disable_parallel_check" ]; then
+ ${make_cmd} ${makejobs} ${make_check_args} ${make_check_target}
+ else
+ ${make_cmd} ${make_check_args} ${make_check_target}
+ fi
}
do_install() {
diff --git a/common/build-style/gnu-makefile.sh b/common/build-style/gnu-makefile.sh
index d7ea148fc483..0f7afa7d3695 100644
--- a/common/build-style/gnu-makefile.sh
+++ b/common/build-style/gnu-makefile.sh
@@ -30,7 +30,11 @@ do_check() {
: ${make_cmd:=make}
: ${make_check_target:=check}
- ${make_cmd} ${make_check_args} ${make_check_target}
+ if [ -z "$disable_parallel_check" ]; then
+ ${make_cmd} ${makejobs} ${make_check_args} ${make_check_target}
+ else
+ ${make_cmd} ${make_check_args} ${make_check_target}
+ fi
}
do_install() {
diff --git a/common/build-style/perl-module.sh b/common/build-style/perl-module.sh
index b9a01b13cfa9..d47945b33d36 100644
--- a/common/build-style/perl-module.sh
+++ b/common/build-style/perl-module.sh
@@ -79,7 +79,11 @@ do_check() {
: ${make_cmd:=make}
: ${make_check_target:=test}
- ${make_cmd} ${make_check_args} ${make_check_target}
+ if [ -z "$disable_parallel_check" ]; then
+ ${make_cmd} ${makejobs} ${make_check_args} ${make_check_target}
+ else
+ ${make_cmd} ${make_check_args} ${make_check_target}
+ fi
}
do_install() {
diff --git a/common/environment/setup/sourcepkg.sh b/common/environment/setup/sourcepkg.sh
index bc06f745b78a..b6caa797138f 100644
--- a/common/environment/setup/sourcepkg.sh
+++ b/common/environment/setup/sourcepkg.sh
@@ -3,7 +3,7 @@
## VARIABLES
unset -v pkgname version revision short_desc homepage license maintainer
-unset -v archs distfiles checksum build_style build_helper nocross broken
+unset -v archs distfiles checksum build_style build_helper nocross disable_parallel_check broken
unset -v configure_script configure_args wrksrc build_wrksrc create_wrksrc
unset -v make_build_args make_check_args make_install_args
unset -v make_build_target make_check_target make_install_target
^ permalink raw reply [flat|nested] 23+ messages in thread* Re: [RFC] common/build-style/: enable running tests in parallel
2021-07-05 1:29 [PR PATCH] common/build-style/: enable running tests in parallel Logarithmus
` (7 preceding siblings ...)
2021-07-14 15:10 ` [PR PATCH] [Updated] " Logarithmus
@ 2021-07-14 15:12 ` Logarithmus
2021-07-14 15:24 ` Chocimier
` (12 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Logarithmus @ 2021-07-14 15:12 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 489 bytes --]
New comment by Logarithmus on void-packages repository
https://github.com/void-linux/void-packages/pull/31811#issuecomment-879976904
Comment:
> Some build styles already run tests in parallel, please add disable_parallel_check there too.
> It can be simpler with `if [ -z "$disable_parallel_check" ] local _check_jobs="${makejobs}"; make ... ${_check_jobs}`
Do you have a list of build systems with parallel tests out of the box? I only know about `cmake+ninja` & probably `meson`.
^ permalink raw reply [flat|nested] 23+ messages in thread* Re: [RFC] common/build-style/: enable running tests in parallel
2021-07-05 1:29 [PR PATCH] common/build-style/: enable running tests in parallel Logarithmus
` (8 preceding siblings ...)
2021-07-14 15:12 ` Logarithmus
@ 2021-07-14 15:24 ` Chocimier
2021-07-14 15:24 ` [PR REVIEW] " sgn
` (11 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Chocimier @ 2021-07-14 15:24 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 159 bytes --]
New comment by Chocimier on void-packages repository
https://github.com/void-linux/void-packages/pull/31811#issuecomment-879987125
Comment:
It's just meson.
^ permalink raw reply [flat|nested] 23+ messages in thread* Re: [PR REVIEW] [RFC] common/build-style/: enable running tests in parallel
2021-07-05 1:29 [PR PATCH] common/build-style/: enable running tests in parallel Logarithmus
` (9 preceding siblings ...)
2021-07-14 15:24 ` Chocimier
@ 2021-07-14 15:24 ` sgn
2021-07-14 15:29 ` sgn
` (10 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: sgn @ 2021-07-14 15:24 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 284 bytes --]
New review comment by sgn on void-packages repository
https://github.com/void-linux/void-packages/pull/31811#discussion_r669719512
Comment:
This is incorrect. `ninja` will run in parallel by default. I would rather change `makejobs` in `common/xbps-src/libexec/xbps-src-docheck.sh`
^ permalink raw reply [flat|nested] 23+ messages in thread* Re: [PR REVIEW] [RFC] common/build-style/: enable running tests in parallel
2021-07-05 1:29 [PR PATCH] common/build-style/: enable running tests in parallel Logarithmus
` (10 preceding siblings ...)
2021-07-14 15:24 ` [PR REVIEW] " sgn
@ 2021-07-14 15:29 ` sgn
2021-07-24 12:01 ` Logarithmus
` (9 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: sgn @ 2021-07-14 15:29 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 187 bytes --]
New review comment by sgn on void-packages repository
https://github.com/void-linux/void-packages/pull/31811#discussion_r669723533
Comment:
Please move to near `disable_parallel_build`
^ permalink raw reply [flat|nested] 23+ messages in thread* Re: [PR REVIEW] [RFC] common/build-style/: enable running tests in parallel
2021-07-05 1:29 [PR PATCH] common/build-style/: enable running tests in parallel Logarithmus
` (11 preceding siblings ...)
2021-07-14 15:29 ` sgn
@ 2021-07-24 12:01 ` Logarithmus
2021-07-24 12:04 ` Logarithmus
` (8 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Logarithmus @ 2021-07-24 12:01 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 409 bytes --]
New review comment by Logarithmus on void-packages repository
https://github.com/void-linux/void-packages/pull/31811#discussion_r675990723
Comment:
> This is incorrect. `ninja` will run in parallel by default. I would rather change `makejobs` in `common/xbps-src/libexec/xbps-src-docheck.sh`
Sorry, but I can't find `makejobs` in `common/xbps-src/libexec/xbps-src-docheck.sh`. Where should I define it?
^ permalink raw reply [flat|nested] 23+ messages in thread* Re: [PR REVIEW] [RFC] common/build-style/: enable running tests in parallel
2021-07-05 1:29 [PR PATCH] common/build-style/: enable running tests in parallel Logarithmus
` (12 preceding siblings ...)
2021-07-24 12:01 ` Logarithmus
@ 2021-07-24 12:04 ` Logarithmus
2021-07-24 12:16 ` sgn
` (7 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Logarithmus @ 2021-07-24 12:04 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 156 bytes --]
New review comment by Logarithmus on void-packages repository
https://github.com/void-linux/void-packages/pull/31811#discussion_r675990963
Comment:
Fixed
^ permalink raw reply [flat|nested] 23+ messages in thread* Re: [PR REVIEW] [RFC] common/build-style/: enable running tests in parallel
2021-07-05 1:29 [PR PATCH] common/build-style/: enable running tests in parallel Logarithmus
` (13 preceding siblings ...)
2021-07-24 12:04 ` Logarithmus
@ 2021-07-24 12:16 ` sgn
2021-07-24 12:30 ` [PR PATCH] [Updated] " Logarithmus
` (6 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: sgn @ 2021-07-24 12:16 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 200 bytes --]
New review comment by sgn on void-packages repository
https://github.com/void-linux/void-packages/pull/31811#discussion_r675992315
Comment:
It's exported from outside. You can change there, though.
^ permalink raw reply [flat|nested] 23+ messages in thread* Re: [PR PATCH] [Updated] [RFC] common/build-style/: enable running tests in parallel
2021-07-05 1:29 [PR PATCH] common/build-style/: enable running tests in parallel Logarithmus
` (14 preceding siblings ...)
2021-07-24 12:16 ` sgn
@ 2021-07-24 12:30 ` Logarithmus
2021-07-24 12:31 ` Logarithmus
` (5 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Logarithmus @ 2021-07-24 12:30 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 413 bytes --]
There is an updated pull request by Logarithmus against master on the void-packages repository
https://github.com/Logarithmus/void-packages common-parallel-tests
https://github.com/void-linux/void-packages/pull/31811
[RFC] common/build-style/: enable running tests in parallel
What do you think? Are there any caveats?
A patch file from https://github.com/void-linux/void-packages/pull/31811.patch is attached
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-common-parallel-tests-31811.patch --]
[-- Type: text/x-diff, Size: 4059 bytes --]
From 48581cf3dc7706f949758675923dd7932dd192ef Mon Sep 17 00:00:00 2001
From: Artur Sinila <freesoftware@logarithmus.dev>
Date: Mon, 5 Jul 2021 03:44:08 +0300
Subject: [PATCH] common/build-style/: enable running tests in parallel
---
common/build-style/cmake.sh | 6 +++++-
common/build-style/configure.sh | 6 +++++-
common/build-style/gnu-configure.sh | 6 +++++-
common/build-style/gnu-makefile.sh | 6 +++++-
common/build-style/perl-module.sh | 6 +++++-
common/environment/setup/sourcepkg.sh | 3 ++-
6 files changed, 27 insertions(+), 6 deletions(-)
diff --git a/common/build-style/cmake.sh b/common/build-style/cmake.sh
index a7627e7dcf21..c3f6143d928a 100644
--- a/common/build-style/cmake.sh
+++ b/common/build-style/cmake.sh
@@ -113,7 +113,11 @@ do_check() {
: ${make_check_target:=test}
- ${make_cmd} ${make_check_args} ${make_check_target}
+ if [ -z "$disable_parallel_check" ]; then
+ ${make_cmd} ${makejobs} ${make_check_args} ${make_check_target}
+ else
+ ${make_cmd} ${make_check_args} ${make_check_target}
+ fi
}
do_install() {
diff --git a/common/build-style/configure.sh b/common/build-style/configure.sh
index 0963fc347934..b846a1a77572 100644
--- a/common/build-style/configure.sh
+++ b/common/build-style/configure.sh
@@ -29,7 +29,11 @@ do_check() {
: ${make_cmd:=make}
: ${make_check_target:=check}
- ${make_cmd} ${make_check_args} ${make_check_target}
+ if [ -z "$disable_parallel_check" ]; then
+ ${make_cmd} ${makejobs} ${make_check_args} ${make_check_target}
+ else
+ ${make_cmd} ${make_check_args} ${make_check_target}
+ fi
}
do_install() {
diff --git a/common/build-style/gnu-configure.sh b/common/build-style/gnu-configure.sh
index 8121136cd121..b09de442b05b 100644
--- a/common/build-style/gnu-configure.sh
+++ b/common/build-style/gnu-configure.sh
@@ -30,7 +30,11 @@ do_check() {
: ${make_cmd:=make}
: ${make_check_target:=check}
- ${make_cmd} ${make_check_args} ${make_check_target}
+ if [ -z "$disable_parallel_check" ]; then
+ ${make_cmd} ${makejobs} ${make_check_args} ${make_check_target}
+ else
+ ${make_cmd} ${make_check_args} ${make_check_target}
+ fi
}
do_install() {
diff --git a/common/build-style/gnu-makefile.sh b/common/build-style/gnu-makefile.sh
index d7ea148fc483..0f7afa7d3695 100644
--- a/common/build-style/gnu-makefile.sh
+++ b/common/build-style/gnu-makefile.sh
@@ -30,7 +30,11 @@ do_check() {
: ${make_cmd:=make}
: ${make_check_target:=check}
- ${make_cmd} ${make_check_args} ${make_check_target}
+ if [ -z "$disable_parallel_check" ]; then
+ ${make_cmd} ${makejobs} ${make_check_args} ${make_check_target}
+ else
+ ${make_cmd} ${make_check_args} ${make_check_target}
+ fi
}
do_install() {
diff --git a/common/build-style/perl-module.sh b/common/build-style/perl-module.sh
index b9a01b13cfa9..d47945b33d36 100644
--- a/common/build-style/perl-module.sh
+++ b/common/build-style/perl-module.sh
@@ -79,7 +79,11 @@ do_check() {
: ${make_cmd:=make}
: ${make_check_target:=test}
- ${make_cmd} ${make_check_args} ${make_check_target}
+ if [ -z "$disable_parallel_check" ]; then
+ ${make_cmd} ${makejobs} ${make_check_args} ${make_check_target}
+ else
+ ${make_cmd} ${make_check_args} ${make_check_target}
+ fi
}
do_install() {
diff --git a/common/environment/setup/sourcepkg.sh b/common/environment/setup/sourcepkg.sh
index bc06f745b78a..1fd541f1e715 100644
--- a/common/environment/setup/sourcepkg.sh
+++ b/common/environment/setup/sourcepkg.sh
@@ -13,7 +13,8 @@ unset -v cmake_builddir meson_builddir
unset -v meson_crossfile
unset -v gemspec
unset -v go_import_path go_package go_mod_mode
-unset -v patch_args disable_parallel_build keep_libtool_archives make_use_env
+unset -v patch_args disable_parallel_build disable_parallel_check
+unset -v keep_libtool_archives make_use_env
unset -v reverts subpackages makedepends hostmakedepends checkdepends depends restricted
unset -v nopie build_options build_options_default bootstrap repository reverts
unset -v CFLAGS CXXFLAGS FFLAGS CPPFLAGS LDFLAGS LD_LIBRARY_PATH
^ permalink raw reply [flat|nested] 23+ messages in thread* Re: [PR PATCH] [Updated] [RFC] common/build-style/: enable running tests in parallel
2021-07-05 1:29 [PR PATCH] common/build-style/: enable running tests in parallel Logarithmus
` (15 preceding siblings ...)
2021-07-24 12:30 ` [PR PATCH] [Updated] " Logarithmus
@ 2021-07-24 12:31 ` Logarithmus
2021-07-25 19:53 ` [PR REVIEW] " Logarithmus
` (4 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Logarithmus @ 2021-07-24 12:31 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 413 bytes --]
There is an updated pull request by Logarithmus against master on the void-packages repository
https://github.com/Logarithmus/void-packages common-parallel-tests
https://github.com/void-linux/void-packages/pull/31811
[RFC] common/build-style/: enable running tests in parallel
What do you think? Are there any caveats?
A patch file from https://github.com/void-linux/void-packages/pull/31811.patch is attached
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-common-parallel-tests-31811.patch --]
[-- Type: text/x-diff, Size: 4059 bytes --]
From da8cbe3c214f078857e836cc7faf0920dc1f322f Mon Sep 17 00:00:00 2001
From: Artur Sinila <freesoftware@logarithmus.dev>
Date: Mon, 5 Jul 2021 03:44:08 +0300
Subject: [PATCH] common/build-style/: enable running tests in parallel
---
common/build-style/cmake.sh | 6 +++++-
common/build-style/configure.sh | 6 +++++-
common/build-style/gnu-configure.sh | 6 +++++-
common/build-style/gnu-makefile.sh | 6 +++++-
common/build-style/perl-module.sh | 6 +++++-
common/environment/setup/sourcepkg.sh | 3 ++-
6 files changed, 27 insertions(+), 6 deletions(-)
diff --git a/common/build-style/cmake.sh b/common/build-style/cmake.sh
index 401dabad31c8..c7ffd16fee21 100644
--- a/common/build-style/cmake.sh
+++ b/common/build-style/cmake.sh
@@ -113,7 +113,11 @@ do_check() {
: ${make_check_target:=test}
- ${make_cmd} ${make_check_args} ${make_check_target}
+ if [ -z "$disable_parallel_check" ]; then
+ ${make_cmd} ${makejobs} ${make_check_args} ${make_check_target}
+ else
+ ${make_cmd} ${make_check_args} ${make_check_target}
+ fi
}
do_install() {
diff --git a/common/build-style/configure.sh b/common/build-style/configure.sh
index 0963fc347934..b846a1a77572 100644
--- a/common/build-style/configure.sh
+++ b/common/build-style/configure.sh
@@ -29,7 +29,11 @@ do_check() {
: ${make_cmd:=make}
: ${make_check_target:=check}
- ${make_cmd} ${make_check_args} ${make_check_target}
+ if [ -z "$disable_parallel_check" ]; then
+ ${make_cmd} ${makejobs} ${make_check_args} ${make_check_target}
+ else
+ ${make_cmd} ${make_check_args} ${make_check_target}
+ fi
}
do_install() {
diff --git a/common/build-style/gnu-configure.sh b/common/build-style/gnu-configure.sh
index 8121136cd121..b09de442b05b 100644
--- a/common/build-style/gnu-configure.sh
+++ b/common/build-style/gnu-configure.sh
@@ -30,7 +30,11 @@ do_check() {
: ${make_cmd:=make}
: ${make_check_target:=check}
- ${make_cmd} ${make_check_args} ${make_check_target}
+ if [ -z "$disable_parallel_check" ]; then
+ ${make_cmd} ${makejobs} ${make_check_args} ${make_check_target}
+ else
+ ${make_cmd} ${make_check_args} ${make_check_target}
+ fi
}
do_install() {
diff --git a/common/build-style/gnu-makefile.sh b/common/build-style/gnu-makefile.sh
index d7ea148fc483..0f7afa7d3695 100644
--- a/common/build-style/gnu-makefile.sh
+++ b/common/build-style/gnu-makefile.sh
@@ -30,7 +30,11 @@ do_check() {
: ${make_cmd:=make}
: ${make_check_target:=check}
- ${make_cmd} ${make_check_args} ${make_check_target}
+ if [ -z "$disable_parallel_check" ]; then
+ ${make_cmd} ${makejobs} ${make_check_args} ${make_check_target}
+ else
+ ${make_cmd} ${make_check_args} ${make_check_target}
+ fi
}
do_install() {
diff --git a/common/build-style/perl-module.sh b/common/build-style/perl-module.sh
index b9a01b13cfa9..d47945b33d36 100644
--- a/common/build-style/perl-module.sh
+++ b/common/build-style/perl-module.sh
@@ -79,7 +79,11 @@ do_check() {
: ${make_cmd:=make}
: ${make_check_target:=test}
- ${make_cmd} ${make_check_args} ${make_check_target}
+ if [ -z "$disable_parallel_check" ]; then
+ ${make_cmd} ${makejobs} ${make_check_args} ${make_check_target}
+ else
+ ${make_cmd} ${make_check_args} ${make_check_target}
+ fi
}
do_install() {
diff --git a/common/environment/setup/sourcepkg.sh b/common/environment/setup/sourcepkg.sh
index bc06f745b78a..1fd541f1e715 100644
--- a/common/environment/setup/sourcepkg.sh
+++ b/common/environment/setup/sourcepkg.sh
@@ -13,7 +13,8 @@ unset -v cmake_builddir meson_builddir
unset -v meson_crossfile
unset -v gemspec
unset -v go_import_path go_package go_mod_mode
-unset -v patch_args disable_parallel_build keep_libtool_archives make_use_env
+unset -v patch_args disable_parallel_build disable_parallel_check
+unset -v keep_libtool_archives make_use_env
unset -v reverts subpackages makedepends hostmakedepends checkdepends depends restricted
unset -v nopie build_options build_options_default bootstrap repository reverts
unset -v CFLAGS CXXFLAGS FFLAGS CPPFLAGS LDFLAGS LD_LIBRARY_PATH
^ permalink raw reply [flat|nested] 23+ messages in thread* Re: [PR REVIEW] [RFC] common/build-style/: enable running tests in parallel
2021-07-05 1:29 [PR PATCH] common/build-style/: enable running tests in parallel Logarithmus
` (16 preceding siblings ...)
2021-07-24 12:31 ` Logarithmus
@ 2021-07-25 19:53 ` Logarithmus
2021-07-25 19:54 ` Logarithmus
` (3 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Logarithmus @ 2021-07-25 19:53 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 714 bytes --]
New review comment by Logarithmus on void-packages repository
https://github.com/void-linux/void-packages/pull/31811#discussion_r675992917
Comment:
> It's exported from outside. You can change there, though.
I've just founded out that `disable_parallel_build` sets `XBPS_MAKEJOBS=1`. Maybe we don't need `disable_parallel_check` altogether? Otherwise in order for everything to work as expected we should revise `disable_parallel_build` to not mess with `XBPS_MAKEJOBS`, because currently `disable_parallel_build` also implies `disable_parallel_check`, which isn't obvious.
https://github.com/void-linux/void-packages/blob/90243383a366d6dc64c51555cfd4f3a63dd7995c/common/xbps-src/shutils/common.sh#L468-L469
^ permalink raw reply [flat|nested] 23+ messages in thread* Re: [PR REVIEW] [RFC] common/build-style/: enable running tests in parallel
2021-07-05 1:29 [PR PATCH] common/build-style/: enable running tests in parallel Logarithmus
` (17 preceding siblings ...)
2021-07-25 19:53 ` [PR REVIEW] " Logarithmus
@ 2021-07-25 19:54 ` Logarithmus
2021-07-30 20:43 ` Chocimier
` (2 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Logarithmus @ 2021-07-25 19:54 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 712 bytes --]
New review comment by Logarithmus on void-packages repository
https://github.com/void-linux/void-packages/pull/31811#discussion_r675992917
Comment:
> It's exported from outside. You can change there, though.
I've just found out that `disable_parallel_build` sets `XBPS_MAKEJOBS=1`. Maybe we don't need `disable_parallel_check` altogether? Otherwise in order for everything to work as expected we should revise `disable_parallel_build` to not mess with `XBPS_MAKEJOBS`, because currently `disable_parallel_build` also implies `disable_parallel_check`, which isn't obvious.
https://github.com/void-linux/void-packages/blob/90243383a366d6dc64c51555cfd4f3a63dd7995c/common/xbps-src/shutils/common.sh#L468-L469
^ permalink raw reply [flat|nested] 23+ messages in thread* Re: [PR REVIEW] [RFC] common/build-style/: enable running tests in parallel
2021-07-05 1:29 [PR PATCH] common/build-style/: enable running tests in parallel Logarithmus
` (18 preceding siblings ...)
2021-07-25 19:54 ` Logarithmus
@ 2021-07-30 20:43 ` Chocimier
2022-05-24 2:13 ` github-actions
2022-06-08 2:11 ` [PR PATCH] [Closed]: " github-actions
21 siblings, 0 replies; 23+ messages in thread
From: Chocimier @ 2021-07-30 20:43 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 606 bytes --]
New review comment by Chocimier on void-packages repository
https://github.com/void-linux/void-packages/pull/31811#discussion_r680208579
Comment:
> Maybe we don't need `disable_parallel_check` altogether?
Successful multicore compilation with tests broken when running in parallel will likely happen.
> `disable_parallel_build` also implies `disable_parallel_check`, which isn't obvious
Documenting that should be enough. You could also export additional XBPS_MAX_MAKEJOBS or so and read it for check if disable_parallel_build is on if you prefer, just make sure to export that in right places.
^ permalink raw reply [flat|nested] 23+ messages in thread* Re: [RFC] common/build-style/: enable running tests in parallel
2021-07-05 1:29 [PR PATCH] common/build-style/: enable running tests in parallel Logarithmus
` (19 preceding siblings ...)
2021-07-30 20:43 ` Chocimier
@ 2022-05-24 2:13 ` github-actions
2022-06-08 2:11 ` [PR PATCH] [Closed]: " github-actions
21 siblings, 0 replies; 23+ messages in thread
From: github-actions @ 2022-05-24 2:13 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 305 bytes --]
New comment by github-actions[bot] on void-packages repository
https://github.com/void-linux/void-packages/pull/31811#issuecomment-1135323202
Comment:
Pull Requests become stale 90 days after last activity and are closed 14 days after that. If this pull request is still relevant bump it or assign it.
^ permalink raw reply [flat|nested] 23+ messages in thread* Re: [PR PATCH] [Closed]: [RFC] common/build-style/: enable running tests in parallel
2021-07-05 1:29 [PR PATCH] common/build-style/: enable running tests in parallel Logarithmus
` (20 preceding siblings ...)
2022-05-24 2:13 ` github-actions
@ 2022-06-08 2:11 ` github-actions
21 siblings, 0 replies; 23+ messages in thread
From: github-actions @ 2022-06-08 2:11 UTC (permalink / raw)
To: ml
[-- Attachment #1: Type: text/plain, Size: 235 bytes --]
There's a closed pull request on the void-packages repository
[RFC] common/build-style/: enable running tests in parallel
https://github.com/void-linux/void-packages/pull/31811
Description:
What do you think? Are there any caveats?
^ permalink raw reply [flat|nested] 23+ messages in thread