Github messages for voidlinux
 help / color / mirror / Atom feed
* [ISSUE] [RFC] Ideas for check and CI
@ 2021-12-30 15:15 tornaria
  2022-06-20  2:13 ` github-actions
                   ` (2 more replies)
  0 siblings, 3 replies; 4+ messages in thread
From: tornaria @ 2021-12-30 15:15 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 2260 bytes --]

New issue by tornaria on void-packages repository

https://github.com/void-linux/void-packages/issues/34766

Description:
I'm listing here some random ideas for check and CI.\


From https://github.com/void-linux/void-packages/pull/34469#issuecomment-991778989:

 - have an option `[check-skip]` that works as `[ci-skip]` but instead of skipping the whole CI it will run it without `-Q`.

From https://github.com/void-linux/void-packages/pull/34469#issuecomment-1002142459:
Random ideas:
 - maybe we should fill issues for failing tests, so it's documented (e.g. when revbump, to know it's not an issue introduced by the lib update)
 - maybe have a way to indicate that some checks are expected to fail for certain archs. For instance, I'd mark bogofilter and inkscape as xfail for x86_64-musl. The check step would still run, but it wouldn't stop with an error on failure so the CI can keep going. E.g. `check_xfail="x86_64-musl"`.
 - maybe have a way to indicate that some checks should be skipped for certain archs. For instance, I'd mark gnuradio to skip checks for i686 so the CI wouldn't get stuck on a loose test. E.g. `check_skip="i686"`.

Policy could be to comment any `check_xfail` or `check_skip` with an issue number where it is well documented.

More random thoughts:
 - maybe useful to have an option for CI such that a failure on check doesn't stop; the pkg step would be run anyway. The whole thing would still show a failure at the end but at least one would know the packaging step runs ok.
 - similarly, if one package fails, keep building the rest (as long as its dependencies didn't fail on build or pkg). Print a complete and concise report at the end about which steps (build/check/pkg) failed for which pkgs, and still give error if anything failed.
 - I wonder if it would be possible to have "dynamic" steps in a CI job, I mean: instead of having one step "Build and check packages" have one separate step for each pkg that is built so it's easy to see logs for each pkg individually
 - Another question: would -t work in the CI? Would it make sense to run using it so building of one pkg doesn't affect building of others (avoid the libgomp issue). If -t doesn't work, could zap the masterdir between pkgs.




^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [RFC] Ideas for check and CI
  2021-12-30 15:15 [ISSUE] [RFC] Ideas for check and CI tornaria
@ 2022-06-20  2:13 ` github-actions
  2022-07-04  2:15 ` [ISSUE] [CLOSED] " github-actions
  2022-11-06 20:08 ` paper42
  2 siblings, 0 replies; 4+ messages in thread
From: github-actions @ 2022-06-20  2:13 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 293 bytes --]

New comment by github-actions[bot] on void-packages repository

https://github.com/void-linux/void-packages/issues/34766#issuecomment-1159892112

Comment:
Issues become stale 90 days after last activity and are closed 14 days after that.  If this issue is still relevant bump it or assign it.

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [ISSUE] [CLOSED] [RFC] Ideas for check and CI
  2021-12-30 15:15 [ISSUE] [RFC] Ideas for check and CI tornaria
  2022-06-20  2:13 ` github-actions
@ 2022-07-04  2:15 ` github-actions
  2022-11-06 20:08 ` paper42
  2 siblings, 0 replies; 4+ messages in thread
From: github-actions @ 2022-07-04  2:15 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 2263 bytes --]

Closed issue by tornaria on void-packages repository

https://github.com/void-linux/void-packages/issues/34766

Description:
I'm listing here some random ideas for check and CI.\


From https://github.com/void-linux/void-packages/pull/34469#issuecomment-991778989:

 - have an option `[check-skip]` that works as `[ci-skip]` but instead of skipping the whole CI it will run it without `-Q`.

From https://github.com/void-linux/void-packages/pull/34469#issuecomment-1002142459:
Random ideas:
 - maybe we should fill issues for failing tests, so it's documented (e.g. when revbump, to know it's not an issue introduced by the lib update)
 - maybe have a way to indicate that some checks are expected to fail for certain archs. For instance, I'd mark bogofilter and inkscape as xfail for x86_64-musl. The check step would still run, but it wouldn't stop with an error on failure so the CI can keep going. E.g. `check_xfail="x86_64-musl"`.
 - maybe have a way to indicate that some checks should be skipped for certain archs. For instance, I'd mark gnuradio to skip checks for i686 so the CI wouldn't get stuck on a loose test. E.g. `check_skip="i686"`.

Policy could be to comment any `check_xfail` or `check_skip` with an issue number where it is well documented.

More random thoughts:
 - maybe useful to have an option for CI such that a failure on check doesn't stop; the pkg step would be run anyway. The whole thing would still show a failure at the end but at least one would know the packaging step runs ok.
 - similarly, if one package fails, keep building the rest (as long as its dependencies didn't fail on build or pkg). Print a complete and concise report at the end about which steps (build/check/pkg) failed for which pkgs, and still give error if anything failed.
 - I wonder if it would be possible to have "dynamic" steps in a CI job, I mean: instead of having one step "Build and check packages" have one separate step for each pkg that is built so it's easy to see logs for each pkg individually
 - Another question: would -t work in the CI? Would it make sense to run using it so building of one pkg doesn't affect building of others (avoid the libgomp issue). If -t doesn't work, could zap the masterdir between pkgs.




^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [RFC] Ideas for check and CI
  2021-12-30 15:15 [ISSUE] [RFC] Ideas for check and CI tornaria
  2022-06-20  2:13 ` github-actions
  2022-07-04  2:15 ` [ISSUE] [CLOSED] " github-actions
@ 2022-11-06 20:08 ` paper42
  2 siblings, 0 replies; 4+ messages in thread
From: paper42 @ 2022-11-06 20:08 UTC (permalink / raw)
  To: ml

[-- Attachment #1: Type: text/plain, Size: 1472 bytes --]

New comment by paper42 on void-packages repository

https://github.com/void-linux/void-packages/issues/34766#issuecomment-1304883171

Comment:
> * have an option `[check-skip]` that works as `[ci-skip]` but instead of skipping the whole CI it will run it without `-Q`.

Tests can be skipped with `make_check=ci-skip` in the template if they don't work in CI or `make_check=extended`.

> maybe have a way to indicate that some checks are expected to fail for certain archs. For instance, I'd mark bogofilter and inkscape as xfail for x86_64-musl. The check step would still run, but it wouldn't stop with an error on failure so the CI can keep going. E.g. check_xfail="x86_64-musl".

> maybe have a way to indicate that some checks should be skipped for certain archs. For instance, I'd mark gnuradio to skip checks for i686 so the CI wouldn't get stuck on a loose test. E.g. check_skip="i686".

This can already be done:
```
case "$XBPS_TARGET_MACHINE" in
	# reason
	i686*) make_check=no ;;
esac
```

It doesn't make much sense to run the tests, but then ignore the result, so I think not running tests is good enough for us.

> Policy could be to comment any check_xfail or check_skip with an issue number where it is well documented.

`$make_check` should always have a comment explaining why it's needed and can link to an upstream issue. `xlint` check for a comment and will throw an error when it's not there which makes the lint step in CI fail.

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2022-11-06 20:08 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-12-30 15:15 [ISSUE] [RFC] Ideas for check and CI tornaria
2022-06-20  2:13 ` github-actions
2022-07-04  2:15 ` [ISSUE] [CLOSED] " github-actions
2022-11-06 20:08 ` paper42

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).