New comment by tornaria on void-packages repository

https://github.com/void-linux/void-packages/pull/34469#issuecomment-1002142459

Comment:
The last CI:
 - x86_64 and aarch64 pass all
 - i686 hangs on test `qa_fecapi_ldpc` as expected (3h30 stuck at that until github kills it)
 - x86_64-musl fails on check inkscape as expected (it was either that one or bogofilter that would fail)
 - the other cross builds don't fail, they just run out of time (note aarch64 just barely made it in 344m while the limit is 360m).

A fix for test qa_fecapi_ldpc on gnuradio so it doesn't hang would be very desirable, but the others... I'm tired of this (I only intended to update gsl and I already fixed several issues on other pkgs :thinking:).

----

Random ideas:
 - maybe we should fill issues for failing tests, so it's documented (e.g. when revbump, to know it's not an issue introduced by the lib update)
 - maybe have a way to indicate that some checks are expected to fail for certain archs. For instance, I'd mark bogofilter and inkscape as xfail for x86_64-musl. The check step would still run, but it wouldn't stop with an error on failure so the CI can keep going. E.g. `check_xfail="x86_64-musl"`.
 - maybe have a way to indicate that some checks should be skipped for certain archs. For instance, I'd mark gnuradio to skip checks for i686 so the CI wouldn't get stuck on a loose test. E.g. `check_skip="i686".

Policy could be to comment any `check_xfail` or `check_skip` with an issue number where it is well documented.

More random thoughts:
 - maybe useful to have an option for CI such that a failure on check doesn't stop; the pkg step would be run anyway. The whole thing would still show a failure at the end but at least one would know the packaging step runs ok.
 - similarly, if one package fails, keep building the rest (as long as its dependencies didn't fail on build or pkg). At the end print a complete and concise report at the end about which steps (build/check/pkg) failed for which pkgs, and still give error if anything failed.
 - I wonder if it would be possible to have "dynamic" steps in a CI job, I mean: instead of having one step "Build and check packages" have one separate step for each pkg that is built so it's easy to see logs for each pkg individually
 - Another question: would -t work in the CI? Would it make sense to run using it so building of one pkg doesn't affect building of others (avoid the libgomp issue). If -t doesn't work, could zap the masterdir between pkgs.