zsh-workers
 help / color / mirror / code / Atom feed
* A suggestion RE Xfail tests
@ 2022-05-04  1:44 Bart Schaefer
  2022-05-04  2:52 ` Daniel Shahaf
  0 siblings, 1 reply; 2+ messages in thread
From: Bart Schaefer @ 2022-05-04  1:44 UTC (permalink / raw)
  To: Zsh hackers list

It would be helpful if tests that are expected to fail could list both
the expected "correct" result (that is, the result if the test were to
succeed, which would cause the Xfail test to fail) and the expected
"wrong" output ... so that if the way a test goes wrong changes, that
too can be seen.

Yes, this can be done now by repeating the test twice, once as an
xfail and once as a regular test but with the "broken" result ... but
it might be easier to keep track of if it were all in one place.

(No action on this expected pre-5.9-release, just musing.)


^ permalink raw reply	[flat|nested] 2+ messages in thread

* Re: A suggestion RE Xfail tests
  2022-05-04  1:44 A suggestion RE Xfail tests Bart Schaefer
@ 2022-05-04  2:52 ` Daniel Shahaf
  0 siblings, 0 replies; 2+ messages in thread
From: Daniel Shahaf @ 2022-05-04  2:52 UTC (permalink / raw)
  To: Zsh hackers list

Bart Schaefer wrote on Wed, 04 May 2022 01:44 +00:00:
> It would be helpful if tests that are expected to fail could list both
> the expected "correct" result (that is, the result if the test were to
> succeed, which would cause the Xfail test to fail) and the expected
> "wrong" output ... so that if the way a test goes wrong changes, that
> too can be seen.
>

Fair request.

FWIW, I don't know of any other test harness that supports this.

> Yes, this can be done now by repeating the test twice, once as an
> xfail and once as a regular test but with the "broken" result ... but
> it might be easier to keep track of if it were all in one place.
>

How about if, for the time being, we use the "duplicate the test"
approach, with F: lines or comments as needed?  Once we have some
experience with that, we may have a better idea of what test harness
functionality to implement.

> (No action on this expected pre-5.9-release, just musing.)


^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2022-05-04  2:52 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-05-04  1:44 A suggestion RE Xfail tests Bart Schaefer
2022-05-04  2:52 ` Daniel Shahaf

Code repositories for project(s) associated with this inbox:

	https://git.vuxu.org/mirror/zsh/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).