* other testsuite problems
@ 2007-12-24 22:23 Clint Adams
2007-12-25 1:36 ` Clint Adams
2007-12-25 2:27 ` Clint Adams
0 siblings, 2 replies; 9+ messages in thread
From: Clint Adams @ 2007-12-24 22:23 UTC (permalink / raw)
To: zsh-workers
In addition to the >&- test for which I have done no followup, I note
the following other problems in testsuiteland:
If $HOME does not exist or is not writable, many tests go awry. Arguably, this is user error.
In some environments, I get this
../../Test/A01grammar.ztst: starting.
*** /tmp/zsh.ztst.err.22699 Mon Dec 24 21:24:33 2007
--- /tmp/zsh.ztst.terr.22699 Mon Dec 24 21:24:33 2007
***************
*** 1,2 ****
- 1) one 2) two 3) three
input> input>
--- 1 ----
Test ../../Test/A01grammar.ztst failed: error output differs from expected as shown above for:
(COLUMNS=80
PS3="input> "
select name in one two three; do
print $name
done)
Was testing: `select' loop
../../Test/A01grammar.ztst: test failed.
Finally, if D07multibyte.ztst is skipped due to a lack of UTF-8 locale, it seems as though that is
being registered as a failure. Perhaps a distinct exit code should be used for skips (and counting
such in runtests.zsh).
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: other testsuite problems
2007-12-24 22:23 other testsuite problems Clint Adams
@ 2007-12-25 1:36 ` Clint Adams
2007-12-25 2:27 ` Clint Adams
1 sibling, 0 replies; 9+ messages in thread
From: Clint Adams @ 2007-12-25 1:36 UTC (permalink / raw)
To: zsh-workers
On Mon, Dec 24, 2007 at 05:23:25PM -0500, Clint Adams wrote:
> Finally, if D07multibyte.ztst is skipped due to a lack of UTF-8 locale, it seems as though that is
> being registered as a failure. Perhaps a distinct exit code should be used for skips (and counting
> such in runtests.zsh).
To that end,
Index: Test/runtests.zsh
===================================================================
RCS file: /cvsroot/zsh/zsh/Test/runtests.zsh,v
retrieving revision 1.1
diff -u -r1.1 runtests.zsh
--- Test/runtests.zsh 18 Dec 2007 21:16:30 -0000 1.1
+++ Test/runtests.zsh 25 Dec 2007 01:35:00 -0000
@@ -7,10 +7,13 @@
# protect from catastrophic failure of an individual test.
# We could probably do that with subshells instead.
-integer success failure
+integer success failure skipped retval
for file in "${(f)ZTST_testlist}"; do
$ZTST_exe +Z -f $ZTST_srcdir/ztst.zsh $file
- if (( $? )); then
+ retval=$?
+ if (( $retval == 2 )); then
+ (( skipped++ ))
+ elif (( $retval )); then
(( failure++ ))
else
(( success++ ))
@@ -18,6 +21,7 @@
done
print "**************************************
$success successful test script${${success:#1}:+s}, \
-$failure failure${${failure:#1}:+s}
+$failure failure${${failure:#1}:+s}, \
+$skipped skipped
**************************************"
return $(( failure ? 1 : 0 ))
Index: Test/ztst.zsh
===================================================================
RCS file: /cvsroot/zsh/zsh/Test/ztst.zsh,v
retrieving revision 1.28
diff -u -r1.28 ztst.zsh
--- Test/ztst.zsh 18 Dec 2007 21:16:30 -0000 1.28
+++ Test/ztst.zsh 25 Dec 2007 01:35:00 -0000
@@ -467,6 +467,7 @@
if [[ -n "$ZTST_unimplemented" ]]; then
print "$ZTST_testname: skipped ($ZTST_unimplemented)"
+ ZTST_testfailed=2
elif (( ! $ZTST_testfailed )); then
print "$ZTST_testname: all tests successful."
fi
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: other testsuite problems
2007-12-24 22:23 other testsuite problems Clint Adams
2007-12-25 1:36 ` Clint Adams
@ 2007-12-25 2:27 ` Clint Adams
2008-01-03 18:17 ` Peter Stephenson
1 sibling, 1 reply; 9+ messages in thread
From: Clint Adams @ 2007-12-25 2:27 UTC (permalink / raw)
To: zsh-workers
On Mon, Dec 24, 2007 at 05:23:25PM -0500, Clint Adams wrote:
> Finally, if D07multibyte.ztst is skipped due to a lack of UTF-8 locale, it seems as though that is
> being registered as a failure. Perhaps a distinct exit code should be used for skips (and counting
This was completely off the mark; there is still a failure counted
despite it not being reported as failed (or I'm blind):
http://buildd.debian.org/fetch.cgi?pkg=zsh;ver=4.3.4-dev-6-3;arch=amd64;stamp=1198548543
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: other testsuite problems
2007-12-25 2:27 ` Clint Adams
@ 2008-01-03 18:17 ` Peter Stephenson
2008-01-04 4:14 ` Clint Adams
0 siblings, 1 reply; 9+ messages in thread
From: Peter Stephenson @ 2008-01-03 18:17 UTC (permalink / raw)
To: zsh-workers
On Mon, 24 Dec 2007 21:27:44 -0500
Clint Adams <clint@zsh.org> wrote:
> On Mon, Dec 24, 2007 at 05:23:25PM -0500, Clint Adams wrote:
> > Finally, if D07multibyte.ztst is skipped due to a lack of UTF-8 locale, it seems as though that is
> > being registered as a failure. Perhaps a distinct exit code should be used for skips (and counting
>
> This was completely off the mark; there is still a failure counted
> despite it not being reported as failed (or I'm blind):
> http://buildd.debian.org/fetch.cgi?pkg=zsh;ver=4.3.4-dev-6-3;arch=amd64;stamp=1198548543
The fix is OK, but V01 is going off into a parallel universe; probably an
exec or similar.
../../Test/V01zmodload.ztst: starting.
../../Test/V02zregexparse.ztst: starting.
Try with more detail.
It might be better to shift the messages (apart from the hash printing,
obviously) to runtests.zsh, as well as making failures more obvious, to
fix this.
--
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2008-01-09 20:52 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2007-12-24 22:23 other testsuite problems Clint Adams
2007-12-25 1:36 ` Clint Adams
2007-12-25 2:27 ` Clint Adams
2008-01-03 18:17 ` Peter Stephenson
2008-01-04 4:14 ` Clint Adams
2008-01-04 12:55 ` Peter Stephenson
2008-01-04 19:05 ` Clint Adams
2008-01-05 12:25 ` Peter Stephenson
2008-01-09 20:51 ` Clint Adams
Code repositories for project(s) associated with this public inbox
https://git.vuxu.org/mirror/zsh/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).