From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 2491 invoked from network); 15 Aug 2005 15:53:22 -0000 Received: from news.dotsrc.org (HELO a.mx.sunsite.dk) (130.225.247.88) by ns1.primenet.com.au with SMTP; 15 Aug 2005 15:53:22 -0000 Received: (qmail 34632 invoked from network); 15 Aug 2005 15:53:15 -0000 Received: from sunsite.dk (130.225.247.90) by a.mx.sunsite.dk with SMTP; 15 Aug 2005 15:53:15 -0000 Received: (qmail 1885 invoked by alias); 15 Aug 2005 15:53:12 -0000 Mailing-List: contact zsh-workers-help@sunsite.dk; run by ezmlm Precedence: bulk X-No-Archive: yes X-Seq: 21617 Received: (qmail 1874 invoked from network); 15 Aug 2005 15:53:12 -0000 Received: from news.dotsrc.org (HELO a.mx.sunsite.dk) (130.225.247.88) by sunsite.dk with SMTP; 15 Aug 2005 15:53:12 -0000 Received: (qmail 34325 invoked from network); 15 Aug 2005 15:53:12 -0000 Received: from mail.gmx.de (HELO mail.gmx.net) (213.165.64.20) by a.mx.sunsite.dk with SMTP; 15 Aug 2005 15:53:08 -0000 Received: (qmail invoked by alias); 15 Aug 2005 15:53:04 -0000 Received: from Bc926.b.pppool.de (EHLO pcdahl4201) [213.7.201.38] by mail.gmx.net (mp013) with SMTP; 15 Aug 2005 17:53:05 +0200 X-Authenticated: #21620914 Message-ID: <019001c5a1b2$c3f21500$e221fea9@pcdahl4201> From: "Thorsten Dahlheimer" To: "Wayne Davison" Cc: References: <028601c59c46$56d0dea0$cc74fea9@pcdahl4201> <20050809073927.GC12952@blorf.net> Subject: Re: PATCH: Slightly improve test harness Date: Mon, 15 Aug 2005 18:02:38 +0200 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_018D_01C5A1C3.85E996A0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1409 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1409 X-Y-GMX-Trusted: 0 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on f.primenet.com.au X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=AWL,BAYES_00, MIME_QP_LONG_LINE autolearn=ham version=3.0.4 This is a multi-part message in MIME format. ------=_NextPart_000_018D_01C5A1C3.85E996A0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Wayne Davison wrote: > If so, it might be best to just leave the abort-after-the-first-failure > behavior unchanged. Ok, I've attached a new patch with this change undone. It also corrects an erroneous testcase in A06assign.ztst and replaces various hardcoded paths to the newly-compiled zsh executable in the test files with $ZTST_exe. > That fixed trap code seems to make it impossible to Ctrl-C out of the > tests, though I'm not sure why yet. Perhaps it's a bug in trap > handling, but zsh hangs when that cleanup trap in place and Ctrl-C is > pressed. I could reproduce this when running on linux, so I left the trap commented out. Thorsten ------=_NextPart_000_018D_01C5A1C3.85E996A0 Content-Type: application/octet-stream; name="ztst_2.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="ztst_2.patch" Index: Test/A01grammar.ztst=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= RCS file: /cvsroot/zsh/zsh/Test/A01grammar.ztst,v=0A= retrieving revision 1.12=0A= diff -u -r1.12 A01grammar.ztst=0A= --- Test/A01grammar.ztst 11 Apr 2005 10:23:57 -0000 1.12=0A= +++ Test/A01grammar.ztst 15 Aug 2005 15:22:35 -0000=0A= @@ -50,7 +50,7 @@=0A= #=0A= # Tests for `Precommand Modifiers'=0A= #=0A= - - $ZTST_testdir/../Src/zsh -fc "[[ \$0 =3D = \"-$ZTST_testdir/../Src/zsh\" ]]"=0A= + - $ZTST_testdir/$ZTST_exe -fc "[[ \$0 =3D = \"-$ZTST_testdir/$ZTST_exe\" ]]"=0A= 0:`-' precommand modifier=0A= =0A= echo f*=0A= @@ -441,18 +441,18 @@=0A= 0:Handling of &&'s and ||'s with a for loop in between=0A= >no=0A= =0A= - $ZTST_testdir/../Src/zsh -f unmatched_quote.txt=0A= + $ZTST_testdir/$ZTST_exe -f unmatched_quote.txt=0A= 1:Parse error with file causes non-zero exit status=0A= ?unmatched_quote.txt:2: unmatched '=0A= =0A= - $ZTST_testdir/../Src/zsh -f 5.7=0A= =0A= - typeset -A hash=0A= - hash=3D(one 1)=0A= + typeset -A h=0A= + h=3D(one 1)=0A= h+=3Dstring=0A= [[ $h[@] =3D=3D string ]]=0A= 0:add scalar to association=0A= Index: Test/D02glob.ztst=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= RCS file: /cvsroot/zsh/zsh/Test/D02glob.ztst,v=0A= retrieving revision 1.11=0A= diff -u -r1.11 D02glob.ztst=0A= --- Test/D02glob.ztst 21 Jun 2005 08:58:46 -0000 1.11=0A= +++ Test/D02glob.ztst 15 Aug 2005 15:22:36 -0000=0A= @@ -6,7 +6,7 @@=0A= mkdir glob.tmp/dir3/subdir=0A= : >glob.tmp/{,{dir1,dir2}/}{a,b,c}=0A= =0A= - globtest () { $ZTST_testdir/../Src/zsh -f $ZTST_srcdir/../Misc/$1 }=0A= + globtest () { $ZTST_testdir/$ZTST_exe -f $ZTST_srcdir/../Misc/$1 }=0A= =0A= regress_absolute_path_and_core_dump() {=0A= local absolute_dir=3D$(cd glob.tmp && pwd -P)=0A= Index: Test/Y01completion.ztst=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= RCS file: /cvsroot/zsh/zsh/Test/Y01completion.ztst,v=0A= retrieving revision 1.4=0A= diff -u -r1.4 Y01completion.ztst=0A= --- Test/Y01completion.ztst 11 Mar 2004 19:14:42 -0000 1.4=0A= +++ Test/Y01completion.ztst 15 Aug 2005 15:22:39 -0000=0A= @@ -5,7 +5,7 @@=0A= . $ZTST_srcdir/comptest=0A= mkdir comp.tmp=0A= cd comp.tmp=0A= - comptestinit -z $ZTST_testdir/../Src/zsh &&=0A= + comptestinit -z $ZTST_testdir/$ZTST_exe &&=0A= {=0A= mkdir dir1 &&=0A= mkdir dir2 &&=0A= Index: Test/Y02compmatch.ztst=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= RCS file: /cvsroot/zsh/zsh/Test/Y02compmatch.ztst,v=0A= retrieving revision 1.3=0A= diff -u -r1.3 Y02compmatch.ztst=0A= --- Test/Y02compmatch.ztst 11 Mar 2004 19:14:42 -0000 1.3=0A= +++ Test/Y02compmatch.ztst 15 Aug 2005 15:22:40 -0000=0A= @@ -15,7 +15,7 @@=0A= . $ZTST_srcdir/comptest=0A= mkdir match.tmp=0A= cd match.tmp=0A= - comptestinit -z $ZTST_testdir/../Src/zsh &&=0A= + comptestinit -z $ZTST_testdir/$ZTST_exe &&=0A= {=0A= list1=3D(IndianRed IndianRed2 IndianRed3 IndianRed4)=0A= test_code () {=0A= Index: Test/Y03arguments.ztst=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= RCS file: /cvsroot/zsh/zsh/Test/Y03arguments.ztst,v=0A= retrieving revision 1.6=0A= diff -u -r1.6 Y03arguments.ztst=0A= --- Test/Y03arguments.ztst 11 Mar 2004 19:14:42 -0000 1.6=0A= +++ Test/Y03arguments.ztst 15 Aug 2005 15:22:41 -0000=0A= @@ -5,7 +5,7 @@=0A= . $ZTST_srcdir/comptest=0A= mkdir comp.tmp=0A= cd comp.tmp=0A= - comptestinit -z $ZTST_testdir/../Src/zsh &&=0A= + comptestinit -z $ZTST_testdir/$ZTST_exe &&=0A= {=0A= comptesteval 'compdef _tst tst'=0A= tst_arguments () { comptesteval "_tst () { _arguments = ${${(@qq)*}} }" }=0A= Index: Test/ztst.zsh=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= RCS file: /cvsroot/zsh/zsh/Test/ztst.zsh,v=0A= retrieving revision 1.22=0A= diff -u -r1.22 ztst.zsh=0A= --- Test/ztst.zsh 9 Aug 2005 06:51:40 -0000 1.22=0A= +++ Test/ztst.zsh 15 Aug 2005 15:22:43 -0000=0A= @@ -118,18 +118,33 @@=0A= # we should try and arrange that any test-specific cleanup=0A= # always gets called as well.=0A= ##trap 'print cleaning up...=0A= -##ZTST_cleanup' INT QUIT TERM=0A= +##ZTST_cleanup=0A= +##exit 2' INT QUIT TERM=0A= # Make sure it's clean now.=0A= rm -rf dummy.tmp *.tmp=0A= =0A= +# Exit from this test run. Print test result, clean up,=0A= +# and set exit status according to $ZTST_testfailed.=0A= +ZTST_exit() {=0A= + if [[ -n "$ZTST_unimplemented" ]]; then=0A= + print -r "$ZTST_testname: skipped ($ZTST_unimplemented)"=0A= + elif (( $ZTST_testfailed )); then=0A= + print -r "$ZTST_testname: test failed."=0A= + else=0A= + print -r "$ZTST_testname: all tests successful."=0A= + fi=0A= + ZTST_cleanup=0A= + exit $(( ZTST_testfailed ))=0A= +}=0A= +=0A= # Report failure. Note that all output regarding the tests goes to = stdout.=0A= # That saves an unpleasant mixture of stdout and stderr to sort out.=0A= ZTST_testfailed() {=0A= + ZTST_clearline=0A= print -r "Test $ZTST_testname failed: $1"=0A= if [[ -n $ZTST_message ]]; then=0A= print -r "Was testing: $ZTST_message"=0A= fi=0A= - print -r "$ZTST_testname: test failed."=0A= if [[ -n $ZTST_failmsg ]]; then=0A= print -r "The following may (or may not) help identifying the cause:=0A= $ZTST_failmsg"=0A= @@ -138,20 +153,40 @@=0A= return 1=0A= }=0A= =0A= +# Report a syntax error in the test file.=0A= +ZTST_syntaxerror() {=0A= + # Don't print the test-specific messages in this case=0A= + local ZTST_message=3D'' ZTST_failmsg=3D''=0A= + if [[ $1 =3D=3D *: ]]; then=0A= + ZTST_testfailed "$1=0A= +line $ZTST_curlineno: $ZTST_curline"=0A= + else=0A= + ZTST_testfailed "$1 (near line $ZTST_curlineno)"=0A= + fi=0A= +}=0A= +=0A= # Print messages if $ZTST_verbose is non-empty=0A= ZTST_verbose() {=0A= local lev=3D$1=0A= shift=0A= [[ -n $ZTST_verbose && $ZTST_verbose -ge $lev ]] && print -r -- $* >&8=0A= }=0A= +=0A= +# Print progress indicator if not verbose...=0A= ZTST_hashmark() {=0A= [[ ZTST_verbose -le 0 && -t 8 ]] && print -nu8 = ${(pl:SECONDS::\#::\#\r:)}=0A= (( SECONDS > COLUMNS+1 && (SECONDS -=3D COLUMNS) ))=0A= }=0A= +# ...and clear it again.=0A= +ZTST_clearline() {=0A= + if [[ ZTST_verbose -le 0 && -t 8 ]]; then=0A= + print -nu8 ${(l:COLUMNS:)}'\r'=0A= + fi=0A= +}=0A= =0A= if [[ ! -r $ZTST_testname ]]; then=0A= ZTST_testfailed "can't read test file."=0A= - exit 1=0A= + ZTST_exit=0A= fi=0A= =0A= exec 8>&1=0A= @@ -159,6 +194,8 @@=0A= =0A= # The current line read from the test file.=0A= ZTST_curline=3D''=0A= +# The current line number=0A= +integer ZTST_curlineno=3D0=0A= # The current section being run=0A= ZTST_cursect=3D''=0A= =0A= @@ -167,6 +204,7 @@=0A= ZTST_getline() {=0A= local IFS=3D=0A= while true; do=0A= + (( ZTST_curlineno++ ))=0A= read -r ZTST_curline <&9 || return 1=0A= [[ $ZTST_curline =3D=3D \#* ]] || return 0=0A= done=0A= @@ -174,19 +212,17 @@=0A= =0A= # Get the name of the section. It may already have been read into=0A= # $curline, or we may have to skip some initial comments to find it.=0A= -# If argument present, it's OK to skip the reset of the current section,=0A= +# If argument present, it's OK to skip the rest of the current section,=0A= # so no error if we find garbage.=0A= ZTST_getsect() {=0A= local match mbegin mend=0A= =0A= while [[ $ZTST_curline !=3D '%'(#b)([[:alnum:]]##)* ]]; do=0A= - ZTST_getline || return 1=0A= - [[ $ZTST_curline =3D [[:blank:]]# ]] && continue=0A= - if [[ $# -eq 0 && $ZTST_curline !=3D '%'[[:alnum:]]##* ]]; then=0A= - ZTST_testfailed "bad line found before or after section:=0A= -$ZTST_curline"=0A= - exit 1=0A= + if [[ $# -eq 0 && $ZTST_curline !=3D [[:blank:]]# ]]; then=0A= + ZTST_syntaxerror "bad line found before or after section:"=0A= + ZTST_exit=0A= fi=0A= + ZTST_getline || return 1=0A= done=0A= # have the next line ready waiting=0A= ZTST_getline=0A= @@ -234,7 +270,7 @@=0A= ;;=0A= ('?') fn=3D$ZTST_err=0A= ;;=0A= - (*) ZTST_testfailed "bad redir operator: $char"=0A= + (*) ZTST_syntaxerror "bad redir operator: $char"=0A= return 1=0A= ;;=0A= esac=0A= @@ -268,7 +304,7 @@=0A= ZTST_execchunk >/dev/null || [[ -n $1 ]] || {=0A= [[ -n "$ZTST_unimplemented" ]] ||=0A= ZTST_testfailed "non-zero status from preparation code:=0A= -$ZTST_code" && return 0=0A= +$ZTST_code" && return 1=0A= }=0A= done=0A= }=0A= @@ -280,7 +316,8 @@=0A= diff_out=3D$(diff "$@")=0A= diff_ret=3D"$?"=0A= if [[ "$diff_ret" !=3D "0" ]]; then=0A= - echo "$diff_out"=0A= + ZTST_clearline=0A= + print -r -- "$diff_out"=0A= fi=0A= =0A= return "$diff_ret"=0A= @@ -288,6 +325,9 @@=0A= =0A= ZTST_test() {=0A= local last match mbegin mend found=0A= + # Make sure messages are reset upon exit from this function=0A= + # to keep ZTST_testfailed output correct=0A= + local ZTST_message ZTST_failmsg=0A= =0A= while true; do=0A= rm -f $ZTST_in $ZTST_out $ZTST_err=0A= @@ -323,8 +363,7 @@=0A= ZTST_flags=3D$match[2]=0A= ZTST_message=3D${match[3]:+${match[3][2,-1]}}=0A= else=0A= - ZTST_testfailed "expecting test status at:=0A= -$ZTST_curline"=0A= + ZTST_syntaxerror "expecting test status at:"=0A= return 1=0A= fi=0A= ZTST_getline=0A= @@ -344,8 +383,7 @@=0A= ZTST_getline=0A= found=3D1=0A= ;;=0A= - (*) ZTST_testfailed "bad line in test block:=0A= -$ZTST_curline"=0A= + (*) ZTST_syntaxerror "bad line in test block:"=0A= return 1=0A= ;;=0A= esac=0A= @@ -393,9 +431,6 @@=0A= done=0A= =0A= ZTST_verbose 2 "ZTST_test: all tests successful"=0A= -=0A= - # reset message to keep ZTST_testfailed output correct=0A= - ZTST_message=3D''=0A= }=0A= =0A= =0A= @@ -403,7 +438,7 @@=0A= typeset -A ZTST_sects=0A= ZTST_sects=3D(prep 0 test 0 clean 0)=0A= =0A= -print "$ZTST_testname: starting."=0A= +print -r "$ZTST_testname: starting."=0A= =0A= # Now go through all the different sections until the end.=0A= # prep section may set ZTST_unimplemented, in this case the actual=0A= @@ -414,16 +449,16 @@=0A= case $ZTST_cursect in=0A= (prep) if (( ${ZTST_sects[prep]} + ${ZTST_sects[test]} + \=0A= ${ZTST_sects[clean]} )); then=0A= - ZTST_testfailed "\`prep' section must come first"=0A= - exit 1=0A= + ZTST_syntaxerror "\`prep' section must come first"=0A= + ZTST_exit=0A= fi=0A= - ZTST_prepclean=0A= + ZTST_prepclean || ZTST_exit=0A= ZTST_sects[prep]=3D1=0A= ;;=0A= (test)=0A= if (( ${ZTST_sects[test]} + ${ZTST_sects[clean]} )); then=0A= - ZTST_testfailed "bad placement of \`test' section"=0A= - exit 1=0A= + ZTST_syntaxerror "bad placement of \`test' section"=0A= + ZTST_exit=0A= fi=0A= # careful here: we can't execute ZTST_test before || or &&=0A= # because that affects the behaviour of traps in the tests.=0A= @@ -433,22 +468,16 @@=0A= ;;=0A= (clean)=0A= if (( ${ZTST_sects[test]} =3D=3D 0 || ${ZTST_sects[clean]} )); then=0A= - ZTST_testfailed "bad use of \`clean' section"=0A= + ZTST_syntaxerror "bad use of \`clean' section"=0A= else=0A= ZTST_prepclean 1=0A= ZTST_sects[clean]=3D1=0A= fi=0A= ZTST_skipok=3D=0A= ;;=0A= - *) ZTST_testfailed "bad section name: $ZTST_cursect"=0A= + (*) ZTST_syntaxerror "bad section name: $ZTST_cursect"=0A= ;;=0A= esac=0A= done=0A= =0A= -if [[ -n "$ZTST_unimplemented" ]]; then=0A= - print "$ZTST_testname: skipped ($ZTST_unimplemented)"=0A= -elif (( ! $ZTST_testfailed )); then=0A= - print "$ZTST_testname: all tests successful."=0A= -fi=0A= -ZTST_cleanup=0A= -exit $(( ZTST_testfailed ))=0A= +ZTST_exit=0A= ------=_NextPart_000_018D_01C5A1C3.85E996A0--