PATCH: 3.1.6-pws-11: basic tests

PATCH: 3.1.6-pws-11: basic tests

[Peter Stephenson]

Here are some tests for basic shell syntax, etc., corresponding to the
`Shell Grammar' texinfo node.  The tests now have numbers in front so that
they will be run in a reasonably sensible order, so that the old
Test/cd.ztst can be removed after applying this.  The driver script has
been tweaked a little, too.

A few issues turned up during this.  I have not looked into these in much
detail.

First, the test that runs this
  { cd /NonExistentDirectory >&/dev/null; } || print false
fails if the semicolon is missing, even though the same test usually works
in an interactive shell or after zsh -fc.  Other shells require the
semicolon, but zsh usually doesn't, and it needs to be handled
consistently.

Second, `select' loops appear to try to open the input from the terminal
even in a non-interactive shell.  According to the manual page, it should
always use stdin.  One or maybe both of these is wrong.  In any case it's
impossible to run a simple non-interactive test for it.

Third, alternate forms of complex commands with braces are described in the
manual as just requiring a list before the opening brace.  This would
allow things like
  if true
  {
    print true
  }
which doesn't work.  Actually, it's already described in the FAQ that the
test has to be `clearly delimited', so in this case the manual just needs
to be clearer, and more explicit about what is allowed.

Fourth, which could be related to the first, this:
  case bravo {
    (alpha) print schmalpha
	    ;;
    (bravo) print schmavo
	    ;;
    (charlie) print schmarlie
	    ;;
  }
doesn't work in the tests although, again, it works interactively or from
zsh -c.

Index: Etc/zsh-development-guide
===================================================================
RCS file: /home/pws/CVSROOT/projects/zsh/Etc/zsh-development-guide,v
retrieving revision 1.4
diff -u -r1.4 zsh-development-guide
--- Etc/zsh-development-guide	1999/12/08 20:08:27	1.4
+++ Etc/zsh-development-guide	1999/12/13 18:44:48
@@ -47,7 +47,7 @@
   a wide range of human and artificial life, it is very difficult to
   test the shell thoroughly.  For this purpose, the Test subdirectory
   exists.  It consists of a driver script (ztst.zsh) and various test
-  files (*.ztst) in a format which is described in cd.ztst, which acts
+  files (*.ztst) in a format which is described in 50cd.ztst, which acts
   as a template.  It is designed to make it easy to provide input to
   chunks of shell code and to test the corresponding standard output,
   error output and exit status.
Index: Test/.distfiles
===================================================================
RCS file: /home/pws/CVSROOT/projects/zsh/Test/.distfiles,v
retrieving revision 1.1
diff -u -r1.1 .distfiles
--- Test/.distfiles	1999/12/07 22:18:16	1.1
+++ Test/.distfiles	1999/12/13 18:44:29
@@ -1,5 +1,5 @@
 DISTFILES_SRC='
     .cvsignore .distfiles Makefile.in
     ztst.zsh
-    cd.ztst
+    01grammar.ztst 02alias.ztst 03quoting.ztst 50cd.ztst
 '
Index: Test/01grammar.ztst
===================================================================
RCS file: 01grammar.ztst
diff -N 01grammar.ztst
--- /dev/null	Tue May  5 21:32:27 1998
+++ 01grammar.ztst	Sun Dec 12 18:39:04 1999
@@ -0,0 +1,288 @@
+#
+# This file contains tests corresponding to the `Shell Grammar' texinfo node.
+#
+
+%prep
+
+  mkdir basic.tmp && cd basic.tmp
+  touch foo bar
+
+%test
+#
+# Tests for `Simple Commands and Pipelines'
+#
+  echo foo | cat | sed 's/foo/bar/'
+0:Basic pipeline handling
+>bar
+
+  false | true
+0:Exit status of pipeline with builtins (true)
+
+  true | false
+1:Exit status of pipeline with builtins (false)
+
+  fn() { local foo; read foo; print $foo; }
+  coproc fn
+  print -p coproc test output
+  read -p bar
+  print $bar
+0:Basic coprocess handling
+>coproc test output
+
+  true | false && print true || print false
+0:Basic sublist (i)
+>false
+
+  false | true && print true || print false
+0:Basic sublist (ii)
+>true
+
+  (cd /NonExistentDirectory >&/dev/null) || print false
+0:Basic subshell list with error
+>false
+
+# Can someone convince me the following is really supposed to fail
+# without the semicolon present?
+  { cd /NonExistentDirectory >&/dev/null; } || print false
+0:Basic current shell list with error
+>false
+
+#
+# Tests for `Precommand Modifiers'
+#
+  - sh -c 'echo $0'
+0:`-' precommand modifier
+>-sh
+
+  echo f*
+  noglob echo f*
+0:`noglob' precommand modifier
+>foo
+>f*
+
+  (exec /bin/sh; echo bar)
+0:`exec' precommand modifier
+
+  cat() { echo Function cat executed; }
+  command cat && unfunction cat
+0:`command' precommand modifier
+<External command cat executed
+>External command cat executed
+
+  cd() { echo Not cd at all; }
+  builtin cd . && unfunction cd
+0:`builtin' precommand modifier
+
+#
+# Tests for `Complex Commands'
+#
+
+  if true; then
+    print true-1
+  elif true; then
+    print true-2
+  else
+    print false
+  fi
+0:`if ...' (i)
+>true-1
+
+  if false; then
+    print true-1
+  elif true; then
+    print true-2
+  else
+    print false
+  fi
+0:`if ...' (ii)
+>true-2
+
+  if false; then
+    print true-1
+  elif false; then
+    print true-2
+  else
+    print false
+  fi
+0:`if ...' (iii)
+>false
+
+  for name in word to term; do
+    print $name
+  done
+0:`for' loop
+>word
+>to
+>term
+
+  for (( name = 0; name < 3; name++ )); do
+    print $name
+  done
+0:arithmetic `for' loop
+>0
+>1
+>2
+
+  name=0
+  while (( name < 3 )); do
+    print $name
+    (( name++ ))
+  done
+0:`while' loop
+>0
+>1
+>2
+
+  name=0
+  until (( name == 3 )); do
+    print $name
+    (( name++ ))
+  done
+0:`until' loop
+>0
+>1
+>2
+
+  repeat 3 do
+    echo over and over
+  done
+0:`repeat' loop
+>over and over
+>over and over
+>over and over
+
+  word=Trinity
+  case $word in
+    Michaelmas) print 0
+                ;;
+    Hilary) print 1
+            ;;
+    Trinity) print 2
+             ;;
+    *) print 3
+       ;;
+  esac
+0:`case' loop, old syntax
+>2
+
+  word=Trinity
+  case $word in
+    (Michaelmas) print 0
+                ;;
+    (Hilary) print 1
+            ;;
+    (Trinity) print 2
+             ;;
+    (*) print 3
+       ;;
+  esac
+0:`case' loop, new syntax
+>2
+
+## This doesn't work, because zsh tries to read from the terminal
+## even in a non-interactive shell.  The manual implies it always reads
+## from stdin, even in an interactive shell.
+#  PS3="input> "
+#  select name in one two three; do
+#    print $name
+#  done
+#0:`select' loop
+#<2
+#>1) one     2) two     3) three   
+#>input> 
+#>two
+
+  function name1 name2 () { print This is $0; }
+  name2
+  name1 name2() { print This is still $0; }
+  name2
+0:`function' keyword
+>This is name2
+>This is still name2
+
+  (time cat) >&/dev/null
+0:`time' keyword (status only)
+
+  if [[ -f foo && -d . && -n $ZTST_testdir ]]; then
+    true
+  else
+    false
+  fi
+0:basic [[ ... ]] test
+
+#
+# Tests for `Alternate Forms For Complex Commands'
+#
+
+## I simply can't get these to work.
+## I suspect that the lists which are allowed here are only syntactically
+## special tests.
+#  if true; { print true-1; } elif true; { print true-2; } else { false; }
+#  if false; { print true-1; } elif true; { print true-2; } else { false; }
+#  if false; { print true-1; } elif false; { print true-2; } else { false; }
+#0:Alternate `if' with braces
+## Are all those semicolons necessary?  If not, what are the rules?
+#>true-1
+#>true-2
+#>false
+
+  if true; print true
+0:Short form of `if'
+>true
+
+  for name ( word1 word2 word3 ) print $name
+0:Form of `for' with parentheses.
+>word1
+>word2
+>word3
+
+  for name in alpha beta gamma; print $name
+0:Short form of `for'
+>alpha
+>beta
+>gamma
+
+  for (( val = 2; val < 10; val *= val )) print $val
+0:Short arithmetic `for'
+>2
+>4
+
+  foreach name ( verbiage words periphrasis )
+    print $name
+  end
+0:Csh-like `for'
+>verbiage
+>words
+>periphrasis
+
+# see comment with braces used in if loops
+  val=0;
+  while (( val < 2 )) { print $((val++)); }
+0:Alternative `while'
+>0
+>1
+
+  val=2;
+  until (( val == 0 )) { print $((val--)); }
+0:Alternative `until'
+>2
+>1
+
+  repeat 3 print Hip hip hooray
+0:Short `repeat'
+>Hip hip hooray
+>Hip hip hooray
+>Hip hip hooray
+
+## Why doesn't this one work here?  It works from the command line
+## or with zsh -fc.
+#  case bravo {
+#    (alpha) print schmalpha
+#	    ;;
+#    (bravo) print schmavo
+#	    ;;
+#    (charlie) print schmarlie
+#	    ;;
+#  }
+#0:`case' with braces
+#>schmavo
Index: Test/02alias.ztst
===================================================================
RCS file: 02alias.ztst
diff -N 02alias.ztst
--- /dev/null	Tue May  5 21:32:27 1998
+++ 02alias.ztst	Sun Dec 12 18:48:27 1999
@@ -0,0 +1,23 @@
+%prep
+  alias foo=echo
+
+  alias -g bar=echo
+
+  alias '\bar=echo'
+
+%test
+  foo foo
+0:Basic aliasing
+>foo
+
+  bar bar
+0:Global aliasing
+>echo
+
+  \foo foo
+1:Not aliasing
+?ZTST_execchunk:2: command not found: foo
+
+  \bar \bar
+0:Aliasing with a backslash
+>bar
Index: Test/03quoting.ztst
===================================================================
RCS file: 03quoting.ztst
diff -N 03quoting.ztst
--- /dev/null	Tue May  5 21:32:27 1998
+++ 03quoting.ztst	Mon Dec 13 18:04:30 1999
@@ -0,0 +1,26 @@
+%test
+  print 'single  quotes'  "double  quotes"  `echo backquotes`
+0:Simple use of quotes
+>single  quotes double  quotes backquotes
+
+  foo=text
+  print -r '$foo\\\' "$foo\$foo\\\"\``echo bar`\`\"" `print -r $foo\\\``
+0:Quoting inside quotes
+>$foo\\\ text$foo\"`bar`" text`
+
+  print -r $'\'ut queant laxis\'\n"resonare fibris"'
+0:$'-style quotes
+>'ut queant laxis'
+>"resonare fibris"
+
+  print -r ''''
+  setopt rcquotes
+# We need to set rcquotes here for the next example since it is
+# needed while parsing.
+0:No RC_QUOTES with single quotes
+>
+
+  print -r ''''
+  unsetopt rcquotes
+0:Yes RC_QUOTES with single quotes
+>'
Index: Test/50cd.ztst
===================================================================
RCS file: 50cd.ztst
diff -N 50cd.ztst
--- /dev/null	Tue May  5 21:32:27 1998
+++ 50cd.ztst	Mon Dec 13 17:54:36 1999
@@ -0,0 +1,104 @@
+# This file serves as a model for how to write tests, so is more heavily
+# commented that the others.  All tests are run in the Test subdirectory
+# of the distribution, which must be writable.  They should end with
+# the suffix `.ztst': this is not required by the test harness itself,
+# but it is needed by the Makefile to run all the tests.
+
+# Blank lines with no other special meaning (e.g. separating chunks of
+# code) and all those with a `#' in the first column are ignored.
+
+# All section names start with a % in the first column.  The names
+# must be in the expected order, though not all sections are required.
+# The sections are %prep (preparatory setup:  code executed should return
+# status 0, but no other tests are performed), %test (the main tests), and
+# %clean (to cleanup: the code is simply unconditionally executed).
+#
+# Literal shell code to be evaluated must be indented with any number
+# of spaces and/or tabs, to differentiate it from tags with a special
+# meaning to the test harness.  Note that this is true even in sections
+# where there are no such tags.  Also note that file descriptor 9
+# is reserved for input from the test script; if ZTST_verbose is set,
+# output is sent to the original stdout via fd 8.  Option settings
+# are preserved between the execution of different code chunks;
+# initially, all standard zsh options (the effect of `emulate -R zsh')
+# are set.
+
+%prep
+# This optional section prepares the test, creating directories and files
+# and so on.  Chunks of code are separated by blank lines (which is not
+# necessary before the end of the section); each chunk of code is evaluated
+# in one go and must return status 0, or the preparation is deemed to have
+# failed and the test ends with an appropriate error message.  Standard
+# output from this section is redirected to /dev/null, but standard error
+# is not redirected.
+#
+# Tests should use subdirectories ending in `.tmp'.  These will be
+# removed with all the contents even if the test is aborted.
+ mkdir cdtst.tmp cdtst.tmp/real cdtst.tmp/sub
+
+ ln -s ../real cdtst.tmp/sub/fake
+
+ mydir=$PWD
+
+%test
+# This is where the tests are run.  It consists of blocks separated
+# by blank lines.  Each block has the same format and there may be any
+# number of them.  It consists of indented code, plus optional sets of lines
+# beginning '<', '>' and '?' which may appear in any order.  These correspond
+# to stdin (fed to the code), stdout (compared with code output) and
+# stderr (compared with code error output) respectively.  These subblocks
+# may occur in any order, but the natural one is: code, stdin, stdout,
+# stderr.
+#
+# The rules for '<', '>' and '?' lines are the same: only the first
+# character is stripped, with subsequent whitespace being significant;
+# lines are not subject to any substitution unless the `q' flags (see
+# below) is set.
+#
+# Each chunk of indented code is to be evaluated in one go and is to
+# be followed by a line starting (in the first column) with
+# the expected status returned by the code when run, or - if it is
+# irrelevant.  An optional set of single-letter flags follows the status
+# or -.  The following are understood:
+#   d   Don't diff stdout against the expected stdout.
+#   D   Don't diff stderr agsinst the expected stderr.
+#   q   All redirection lines given in the test script (not the lines
+#       actually produced by the test) are subject to ordinary quoted shell
+#       expansion (i.e. not globbing).
+# This can be followed by a `:' and a message describing the
+# test, which will be printed if the test fails, along with a
+# description of the failure that occurred.  The `:' and message are
+# optional, but highly recommended.
+# Hence a complete status line looks something like:
+#  0dDq:Checking whether the world will end with a bang or a whimper
+#
+# If either or both of the '>' and '?' sets of lines is absent, it is
+# assumed the corresponding output should be empty and it is an error if it
+# is not.  If '<' is empty, stdin is an empty (but opened) file.
+ cd cdtst.tmp/sub/fake &&
+ pwd &&
+ print $PWD
+0q:Preserving symbolic links in the current directory string
+>$mydir/cdtst.tmp/sub/fake
+>$mydir/cdtst.tmp/sub/fake
+
+ cd ../../.. &&
+ pwd &&
+ print $PWD
+0q:Changing directory up through symbolic links without following them
+>$mydir
+>$mydir
+
+ setopt chaselinks
+ cd cdtst.tmp/sub/fake &&
+ pwd &&
+ print $PWD
+0q:Resolving symbolic links with chaselinks set
+>$mydir/cdtst.tmp/real
+>$mydir/cdtst.tmp/real
+
+%clean
+# This optional section cleans up after the test, if necessary,
+# e.g. killing processes etc.  This is in addition to the removal of *.tmp
+# subdirectories.  This is essentially like %prep, except that status
+# return values are ignored.
Index: Test/cd.ztst
===================================================================
RCS file: /home/pws/CVSROOT/projects/zsh/Test/cd.ztst,v
retrieving revision 1.1
diff -u -r1.1 cd.ztst
--- Test/cd.ztst	1999/12/07 22:18:16	1.1
+++ Test/cd.ztst	1999/12/13 18:45:33
@@ -1,97 +0,0 @@
-# This file serves as a model for how to write tests, so is more heavily
-# commented that the others.  All tests are run in the Test subdirectory
-# of the distribution, which must be writable.  They should end with
-# the suffix `.ztst': this is not required by the test harness itself,
-# but it is needed by the Makefile to run all the tests.
-
-# Blank lines with no other special meaning (e.g. separating chunks of
-# code) and all those with a `#' in the first column are ignored.
-
-# All section names start with a % in the first column.  The names
-# must be in the expected order, though not all sections are required.
-# The sections are %prep (preparatory setup:  code executed should return
-# status 0, but no other tests are performed), %test (the main tests), and
-# %clean (to cleanup: the code is simply unconditionally executed).
-#
-# Literal shell code to be evaluated must be indented with any number
-# of spaces and/or tabs, to differentiate it from tags with a special
-# meaning to the test harness.  Note that this is true even in sections
-# where there are no such tags.  Also note that file descriptor 9
-# is reserved for input from the test script; if ZTST_verbose is set,
-# output is sent to the original stdout via fd 8.  Option settings
-# are preserved between the execution of different code chunks;
-# initially, all standard zsh options (the effect of `emulate -R zsh')
-# are set.
-
-%prep
-# This optional section prepares the test, creating directories and files
-# and so on.  Chunks of code are separated by blank lines (which is not
-# necessary before the end of the section); each chunk of code is evaluated
-# in one go and must return status 0, or the preparation is deemed to have
-# failed and the test ends with an appropriate error message.  Standard
-# output from this section is redirected to /dev/null, but standard error
-# is not redirected.
-#
-# Tests should use subdirectories ending in `.tmp'.  These will be
-# removed with all the contents even if the test is aborted.
- mkdir cdtst.tmp cdtst.tmp/real cdtst.tmp/sub
-
- ln -s ../real cdtst.tmp/sub/fake
-
- mydir=$PWD
-
-%test
-# This is where the tests are run.  It consists of blocks separated
-# by blank lines.  Each block has the same format and there may be any
-# number of them.  It consists of indented code, plus optional sets of lines
-# beginning '<', '>' and '?' which may appear in any order.  These correspond
-# to stdin (fed to the code), stdout (compared with code output) and
-# stderr (compared with code error output) respectively.  These subblocks
-# may occur in any order, but the natural one is: code, stdin, stdout,
-# stderr.
-#
-# The rules for '<', '>' and '?' lines are the same: only the first
-# character is stripped, with subsequent whitespace being significant;
-# lines are subject to ordinary quoted shell expansion (i.e. not globbing).
-#
-# Each chunk of indented code is to be evaluated in one go and is to
-# be followed by a line starting (in the first column) with
-# the expected status returned by the code when run, or - if it is
-# irrelevant.  This can be followed by a `:' and a message describing the
-# test, which will be printed if the test fails, along with a
-# description of the failure that occurred.  The `:' and message are
-# optional, but highly recommended.
-#
-# If either or both of the '>' and '?' sets of lines is absent, it is
-# assumed the corresponding output should be empty and it is an error if it
-# is not.  If '<' is empty, stdin is an empty (but opened) file.
-#
-# TODO: flags to the post-code status line indicating that diffs are
-# not to be performed.
- cd cdtst.tmp/sub/fake &&
- pwd &&
- print $PWD
-0:Preserving symbolic links in the current directory string
->$mydir/cdtst.tmp/sub/fake
->$mydir/cdtst.tmp/sub/fake
-
- cd ../../.. &&
- pwd &&
- print $PWD
-0:Changing directory up through symbolic links without following them
->$mydir
->$mydir
-
- setopt chaselinks
- cd cdtst.tmp/sub/fake &&
- pwd &&
- print $PWD
-0:Resolving symbolic links with chaselinks set
->$mydir/cdtst.tmp/real
->$mydir/cdtst.tmp/real
-
-%clean
-# This optional section cleans up after the test, if necessary,
-# e.g. killing processes etc.  This is in addition to the removal of *.tmp
-# subdirectories.  This is essentially like %prep, except that status
-# return values are ignored.
Index: Test/ztst.zsh
===================================================================
RCS file: /home/pws/CVSROOT/projects/zsh/Test/ztst.zsh,v
retrieving revision 1.2
diff -u -r1.2 ztst.zsh
--- Test/ztst.zsh	1999/12/09 23:05:23	1.2
+++ Test/ztst.zsh	1999/12/13 18:48:02
@@ -14,7 +14,8 @@
 
 # Produce verbose messages if non-zero.
 # If 1, produce reports of tests executed; if 2, also report on progress.
-ZTST_verbose=0
+# Defined in such a way that any value from the environment is used.
+: ${ZTST_verbose:=0}
 
 # We require all options to be reset, not just emulation options.
 # Unfortunately, due to the crud which may be in /etc/zshenv this might
@@ -42,19 +43,19 @@
 ZTST_testdir=$PWD
 ZTST_testname=$1
 
+: ${TMPPREFIX:=/tmp/zsh}
 # Temporary files for redirection inside tests.
-ZTST_in=${TMPPREFIX-:/tmp/zsh}.ztst.in.$$
+ZTST_in=${TMPPREFIX}.ztst.in.$$
 # hold the expected output
-ZTST_out=${TMPPREFIX-:/tmp/zsh}.ztst.out.$$
-ZTST_err=${TMPPREFIX-:/tmp/zsh}.ztst.err.$$
+ZTST_out=${TMPPREFIX}.ztst.out.$$
+ZTST_err=${TMPPREFIX}.ztst.err.$$
 # hold the actual output from the test
-ZTST_tout=${TMPPREFIX-:/tmp/zsh}.ztst.tout.$$
-ZTST_terr=${TMPPREFIX-:/tmp/zsh}.ztst.terr.$$
+ZTST_tout=${TMPPREFIX}.ztst.tout.$$
+ZTST_terr=${TMPPREFIX}.ztst.terr.$$
 
 ZTST_cleanup() {
   cd $ZTST_testdir
-  rm -rf $ZTST_testdir/dummy.tmp $ZTST_testdir/*.tmp \
-         $ZTST_in $ZTST_out $ZTST_err $ZTST_tout $ZTST_terr
+  rm -rf $ZTST_testdir/dummy.tmp $ZTST_testdir/*.tmp ${TMPPREFIX}.ztst*$$
 }
 
 # This cleanup always gets performed, even if we abort.  Later,
@@ -72,6 +73,7 @@
   if [[ -n $ZTST_message ]]; then
     print "Was testing: $ZTST_message"
   fi
+  print "$ZTST_testname: test failed."
   ZTST_cleanup
   exit 1
 }
@@ -80,7 +82,7 @@
 ZTST_verbose() {
   local lev=$1
   shift
-  [[ -n $ZTST_verbose && $ZTST_verbose -ge $lev ]] && print $* >&8
+  [[ -n $ZTST_verbose && $ZTST_verbose -ge $lev ]] && print -- $* >&8
 }
 
 [[ ! -r $ZTST_testname ]] && ZTST_testfailed "can't read test file."
@@ -98,7 +100,7 @@
 ZTST_getline() {
   local IFS=
   while true; do
-    read ZTST_curline <&9 || return 1
+    read -r ZTST_curline <&9 || return 1
     [[ $ZTST_curline == \#* ]] || return 0
   done
 }
@@ -145,7 +147,7 @@
 
 # Read in a piece for redirection.
 ZTST_getredir() {
-  local char=${ZTST_curline[1]}
+  local char=${ZTST_curline[1]} fn
   ZTST_redir=${ZTST_curline[2,-1]}
   while ZTST_getline; do
     [[ $ZTST_curline[1] = $char ]] || break
@@ -154,6 +156,22 @@
   done
   ZTST_verbose 2 "ZTST_getredir: read redir for '$char':
 $ZTST_redir"
+
+case $char in
+  '<') fn=$ZTST_in
+       ;;
+  '>') fn=$ZTST_out
+       ;;
+  '?') fn=$ZTST_err
+       ;;
+   *)  ZTST_testfailed "bad redir operator: $char"
+       ;;
+esac
+if [[ $ZTST_flags = *q* ]]; then
+  print -r -- "${(e)ZTST_redir}" >>$fn
+else
+  print -r -- "$ZTST_redir" >>$fn
+fi
 }
 
 # Execute an indented chunk.  Redirections will already have
@@ -210,27 +228,24 @@
 	    fi
 	    ;;
 	[[:space:]]##[^[:space:]]*) ZTST_getchunk
-	  [[ $ZTST_curline != [-0-9]* ]] &&
-	  ZTST_testfailed "expecting test status at:
-$ZTST_curline"
-          ZTST_xstatus=$ZTST_curline
-	  if [[ $ZTST_curline == (#b)([^:]##):(*) ]]; then
+	  if [[ $ZTST_curline == (#b)([-0-9]##)([[:alpha:]]#)(:*)# ]]; then
 	    ZTST_xstatus=$match[1]
-	    ZTST_message=$match[2]
+	    ZTST_flags=$match[2]
+	    ZTST_message=${match[3]:+${match[3][2,-1]}}
+	  else
+	    ZTST_testfailed "expecting test status at:
+$ZTST_curline"
 	  fi
 	  ZTST_getline
 	  found=1
 	  ;;
 	'<'*) ZTST_getredir
-	  print -r "${(e)ZTST_redir}" >>$ZTST_in
 	  found=1
 	  ;;
 	'>'*) ZTST_getredir
-          print -r "${(e)ZTST_redir}" >>$ZTST_out
 	  found=1
 	  ;;
 	'?'*) ZTST_getredir
-	  print -r "${(e)ZTST_redir}" >>$ZTST_err
 	  found=1
 	  ;;
 	*) ZTST_testfailed "bad line in test block:
@@ -241,8 +256,7 @@
 
     # If we found some code to execute...
     if [[ -n $ZTST_code ]]; then
-      ZTST_verbose 1 "Running test:
-$ZTST_message"
+      ZTST_verbose 1 "Running test: $ZTST_message"
       ZTST_verbose 2 "ZTST_test: expecting status: $ZTST_xstatus"
 
       ZTST_execchunk <$ZTST_in >$ZTST_tout 2>$ZTST_terr
@@ -250,7 +264,9 @@
       # First check we got the right status, if specified.
       if [[ $ZTST_xstatus != - && $ZTST_xstatus != $ZTST_status ]]; then
 	ZTST_testfailed "bad status $ZTST_status, expected $ZTST_xstatus from:
-$ZTST_code"
+$ZTST_code${$(<$ZTST_terr):+
+Error output:
+$(<$ZTST_terr)}"
       fi
 
       ZTST_verbose 2 "ZTST_test: test produced standard output:
@@ -259,11 +275,13 @@
 $(<$ZTST_terr)"
 
       # Now check output and error.
-      if ! diff -c $ZTST_out $ZTST_tout; then
+      if [[ $ZTST_flags != *d* ]] && ! diff -c $ZTST_out $ZTST_tout; then
 	ZTST_testfailed "output differs from expected as shown above for:
-$ZTST_code"
+$ZTST_code${$(<$ZTST_terr):+
+Error output:
+$(<$ZTST_terr)}"
       fi
-      if ! diff -c $ZTST_err $ZTST_terr; then
+      if [[ $ZTST_flags != *D* ]] && ! diff -c $ZTST_err $ZTST_terr; then
 	ZTST_testfailed "error output differs from expected as shown above for:
 $ZTST_code"
       fi

-- 
Peter Stephenson <pws@pwstephenson.fsnet.co.uk>

PATCH: Re: PATCH: 3.1.6-pws-11: basic tests

[Peter Stephenson]

Peter Stephenson wrote:
> Here are some tests for basic shell syntax, etc.
> ...
> A few issues turned up during this.  I have not looked into these in much
> detail.

I've now fixed these.

> First, the test that runs this
>   { cd /NonExistentDirectory >&/dev/null; } || print false
> fails if the semicolon is missing, even though the same test usually works
> in an interactive shell or after zsh -fc.

This traces to bad handling of values when retrieving the $options hash for
options with negative number, which is supposed to mean negate.  The
problem was triggered by braceexpand, a negated alias for ignorebraces.

> Second, `select' loops appear to try to open the input from the terminal
> even in a non-interactive shell.  According to the manual page, it should
> always use stdin.  One or maybe both of these is wrong.  In any case it's
> impossible to run a simple non-interactive test for it.

The test for whether to use the terminal was different from the test for
whether to use zle, so sometimes the terminal was used instead of stdin
even when zle wasn't.  I've used the more restrictive test in both cases
and altered the manual.

> Third, alternate forms of complex commands with braces are described in the
> manual as just requiring a list before the opening brace.  This would
> allow things like
>   if true
>   {
>     print true
>   }
> which doesn't work.  Actually, it's already described in the FAQ that the
> test has to be `clearly delimited', so in this case the manual just needs
> to be clearer, and more explicit about what is allowed.

I've made the manual much clearer about this.

> Fourth, which could be related to the first, this:
>   case bravo {
>     (alpha) print schmalpha
> 	    ;;
>     (bravo) print schmavo
> 	    ;;
>     (charlie) print schmarlie
> 	    ;;
>   }
> doesn't work in the tests although, again, it works interactively or from
> zsh -c.

This is the same problem with retrieving the braceexpand element of $options.

Index: Doc/Zsh/grammar.yo
===================================================================
RCS file: /home/pws/CVSROOT/projects/zsh/Doc/Zsh/grammar.yo,v
retrieving revision 1.1.1.1
diff -u -r1.1.1.1 grammar.yo
--- Doc/Zsh/grammar.yo	1999/11/28 17:42:27	1.1.1.1
+++ Doc/Zsh/grammar.yo	2000/01/11 21:21:20
@@ -205,9 +205,10 @@
 where var(term) is one or more newline or tt(;) to terminate the var(word)s.
 Print the set of var(word)s, each preceded by a number.
 If the tt(in) var(word) is omitted, use the positional parameters.
-The tt(PROMPT3) prompt is printed and a line is read from standard
-input.  If this line consists of the number of one of the listed
-var(word)s, then the parameter var(name)
+The tt(PROMPT3) prompt is printed and a line is read from the line editor
+if the shell is interactive and that is active, or else standard input.
+If this line consists of the
+number of one of the listed var(word)s, then the parameter var(name)
 is set to the var(word) corresponding to this number.
 If this line is empty, the selection list is printed again.
 Otherwise, the value of the parameter var(name) is set to null.
@@ -263,11 +264,26 @@
 removed in the future.  The versions in the previous section should be
 preferred instead.  The short versions below only work if var(sublist)
 is of the form `tt({) var(list) tt(})' or if the tt(SHORT_LOOPS)
-option is set.
+option is set.  In this case, the test part of the loop must also be
+suitably delimited, such as by `tt([[ ... ]])' or `tt((( ... )))', else
+the end of the test will not be recognized.
 
 startitem()
 item(tt(if) var(list) tt({) var(list) tt(}) [ tt(elif) var(list) tt({) var(list) tt(}) ] ... [ tt(else {) var(list) tt(}) ])(
-An alternate form of tt(if).
+An alternate form of tt(if).  The rules mean that
+
+example(if [[ -o ignorebraces ]] {
+  print yes
+})
+
+works, but
+
+example(if true {  # Does not work!
+  print yes
+}
+)
+
+does em(not), since the test is not suitably delimited.
 )
 item(tt(if) var(list) var(sublist))(
 A short form of the alternate `if'.
Index: Src/loop.c
===================================================================
RCS file: /home/pws/CVSROOT/projects/zsh/Src/loop.c,v
retrieving revision 1.3
diff -u -r1.3 loop.c
--- Src/loop.c	2000/01/04 22:04:35	1.3
+++ Src/loop.c	2000/01/11 21:15:05
@@ -154,7 +154,7 @@
     Forcmd node;
     char *str, *s;
     LinkNode n;
-    int i;
+    int i, usezle;
     FILE *inp;
     size_t more;
 
@@ -172,12 +172,13 @@
     lastval = 0;
     pushheap();
     cmdpush(CS_SELECT);
-    inp = fdopen(dup((SHTTY == -1) ? 0 : SHTTY), "r");
+    usezle = interact && SHTTY != -1 && isset(USEZLE);
+    inp = fdopen(dup(usezle ? SHTTY : 0), "r");
     more = selectlist(args, 0);
     for (;;) {
 	for (;;) {
 	    if (empty(bufstack)) {
-	    	if (interact && SHTTY != -1 && isset(USEZLE)) {
+	    	if (usezle) {
 		    int oef = errflag;
 
 		    isfirstln = 1;
Index: Src/Modules/parameter.c
===================================================================
RCS file: /home/pws/CVSROOT/projects/zsh/Src/Modules/parameter.c,v
retrieving revision 1.4
diff -u -r1.4 parameter.c
--- Src/Modules/parameter.c	1999/12/21 15:18:28	1.4
+++ Src/Modules/parameter.c	2000/01/11 20:54:06
@@ -838,8 +838,10 @@
 
     for (i = 0; i < optiontab->hsize; i++)
 	for (hn = optiontab->nodes[i]; hn; hn = hn->next) {
+	    int optno = ((Optname) hn)->optno, ison;
 	    pm.nam = hn->nam;
-	    pm.u.str = dupstring(opts[((Optname) hn)->optno] ? "on" : "off");
+	    ison = optno < 0 ? !opts[-optno] : opts[optno];
+	    pm.u.str = dupstring(ison ? "on" : "off");
 	    func((HashNode) &pm, flags);
 	}
 }

-- 
Peter Stephenson <pws@pwstephenson.fsnet.co.uk>