Multiple as-installed test suite failures under different circumstances with 5.1-test-1

[Axel Beckert <abe@deuxchevaux.org>]

Hi,

while zsh's test suite seems to pass at build time, when running zsh's
tests on the installed zsh (package) as done in Debian now for quite a
while (since 5.0.6, see
http://ci.debian.net/packages/z/zsh/unstable/amd64/), the test suite fails at two
different places dependening on where I run test suite:

In our Jenkins CI, at least three failures are shown on
http://jenkins.grml.org/view/Debian/job/zsh-binaries/architecture=amd64/209/console:

/tmp/adttmp.mhKVzv/Test/A02alias.ztst: starting.
This test hangs the shell when it fails...
*** /tmp/zsh.ztst.out.546 Tue Sep  8 21:07:04 2015
--- /tmp/zsh.ztst.tout.546    Tue Sep  8 21:07:04 2015
***************
*** 1,2 ****
  foo is a suffix alias for print
! foo: suffix alias
--- 1,2 ----
  foo is a suffix alias for print
! foo: alias
Test /tmp/adttmp.mhKVzv/Test/A02alias.ztst failed: output differs from expected as shown above for:
  alias -s foo=print
  type bar.foo; type -w bar.foo
  unalias -as
Was testing: unalias -as
/tmp/adttmp.mhKVzv/Test/A02alias.ztst: test failed.

/tmp/adttmp.mhKVzv/Test/A04redirect.ztst: starting.
*** /tmp/zsh.ztst.out.646		  Tue Sep  8 21:07:05 2015
--- /tmp/zsh.ztst.tout.646		  Tue Sep  8 21:07:05 2015
***************
*** 1,8 ****
  fn-two-heres () {
    cat <<x <<y
  foo
! x
! bar
  y
  }
  foo
--- 1,7 ----
  fn-two-heres () {
    cat <<x <<y
  foo
! xbar
  y
  }
  foo
***************
*** 11,16 ****
  bar
	cat <<x <<y
  foo
! x
! bar
  y
--- 10,14 ----
  bar
	cat <<x <<y
  foo
! xbar
  y
Test /tmp/adttmp.mhKVzv/Test/A04redirect.ztst failed: output differs from expected as shown above for:
  fn-two-heres() {
    cat <<-x <<-y
    foo
    x
    bar
    y
  }
  which -x2 fn-two-heres
  fn-two-heres
  eval "$(which -x2 fn-two-heres)"
  fn-two-heres
  print $functions[fn-two-heres]
Error output:
(eval):6: parse error near `\n'
Was testing: Two here-documents in a line are shown correctly.
/tmp/adttmp.mhKVzv/Test/A04redirect.ztst: test failed.

/tmp/adttmp.mhKVzv/Test/A05execution.ztst: starting.
Unable to change MONITOR option
This test takes 5 seconds to fail...
Skipping pipe leak test, requires MONITOR option
This test takes 3 seconds and hangs the shell when it fails...
[and then it hangs]

Running the test suite against my locally built and installed Debian
package of zsh 5.1-test-1, it passes all the tests above, but fails
later as follows:

/tmp/adttmp.ga8JYP/Test/V06parameter.ztst: starting.
*** /tmp/zsh.ztst.out.22805     Tue Sep  8 23:25:46 2015
--- /tmp/zsh.ztst.tout.22805    Tue Sep  8 23:25:46 2015
***************
*** 1,10 ****
  Started functrace.zsh
! 3 + +
  Inside function fn
! 2 + ./functrace.zsh:10 + ./functrace.zsh:5
  Inside autofn
! 2 + ./functrace.zsh:20 + ./autofn:0
  Inside autofn
! 2 + ./functrace.zsh:21 + ./autofn:0
  In sourced file
! 2 + ./functrace.zsh:22 + ./sourcedfile:0
--- 1,10 ----
  Started functrace.zsh
! 3 + zsh/parameter + zsh/parameter
  Inside function fn
! 2 + zsh/parameter + zsh/parameter
  Inside autofn
! 2 + zsh/parameter + zsh/parameter
  Inside autofn
! 2 + zsh/parameter + zsh/parameter
  In sourced file
! 2 + zsh/parameter + zsh/parameter
Test /tmp/adttmp.ga8JYP/Test/V06parameter.ztst failed: output differs from expected as shown above for:
  print 'print In sourced file
  print $LINENO + $functrace + $funcsourcetrace
  ' >sourcedfile
  print -r -- 'print Started functrace.zsh
  module_path=(./Modules)
  print $LINENO + $functrace + $funcsourcetrace
  :
  fn() {
    print Inside function $0
    print $LINENO + $functrace + $funcsourcetrace
  }
  :
  fn
  :
  fpath=(. $fpath)
  :
  echo '\''print Inside $0
    print $LINENO + $functrace + $funcsourcetrace
  '\'' >autofn
  :
  autoload autofn
  :
  autofn
  autofn
  . ./sourcedfile' >functrace.zsh
  $ZTST_testdir/../Src/zsh +Z -f ./functrace.zsh
Error output:
./functrace.zsh:3: failed to load module `zsh/parameter': ./Modules/zsh/parameter.so: cannot open shared object file: No such file or directory
./functrace.zsh:3: failed to load module `zsh/parameter': ./Modules/zsh/parameter.so: cannot open shared object file: No such file or directory
fn:2: failed to load module `zsh/parameter': ./Modules/zsh/parameter.so: cannot open shared object file: No such file or directory
fn:2: failed to load module `zsh/parameter': ./Modules/zsh/parameter.so: cannot open shared object file: No such file or directory
autofn:2: failed to load module `zsh/parameter': ./Modules/zsh/parameter.so: cannot open shared object file: No such file or directory
autofn:2: failed to load module `zsh/parameter': ./Modules/zsh/parameter.so: cannot open shared object file: No such file or directory
autofn:2: failed to load module `zsh/parameter': ./Modules/zsh/parameter.so: cannot open shared object file: No such file or directory
autofn:2: failed to load module `zsh/parameter': ./Modules/zsh/parameter.so: cannot open shared object file: No such file or directory
./sourcedfile:2: failed to load module `zsh/parameter': ./Modules/zsh/parameter.so: cannot open shared object file: No such file or directory
./sourcedfile:2: failed to load module `zsh/parameter': ./Modules/zsh/parameter.so: cannot open shared object file: No such file or directory
Was testing: Function tracing
/tmp/adttmp.ga8JYP/Test/V06parameter.ztst: test failed.

(It is possible that not all necessary files have been copied from the
zsh source directory to the directory from which the tests are run if
there are additional files necessary since 5.1 -- which passed these
tests at least on our Jenkins CI.)

		Kind regards, Axel
-- 
/~\  Plain Text Ribbon Campaign                   | Axel Beckert
\ /  Say No to HTML in E-Mail and News            | abe@deuxchevaux.org  (Mail)
 X   See http://www.nonhtmlmail.org/campaign.html | abe@noone.org (Mail+Jabber)
/ \  I love long mails: http://email.is-not-s.ms/ | http://abe.noone.org/ (Web)