zsh 5.1.1-test-1

zsh 5.1.1-test-1

[Peter Stephenson]

I've upoloaded a test version 5.1.1-test-1 preparatory to releasing
5.2.  You can find it at

http://www.zsh.org/pub/development/

and also as a tag in the repository.

I'm not expecting unexpected problems, but then logically I can't, so
let me know.

pws

Re: zsh 5.1.1-test-1

[Pascal Wittmann]

[-- Attachment #1: Type: text/plain, Size: 174 bytes --]

On 11/21/2015 07:35 PM, Peter Stephenson wrote:
> I'm not expecting unexpected problems, but then logically I can't, so
> let me know.

Builds and runs fine on NixOS.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: zsh 5.1.1-test-1

[Simon Ruderich]

[-- Attachment #1: Type: text/plain, Size: 1111 bytes --]

On Sat, Nov 21, 2015 at 06:35:14PM +0000, Peter Stephenson wrote:
> I've upoloaded a test version 5.1.1-test-1 preparatory to releasing
> 5.2.  You can find it at
>
> http://www.zsh.org/pub/development/
>
> and also as a tag in the repository.

Test suite runs fine on Debian sid.

However on startup with my zshrc I get the following error from
VCS_info:

    VCS_INFO_maxexports:7: array parameter zle_bracketed_paste created globally in function VCS_INFO_maxexports
    VCS_INFO_nvcsformats:13: array parameter msgs created globally in function VCS_INFO_nvcsformats

I reproduced this with the following zshrc:

    setopt warn_create_global
    autoload -Uz vcs_info
    vcs_info


Btw. it would be really nice if we could get signed releases
(signed git tag and signature files for the tarballs). Having to
download untrusted code and just running it is something I'd like
to avoid. Especially useful for distributions which provide those
sources to many users.

Regards
Simon
-- 
+ privacy is necessary
+ using gnupg http://gnupg.org
+ public key id: 0x92FEFDB7E44C32F9

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: zsh 5.1.1-test-1

[Bart Schaefer]

On Nov 21,  8:09pm, Simon Ruderich wrote:
}
}     VCS_INFO_maxexports:7: array parameter zle_bracketed_paste created globally in function VCS_INFO_maxexports

That's actually coming directly from Src/Zle/zle_main.c where there is a
call to setaparam("zle_bracketed_paste", ...) -- that function now issues
a warning if used at top-level, and should be replaced by assignaparam().

We're going to have to do a sweep of the sources for non-special variables
that are set with set?param() after dotfiles have been loaded, especially
various autoloaded nonspecial parameters from modules [see Src/module.c
add_autoparam()].

}     VCS_INFO_nvcsformats:13: array parameter msgs created globally in function VCS_INFO_nvcsformats

This one is coming from a "zstyle" call.  Another unexpected side-effect
of testing warn_create_global at a lower level.

Re: zsh 5.1.1-test-1

[Christian Heinrich]

[-- Attachment #1: Type: text/plain, Size: 612 bytes --]

On Sat, 2015-11-21 at 20:09 +0100, Simon Ruderich wrote:
> Btw. it would be really nice if we could get signed releases
> (signed git tag and signature files for the tarballs). Having to
> download untrusted code and just running it is something I'd like
> to avoid. Especially useful for distributions which provide those
> sources to many users.

I totally agree with Simon here. It's not much of a hassle to do that;
in fact, even signed commits are nice and very easy with git, just use
git commit -S + the commit.signingkey option:

git config --global user.signingkey <key>

Best 
Christian

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: zsh 5.1.1-test-1

[Peter Stephenson]

On Sat, 21 Nov 2015 20:09:08 +0100
Simon Ruderich <simon@ruderich.org> wrote:
> However on startup with my zshrc I get the following error from
> VCS_info:
> 
>     VCS_INFO_maxexports:7: array parameter zle_bracketed_paste created globally in function VCS_INFO_maxexports

That's an exception --- the variable so created is intended to be
global, so needs to be treated specially.

>     VCS_INFO_nvcsformats:13: array parameter msgs created globally in function VCS_INFO_nvcsformats

That's not --- msgs needs to be local within the function, so this is
the warning doing its job properly.

Thanks for these --- I'm sure there are more when people get around to
running their favourite distributed functions with the latest coe.

> Btw. it would be really nice if we could get signed releases
> (signed git tag and signature files for the tarballs). Having to
> download untrusted code and just running it is something I'd like
> to avoid. Especially useful for distributions which provide those
> sources to many users.

This has been mentioned before and I'm happy to go along with it if
someone who knows what they're doing wants to set it up / establish
ground rules.  I'll need to set up a gpg key as it doesn't like my
existing PGP key.

(The idea that a tag signed by me is somehow "safer" than anything else
on the master branch in the git repository is a bit far-fetched, but
that's a different issue; nothing wrong with using the state of the art
technology.)

pws

diff --git a/Functions/VCS_Info/VCS_INFO_nvcsformats b/Functions/VCS_Info/VCS_INFO_nvcsformats
index 581aa5a..203a86d 100644
--- a/Functions/VCS_Info/VCS_INFO_nvcsformats
+++ b/Functions/VCS_Info/VCS_INFO_nvcsformats
@@ -4,6 +4,7 @@
 
 setopt localoptions noksharrays NO_shwordsplit
 local c v rr
+local -a msgs
 
 if [[ $1 == '-preinit-' ]] ; then
     c='default'
diff --git a/Src/Zle/zle_main.c b/Src/Zle/zle_main.c
index 593d636..38427e8 100644
--- a/Src/Zle/zle_main.c
+++ b/Src/Zle/zle_main.c
@@ -2045,7 +2045,8 @@ setup_(UNUSED(Module m))
     bpaste = zshcalloc(3*sizeof(char *));
     bpaste[0] = ztrdup("\033[?2004h");
     bpaste[1] = ztrdup("\033[?2004l");
-    setaparam("zle_bracketed_paste", bpaste);
+    /* Intended to be global, no WARNCREATEGLOBAL check. */
+    assignaparam("zle_bracketed_paste", bpaste, 0);
 
     return 0;
 }

Re: zsh 5.1.1-test-1

[Simon Ruderich]

[-- Attachment #1.1: Type: text/plain, Size: 1813 bytes --]

On Sun, Nov 22, 2015 at 03:31:08PM +0000, Peter Stephenson wrote:
> This has been mentioned before and I'm happy to go along with it if
> someone who knows what they're doing wants to set it up / establish
> ground rules.  I'll need to set up a gpg key as it doesn't like my
> existing PGP key.

Hello,

Thanks for considering it.

The guide at [1] has all the necessary information to create a
secure GPG key. The important parts: Put the following in your
~/.gnupg/gpg.conf (or use the version from [2]):

    personal-digest-preferences SHA256
    cert-digest-algo SHA256
    default-preference-list SHA512 SHA384 SHA256 SHA224 AES256 AES192 AES CAST5 ZLIB BZIP2 ZIP Uncompressed

These config settings are important to prevent gpg from using
SHA-1, which might become insecure in the future.

Then run gpg --gen-key and accept the defaults (or change them as
you see fit; but the key should be >= 2048 bit).

Now you can sign all tarballs with gpg --armor --detach-sign and
tag the commits with git tag -s (add -u keyid if you have
multiple keys).

I attached a small patch which will take care of the signing of
the tarballs.

> (The idea that a tag signed by me is somehow "safer" than anything else
> on the master branch in the git repository is a bit far-fetched, but
> that's a different issue; nothing wrong with using the state of the art
> technology.)

The idea is not safer, but at least attributable to you. Same for
the tarball. It ensures that everybody gets the same, hopefully
trustable, version.

Regards
Simon

[1]: https://help.riseup.net/en/gpg-best-practices
[2]: https://raw.githubusercontent.com/ioerror/duraconf/master/configs/gnupg/gpg.conf
-- 
+ privacy is necessary
+ using gnupg http://gnupg.org
+ public key id: 0x92FEFDB7E44C32F9

[-- Attachment #1.2: 0001-Makefile.in-sign-tarballs-with-gpg.patch --]
[-- Type: text/x-diff, Size: 1001 bytes --]

From 42cbfc9e606250e3bf3d8d8930f06793429a925c Mon Sep 17 00:00:00 2001
Message-Id: <42cbfc9e606250e3bf3d8d8930f06793429a925c.1448350522.git.simon@ruderich.org>
From: Simon Ruderich <simon@ruderich.org>
Date: Tue, 24 Nov 2015 08:35:12 +0100
Subject: [PATCH] Makefile.in: sign tarballs with gpg

---
 Makefile.in | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/Makefile.in b/Makefile.in
index cb74e94..dc86264 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -167,6 +167,7 @@ $(DISTNAME).tar.gz: FORCE
 	echo '#define ZSH_PATCHLEVEL "'`cd $(sdir_top) && git describe --tags --long`'"' >$(DISTNAME)/Src/patchlevel.h.release
 	tar cf - $(DISTNAME) | gzip -9 > $@
 	rm -rf $(DISTNAME)
+	gpg --armor --detach-sign $@
 
 targz-doc: $(DISTNAME)-doc.tar.gz
 $(DISTNAME)-doc.tar.gz: FORCE
@@ -174,5 +175,6 @@ $(DISTNAME)-doc.tar.gz: FORCE
 	    $(MAKE) $(MAKEDEFS)
 	tar cf - $(DISTNAME) | gzip -9 > $@
 	rm -rf $(DISTNAME)
+	gpg --armor --detach-sign $@
 
 FORCE:
-- 
2.6.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Test/V01private.ztst skipped (was: zsh 5.1.1-test-1)

[Daniel Shahaf]

Peter Stephenson wrote on Sat, Nov 21, 2015 at 18:35:14 +0000:
> I'm not expecting unexpected problems, but then logically I can't, so
> let me know.

    % make test
    (eval):4: failed to load module `zsh/param/private': /home/daniel/src/zsh/Test/Modules/zsh/param/private.so: cannot open shared object file: No such file or directory
    ./V10private.ztst: skipped (can't load the zsh/param/private module for testing)

This is both before and after running 'make install'.  After 'git clean -fdx'
and rebuild/reinstall, it runs and passes.

Did I miss a memo about having to rerun Util/preconfig at some point?

Re: Test/V01private.ztst skipped (was: zsh 5.1.1-test-1)

[Bart Schaefer]

On Nov 21, 11:01pm, Daniel Shahaf wrote:
}
} Did I miss a memo about having to rerun Util/preconfig at some point?

I was grumbling about a related thing some weeks ago, wherein once a
module has been compiled in, it's very difficult to get the build to
stop trying to link it even though it has been removed.

I *think* the answer to your issue would be to remove config.modules
and allow it to be rebuilt; I don't think Util/preconfig has anything
directly to do with it.

Build-time module dependencies are a bit of a nightmare.

Re: Test/V01private.ztst skipped (was: zsh 5.1.1-test-1)

[Daniel Shahaf]

Bart Schaefer wrote on Sat, Nov 21, 2015 at 17:09:55 -0800:
> I *think* the answer to your issue would be to remove config.modules
> and allow it to be rebuilt; I don't think Util/preconfig has anything
> directly to do with it.

Thanks for the answer.  (I can't confirm it works because I didn't keep
a copy of the pre-'git clean' tree.)

Re: Test/V01private.ztst skipped (was: zsh 5.1.1-test-1)

[Bart Schaefer]

On Nov 21, 11:01pm, Daniel Shahaf wrote:
}
}     % make test
}     (eval):4: failed to load module `zsh/param/private': /home/daniel/src/zsh/Test/Modules/zsh/param/private.so: cannot open shared object file: No such file or directory

So obviously the 2>/dev/null is in the wrong place.

No response to my grumbling in workers/37086 ...


diff --git a/Test/V10private.ztst b/Test/V10private.ztst
index 6c38e39..cefeba9 100644
--- a/Test/V10private.ztst
+++ b/Test/V10private.ztst
@@ -2,10 +2,11 @@
 
 %prep
 
- if ! (zmodload zsh/param/private >/dev/null 2>/dev/null); then
+ if ! (zmodload zsh/param/private) >/dev/null 2>/dev/null; then
    ZTST_unimplemented="can't load the zsh/param/private module for testing"
+ else
+   zmodload zsh/param/private
  fi
- zmodload zsh/param/private
 
 %test
 

Re: Test/V01private.ztst skipped (was: zsh 5.1.1-test-1)

[Bart Schaefer]

On Nov 21,  5:14pm, Bart Schaefer wrote:
}
} So obviously the 2>/dev/null is in the wrong place.
} 
} No response to my grumbling in workers/37086 ...

I went through all the tests that use zmodload, forced zmodload to fail
by copying /dev/null on top of the corresponding .so file, and then
checked the output of the test.

The following patch makes all tests as consistent as context permits.
This replaces 37178.  "make check" passes all with these changes. 

One interesting point:  If zsh/datetime cannot be loaded, we don't
test any of the "features" functionality in V04features.ztst.  Should
this be considered failure rather than unimplemented?  I left it as
it was for now.


diff --git a/Test/C02cond.ztst b/Test/C02cond.ztst
index e9a596a..40bbf42 100644
--- a/Test/C02cond.ztst
+++ b/Test/C02cond.ztst
@@ -257,7 +257,7 @@ F:Failures in these cases do not indicate a problem in the shell.
 >status = 1
 
 # core dumps on failure
-  if zmodload -i zsh/regex 2>/dev/null; then
+  if zmodload zsh/regex 2>/dev/null; then
      echo >regex_test.sh 'if [[ $# = 1 ]]; then
 	if [[ $1 =~ /?[^/]+:[0-9]+:$ ]]; then
 	  :
@@ -268,7 +268,8 @@ F:Failures in these cases do not indicate a problem in the shell.
   fi
 0:regex tests shouldn't crash
 
-  (if zmodload -i zsh/regex 2>/dev/null; then
+  if zmodload zsh/regex 2>/dev/null; then
+   ( # subshell in case coredump test failed
     string="this has stuff in it"
     bad_regex=0
     if [[ $string =~ "h([a-z]*) s([a-z]*) " ]]; then
@@ -295,23 +296,26 @@ F:Failures in these cases do not indicate a problem in the shell.
     else
       print -r "regex failed to match '$string'"
     fi
+   )
   else
     # if it didn't load, tough, but not a test error
-    print OK
-  fi)
+    ZTST_skip="regexp library not found."
+  fi
 0:MATCH, MBEGIN, MEND, match, mbegin, mend
 >OK
 
-  (if zmodload -i zsh/regex 2>/dev/null; then
+  if zmodload zsh/regex 2>/dev/null; then
+   ( # subshell because regex module may dump core, see above
     if [[ a =~ a && b == b ]]; then
       print OK
     else
       print "regex =~ inverted following test"
     fi
+   )
   else
     # not a test error
-    print OK
-  fi)
+    ZTST_skip="regexp library not found."
+  fi
 0:regex infix operator should not invert following conditions
 >OK
 
diff --git a/Test/D07multibyte.ztst b/Test/D07multibyte.ztst
index dff2ec1..f95c06d 100644
--- a/Test/D07multibyte.ztst
+++ b/Test/D07multibyte.ztst
@@ -500,7 +500,7 @@
 # aren't quite double width, but the arithmetic is correct.
 # It appears just to be an effect of the font.
 
-  if zmodload -i zsh/regex 2>/dev/null; then
+  if zmodload zsh/regex 2>/dev/null; then
     [[ $'\ua0' =~ '^.$' ]] && print OK
     [[ $'\ua0' =~ $'^\ua0$' ]] && print OK
     [[ $'\ua0'X =~ '^X$' ]] || print OK
diff --git a/Test/V01zmodload.ztst b/Test/V01zmodload.ztst
index 3580bac..349ae9c 100644
--- a/Test/V01zmodload.ztst
+++ b/Test/V01zmodload.ztst
@@ -52,7 +52,7 @@
 >zmodload zsh/main
 >zmodload zsh/parameter
 
-# You use to need zmodload -i to avoid an error.
+# You used to need zmodload -i to avoid an error.
 # That has been deemed pointless, so now an attempt
 # to load a loaded module should succeed.
  zmodload zsh/main
@@ -64,7 +64,7 @@
 
  for m in $mods
  do
-   zmodload -i $m || mods[(r)$m]=()
+   zmodload $m || mods[(r)$m]=()
  done
 0d:Test loading of all compiled modules
 
@@ -74,7 +74,7 @@
 # Now check for proper failure conditions by trying some operations on
 # a nonexistent module.
 
- zmodload -i bogus/notamodule
+ zmodload bogus/notamodule
 1D:Check that loading a nonexistent module fails
 
  zmodload -u bogus/notamodule
diff --git a/Test/V02zregexparse.ztst b/Test/V02zregexparse.ztst
index ddca3c9..b4cec42 100644
--- a/Test/V02zregexparse.ztst
+++ b/Test/V02zregexparse.ztst
@@ -2,7 +2,9 @@
 
 %prep
 
-  zmodload zsh/zutil
+ if ! zmodload zsh/zutil 2>/dev/null; then
+   ZTST_unimplemented="can't load the zsh/zutil module for testing"
+ fi
 
 %test
 
diff --git a/Test/V03mathfunc.ztst b/Test/V03mathfunc.ztst
index ab383db..1edb7a2 100644
--- a/Test/V03mathfunc.ztst
+++ b/Test/V03mathfunc.ztst
@@ -1,9 +1,7 @@
 # Tests for the module zsh/mathfunc
 
 %prep
-  if ( zmodload -i zsh/mathfunc ) >/dev/null 2>&1; then
-    zmodload -i zsh/mathfunc
-  else
+  if ! zmodload zsh/mathfunc 2>/dev/null; then
     ZTST_unimplemented="The module zsh/mathfunc is not available."
   fi
 
@@ -112,7 +110,6 @@ F:This test fails if your math library doesn't have erand48().
   float -F f sum sumsq max max2 av sd
   typeset -a randoms
   randoms=('f = RANDOM' 'f = rand48()')
-  zmodload -i zsh/mathfunc
   for isource in 1 2; do
     (( sum = sumsq = max = 0 ))
     repeat $N; do
diff --git a/Test/V04features.ztst b/Test/V04features.ztst
index 2790456..6939053 100644
--- a/Test/V04features.ztst
+++ b/Test/V04features.ztst
@@ -7,7 +7,8 @@
 # We use zsh/datetime because it has a list of features that is short
 # but contains two types.
 
-  if ! (zmodload zsh/datetime >/dev/null 2>/dev/null); then
+  # Subshell for prep test so we can load individual features later
+  if ! (zmodload zsh/datetime 2>/dev/null); then
     ZTST_unimplemented="can't load the zsh/datetime module for testing"
   fi
 
diff --git a/Test/V05styles.ztst b/Test/V05styles.ztst
index e932b54..ca95b63 100644
--- a/Test/V05styles.ztst
+++ b/Test/V05styles.ztst
@@ -2,9 +2,7 @@
 
 # Test the use of styles, if the zsh/zutil module is available.
 
-  if (zmodload zsh/zutil >/dev/null 2>/dev/null); then
-    zmodload zsh/zutil
-  else
+  if ! zmodload zsh/zutil 2>/dev/null; then
     ZTST_unimplemented="can't load the zsh/zutil module for testing"
   fi
 
diff --git a/Test/V09datetime.ztst b/Test/V09datetime.ztst
index 1e677cd..63ff4ee 100644
--- a/Test/V09datetime.ztst
+++ b/Test/V09datetime.ztst
@@ -1,18 +1,18 @@
 %prep
 
-  if ! (zmodload zsh/datetime >/dev/null 2>/dev/null); then
+  if zmodload zsh/datetime 2>/dev/null; then
+    setopt multibyte
+    unset LC_ALL
+    LC_TIME=C
+    TZ=UTC+0
+    # It's not clear this skip_extensions is correct, but the
+    # format in question is causing problems on Solaris.
+    # We'll revist this after the release.
+    [[ "$(strftime %^_10B 0)" = "   JANUARY" ]] || skip_extensions=1
+    [[ "$(LC_TIME=ja_JP.UTF-8 strftime %OS 1)" = 一 ]] || skip_japanese=1
+  else
     ZTST_unimplemented="can't load the zsh/datetime module for testing"
   fi
-  setopt multibyte
-  zmodload zsh/datetime
-  unset LC_ALL
-  LC_TIME=C
-  TZ=UTC+0
-  # It's not clear this skip_extensions is correct, but the
-  # format in question is causing problems on Solaris.
-  # We'll revist this after the release.
-  [[ "$(strftime %^_10B 0)" = "   JANUARY" ]] || skip_extensions=1
-  [[ "$(LC_TIME=ja_JP.UTF-8 strftime %OS 1)" = 一 ]] || skip_japanese=1
 
 %test
 
diff --git a/Test/V10private.ztst b/Test/V10private.ztst
index 6c38e39..c66f175 100644
--- a/Test/V10private.ztst
+++ b/Test/V10private.ztst
@@ -2,10 +2,9 @@
 
 %prep
 
- if ! (zmodload zsh/param/private >/dev/null 2>/dev/null); then
+ if ! zmodload zsh/param/private 2>/dev/null; then
    ZTST_unimplemented="can't load the zsh/param/private module for testing"
  fi
- zmodload zsh/param/private
 
 %test
 
diff --git a/Test/X02zlevi.ztst b/Test/X02zlevi.ztst
index 14bc02e..ced7030 100644
--- a/Test/X02zlevi.ztst
+++ b/Test/X02zlevi.ztst
@@ -3,7 +3,7 @@
 %prep
   if [[ $OSTYPE = cygwin ]]; then
     ZTST_unimplemented="the zsh/zpty module does not work on Cygwin"
-  elif ( zmodload -i zsh/zpty ) >/dev/null 2>&1; then
+  elif ( zmodload zsh/zpty 2>/dev/null ); then
     . $ZTST_srcdir/comptest
     comptestinit -v -z $ZTST_testdir/../Src/zsh
   else
diff --git a/Test/Y01completion.ztst b/Test/Y01completion.ztst
index 383e652..1568369 100644
--- a/Test/Y01completion.ztst
+++ b/Test/Y01completion.ztst
@@ -3,7 +3,7 @@
 %prep
   if [[ $OSTYPE = cygwin ]]; then
     ZTST_unimplemented="the zsh/zpty module does not work on Cygwin"
-  elif ( zmodload -i zsh/zpty ) >/dev/null 2>&1; then
+  elif ( zmodload zsh/zpty 2>/dev/null ); then
     . $ZTST_srcdir/comptest
     mkdir comp.tmp
     cd comp.tmp
diff --git a/Test/Y02compmatch.ztst b/Test/Y02compmatch.ztst
index db734fa..e2f8e1a 100644
--- a/Test/Y02compmatch.ztst
+++ b/Test/Y02compmatch.ztst
@@ -13,7 +13,7 @@
 %prep
   if [[ $OSTYPE = cygwin ]]; then
     ZTST_unimplemented="the zsh/zpty module does not work on Cygwin"
-  elif ( zmodload -i zsh/zpty ) >/dev/null 2>&1; then
+  elif ( zmodload zsh/zpty 2>/dev/null ); then
     . $ZTST_srcdir/comptest
     mkdir match.tmp
     cd match.tmp
diff --git a/Test/Y03arguments.ztst b/Test/Y03arguments.ztst
index 0627104..0147c7d 100644
--- a/Test/Y03arguments.ztst
+++ b/Test/Y03arguments.ztst
@@ -3,7 +3,7 @@
 %prep
   if [[ $OSTYPE = cygwin ]]; then
     ZTST_unimplemented="the zsh/zpty module does not work on Cygwin"
-  elif ( zmodload -i zsh/zpty ) >/dev/null 2>&1; then
+  elif ( zmodload zsh/zpty 2>/dev/null ); then
     . $ZTST_srcdir/comptest
     mkdir comp.tmp
     cd comp.tmp
diff --git a/Test/comptest b/Test/comptest
index 20a3a5b..f10739a 100644
--- a/Test/comptest
+++ b/Test/comptest
@@ -5,7 +5,7 @@ comptestinit () {
           $ZTST_srcdir/../Completion
           $ZTST_srcdir/../Completion/*/*~*/CVS(/) )
 
-  zmodload -i zsh/zpty || return $?
+  zmodload zsh/zpty || return $?
 
   comptest_zsh=${ZSH:-zsh}
   comptest_keymap=e
diff --git a/Test/ztst.zsh b/Test/ztst.zsh
index ce89a83..cdd84b5 100755
--- a/Test/ztst.zsh
+++ b/Test/ztst.zsh
@@ -41,7 +41,7 @@ export MODULE_PATH
 
 # We need to be able to save and restore the options used in the test.
 # We use the $options variable of the parameter module for this.
-zmodload -i zsh/parameter
+zmodload zsh/parameter
 
 # Note that both the following are regular arrays, since we only use them
 # in whole array assignments to/from $options.

-- 
Barton E. Schaefer

Re: zsh 5.1.1-test-1

[Sebastian Gniazdowski]

[-- Attachment #1: Type: text/plain, Size: 662 bytes --]

Test suite runs fine on OS X 10.11.1. I've build two versions, without
pcre and with it (so that V07pcre.ztst could be ran).

I also ran my performance and memory test against pcre test1, slightly
earlier version (d5ba08a) with no pcre and 5.0.2, results are good.

Best regards,
Sebastian Gniazdowski

On 21 November 2015 at 19:35, Peter Stephenson
<p.w.stephenson@ntlworld.com> wrote:
> I've upoloaded a test version 5.1.1-test-1 preparatory to releasing
> 5.2.  You can find it at
>
> http://www.zsh.org/pub/development/
>
> and also as a tag in the repository.
>
> I'm not expecting unexpected problems, but then logically I can't, so
> let me know.
>
> pws

[-- Attachment #2: test1_mem_results.txt --]
[-- Type: text/plain, Size: 1208 bytes --]

# Tests for zsh-5.1.1-test-1
# string_test 6.6, 9.7, 12, 12, 12, 13, 12, 12, last: 11
string_test 11.0
# array_test 1.5, 2.4, 3.1, 3.5, 3.9, 4.3, 4.6, 5.1, 6, 6.7, 7.4, 8, 8.2, 8.4, 8.6, 8.8, 9, 9.2, last: 9.2
array_test 5.9
# function_test 2.7, 2.5, 2.7, 2.8, 2.5, 2.5, 2.6, 2.6, 2.6, 2.8, 2.6, 2.8, last: 2.4
function_test 2.6
# search_test 111, 196, last: 274
search_test 111.0


# Tests for zsh-5.0.2
# string_test 73, 161, 217, 262, 300, 334, 364, 392, 417, 440, 463, 483, 505, 522, 541, 557, 573, last: 586
string_test 376.9
# array_test 1.5, 2.4, 3, 3.5, 3.8, 4.3, 4.4, 4.9, 5.8, 6.5, 7.2, 7.6, 8, 8.2, 8.4, 8.6, 8.8, 9, last: 9
array_test 5.7
# function_test 2.4, 2.4, 2.4, 2.5, 2.5, 2.5, 2.4, 2.4, 2.5, last: 2.4
function_test 2.4
# search_test 62, 108, 138, 162, 183, 202, 220, 237, last: 245
search_test 153.6


# Tests for zsh-d5ba08a-clean
# string_test 6.3, 9.4, 11, 11, 12, 12, 12, 12, last: 11
string_test 10.5
# array_test 1.6, 2.4, 3, 3.5, 4, 4.4, 4.5, 5.2, 6.1, 6.7, 7.4, 8.1, 8.3, 8.5, 8.7, 8.8, 9.1, 9.2, last: 9.2
array_test 5.9
# function_test 2.8, 2.5, 2.8, 2.8, 2.8, 2.5, 2.6, 2.5, 2.8, 2.6, 2.6, last: 2.5
function_test 2.7
# search_test 111, 190, 251, last: 264
search_test 150.5


[-- Attachment #3: test1_perf_results.txt --]
[-- Type: text/plain, Size: 807 bytes --]

Running [zsh-5.1.1-test-1]:              string_test       1602,18
Running [zsh-5.0.2]:                     string_test       4211,40
Running [zsh-d5ba08a-clean]:             string_test       1641,52

Running [zsh-5.1.1-test-1]:              array_test        4920,74
Running [zsh-5.0.2]:                     array_test        5126,64
Running [zsh-d5ba08a-clean]:             array_test        5041,16

Running [zsh-5.1.1-test-1]:              function_test    13866,95
Running [zsh-5.0.2]:                     function_test    11229,21
Running [zsh-d5ba08a-clean]:             function_test    13736,76

Running [zsh-5.1.1-test-1]:              search_test       2532,93
Running [zsh-5.0.2]:                     search_test       4653,87
Running [zsh-d5ba08a-clean]:             search_test       2471,28

[-- Attachment #4: perf-test.zsh --]
[-- Type: application/octet-stream, Size: 2143 bytes --]

#!/bin/zsh

emulate -L zsh
setopt extendedglob

zmodload zsh/zprof

zshs=( zsh-5.1.1-test-1 zsh-5.0.2 zsh-d5ba08a-clean )

#
# Children administration
#

trap "finished" SIGUSR1

finished() {
    FINISHED=1
}

# Waits for signal from child process
wait_for_end_of_test() {
    while [ "$FINISHED" -eq 0 ]; do
        sleep 1
    done
    kill -15 "$SUB_PID"
}

_finished_signal_wait() {
    kill -SIGUSR1 "$MAIN_PID"
    sleep 60
}

#
# Tests
#

tests=( string_test array_test function_test search_test )
#tests=( string_test array_test )

float multiplier=0.3

string_test() {
    local a=""
    integer i=$(( 150000*multiplier ))
    repeat $i; do a+="$i"; done
}

array_test() {
    typeset -a a
    integer i=$(( 25000*multiplier ))
    repeat $i; do a+=( $i ); done
}

function_test() {
    local count
    integer i=$(( 10000*multiplier ))

    if [ -z "$1" ]; then
        repeat $i; do function_test 100; done
    else
        count="$1"
    fi

    if (( count -- > 0 )); then
        function_test "$count"
    fi
}

search_test() {
    integer elements=$(( 800000 * multiplier ))
    a="${(r:elements:: _:)b}"
    a=( $=a )
    a=( "${(@M)a:#(#i)*_*}" )
    a=( "${(@)a//(#mi)(_|a)/-${MATCH}-}" )
    a=( "${(@)a//(#bi)(_|-)/|${match[1]}|}" )
}

#
# Main code
#

# Detect main vs. for-test invocation
if [ -z "$1" ]; then

    for test in "$tests[@]"; do
        for current_zsh in "$zshs[@]"; do

            type "$current_zsh" 2>/dev/null 1>&2 || { echo >&2 "Skipping non-accessible $current_zsh"; continue }
            zsh_binary="${current_zsh##*/}"
            FINISHED=0
            TEST="$test"

            "$current_zsh" -c "source ./$0 $$ \"$current_zsh\" $test" &

            SUB_PID=$!
            wait_for_end_of_test
        done

        echo

    done

else
    MAIN_PID="$1"
    zsh_binary="${2##*/}"
    shift
    shift
    txt="Running [$zsh_binary]: "
    echo -n "${(r:40:: :)txt}" "${(r:15:: :)*}"

    # Run the test
    zprof -c
    "$@"
    zprof_out=( "${(@f)"$( zprof )"}" )
    zprof_out="$zprof_out[3]"
    zprof_out=( $=zprof_out )
    echo "${(l:10:: :)zprof_out[3]}"

    _finished_signal_wait
fi

[-- Attachment #5: mem-test.zsh --]
[-- Type: application/octet-stream, Size: 4001 bytes --]

#!/bin/zsh

emulate -L zsh
setopt extendedglob

zshs=( zsh-5.1.1-test-1 zsh-5.0.2 zsh-d5ba08a-clean )

# Convert sizes to number of megabytes
to_mbytes() {
    local size="$1"
    #echo Converting: "$1"
    if [[ "$size" = [0-9]#[Mm]* ]]; then
        size="${size%[Mm]*}"
    elif [[ "$size" = [0-9]#[Kk]* ]]; then
        size="${size%[Kk]*}"
        (( size = size / 1024.0 ))
    elif [[ "$size" = [0-9]# ]]; then
        case $( uname ) in
            *Linux*)
                (( size = size / 1024.0 ))
                ;;
            *)
                (( size = size / (1024.0 * 1024.0) ))
                ;;
        esac
    else
        echo "Bad size occured: $size"
    fi

    REPLY="$size"
}

#
# Children administration
#

trap "finished" SIGUSR1

finished() {
    FINISHED=1
}

# Gets memory size of the zsh that runs the test
get_mem() {
    case $( uname ) in
        *Darwin*)
            output=( "${(@f)"$( top -pid "$SUB_PID" -stats mem -l 1 )"}" )
            to_mbytes "$output[-1]"
            ;;
        *Linux*)
            output=( "${(@f)"$( top -p "$SUB_PID" -bn 1 )"}" )
            output=$output[-1]
            output=( $=output )
            to_mbytes "$output[6]"
            ;;
    esac
}

# Waits for signal from child process
wait_get_mem() {
    local number
    float average=0.0
    integer count=0

    echo -n "# $TEST "

    sleep 1
    while [ "$FINISHED" -eq 0 ]; do
        get_mem
        number=`LANG=C printf "%.1f" "$REPLY"`
        echo -n "${number%.0}, "

        average+=number
        count+=1
        sleep 3
    done

    # Remove last result - to be certain that the average
    # is only above data from actively working test function
    if [ "$count" -gt 1 ]; then
        average=average-number
        count=count-1
    fi
    average=average/count

    # Take the explicit last result, it's telling
    # how zsh behaves when computation is stopped
    sleep 1
    get_mem
    number=`LANG=C printf "%.1f" "$REPLY"`
    echo "last: ${number%.0}"
    kill -15 "$SUB_PID"

    # Suitable for gnuplot - X Y
    LANG=C printf "$TEST %.1f\n" $average
}

_finished_signal_wait() {
    kill -SIGUSR1 "$MAIN_PID"
    sleep 60
}

#
# Tests
#

tests=( string_test array_test function_test search_test )

string_test() {
    local a=""
    integer i=150000
    repeat $i; do a+="$i"; done

    _finished_signal_wait
}

array_test() {
    typeset -a a
    integer i=25000
    repeat $i; do a+=( $i ); done

    _finished_signal_wait
}

function_test() {
    local count

    if [ -z "$1" ]; then
        repeat 10000; do function_test 100; done
        _finished_signal_wait
    else
        count="$1"
    fi

    if (( count -- > 0 )); then
        function_test "$count"
    fi
}

search_test() {
    integer elements=700000
    a="${(r:elements:: _:)b}"
    a=( $=a )
    a=( "${(@M)a:#(#i)*_*}" )
    a=( "${(@)a//(#mi)(_|a)/-${MATCH}-}" )
    a=( "${(@)a//(#bi)(_|-)/|${match[1]}|}" )

    _finished_signal_wait
}

#
# Main code
#

# Detect main vs. for-test invocation
if [ -z "$1" ]; then
    for current_zsh in "$zshs[@]"; do
        type "$current_zsh" 2>/dev/null 1>&2 || { echo >&2 "Skipping non-accessible $current_zsh"; continue }
        zsh_binary="${current_zsh##*/}"

        echo "# Tests for $zsh_binary"
        for test in "$tests[@]"; do
            FINISHED=0
            TEST="$test"

            "$current_zsh" -c "source ./$0 $$ \"$current_zsh\" $test" &

            SUB_PID=$!
            wait_get_mem
        done
        echo
        echo

    done

    # Example gnuplot invocation:
    #set style data histogram
    #set style fill solid border rgb "black"
    #plot "result" index 0 using 2: xtic(1), "result" index 1 using 2: xtic(1), "result" index 2 using 2: xtic(1)
else
    MAIN_PID="$1"
    zsh_binary="${2##*/}"
    shift
    shift
    # Echo status only when output is not to terminal
    [ ! -t 1 ] && echo >&2 "Running [$zsh_binary]: $@"

    # Run the test
    eval "run_test() { $@ }"
    run_test
fi

Re: zsh 5.1.1-test-1

[Manuel Presnitz]

I experience problems with this command
  $ hash -m 'foo'
which hangs infinitely (? -- at least a very long time) using "zsh-5.1.1-test-1-1-gb168830" with no rcfiles loaded (zsh -f).

I can observe this behaviour on cygwin-i686 as well as on linux-x86_64 usind my usual configure options:
  $ ./configure --prefix=/usr --enable-multibyte --enable-pcre
Disabling pcre does not cure the problem (only tested on linux-x86_64).

Unfortunately I wasn't up-to-date the last months with my zsh, so at the moment I can only say that "zsh-5.0.8-17-ga253ada" does not have this problem. I can try to narrow it down to a specific patch later that day.

---Manuel.


Peter Stephenson hat am 21.11.2015 um 18:35 folgendes geschrieben:

> I've upoloaded a test version 5.1.1-test-1 preparatory to releasing
> 5.2.  You can find it at
>
> http://www.zsh.org/pub/development/
>
> and also as a tag in the repository.
>
> I'm not expecting unexpected problems, but then logically I can't, so
> let me know.
>
> pws

Re: zsh 5.1.1-test-1

[Mikael Magnusson]

On Sun, Nov 22, 2015 at 12:48 PM, Manuel Presnitz <mpy@gmx.net> wrote:
> I experience problems with this command
>   $ hash -m 'foo'
> which hangs infinitely (? -- at least a very long time) using "zsh-5.1.1-test-1-1-gb168830" with no rcfiles loaded (zsh -f).
>
> I can observe this behaviour on cygwin-i686 as well as on linux-x86_64 usind my usual configure options:
>   $ ./configure --prefix=/usr --enable-multibyte --enable-pcre
> Disabling pcre does not cure the problem (only tested on linux-x86_64).
>
> Unfortunately I wasn't up-to-date the last months with my zsh, so at the moment I can only say that "zsh-5.0.8-17-ga253ada" does not have this problem. I can try to narrow it down to a specific patch later that day.

Unfortunately for Peter,

39b28980f38e83e15cdeb19a489b5659af97fe93 is the first bad commit
commit 39b28980f38e83e15cdeb19a489b5659af97fe93
Author: Peter Stephenson <pws@zsh.org>
Date:   Thu Jun 18 14:54:41 2015 +0100

    various posts: Implement assignment parsing for typeset.

-- 
Mikael Magnusson

Re: zsh 5.1.1-test-1

[Peter Stephenson]

On Sun, 22 Nov 2015 12:48:36 +0100
"Manuel Presnitz" <mpy@gmx.net> wrote:
> I experience problems with this command
>   $ hash -m 'foo'
> which hangs infinitely

Yes, it's infinite; it's a simple missing increment.  (Took a while for
the penny to break out of the non-dropping loop while I scratched my
head over search algorithms...)

I'm wondering, having just looked, whether it's safe to use a variable-size
array for all the hash nodes in the command table when sorting?  That's
using up quite a lot of stack.

pws

diff --git a/Src/builtin.c b/Src/builtin.c
index 01eb5b8..cac4f42 100644
--- a/Src/builtin.c
+++ b/Src/builtin.c
@@ -3712,6 +3712,7 @@ bin_hash(char *name, char **argv, Options ops, UNUSED(int func))
 		zwarnnam(name, "bad pattern : %s", *argv);
 		returnval = 1;
 	    }
+	    argv++;
             continue;
 	}
         if (!(asg = getasg(&argv, NULL))) {

Re: zsh 5.1.1-test-1

[Peter Stephenson]

Test for the hash command.

pws

diff --git a/Test/B09hash.ztst b/Test/B09hash.ztst
new file mode 100644
index 0000000..49f3048
--- /dev/null
+++ b/Test/B09hash.ztst
@@ -0,0 +1,71 @@
+# The hash builtin is most used for the command hash table, which is
+# populated automatically.  This is therefore highly system specific,
+# so mostly we'll test with the directory hash table: the logic is
+# virtually identical but with the different table, and furthermore
+# the shell doesn't care whether the directory exists unless you refer
+# to it in a context that needs one.
+
+%prep
+  populate_hash() {
+    hash -d one=/first/directory
+    hash -d two=/directory/the/second
+    hash -d three=/noch/ein/verzeichnis
+    hash -d four=/bored/with/this/now
+  }
+
+%test
+
+  hash -d
+0:Directory hash initially empty
+
+  populate_hash
+  hash -d
+0:Populating directory hash and output with sort
+>four=/bored/with/this/now
+>one=/first/directory
+>three=/noch/ein/verzeichnis
+>two=/directory/the/second
+  
+  hash -rd
+  hash -d
+0:Empty hash
+
+  populate_hash
+  hash -d
+0:Refill hash
+>four=/bored/with/this/now
+>one=/first/directory
+>three=/noch/ein/verzeichnis
+>two=/directory/the/second
+ 
+  hash -dL
+0:hash -L option
+>hash -d four=/bored/with/this/now
+>hash -d one=/first/directory
+>hash -d three=/noch/ein/verzeichnis
+>hash -d two=/directory/the/second
+
+  hash -dm 't*'
+0:hash -m option
+>three=/noch/ein/verzeichnis
+>two=/directory/the/second
+
+  hash -d five=/yet/more six=/here/we/go seven=/not/yet/eight
+  hash -d
+0:Multiple assignments
+>five=/yet/more
+>four=/bored/with/this/now
+>one=/first/directory
+>seven=/not/yet/eight
+>six=/here/we/go
+>three=/noch/ein/verzeichnis
+>two=/directory/the/second
+
+  hash -d one two three
+0:Multiple arguments with no assignment not in verbose mode
+
+  hash -vd one two three
+0:Multiple arguments with no assignment in verbose mode
+>one=/first/directory
+>two=/directory/the/second
+>three=/noch/ein/verzeichnis