Changes for dev-4

Changes for dev-4

[Peter Stephenson]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 10467 bytes --]

With Wayne's fix there's scope for a 4.3.0-dev-4 before 4.3.1.  Here are
some minor tweaks to various bits of documentation.

Index: INSTALL
===================================================================
RCS file: /cvsroot/zsh/zsh/INSTALL,v
retrieving revision 1.24
diff -u -r1.24 INSTALL
--- INSTALL	6 Feb 2006 12:05:32 -0000	1.24
+++ INSTALL	16 Feb 2006 10:57:35 -0000
@@ -266,7 +266,11 @@
 Support for multibyte character sets that extend ASCII, such as UTF-8, is
 under development but the code in the line editor is sufficiently stable to
 be turned on by default in environments that provide full ISO 10646 support
-including the preprocessor definition __STDC_ISO_10646__.
+including the preprocessor definition __STDC_ISO_10646__.  In principle
+this definition does not guarantee the full environment, but in practice
+systems with this defined also provide suitable library support.  The shell
+does not probe for all the features, so on other systems use of multibyte
+support must be explicitly enabled when it is available.
 
 The support can be explicitly enabled or disable with --enable-multibyte or
 --disable-multibyte.  Reports of systems where multibyte support was not
Index: README
===================================================================
RCS file: /cvsroot/zsh/zsh/README,v
retrieving revision 1.26
diff -u -r1.26 README
--- README	7 Feb 2006 11:29:31 -0000	1.26
+++ README	16 Feb 2006 10:57:35 -0000
@@ -26,6 +26,17 @@
 
 Since 4.2:
 
+The option SH_WORD_SPLIT, used in Bourne/Korn/Posix shell compatibility
+mode, has been made more like other shells in the case of substitutions of
+the form ${1+"$@"} (a common trick used to work around problems in older
+Bourne shells) or any of the related forms with the + replaced by - or =
+with an optional colon following.  Previously, with SH_WORD_SPLIT in
+effect, this expression would cause splitting on all white space in the
+shell arguments.  (This was always regarded as a bug but was long-standing
+behaviour.)  Now it is treated identically to "$@".  The same change
+applies to expressions with forced splitting such as ${=1+"$@"}, but
+otherwise the case where SH_WORD_SPLIT is not set is unaffected.
+
 The "unset" builtin now does not regard the unsetting of non-existent
 variables as an error, so can still return status 0 (depending on the
 handling of other arguments).  This appears to be the standard shell
Index: Config/version.mk
===================================================================
RCS file: /cvsroot/zsh/zsh/Config/version.mk,v
retrieving revision 1.54
diff -u -r1.54 version.mk
--- Config/version.mk	6 Feb 2006 12:05:32 -0000	1.54
+++ Config/version.mk	16 Feb 2006 10:57:35 -0000
@@ -27,5 +27,5 @@
 # This must also serve as a shell script, so do not add spaces around the
 # `=' signs.
 
-VERSION=4.3.0-dev-3
-VERSION_DATE='February 6, 2006'
+VERSION=4.3.0-dev-4
+VERSION_DATE='February 15, 2006'
Index: Etc/CONTRIBUTORS
===================================================================
RCS file: /cvsroot/zsh/zsh/Etc/CONTRIBUTORS,v
retrieving revision 1.13
diff -u -r1.13 CONTRIBUTORS
--- Etc/CONTRIBUTORS	26 Feb 2004 15:09:12 -0000	1.13
+++ Etc/CONTRIBUTORS	16 Feb 2006 10:57:36 -0000
@@ -12,8 +12,22 @@
 patches.  Note that email addresses below and in the ChangeLog file
 are included for disambiguation purposes only, and are not guaranteed
 to be currently accurate.  If you feel that you or someone else have
-been unfairly omitted from this list please mail the current maintainer
-at <coordinator@zsh.org>.
+been unfairly omitted from this list please mail the list
+<zsh-workers@sunsite.dk>.
+
+Version 4.3
+-----------
+
+Contributors to the main shell include Clint Adams, Andrey Borzenkov, Dan
+Bullok, Thorsten Dahlheimer, Wayne Davison, Oliver Kiddle, Matthias
+Kopferman, Dan Nelson, Toby Peterson, R. Ramkumar, Bart Schaefer, Joerg
+Sonnenberger, Peter Stephenson, Philippe Troin, Geoff Wing.
+
+Contributors to completion functions include those above plus Zack Cerza,
+Danek Duvall, Tobias Gruetzmacher, Dough Kearns, Hannu Koivisto, Henryk
+Konsek, Scott Murray, Carlos Phillips, Haakon Riiser, Felix Rosencrantz,
+Stephen Rüger, Kris Shannon, Travis Spencer, Mårten Svantesson, Motoi
+Washida, Jess Weinstein.
 
 Version 4.2
 -----------
Index: Etc/FAQ.yo
===================================================================
RCS file: /cvsroot/zsh/zsh/Etc/FAQ.yo,v
retrieving revision 1.31
diff -u -r1.31 FAQ.yo
--- Etc/FAQ.yo	18 Dec 2005 20:06:00 -0000	1.31
+++ Etc/FAQ.yo	16 Feb 2006 10:57:37 -0000
@@ -95,7 +95,7 @@
 3.2. In which startup file do I put...?
 3.3. What is the difference between `export' and the ALL_EXPORT option?
 3.4. How do I turn off spelling correction/globbing for a single command?
-3.5. How do I get the meta key to work on my xterm?
+3.5. How do I get the Meta key to work on my xterm?
 3.6. How do I automatically display the directory in my xterm title bar?
 3.7. How do I make the completion list use eight bit characters?
 3.8. Why do the cursor (arrow) keys not work?
@@ -1163,13 +1163,20 @@
   be expanded before the rest of the command line is parsed.
 
 
-sect(How do I get the meta key to work on my xterm?)
+sect(How do I get the Meta key to work on my xterm?)
 label(35)
 
-  As stated in the manual, zsh needs to be told about the meta key by
+  The Meta key isn't present on a lot of keyboards, but on some
+  the Alt key has the same effect.  If a character is typed on the
+  keyboard while the Meta key is held down, the characters is sent
+  as terminal input with its eighth bit set.  For example, ASCII
+  mytt(A), hex 65, becomes hex E5.  This is sometimes used to provide
+  extra editing commands.
+
+  As stated in the manual, zsh needs to be told about the Meta key by
   using mytt(bindkey -me) or mytt(bindkey -mv) in your .zshrc or on the
   command line.  You probably also need to tell the terminal driver to
-  allow the `meta' bit of the character through; mytt(stty pass8) is the
+  allow the `Meta' bit of the character through; mytt(stty pass8) is the
   usual incantation.  Sample .zshrc entry:
   verb(
     [[ $TERM = "xterm" ]] && stty pass8 && bindkey -me
@@ -1186,7 +1193,12 @@
   have tinkered with it.
 
   You don't need the mytt(bindkey) to be able to define your own sequences
-  with the meta key, though you still need the mytt(stty).
+  with the Meta key, though you still need the mytt(stty).
+
+  If you are using multibyte input directly from the keyboard you
+  probably don't want to use this feature since the eighth bit in
+  each byte is used to indicate a part of a multibyte character.  See
+  link(chapter 5)(c5).
 
 
 sect(How do I automatically display the directory in my xterm title bar?)
@@ -1953,6 +1965,7 @@
 
 
 chapter(Multibyte input)
+label(c5)
 
 sect(What is multibyte input?)
 
@@ -1994,7 +2007,7 @@
   characters.  However, in the case of Unicode, UTF-8 is the only one you
   are likely to enounter.
 
-  (In case you're confused: Unicode is the characters set, while UTF-8 is
+  (In case you're confused: Unicode is the character set, while UTF-8 is
   an encoding of it.  You might hear about other encodings, such as UCS-2
   and UCS-4 which are basically the character's index in the character set
   as a two-octet or four-octet integer.  You might see files encoded this
@@ -2015,14 +2028,21 @@
   has been compiled with the appropriate definitions.  This will happen
   automatically if the compiler defines __STDC_ISO_10646__, which is true
   for many recent GNU-based systems.  On other systems you must configure
-  zsh with the argument --enable-multibyte to configure.  (The reason for
-  this is that the presence of __STDC_ISO_10646__ ensures all the required
-  library support is present, short-circuiting a large number of
-  configuration tests.)  Explicit use of --enable-multibyte should work on
-  many other recent UNIX systems; if it works on yours, and that's not
-  mentioned in the shell documentation, please report this to
-  zsh-workers@sunsite.dk, and if it doesn't but you can work out why not
-  we'd also be interested in hearing.
+  zsh with the argument --enable-multibyte to configure.  Explicit use of
+  --enable-multibyte should work on many other recent UNIX systems; if it
+  works on yours, and that's not mentioned in the shell documentation,
+  please report this to zsh-workers@sunsite.dk, and if it doesn't but you
+  can work out why not we'd also be interested in hearing.
+
+  (The reason for the test for __STDC_ISO_10646__ is that its presence
+  happens to indicate that the required library support is likely to be
+  present, short-circuiting a large number of configuration tests.  This
+  isn't strictly guaranteed, since the definition indicates the rather more
+  limited fact that the wide character representation used internally by
+  the shell is Unicode.  However, in practice such systems provide the
+  right level of support for zsh to use.  It would be better to test
+  individually for the library features the shell needs; unfortunately
+  there are a lot of them.)
 
   You can test if multibyte handling is compiled into your version of the
   shell by running:
@@ -2100,6 +2120,14 @@
       Unicode characters than others.
   )
 
+  As mentioned in the previous section, mytt(bindkey -m) now outputs
+  a warning message telling you that multibyte input from the terminal
+  is likely not to work.  (See link(3.5)(35) if you don't know what
+  this feature does.)  If your terminal doesn't have characters
+  that need to be input as multibyte, however, you can still use
+  the meta bindings and can ignore the warning message.  Use
+  mytt(bindkey -m 2>/dev/null) to suprress it.
+
 
 sect(How can I input characters that aren't on my keyboard?)
 
@@ -2119,7 +2147,7 @@
 
   tt(insert-composed-char) is followed by two characters that
   are a mnemonic for a multibyte character.  For example mytt(a:)
-  is a with an umlaut; mytt(cH) is the symbol for hearts on a playing
+  is a with an Umlaut; mytt(cH) is the symbol for hearts on a playing
   card.  Various accented characters, European and related alphabets,
   and punctuation and mathematical symbols are available.  The
   mnemonics are mostly those given by RFC 1345, see

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070

Re: Changes for dev-4

[Wayne Davison]

[-- Attachment #1: Type: text/plain, Size: 565 bytes --]

On Thu, Feb 16, 2006 at 11:00:54AM +0000, Peter Stephenson wrote:
> +Bourne shells) or any of the related forms with the + replaced by - or =

Actually, I didn't change = because I was thinking that it would just
turn into a scalar anyway.  However, since the assignment might be to an
array, it could also benefit from the new array-split-preservation code.

Attached is a patch that fixes this.  Let me know if you agree that this
is something that should make it into 4.3.1.  (I'll check-in the first
tweak to the README file, since it fixes a typo.)

..wayne..

[-- Attachment #2: equals-too.patch --]
[-- Type: text/plain, Size: 2625 bytes --]

--- README	16 Feb 2006 14:28:54 -0000	1.27
+++ README	16 Feb 2006 18:50:28 -0000
@@ -30,7 +30,7 @@ The option SH_WORD_SPLIT, used in Bourne
 mode, has been made more like other shells in the case of substitutions of
 the form ${1+"$@"} (a common trick used to work around problems in older
 Bourne shells) or any of the related forms with the + replaced by - or =
-with an optional colon following.  Previously, with SH_WORD_SPLIT in
+with an optional colon preceding.  Previously, with SH_WORD_SPLIT in
 effect, this expression would cause splitting on all white space in the
 shell arguments.  (This was always regarded as a bug but was long-standing
 behaviour.)  Now it is treated identically to "$@".  The same change
--- Src/subst.c	15 Feb 2006 18:35:35 -0000	1.46
+++ Src/subst.c	16 Feb 2006 18:50:29 -0000
@@ -2070,27 +2070,23 @@ paramsubst(LinkList l, LinkNode n, char 
 	case '=':
 	case Equals:
 	    if (vunset) {
+		int ws = opts[SHWORDSPLIT];
 		char sav = *idend;
 		int l;
 
 		*idend = '\0';
 		val = dupstring(s);
-		/*
-		 * TODO: this is one of those places where I don't
-		 * think we want to do the joining until later on.
-		 * We also need to handle spbreak and spsep at this
-		 * point and unset them.
-		 */
-		if (spsep || spbreak || !arrasg)
+		if (spsep || !arrasg) {
+		    opts[SHWORDSPLIT] = 0;
 		    multsub(&val, 0, NULL, &isarr, NULL);
-		else
-		    multsub(&val, 0, &aval, &isarr, NULL);
+		} else {
+		    opts[SHWORDSPLIT] = spbreak;
+		    multsub(&val, spbreak, &aval, &isarr, NULL);
+		    spbreak = 0;
+		}
+		opts[SHWORDSPLIT] = ws;
 		if (arrasg) {
-		    /*
-		     * This is an array assignment in a context
-		     * where we have no syntactic way of finding
-		     * out what an array element is.  So we just guess.
-		     */
+		    /* This is an array assignment. */
 		    char *arr[2], **t, **a, **p;
 		    if (spsep || spbreak) {
 			aval = sepsplit(val, spsep, 0, 1);
--- Test/D04parameter.ztst	15 Feb 2006 18:38:58 -0000	1.15
+++ Test/D04parameter.ztst	16 Feb 2006 18:50:30 -0000
@@ -551,9 +551,10 @@
 
   local sure_that='sure that' varieties_of='varieties of' one=1 two=2
   extra=(5 4 3)
+  unset foo
   set Make $sure_that "this test keeps" on 'preserving all' "$varieties_of" quoted whitespace
   print -l ${=1+"$@"}
-  print -l ${=1+Make $sure_that "this test keeps" on 'preserving all' "$varieties_of" quoted whitespace}
+  print -l ${(A)=foo=Make $sure_that "this test keeps" on 'preserving all' "$varieties_of" quoted whitespace}
   print -l ${=1+$one $two}
   print -l ${1+$extra$two$one}
 0:Regression test of ${=1+"$@"} bug and some related expansions

Re: Changes for dev-4

[Peter Stephenson]

Wayne Davison wrote:
> Actually, I didn't change = because I was thinking that it would just
> turn into a scalar anyway.  However, since the assignment might be to an
> array, it could also benefit from the new array-split-preservation code.
> 
> Attached is a patch that fixes this.  Let me know if you agree that this
> is something that should make it into 4.3.1.  (I'll check-in the first
> tweak to the README file, since it fixes a typo.)

I certainly missed the fact that it doesn't fix a compatibility problem
since you can't assign to an array in the other shells that way.
Ideally we probably want it to be whatever would happen in bash or ksh,
seeing as sh doesn't have arrays, though by the looks of things it's the
same.

In ksh I get:
$ set -A foo
$ set "one word" "two words"
$ : ${foo:="$@"}
$ echo ${foo[0]}
one word two words

and in zsh I get:
% setopt shwordsplit
% set -A foo
% set "one word" "two words"
% : ${foo:="$@"}
% print ${(t)foo}
scalar

So it looks like from the compatibility point of view we're OK either
way.

I would guess that with zsh's ability to assign arrays it would be
better to keep this consistent with the other forms, as in your patch,
however.

(We should probably hold off on the other one, that moves the flag into
the link node, until 4.3.1, though it is useful simplification to have
later.)

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070


To access the latest news from CSR copy this link into a web browser:  http://www.csr.com/email_sig.php

Re: Changes for dev-4

[Wayne Davison]

[-- Attachment #1: Type: text/plain, Size: 348 bytes --]

On Thu, Feb 16, 2006 at 07:35:26PM +0000, Peter Stephenson wrote:
> I would guess that with zsh's ability to assign arrays it would be
> better to keep this consistent with the other forms, as in your patch,
> however.

OK, I've checked it in.  I also made a few more tweaks to the zshexpn
man page where it talks about field splitting.

..wayne..

[-- Attachment #2: zshexpn.patch --]
[-- Type: text/plain, Size: 2472 bytes --]

--- Doc/Zsh/expn.yo	15 Feb 2006 10:13:46 -0000	1.61
+++ Doc/Zsh/expn.yo	16 Feb 2006 20:23:05 -0000
@@ -444,6 +444,7 @@ element of var(name) is substituted, one
 expansion results in one word only; with tt(KSH_ARRAYS), this is the first
 element of an array.  No field splitting is done on the result unless the
 tt(SH_WORD_SPLIT) option is set.
+See also the flags tt(=) and tt(s:)var(string)tt(:).
 )
 item(tt(${PLUS())var(name)tt(}))(
 If var(name) is the name of a set parameter `tt(1)' is substituted,
@@ -454,7 +455,8 @@ If var(name) is set and is non-null then
 value; otherwise substitute var(word). If var(name) is
 missing, substitute var(word).
 Note that you can use standard shell quoting in the var(word) value to
-selectively override any field splitting that may be active.
+selectively override the splitting done by the tt(SH_WORD_SPLIT) option
+and the tt(=) flag, but not the tt(s:)var(string)tt(:) flag.
 )
 xitem(tt(${)var(name)tt(:=)var(word)tt(}))
 item(tt(${)var(name)tt(::=)var(word)tt(}))(
@@ -473,7 +475,8 @@ item(tt(${)var(name)tt(:PLUS())var(word)
 If var(name) is set and is non-null then substitute
 var(word); otherwise substitute nothing.
 Note that you can use standard shell quoting in the var(word) value to
-selectively override any field splitting that may be active.
+selectively override the splitting done by the tt(SH_WORD_SPLIT) option
+and the tt(=) flag, but not the tt(s:)var(string)tt(:) flag.
 )
 enditem()
 
@@ -839,8 +842,8 @@ in string arguments to any of the flags 
 item(tt(j:)var(string)tt(:))(
 Join the words of arrays together using var(string) as a separator.
 pindex(SH_WORD_SPLIT, use of)
-Note that this occurs before field splitting by the tt(SH_WORD_SPLIT)
-option.
+Note that this occurs before field splitting by the tt(s:)var(string)tt(:)
+flag or the tt(SH_WORD_SPLIT) option.
 )
 item(tt(l:)var(expr)tt(::)var(string1)tt(::)var(string2)tt(:))(
 Pad the resulting words on the left.  Each word will be truncated if
@@ -855,10 +858,11 @@ As tt(l), but pad the words on the right
 on the right.
 )
 item(tt(s:)var(string)tt(:))(
-Force field splitting (see the option tt(SH_WORD_SPLIT)) at the
+Force field splitting at the
 separator var(string).  Note that a var(string) of two or more
 characters means that all of them must match in sequence; this differs from
 the treatment of two or more characters in the tt(IFS) parameter.
+See also the tt(=) flag and the tt(SH_WORD_SPLIT) option.
 )
 enditem()