Bug / error in manpage.

Bug / error in manpage.

[Larry Schrof]

[-- Attachment #1: Type: text/plain, Size: 1668 bytes --]

I have found an error in the manpage, or incomplete functionality -
whatever you'd like to call it.

If you are new to zsh and want to learn a couple of things, read
on. If you are a veteran and just want to read about the bug,
jump to the string '===' below in this email.

Context

Let's say I have a string that contains words separated by some
arbitrary token. I'll pick a line from /etc/passwd:

zsh% string=$(grep '^nobody' /etc/passwd)

$() is the same as backticks, but better.

zsh% print $string
nobody:*:-2:-2:Unprivileged User:/var/empty:/usr/bin/false

Let's say we want to get the value of the second field.
Let's try something naive:

zsh% print $string[2]
o
zsh%

That doesn't work - by default it indexes on characters if
it's a scalar. Let's use the split-on-word flag:

zsh% print $string[(w)2]
User:/var/empty:/usr/bin/false

Nope - that's not what we want. It split on whitespace.
We want to tell it to split on colons:

zsh% print $string[(ws{:})2]
*
zsh%

Excellent that's what we want!

===
The man page for subscripting flags is incorrect. Here is the excerpt:

       The flags s, n and b take an argument; the delimiter is shown below  as
       `:',  but  any  character,  or  the  matching  pairs  `(...)', `{...}',
       `[...]', or `<...>', may be used.

The '< >' brackets do not work as separators:
zsh% print $string[(ws<:>)2]
zsh: parse error near `)'
zsh%

This is exactly like the previous command, except the curly braces were changed
to arrow brackets.

Can we either update the manpage, removuing mention of '<...>', or add the ability
to parse them?

Thanks!
===

Re: Bug / error in manpage.

[Peter Stephenson]

(By the way, my work email is about to change, so if you're emailing me
directly, which isn't usually necessary for zsh discussions, use my
home email, p.w.stephenson@ntlworld.com.  You should get a useful
bounce message when it does change, however.)

On Mon, 1 Oct 2012 18:08:36 +0000
Larry Schrof <larrys@fb.com> wrote:
> The man page for subscripting flags is incorrect. Here is the excerpt:
> 
>        The flags s, n and b take an argument; the delimiter is shown below  as
>        `:',  but  any  character,  or  the  matching  pairs  `(...)', `{...}',
>        `[...]', or `<...>', may be used.
> 
> The '< >' brackets do not work as separators:
> zsh% print $string[(ws<:>)2]
> zsh: parse error near `)'
> zsh%

I don't think there's any reason this shouldn't be made to work; there's
currently no check to make the square bracket context special for angle
brackets, except if you used braces in the parameter substitution.

The change also makes things like this work:

[[ '<bleagh>' = [<>]*[<>] ]]

I interpret that as a good thing.  Angle brackets only need to do
special things when used as redirection operators and (to a more limited
extent, since they're then part of a string) numeric glob ranges.

Index: Src/lex.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/lex.c,v
retrieving revision 1.70
diff -p -u -r1.70 lex.c
--- Src/lex.c	8 Dec 2011 19:42:07 -0000	1.70
+++ Src/lex.c	2 Oct 2012 11:45:19 -0000
@@ -1218,7 +1218,7 @@ gettokstr(int c, int sub)
 		c = Comma;
 	    break;
 	case LX2_OUTANG:
-	    if (in_brace_param || sub)
+	    if (in_brace_param || sub || brct)
 		break;
 	    e = hgetc();
 	    if (e != '(') {
@@ -1255,7 +1255,7 @@ gettokstr(int c, int sub)
 		break;
 	    }
 	    lexstop = 0;
-	    if (in_brace_param || sub)
+	    if (in_brace_param || sub || brct)
 		break;
 	    goto brk;
 	case LX2_EQUALS:
Index: Test/D06subscript.ztst
===================================================================
RCS file: /cvsroot/zsh/zsh/Test/D06subscript.ztst,v
retrieving revision 1.13
diff -p -u -r1.13 D06subscript.ztst
--- Test/D06subscript.ztst	31 Jul 2008 13:53:28 -0000	1.13
+++ Test/D06subscript.ztst	2 Oct 2012 11:45:20 -0000
@@ -249,3 +249,23 @@
   string[0]=!
 1:Can't set only element zero of string
 ?(eval):1: string: assignment to invalid subscript range
+
+  string="Life,+like+a+dome+of+many-colour'd+glass"
+  for delims in "()" "{}" "[]" "<>"; do
+    eval 'print ${string[(ws'$delims[1]'+'$delims[2]')2]}'
+    eval 'print $string[(ws'$delims[1]'+'$delims[2]')3]'
+    eval 'print "$string[(ws'$delims[1]'+'$delims[2]')4]"'
+  done
+0:Check matched delimiters in subscripts
+>like
+>a
+>dome
+>like
+>a
+>dome
+>like
+>a
+>dome
+>like
+>a
+>dome

-- 
Peter Stephenson <pws@csr.com>            Software Engineer
Tel: +44 (0)1223 692070                   Cambridge Silicon Radio Limited
Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, UK


Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom
More information can be found at www.csr.com. Follow CSR on Twitter at http://twitter.com/CSR_PLC and read our blog at www.csr.com/blog

Re: Bug / error in manpage.

[Bart Schaefer]

On Oct 1,  6:08pm, Larry Schrof wrote:
}
} The man page for subscripting flags is incorrect. Here is the excerpt:
} 
}        The flags s, n and b take an argument; the delimiter is shown below  as
}        `:',  but  any  character,  or  the  matching  pairs  `(...)', `{...}',
}        `[...]', or `<...>', may be used.
} 
} The '< >' brackets do not work as separators:
} zsh% print $string[(ws<:>)2]
} zsh: parse error near `)'
} zsh%

I see PWS has made a patch, but I just wanted to point out that the man
page was NOT incorrect before.

It works if you quote it:

torch% print "$string[(ws<:>)2]"
*

What PWS's patch does, in effect, is arrange that [ ] will quote the
redirection operators so you no longer need to explicitly quote them in
that particular context.  This is probably breaking some rule or other
of POSIX shell syntax but I can't immediately come up with any example
where it will matter.

Re: Bug / error in manpage.

[Larry Schrof]

Thanks much for taking a look into it and the prompt patch!

On 10/2/12 7:06 AM, "Bart Schaefer" <schaefer@brasslantern.com> wrote:

>On Oct 1,  6:08pm, Larry Schrof wrote:
>}
>} The man page for subscripting flags is incorrect. Here is the excerpt:
>} 
>}        The flags s, n and b take an argument; the delimiter is shown
>below  as
>}        `:',  but  any  character,  or  the  matching  pairs  `(...)',
>`{...}',
>}        `[...]', or `<...>', may be used.
>} 
>} The '< >' brackets do not work as separators:
>} zsh% print $string[(ws<:>)2]
>} zsh: parse error near `)'
>} zsh%
>
>I see PWS has made a patch, but I just wanted to point out that the man
>page was NOT incorrect before.
>
>It works if you quote it:
>
>torch% print "$string[(ws<:>)2]"
>*
>
>What PWS's patch does, in effect, is arrange that [ ] will quote the
>redirection operators so you no longer need to explicitly quote them in
>that particular context.  This is probably breaking some rule or other
>of POSIX shell syntax but I can't immediately come up with any example
>where it will matter.

Re: Bug / error in manpage.

[Peter Stephenson]

On Tue, 02 Oct 2012 07:06:43 -0700
Bart Schaefer <schaefer@brasslantern.com> wrote:
> What PWS's patch does, in effect, is arrange that [ ] will quote the
> redirection operators so you no longer need to explicitly quote them in
> that particular context.  This is probably breaking some rule or other
> of POSIX shell syntax but I can't immediately come up with any example
> where it will matter.

Here's one:

[</dev/null ]

Before returned false, now says

zsh: no such file or directory: [</dev/null

That looks likes something of a dealbreaker; there's no simple
syntactic distinction between that and the cases that we'd like to work.
It might be better just to document the limitations.

-- 
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/

Re: Bug / error in manpage.

[Larry Schrof]

Agreed - I was thinking it was probably a matter of time before one of you
came
up with a good example.

Are you thinking mentioning angle-brackets needing special quoting, or are
you thinking about just removing mention of them from that section of
the manpage?

On 10/2/12 11:35 AM, "Peter Stephenson" <p.w.stephenson@ntlworld.com>
wrote:

>On Tue, 02 Oct 2012 07:06:43 -0700
>Bart Schaefer <schaefer@brasslantern.com> wrote:
>> What PWS's patch does, in effect, is arrange that [ ] will quote the
>> redirection operators so you no longer need to explicitly quote them in
>> that particular context.  This is probably breaking some rule or other
>> of POSIX shell syntax but I can't immediately come up with any example
>> where it will matter.
>
>Here's one:
>
>[</dev/null ]
>
>Before returned false, now says
>
>zsh: no such file or directory: [</dev/null
>
>That looks likes something of a dealbreaker; there's no simple
>syntactic distinction between that and the cases that we'd like to work.
>It might be better just to document the limitations.
>
>-- 
>Peter Stephenson <p.w.stephenson@ntlworld.com>
>Web page now at http://homepage.ntlworld.com/p.w.stephenson/

Re: Bug / error in manpage.

[Peter Stephenson]

On Tue, 2 Oct 2012 18:39:31 +0000
Larry Schrof <larrys@fb.com> wrote:
> Agreed - I was thinking it was probably a matter of time before one of you
> came
> up with a good example.
> 
> Are you thinking mentioning angle-brackets needing special quoting, or are
> you thinking about just removing mention of them from that section of
> the manpage?

I was going to mention the problem...

Index: Doc/Zsh/params.yo
===================================================================
RCS file: /cvsroot/zsh/zsh/Doc/Zsh/params.yo,v
retrieving revision 1.71
diff -p -u -r1.71 params.yo
--- Doc/Zsh/params.yo	21 Sep 2012 12:45:27 -0000	1.71
+++ Doc/Zsh/params.yo	2 Oct 2012 19:59:27 -0000
@@ -182,7 +182,10 @@ to the matching closing one is considere
 The flags tt(s), tt(n) and tt(b) take an argument; the delimiter
 is shown below as `tt(:)', but any character, or the matching pairs
 `tt(LPAR())...tt(RPAR())', `tt({)...tt(})', `tt([)...tt(])', or
-`tt(<)...tt(>)', may be used.
+`tt(<)...tt(>)', may be used, but note that `tt(<)...tt(>)' can only be
+used if the subscript is inside a double quoted expression or a
+parameter substitution enclosed in braces as otherwise the expression is
+interpreted as a redirection.
 
 The flags currently understood are:
 
Index: Src/lex.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/lex.c,v
retrieving revision 1.71
diff -p -u -r1.71 lex.c
--- Src/lex.c	2 Oct 2012 12:46:46 -0000	1.71
+++ Src/lex.c	2 Oct 2012 19:59:27 -0000
@@ -1218,7 +1218,7 @@ gettokstr(int c, int sub)
 		c = Comma;
 	    break;
 	case LX2_OUTANG:
-	    if (in_brace_param || sub || brct)
+	    if (in_brace_param || sub)
 		break;
 	    e = hgetc();
 	    if (e != '(') {
@@ -1255,7 +1255,7 @@ gettokstr(int c, int sub)
 		break;
 	    }
 	    lexstop = 0;
-	    if (in_brace_param || sub || brct)
+	    if (in_brace_param || sub)
 		break;
 	    goto brk;
 	case LX2_EQUALS:
Index: Test/A04redirect.ztst
===================================================================
RCS file: /cvsroot/zsh/zsh/Test/A04redirect.ztst,v
retrieving revision 1.23
diff -p -u -r1.23 A04redirect.ztst
--- Test/A04redirect.ztst	8 Dec 2011 15:55:11 -0000	1.23
+++ Test/A04redirect.ztst	2 Oct 2012 19:59:27 -0000
@@ -441,3 +441,6 @@
 0:failed assignment on non-posix-special, POSIX_BUILTINS
 >output
 ?zsh: read-only variable: foo
+
+  [</dev/null ]
+1:check behaviour with square brackets
Index: Test/D06subscript.ztst
===================================================================
RCS file: /cvsroot/zsh/zsh/Test/D06subscript.ztst,v
retrieving revision 1.14
diff -p -u -r1.14 D06subscript.ztst
--- Test/D06subscript.ztst	2 Oct 2012 12:46:46 -0000	1.14
+++ Test/D06subscript.ztst	2 Oct 2012 19:59:27 -0000
@@ -249,23 +249,3 @@
   string[0]=!
 1:Can't set only element zero of string
 ?(eval):1: string: assignment to invalid subscript range
-
-  string="Life,+like+a+dome+of+many-colour'd+glass"
-  for delims in "()" "{}" "[]" "<>"; do
-    eval 'print ${string[(ws'$delims[1]'+'$delims[2]')2]}'
-    eval 'print $string[(ws'$delims[1]'+'$delims[2]')3]'
-    eval 'print "$string[(ws'$delims[1]'+'$delims[2]')4]"'
-  done
-0:Check matched delimiters in subscripts
->like
->a
->dome
->like
->a
->dome
->like
->a
->dome
->like
->a
->dome

-- 
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/