_subscript quotes too much

_subscript quotes too much

[Mikael Magnusson]

% zsh -f
% autoload -U compinit; compinit
% typeset -A foo
% foo[bar]=baz
% foo[*.txt]=blue
% echo $foo[<tab><tab><tab>
% echo $foo[\*.txt]

% echo $foo[*.txt]
blue

I had a look at _subscript but it didn't mean much to me. I thought
maybe there was a (q) too many but there are none at all.

-- 
Mikael Magnusson

Re: _subscript quotes too much

[Mikael Magnusson]

On 09/04/2008, Mikael Magnusson <mikachu@gmail.com> wrote:
> % zsh -f
>  % autoload -U compinit; compinit
>  % typeset -A foo
>  % foo[bar]=baz
>  % foo[*.txt]=blue
>  % echo $foo[<tab><tab><tab>
>  % echo $foo[\*.txt]
>
>  % echo $foo[*.txt]
>  blue
>
>  I had a look at _subscript but it didn't mean much to me. I thought
>  maybe there was a (q) too many but there are none at all.

Also a bit weird,
% foo[{]=bar
zsh: no matches found: foo[{]=bar
% foo[{a,b}]=bar
% echo $foo[\{a,b\}] #produced by tab complete
bar
% echo $foo[\{a,b}]
bar
% echo $foo[{a,b}]
bar
% echo $foo['{a,b}']


I guess I'm wandering into the land of very subtle rules here... :)

-- 
Mikael Magnusson

Re: _subscript quotes too much

[Peter Stephenson]

"Mikael Magnusson" wrote:
> On 09/04/2008, Mikael Magnusson <mikachu@gmail.com> wrote:
> > % zsh -f
> >  % autoload -U compinit; compinit
> >  % typeset -A foo
> >  % foo[bar]=baz
> >  % foo[*.txt]=blue
> >  % echo $foo[<tab><tab><tab>
> >  % echo $foo[\*.txt]
> >
> >  % echo $foo[*.txt]
> >  blue
> >
> >  I had a look at _subscript but it didn't mean much to me. I thought
> >  maybe there was a (q) too many but there are none at all.

As there are no special rules for subscripts once it gets to the adding
of completions (obviously the list is generated specially), they use the
normal rules that would apply for quoting command line arguments, which
quote globbing characters.

I've a vague feeling that not handling \* the same as * is a bug, but
quoting in subscripts is a horrible can of worms.  I would expect
backslash-quoting, at least, to be uniformly applied.

> Also a bit weird,
> % foo[{]=bar
> zsh: no matches found: foo[{]=bar
> % foo[{a,b}]=bar

I would certainly expect you to need quoting here, and foo[\{]=bar does
work.  The fact you get away with it the second time is luck.  Note
that only single argument substitution happens in subscripts.

> % echo $foo[\{a,b\}] #produced by tab complete
> bar
> % echo $foo[\{a,b}]
> bar
> % echo $foo[{a,b}]
> bar

Same comment, essentially, reinforcing my notion that not handling
$foo[\*.txt] is a bug.

> % echo $foo['{a,b}']

This is a side-effect of the fact that we're only doing single argument
substitution and not full argument parsing.  The quotes don't get parsed
as quotes.

> I guess I'm wandering into the land of very subtle rules here... :)

Oh, yes.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070

Re: _subscript quotes too much

[Peter Stephenson]

On Wed, 09 Apr 2008 09:49:04 +0100
Peter Stephenson <pws@csr.com> wrote:
> > >  % typeset -A foo
> > >  % foo[*.txt]=blue
> > >  % echo $foo[\*.txt]
> > >
> > >  % echo $foo[*.txt]
> > >  blue
>
> I've a vague feeling that not handling \* the same as * is a bug, but
> quoting in subscripts is a horrible can of worms.  I would expect
> backslash-quoting, at least, to be uniformly applied.

Hmm... it isn't, and this appears to be deliberate at at least two points
in the code: lex.c:dquote_parse() (where we carefully choose when to make
backslashes active) and params.c:getarg() (where we put most of the active
backslashes back to real backslashes anyway).  So we're actually going to
some lengths to make sure backslashes are *not* handled as quote characters
in subscripts but are retained.  This is documented in Subscript Parsing in
the zshparam manual, in fact, where there's a long essay about why it's so
difficult.  The paradigm is double quotes, where only some backslashed
characters are treated specially:

% print -r "\*"
\*

The usual workaround is to use a parameter.

As far as _subscript is concerned, it would have to be taught the rules for
which characters can be backslashed, this applied with ${...//.../...}, and
then the -Q flag passed to compadd.  This shouldn't be too hard.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070

Re: _subscript quotes too much

[Peter Stephenson]

On Wed, 9 Apr 2008 12:00:25 +0100
Peter Stephenson <pws@csr.com> wrote:
> As far as _subscript is concerned, it would have to be taught the rules for
> which characters can be backslashed, this applied with ${...//.../...}, and
> then the -Q flag passed to compadd.  This shouldn't be too hard.

It wasn't.  If you can find cases that still don't complete successfully
it can be tweaked further.

Index: Completion/Zsh/Context/_subscript
===================================================================
RCS file: /cvsroot/zsh/zsh/Completion/Zsh/Context/_subscript,v
retrieving revision 1.12
diff -u -r1.12 _subscript
--- Completion/Zsh/Context/_subscript	13 Dec 2007 21:38:42 -0000	1.12
+++ Completion/Zsh/Context/_subscript	9 Apr 2008 11:18:39 -0000
@@ -67,12 +67,14 @@
 
   _values -s '' 'subscript flags' $flags
 elif [[ ${(Pt)${compstate[parameter]}} = assoc* ]]; then
-  local suf
+  local suf MATCH MBEGIN MEND
+  local -a keys
+  keys=(${${(kP)compstate[parameter]}//(#m)[\$\\\[\]\(\)\[\{\}]/\\$MATCH})
 
   [[ "$RBUFFER" != (|\\)\]* ]] && suf="$osuf"
 
   _wanted association-keys expl 'association key' \
-      compadd -S "$suf" -k "$compstate[parameter]"
+      compadd -Q -S "$suf" -a keys
 elif [[ ${(Pt)${compstate[parameter]}} = array* ]]; then
   local list i j ret=1 disp
 



-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070

Re: _subscript quotes too much

[Mikael Magnusson]

On 09/04/2008, Peter Stephenson <pws@csr.com> wrote:
> On Wed, 9 Apr 2008 12:00:25 +0100
>
> Peter Stephenson <pws@csr.com> wrote:
>
> > As far as _subscript is concerned, it would have to be taught the rules for
>  > which characters can be backslashed, this applied with ${...//.../...}, and
>  > then the -Q flag passed to compadd.  This shouldn't be too hard.
> It wasn't.

Thanks for the explanations and the fix. I read the section in
zshparam you referred to, not sure if i'm more or less confused now
but I think I found two typos.

"This  is  because parameter  expansions  may be surrounded balanced
braces, and subscript flags are introduced by balanced parenthesis."
should probably be 'surrounded by' and 'balanced parentheses'.

>  +  keys=(${${(kP)compstate[parameter]}//(#m)[\$\\\[\]\(\)\[\{\}]/\\$MATCH})
wow :).

> If you can find cases that still don't complete successfully
> it can be tweaked further.

Well,
typeset -A hi
hi=(\" hello)
echo $hi[<tab>

should ideally produce "$hi[\"]" but I don't even know if that's
possible or worth it. :) Or even
hi=(\* hello)
echo $hi[<tab>
would produce $hi[(e)*], but I guess at some point I just have to read
the manual and learn some of the subtler rules instead of depending on
the completion system as these are explained quite clearly there.

-- 
Mikael Magnusson

Re: _subscript quotes too much

[Peter Stephenson]

On Wed, 9 Apr 2008 13:53:00 +0200
"Mikael Magnusson" <mikachu@gmail.com> wrote:
> Well,
> typeset -A hi
> hi=(\" hello)
> echo $hi[<tab>
> 
> should ideally produce "$hi[\"]" but I don't even know if that's
> possible or worth it. :) Or even

Double quotes are really hard and I think I'll pretend they don't exist.
The fact that foo["]=stuff doesn't work (it's waiting for a closing quote)
while foo[\"]=stuff assigns to an element \", including the backslash,
doesn't fill me with zeal for handling this in completion.

> hi=(\* hello)
> echo $hi[<tab>
> would produce $hi[(e)*]

That's easy.

Index: Completion/Zsh/Context/_subscript
===================================================================
RCS file: /cvsroot/zsh/zsh/Completion/Zsh/Context/_subscript,v
retrieving revision 1.13
diff -u -r1.13 _subscript
--- Completion/Zsh/Context/_subscript	9 Apr 2008 11:25:41 -0000	1.13
+++ Completion/Zsh/Context/_subscript	9 Apr 2008 12:06:21 -0000
@@ -70,7 +70,7 @@
   local suf MATCH MBEGIN MEND
   local -a keys
   keys=(${${(kP)compstate[parameter]}//(#m)[\$\\\[\]\(\)\[\{\}]/\\$MATCH})
-
+  keys=(${keys//#%(#m)[*@]/(e)$MATCH})
   [[ "$RBUFFER" != (|\\)\]* ]] && suf="$osuf"
 
   _wanted association-keys expl 'association key' \
Index: Doc/Zsh/params.yo
===================================================================
RCS file: /cvsroot/zsh/zsh/Doc/Zsh/params.yo,v
retrieving revision 1.46
diff -u -r1.46 params.yo
--- Doc/Zsh/params.yo	3 Apr 2008 08:46:50 -0000	1.46
+++ Doc/Zsh/params.yo	9 Apr 2008 12:06:23 -0000
@@ -344,8 +344,8 @@
 braces (`tt({)' and `tt(})'): they must appear either in balanced pairs or
 preceded by a backslash, and backslashes that protect parentheses or
 braces are removed during parsing.  This is because parameter expansions
-may be surrounded balanced braces, and subscript flags are introduced by
-balanced parenthesis.
+may be surrounded by balanced braces, and subscript flags are introduced by
+balanced parentheses.
 
 The second difference is that a double-quote (`tt(")') may appear as part
 of a subscript expression without being preceded by a backslash, and




-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070