PATCH: Assorted parameter stuff

PATCH: Assorted parameter stuff

[Bart Schaefer]

The main point of this patch is to address Andrej's message from last August
("SourceForge bug id 104052 - case study") and related issues with parsing
array subscripts.  Some other stuff got tweaked in passing.

I'll hold off committing this until a couple of you have applied it and say
it seems OK.  The "make check" tests all pass for me, at least (and it was
the completion tests, rather than the parameter tests, that gave it the best
workout).

Andrej noted three cases:

} Case 1.
} 
} bor@itsrm2% typeset -A foo
} bor@itsrm2% foo[\]]=bar
} zsh: not an identifier: foo[\]]

With the patch below, that correctly assigns `bar' to the array element
with the index `]' (not `\]').

} Case 2.
} 
} bor@itsrm2% print $foo[\]]
} ]
} bor@itsrm2% print ${foo[\]]}
} zsh: bad substitution

Both of these are fixed, too, and should both print `bar'.

} Case 3.
} 
} bor@itsrm2% print ${(kv)foo}
} a bar abc baz
} bor@itsrm2% print $foo[(I)[^]]]
} bar] <== that I do not understand at all!

This happens in part because the following also happens:

% print z]
z]

Note that an unmatched close-bracket is not an error, only an unmatched open
bracket is (bad pattern). $foo[(I)[^]]] should give "bad pattern" for `[^]',
but that error is ignored during subscript parsing (and I did not find any
reasonable way to propagate it).  Ignoring the error means that the (I) flag
is also ignored, with the result that ${${foo}[0]} is returned, which in the
example is `bar' -- hence `bar]' is printed.

This bug is not fixed, but braces around the original substitution reveal
the error:

% print ${foo[(I)[^]]]} 
zsh: bad substitution

However, with a correct pattern the correct result is obtained:

% print $foo[(I)[^\]]]
a
% print ${foo[(I)[^\]]]}
a

Note that one must use [^\]] so that the "unquoted" [ ] will balance.  The
backslash is now removed when the subscript is processed.

There is one remaining problem:  foo[*] of course refers to the entire
array foo, but foo[\*] refers to the element indexed by `\*'; so it is
still the case that the only way to assign or refer to to an element
named `*' is to use `x="*"; foo[$x]=value; : $foo[$x]'.  Maybe someone
else will see an obvious solution to that one.

Andrej said:

} Case 1 is caused by the fact, that setsparam() gets unmetafied
} parameter name. [...] Case 1 is basically independent of other two.
} Unfortunately, to solve it we need to either remove call to isident()
} in setsparam() or pass metafied parameter name to setsparam() and make
} isident() smart enough.

In fact, I did make isident() smarter, but I didn't pass it a metafied
parameter name.  Instead it calls a new function parse_subscript(),
which invokes dquote_parse() to process the matching [ ] (as if they
were a $[...] math expression, which turns out to work just fine for
subscripting).

} Case 2 comes from the fact, that paramsubst() and getindex() treat
} both quoted and unquoted brackets the same. [... The] lexer should
} return different token for "]" as for \], e.g. Qoutbrack like Qstring.

This turns out not to be necessary; subscripts are parsed recursively
(after a very tangled fashion), so as long as each level can be parsed
properly everything will work out fine.  getindex() now also calls the
parse_subscript() function to manage this.  However, I have not checked
how many backslashes are necessary to protect `]' when it's in a deeply
nested subscript expression ...

There's probably a lot of room for optimization here, particularly in
the `needtok' computation in getarg(), which may not be necessary any
more when the entire subscript has already been parsed.

} Case 3 the problem seems to be getarg(). It blindly skips over
} balanced brackets that is of course wrong in this case.

Because getindex() now does a more sophisticated parse of the subscript,
getarg() always gets the subscript untokenized and delimited by Inbrack
and Outbrack, and so it no longer needs to count matching pairs of both
tokenized and untokenized brackets.  I didn't remove the code to count
the matching pairs, but it now looks only for Inbrack/Outbrack and I
don't think it can ever find more than the one Outbrack (but I wasn't
entirely sure).

The final touch was to leave INULL() characters alone in the untokenize
loop in getindex() and then kill them with a well-placed remnulargs()
before using the subscript to index the array or hash.  I was hopeful
that this would also fix the problem with $scalar[(i)${(q)pat}], but
it does not ...

The only side-effect I found of all this was that math expressions that
assign to subscripted values need to untokenize() because some of the
manipulations done by getindex() are done in-place in the input string.

Finally, while looking at this, I noticed that `typeset' didn't seem to
be tracking the Param structs all the way through, and that once it did
so, the isident() test before calling typeset_single() became redundant.

Then I kept being baffled about why $HISTFILE was being fetched in the
midst of many other paramter substitutions.  Well, gee, some of those
substitutions use the parser, and hence initialized the history buffers;
but they never need the file written, so fetching $HISTFILE was a bit
premature at the point where it was done.

There's one last little hunk that fixes Michal's Debian complaint about
$scalar[(i)notfound] returning 0 instead of $(($#scalar + 1)).  Whew.

Index: Src/builtin.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/builtin.c,v
retrieving revision 1.43
diff -u -r1.43 builtin.c
--- Src/builtin.c	2001/03/29 10:52:15	1.43
+++ Src/builtin.c	2001/04/15 06:28:53
@@ -1690,8 +1690,8 @@
 		delenv(pm->env);
 		pm->env = NULL;
 	    }
-	    if (value)
-		setsparam(pname, ztrdup(value));
+	    if (value && !(pm = setsparam(pname, ztrdup(value))))
+		return 0;
 	} else if (value) {
 	    zwarnnam(cname, "can't assign new value for array %s", pname, 0);
 	    return NULL;
@@ -1807,9 +1807,10 @@
 	pm->level = keeplocal;
     else if (on & PM_LOCAL)
 	pm->level = locallevel;
-    if (value && !(pm->flags & (PM_ARRAY|PM_HASHED)))
-	setsparam(pname, ztrdup(value));
-    else if (newspecial && !(pm->old->flags & PM_NORESTORE)) {
+    if (value && !(pm->flags & (PM_ARRAY|PM_HASHED))) {
+	if (!(pm = setsparam(pname, ztrdup(value))))
+	    return 0;
+    } else if (newspecial && !(pm->old->flags & PM_NORESTORE)) {
 	/*
 	 * We need to use the special setting function to re-initialise
 	 * the special parameter to empty.
@@ -2061,12 +2062,6 @@
 
     /* Take arguments literally.  Don't glob */
     while ((asg = getasg(*argv++))) {
-	/* check if argument is a valid identifier */
-	if (!isident(asg->name)) {
-	    zerr("not an identifier: %s", asg->name, 0);
-	    returnval = 1;
-	    continue;
-	}
 	if (!typeset_single(name, asg->name,
 			    (Param) (paramtab == realparamtab ?
 				     gethashnode2(paramtab, asg->name) :
Index: Src/hist.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/hist.c,v
retrieving revision 1.24
diff -u -r1.24 hist.c
--- Src/hist.c	2001/04/10 18:03:58	1.24
+++ Src/hist.c	2001/04/15 06:29:08
@@ -1011,7 +1011,6 @@
     DPUTS(stophist != 2 && !(inbufflags & INP_ALIAS) && !chline,
 	  "BUG: chline is NULL in hend()");
     queue_signals();
-    hf = getsparam("HISTFILE");
     if (histdone & HISTFLAG_SETTY)
 	settyinfo(&shttyinfo);
     if (!(histactive & HA_NOINC))
@@ -1028,6 +1027,7 @@
      && (hist_ignore_all_dups = isset(HISTIGNOREALLDUPS)) != 0)
 	histremovedups();
     /* For history sharing, lock history file once for both read and write */
+    hf = getsparam("HISTFILE");
     if (isset(SHAREHISTORY) && lockhistfile(hf, 0)) {
 	readhistfile(hf, 0, HFILE_USE_OPTIONS | HFILE_FAST);
 	curline.histnum = curhist+1;
Index: Src/lex.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/lex.c,v
retrieving revision 1.16
diff -u -r1.16 lex.c
--- Src/lex.c	2001/03/06 13:00:41	1.16
+++ Src/lex.c	2001/04/15 06:29:18
@@ -1458,6 +1458,38 @@
     return err;
 }
 
+/**/
+mod_export char *
+parse_subscript(char *s)
+{
+    int l = strlen(s), err;
+    char *t;
+
+    if (!*s || *s == ']')
+	return 0;
+    lexsave();
+    untokenize(t = dupstring(s));
+    inpush(t, 0, NULL);
+    strinbeg(0);
+    len = 0;
+    bptr = tokstr = s;
+    bsiz = l + 1;
+    err = dquote_parse(']', 1);
+    if (err) {
+	err = *bptr;
+	*bptr = 0;
+	untokenize(s);
+	*bptr = err;
+	s = 0;
+    } else
+	s = bptr;
+    strinend();
+    inpop();
+    DPUTS(cmdsp, "BUG: parse_subscript: cmdstack not empty.");
+    lexrestore();
+    return s;
+}
+
 /* Tokenize a string given in s. Parsing is done as if s were a normal *
  * command-line argument but it may contain separators.  This is used  *
  * to parse the right-hand side of ${...%...} substitutions.           */
Index: Src/math.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/math.c,v
retrieving revision 1.8
diff -u -r1.8 math.c
--- Src/math.c	2001/01/16 13:44:20	1.8
+++ Src/math.c	2001/04/15 06:29:26
@@ -517,6 +517,7 @@
     }
     if (noeval)
 	return v;
+    untokenize(s);
     setnparam(s, v);
     return v;
 }
Index: Src/params.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/params.c,v
retrieving revision 1.36
diff -u -r1.36 params.c
--- Src/params.c	2001/04/06 07:55:13	1.36
+++ Src/params.c	2001/04/15 06:29:49
@@ -770,38 +770,18 @@
 	if (!iident(*ss))
 	    break;
 
-#if 0
-    /* If this exhaust `s' or the next two characters *
-     * are [(, then it is a valid identifier.         */
-    if (!*ss || (*ss == '[' && ss[1] == '('))
-	return 1;
-
-    /* Else if the next character is not [, then it is *
-     * definitely not a valid identifier.              */
-    if (*ss != '[')
-	return 0;
-
-    noeval = 1;
-    (void)mathevalarg(++ss, &ss);
-    if (*ss == ',')
-	(void)mathevalarg(++ss, &ss);
-    noeval = ne;		/* restore the value of noeval */
-    if (*ss != ']' || ss[1])
-	return 0;
-    return 1;
-#else
     /* If the next character is not [, then it is *
-     * definitely not a valid identifier.              */
+     * definitely not a valid identifier.         */
     if (!*ss)
 	return 1;
     if (*ss != '[')
 	return 0;
 
-    /* Require balanced [ ] pairs */
-    if (skipparens('[', ']', &ss))
+    /* Require balanced [ ] pairs with something between */
+    if (!(ss = parse_subscript(++ss)))
 	return 0;
-    return !*ss;
-#endif
+    untokenize(s);
+    return !ss[1];
 }
 
 /**/
@@ -933,11 +913,11 @@
     }
 
     for (t = s, i = 0;
-	 (c = *t) && ((c != ']' && c != Outbrack &&
+	 (c = *t) && ((c != Outbrack &&
 		       (ishash || c != ',')) || i); t++) {
-	if (c == '[' || c == Inbrack)
+	if (c == Inbrack)
 	    i++;
-	else if (c == ']' || c == Outbrack)
+	else if (c == Outbrack)
 	    i--;
 	if (ispecial(c))
 	    needtok = 1;
@@ -945,6 +925,7 @@
     if (!c)
 	return 0;
     s = dupstrpfx(s, t - s);
+    remnulargs(s);
     *str = tt = t;
     if (needtok) {
 	if (parsestr(s))
@@ -1156,7 +1137,7 @@
 				    return r;
 		    }
 		}
-		return 0;
+		return down ? 0 : len + 1;
 	    }
 	}
     }
@@ -1170,13 +1151,17 @@
     int start, end, inv = 0;
     char *s = *pptr, *tbrack;
 
-    *s++ = '[';
-    for (tbrack = s; *tbrack && *tbrack != ']' && *tbrack != Outbrack; tbrack++)
+    *s++ = Inbrack;
+    s = parse_subscript(s); /* Error handled elsewhere */
+    for (tbrack = *pptr + 1; *tbrack && tbrack != s; tbrack++) {
+	if (INULL(*tbrack) && !*++tbrack)
+	    break;
 	if (itok(*tbrack))
 	    *tbrack = ztokens[*tbrack - Pound];
-    if (*tbrack == Outbrack)
-	*tbrack = ']';
-    if ((s[0] == '*' || s[0] == '@') && s[1] == ']') {
+    }
+    *tbrack = Outbrack;
+    s = *pptr + 1;
+    if ((s[0] == '*' || s[0] == '@') && s[1] == Outbrack) {
 	if ((v->isarr || IS_UNSET_VALUE(v)) && s[0] == '@')
 	    v->isarr |= SCANPM_ISVAR_AT;
 	v->start = 0;
@@ -1208,12 +1193,12 @@
 	    }
 	    if (*s == ',') {
 		zerr("invalid subscript", NULL, 0);
-		while (*s != ']' && *s != Outbrack)
+		while (*s != Outbrack)
 		    s++;
 		*pptr = s;
 		return 1;
 	    }
-	    if (*s == ']' || *s == Outbrack)
+	    if (*s == Outbrack)
 		s++;
 	} else {
 	    int com;
@@ -1228,7 +1213,7 @@
 		start--;
 	    else if (start == 0 && end == 0)
 		end++;
-	    if (*s == ']' || *s == Outbrack) {
+	    if (*s == Outbrack) {
 		s++;
 		if (v->isarr && start == end-1 && !com &&
 		    (!(v->isarr & SCANPM_MATCHMANY) ||


-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Example from manual seems broken

[F. G. Marx]

1. The following example from zshexpn.1 FILENAME_EXPANSION is broken in
   zsh-4.0.1-pre-3:

    zsh -f
    $> setopt extendedglob
    $> foo="a string with a message"
    $> if [[ $foo = (a|an)' '(#b)(*)' '* ]]; then
    then> print ${foo[$mbegin[1],$mend[1]]}
    then> fi
    $> zsh: bad math expression: operand expected at `$mend[1]'

    print ${foo[$mbegin[1],15]}
     and
    print ${foo[3,$mend[1]]}
     work fine.

2. I've got _normal complaining about _use_lo not being found after
   upgrading from dev-2 to dev-3.

    $> objdump --<C-d>
    _use_lo not found at _normal:100

    _use_lo from dev-2 works fine.

3. I'm sorry, I wasn't paying attention: why are the completion functions
   in new sub-subdirectories in source, but amalgamated in --fndir ?

Thanks,

/f

Re: Example from manual seems broken

[Bart Schaefer]

On Apr 16, 11:54pm, F. G. Marx wrote:
}
} 
} 1. The following example from zshexpn.1 FILENAME_EXPANSION is broken in
}    zsh-4.0.1-pre-3:
} 
}     zsh -f
}     $> setopt extendedglob
}     $> foo="a string with a message"
}     $> if [[ $foo = (a|an)' '(#b)(*)' '* ]]; then
}     then> print ${foo[$mbegin[1],$mend[1]]}
}     then> fi
}     $> zsh: bad math expression: operand expected at `$mend[1]'

I don't know exactly why this was broken, but my patch in 13992 fixes it.
So maybe I should commit that patch after all.

} 2. I've got _normal complaining about _use_lo not being found after
}    upgrading from dev-2 to dev-3.

Remove your ~/.zcompdump file and let compinit rebuild it.  _use_lo has
been renamed to _gnu_generic.

} 3. I'm sorry, I wasn't paying attention: why are the completion functions
}    in new sub-subdirectories in source, but amalgamated in --fndir ?

The sub-subdirs are for organizational convenience of the developers, for
the most part.  If one is writing a new completion function it helps to
be able to find other functions that serve a similar purpose.  The subdirs
(one level up) are for selective installation, so that e.g. you can leave
out the X11 functions on a system with no X display, or leave out the BSD
functions on Linux, etc.

However, it was deemed unlikely that anyone would e.g. want to install the
Unix/Command collection without also installing the Unix/Type collection
on which it depends, etc.; and the $fpath gets rather unwieldy with all
those subdirectories.  So installation removes one level if you give the
--enable-function-subdirs configuration option, and removes both levels
otherwise.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Patterns quoting in subscript (was: Re: PATCH: Assorted parameter stuff)

[Andrej Borsenkow]

On Sun, 15 Apr 2001, Bart Schaefer wrote:

> The main point of this patch is to address Andrej's message from last August
> ("SourceForge bug id 104052 - case study") and related issues with parsing
> array subscripts.  Some other stuff got tweaked in passing.
>
> I'll hold off committing this until a couple of you have applied it and say
> it seems OK.  The "make check" tests all pass for me, at least (and it was
> the completion tests, rather than the parameter tests, that gave it the best
> workout).
>

At least one thing has changed (but I consider behaviour broken in
anyway):

bor@itsrm2:~%> foo=(a '?' '\?')
bor@itsrm2:~%> print -r $foo[(r)?]
a
bor@itsrm2:~%> print -r $foo[(r)\?]
?
bor@itsrm2:~%> print -r $foo[(r)\\?]
?
bor@itsrm2:~%> print -r $foo[(r)\\\?]
?

without  patch you get:

bor@itsrm2:~%> foo=(a '?' '\?')
bor@itsrm2:~%> print -r $foo[(r)?]
a
bor@itsrm2:~%> print -r $foo[(r)\?]
?
bor@itsrm2:~%> print -r $foo[(r)\\?]
?
bor@itsrm2:~%> print -r $foo[(r)\\\?]
\?
bor@itsrm2:~%> print -r $foo[(r)\\\\?]
\?
bor@itsrm2:~%> print -r $foo[(r)\\\\\?]
\?
etc etc

Both are broken. Without patch you have at least some (albeit completely
unclear) possibility to quote patterns in subscript - with patch even this
one is gone.

I think, we have to decide on how quoting is actually done and finally
implement it. I have very strong inclination to not reinvent te whell and
simply use current quoting rules *including* $~foo (that as response to
another message). We still need special treatment for patterns inside
double quotes (else no subscripting is possible) - with this patch parsing
of subscription is already done in seperate function so it should be
possible.

Suggestion is something like

$foo[()pattern] - is parsed just like a "normal" globbing pattern, taking
in account any quoting. I.e. in $foo[(r)\*$bar] neither ``*'' nor contents
of $bar is taken as pattern.

Special consideration is needed for double quotes. Because normally
"$foo[(r)\?]" is parsed as *two* characters - ``\'' and ``?'', to allow
quoting inside double-quotes extension of quoting rules is needed. I.e.
``\'' should be allowed to quote pattern metacharacters inside of
subscript. I am not sure what impact it possibly have on compatibility ...
OTOH if foo is array, zsh will always parse "$foo[*]" differently from
/bin/sh even in compatibility mode.


-andrej

Re: PATCH: Assorted parameter stuff

[Bart Schaefer]

On Apr 15,  8:06am, Bart Schaefer wrote:
}
} I'll hold off committing this until a couple of you have applied it
} and say it seems OK.

Everybody must be on spring break.

On Apr 17,  4:45am, Bart Schaefer wrote:
}
} On Apr 16, 11:54pm, F. G. Marx wrote:
} }
} }     then> print ${foo[$mbegin[1],$mend[1]]}
} }     $> zsh: bad math expression: operand expected at `$mend[1]'
} 
} I don't know exactly why this was broken, but my patch in 13992 fixes it.
} So maybe I should commit that patch after all.

I've now got one additional thing fixed.  In any version of zsh up to
4.0.1-pre-3, try this:

% foo='any random string'
% echo $foo[(i)ran
0(i)ran

The correct output would be one of these:

    zsh: invalid subscript

    zsh: bad pattern: any random string[(i)ran

    zsh: bad pattern: string[(i)ran			(shwordsplit)

    any random string[(i)ran				(nobadpattern)

It's fairly difficult to get zsh to treat that open-bracket as anything
other than the start of a subscript, so the first of those is easiest,
and is what I've done.  However, that also means that:

% echo $foo[ran
zsh: invalid subscript

In 4.0.1-pre-2 that gives `any random string[ran' (regardless of the
setting of badpattern, which is arguably also wrong).

One additional question:  I think it'd be straightforward to make this
be parsed as all one word:

% echo $foo[(i)dom str]

I.e. the space inside the [ ] could implicitly be quoted.  Currently
it's necessary to use "$foo[(i)dom str]" to get an embedded space in a
subscript.  I can't think of any way in which such a change would break
a currently-working script.  Is it worth trying?

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Re: Patterns quoting in subscript (was: Re: PATCH: Assorted parameter stuff)

[Bart Schaefer]

On Apr 17, 10:30pm, Andrej Borsenkow wrote:
}
} At least one thing has changed (but I consider behaviour broken in
} anyway):
} 
} bor@itsrm2:~%> foo=(a '?' '\?')
} bor@itsrm2:~%> print -r $foo[(r)?]
} a
} bor@itsrm2:~%> print -r $foo[(r)\?]
} ?
} bor@itsrm2:~%> print -r $foo[(r)\\?]
} ?
} bor@itsrm2:~%> print -r $foo[(r)\\\?]
} ?

You just didn't try enough backslashes yet:

schaefer[507] print -r $foo[(r)\\\\\?]
\?

The reason you need five is that [...] is parsed as if double quoted,
except that \] is magic instead of \".  Then the pattern inside it is
tokenized by (r) before being passed to patcompile().  So it goes like:

     parse        tokenize     pattern
    \\ \\ \?  -->  \\ \?   -->   \?

(In double quotes, \\ --> \ but \? --> \?.)

} Both are broken. Without patch you have at least some (albeit completely
} unclear) possibility to quote patterns in subscript - with patch even this
} one is gone.

Not true.  Without the patch you can't explain the rationale for quoting,
but with the patch it's predictable.  Similarly, without the patch this
fails utterly:

schaefer[508] x='\?'
schaefer[509] print -r $foo[(r)${(q)x}]
\?

But with the patch it's possible to explain why it works:

     parse       tokenize     pattern
    ${(q)x}  -->  \\ \?   -->   \?
 
} Suggestion is something like
} 
} $foo[()pattern] - is parsed just like a "normal" globbing pattern, taking
} in account any quoting. I.e. in $foo[(r)\*$bar] neither ``*'' nor contents
} of $bar is taken as pattern.

Unfortunately, this is a significant change in zsh semantics.  I don't
think we have the opportunity to do this any longer.  It may be too bad
that this wasn't originally the spec, but I think we're stuck with the
current interpretation of [(r)$bar].
 
} Special consideration is needed for double quotes. Because normally
} "$foo[(r)\?]" is parsed as *two* characters - ``\'' and ``?'', to allow
} quoting inside double-quotes extension of quoting rules is needed. I.e.
} ``\'' should be allowed to quote pattern metacharacters inside of
} subscript.

You'll note that with my patch the quoting inside and outside of double
quotes works exactly the same (except of course for the double-quote
character itself).  That is,

schaefer[513] print -r "$foo[(r)\\\\\?]"
\?

Five backslashes, just as before; same result.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Re: Patterns quoting in subscript (was: Re: PATCH: Assorted parameter stuff)

[Bart Schaefer]

I wrote:
}
} Not true.  Without the patch you can't explain the rationale for quoting,
} but with the patch it's predictable.  Similarly, without the patch this
} fails utterly:
} 
} schaefer[508] x='\?'
} schaefer[509] print -r $foo[(r)${(q)x}]
} \?

I'm sorry, I got my examples confused.  The pattern that fails without
my patch is any one with a close-bracket; with my patch, it works just
like the one with '\?', i.e. five backslashes:

schaefer[514] foo=(a '?' '\?' '\]')
schaefer[515] print -r $foo[(r)\\\\\]]
\]

Sorry about the confusion.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Re: Patterns quoting in subscript (was: Re: PATCH: Assorted parameter stuff)

[Andrej Borsenkow]

On Wed, 18 Apr 2001, Bart Schaefer wrote:


>
> You just didn't try enough backslashes yet:
>
> schaefer[507] print -r $foo[(r)\\\\\?]
> \?
>
> The reason you need five is that [...] is parsed as if double quoted,
> except that \] is magic instead of \".  Then the pattern inside it is
> tokenized by (r) before being passed to patcompile().  So it goes like:
>
>      parse        tokenize     pattern
>     \\ \\ \?  -->  \\ \?   -->   \?
>
> (In double quotes, \\ --> \ but \? --> \?.)
>

O.K., any chance to explain the above in Zsh doc?

Just to make sure that I understand:

- "as if in double quotes" means, that I cannot use "..." or '...' quoting
inside of subscript, only back-slash is possible?

- the rule for ``\\'' and ``\$'' still aplies? But ``\"'' is parsed as
*two* characters?


-andrej

Re: PATCH: Assorted parameter stuff

[Sven Wischnowsky]

Bart Schaefer wrote:

> Everybody must be on spring break.

I forgot to copy it onto my laptop yesterday, sorry.  I'll give 14008 a try.

> ...
> 
> One additional question:  I think it'd be straightforward to make this
> be parsed as all one word:
> 
> % echo $foo[(i)dom str]
> 
> I.e. the space inside the [ ] could implicitly be quoted.  Currently
> it's necessary to use "$foo[(i)dom str]" to get an embedded space in a
> subscript.  I can't think of any way in which such a change would break
> a currently-working script.  Is it worth trying?

Oh, that would be wonderful.  I remember me fighting with that several
times (it can get pretty ugly with nested expansions and arrays).

Bye
  Sven


-- 
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

Re: Patterns quoting in subscript (was: Re: PATCH: Assorted parameter stuff)

[Andrej Borsenkow]

On Wed, 18 Apr 2001, Bart Schaefer wrote:

> On Apr 17, 10:30pm, Andrej Borsenkow wrote:
> }
> } At least one thing has changed (but I consider behaviour broken in
> } anyway):
> }
> } bor@itsrm2:~%> foo=(a '?' '\?')
> } bor@itsrm2:~%> print -r $foo[(r)?]
> } a
> } bor@itsrm2:~%> print -r $foo[(r)\?]
> } ?
> } bor@itsrm2:~%> print -r $foo[(r)\\?]
> } ?
> } bor@itsrm2:~%> print -r $foo[(r)\\\?]
> } ?
>
> You just didn't try enough backslashes yet:
>
> schaefer[507] print -r $foo[(r)\\\\\?]
> \?
>

O.K., but why when I add additional backslashes I still get a match?:

bor@itsrm2:~%> print -r $foo[(r)\\\\\\\\\?]
\?

?? It seems to happen for any number of extra backslashes.

-andrej

Re: Patterns quoting in subscript (was: Re: PATCH: Assorted parameter stuff)

[Bart Schaefer]

On Apr 18, 12:34pm, Andrej Borsenkow wrote:
} Subject: Re: Patterns quoting in subscript (was: Re: PATCH: Assorted param
}
} On Wed, 18 Apr 2001, Bart Schaefer wrote:
} >
} > schaefer[507] print -r $foo[(r)\\\\\?]
} > \?
} >
} > The reason you need five is that [...] is parsed as if double quoted,
} > except that \] is magic instead of \".  Then the pattern inside it is
} > tokenized by (r) before being passed to patcompile().  So it goes like:
} >
} >      parse        tokenize     pattern
} >     \\ \\ \?  -->  \\ \?   -->   \?
} >
} > (In double quotes, \\ --> \ but \? --> \?.)
} 
} O.K., any chance to explain the above in Zsh doc?

Yes, I just don't want to spend time writing doc if this change is going
to cause some other problem and therefore not be included.

} Just to make sure that I understand:
} 
} - "as if in double quotes" means, that I cannot use "..." or '...' quoting
} inside of subscript, only back-slash is possible?

It's exactly like quoting for $[...] math expressions, except that (as
of 14008) spaces are not permitted.  Which for a traditional numeric
subscript makes sense, as it *is* a math expression.
 
} - the rule for ``\\'' and ``\$'' still aplies? But ``\"'' is parsed as
} *two* characters?

Yes.

On Apr 18, 12:45pm, Andrej Borsenkow wrote:
}
} O.K., but why when I add additional backslashes I still get a match?:
} 
} bor@itsrm2:~%> print -r $foo[(r)\\\\\\\\\?]
} \?
} 
} ?? It seems to happen for any number of extra backslashes.

Well, sigh, patcompile() also apparently does a level of backslash-removal,
so i've forgotten one step in the explanation above.  It's something like:

       parse           tokenize      pattern     match
    \\ \\ \?       --> \\ \?     -->   \?    -->  \?
    \\ \\ \\ \\ \? --> \\ \\ \?  -->  \\ ?   -->  \?

Then you also have to remember that `?' as a metacharacter matches any
character, so one extra backslash will result in a pattern that matches
by wildcard rather than by a literal question-mark.  So if you want to
match 4 backslashes, you need 16 backslashes; but if you want to match
4 backslashes followed by a literal question mark, you need a whopping
34 backslashes, because you have to double all 16 of those backslashes
and then add two more to quote the `?' itself.

Clearly this is silly; it results because of the extra call to parsestr()
that happens when the `needtok' loop in getarg() detects an ispecial()
(the `?' in this example).  I'll see if that can't be made to behave a
bit more intelligibly.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Re: Patterns quoting in subscript (was: Re: PATCH: Assorted parameter stuff)

[Bart Schaefer]

On Apr 18,  5:26pm, Bart Schaefer wrote:
}
} [...] So if you want to match 4 backslashes, you need 16 backslashes;
} but if you want to match 4 backslashes followed by a literal question
} mark, you need a whopping 34 backslashes, because you have to double
} all 16 of those backslashes and then add two more to quote the `?'
} itself.
}
} Clearly this is silly; it results because of the extra call to
} parsestr() that happens when the `needtok' loop in getarg() detects an
} ispecial() (the `?' in this example). I'll see if that can't be made
} to behave a bit more intelligibly.

By relocating one remnulargs() call relative to 14008 and removing one
INULL() "optimization", it's possible to get a consistent 4-to-1 ratio
of backslashes in the input subscript to match one backslash in the
array value by pattern match.

That applies only with (i) (I) (r) (R); it's the usual 2-to-1 ratio for
a direct associative-array lookup.

The following goes on top of 14008.

--- current/Src/params.c	Tue Apr 17 23:30:50 2001
+++ ../zsh-4.0/Src/params.c	Wed Apr 18 11:56:33 2001
@@ -919,22 +919,19 @@
 	    i++;
 	else if (c == Outbrack)
 	    i--;
-	if (INULL(c)) {
-	    if (!*++t)
-		break;
-	} else if (ispecial(c))
+	if (ispecial(c))
 	    needtok = 1;
     }
     if (!c)
 	return 0;
     s = dupstrpfx(s, t - s);
-    remnulargs(s);
     *str = tt = t;
     if (needtok) {
 	if (parsestr(s))
 	    return 0;
 	singsub(&s);
     }
+    remnulargs(s);
     if (!rev) {
 	if (ishash) {
 	    HashTable ht = v->pm->gets.hfn(v->pm);
@@ -942,7 +939,6 @@
 		ht = newparamtable(17, v->pm->nam);
 		v->pm->sets.hfn(v->pm, ht);
 	    }
-	    remnulargs(s);
 	    untokenize(s);
 	    if (!(v->pm = (Param) ht->getnode(ht, s))) {
 		HashTable tht = paramtab;

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Re: PATCH: Assorted parameter stuff

[Bart Schaefer]

On Apr 18, 10:44am, Sven Wischnowsky wrote:
} Subject: Re: PATCH: Assorted parameter stuff
}
} Bart Schaefer wrote:
} 
} > One additional question:  I think it'd be straightforward to make this
} > be parsed as all one word:
} > 
} > % echo $foo[(i)dom str]
} > 
} > I.e. the space inside the [ ] could implicitly be quoted.
} 
} Oh, that would be wonderful.  I remember me fighting with that several
} times (it can get pretty ugly with nested expansions and arrays).

Well, it's not as straightforward as I thought -- the parse gets tangled
up the with parsing of [[ ... ]], among other things -- and I just noticed
that you can get the equivalent effect by surrounding the whole thing with
braces:

% foo='any random string'
% echo ${foo[(i)dom str]}
8

So I think I'm going to leave it at 14008+14016 unless somebody comes up
with other bugs to zap.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Re: PATCH: Assorted parameter stuff

[Sven Wischnowsky]

Bart Schaefer wrote:

> ...
> 
> Well, it's not as straightforward as I thought -- the parse gets tangled
> up the with parsing of [[ ... ]], among other things --

I feared that...

> and I just noticed
> that you can get the equivalent effect by surrounding the whole thing with
> braces:
> 
> % foo='any random string'
> % echo ${foo[(i)dom str]}
> 8
> 
> So I think I'm going to leave it at 14008+14016 unless somebody comes up
> with other bugs to zap.

[Blink, hm, why didn't I know that?...]


Anyway, about 14008: I only found one problem with a complicated
subscript in _arguments, line 81, which can be seen by completing after
`./configure --<TAB>'.  14016 doesn't seem to fix it either and I didn't
look closely enough to be able to say what exactly the problem is.

There may be other complicated subscripts in the completion code that
can serve as good tests, but none of the other completion functions I
tried failed and grepping for `complicated subscripts' isn't exactly easy.


Bye
  Sven


-- 
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

fpath problem on clean install

[Andrej Borsenkow]

I did almost clean install on Linux Mdk (hard to tell version - it is
mostly updated to the 8.0 state from 7.2). The fpath agains was set
incorrectly:

bor@localhost% zsh -f
localhost% echo $fpath
/usr/share/zsh/site-functions
/usr/share/zsh/4.0.1-pre-3/functions/Completion/*
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/usr/share/zsh/4.0.1-pre-3/functions/Completion
/usr/share/zsh/4.0.1-pre-3/functions/Misc
/usr/share/zsh/4.0.1-pre-3/functions/Prompts
/usr/share/zsh/4.0.1-pre-3/functions/Zftp
/usr/share/zsh/4.0.1-pre-3/functions/Zle

Looking in Makemod, it is expected result - I do not quite understand, how
I can get something different:

zshpaths.h: Makemod $(CONFIG_INCS)
......
            fpath_tmp="`grep ' functions=.' \
            $(dir_top)/config.modules | sed -e '/^#/d' -e '/ link=no/d' \
            -e 's/^.* functions=//'`"; \
            fpath_tmp=`for f in $$fpath_tmp; do \
              echo $$f | sed -e 's%^Functions/%%' -e 's%/[^/]*$$%%' -e
's%/\*%%'; \
            done | sort | uniq`; \

PLease, note, that it assumes dir/* syntax for functions. In reality we
have

bor@localhost% grep complete config.modules
name=zsh/complete modfile=Src/Zle/complete.mdd link=dynamic auto=yes
load=yes functions=Completion/comp* Completion/*/*/*

I wonder, how it can get installed at all ... O.K. without function
subdirs it works.

-andrej

Re: PATCH: Assorted parameter stuff

[Andrej Borsenkow]

On Thu, 19 Apr 2001, Sven Wischnowsky wrote:


>
>
> Anyway, about 14008: I only found one problem with a complicated
> subscript in _arguments, line 81, which can be seen by completing after
> `./configure --<TAB>'.  14016 doesn't seem to fix it either and I didn't
> look closely enough to be able to say what exactly the problem is.
>

There seems to be a problem quoting open bracket:

bor@localhost% foo=(a b c)
bor@localhost% print -r "$foo[(I)\]]"
bor@localhost% print -r "$foo[(I)[]"
zsh: invalid subscript
bor@localhost% print -r "$foo[(I)\[]"
zsh: invalid subscript
bor@localhost% print -r "$foo[(I)\\[]"
zsh: invalid subscript
bor@localhost% print -r "$foo[(I)\\\[]"
zsh: invalid subscript
bor@localhost% print -r "$foo[(I)\\\\[]"
zsh: invalid subscript

O.K. I think it already exceeds any reasonable number of backslashes :-)
This is with both patches of Bart.

4.0.1-pre-1 is even more funnier ...


bor@localhost% zsh-4.0.1-pre-1
bor@localhost% foo=(a b c)
bor@localhost% print $foo[(I)\[]
0(I)[]

so I ask myself if it ever worked

-andrej

Re: PATCH: Assorted parameter stuff

[Bart Schaefer]

On Apr 19,  9:05am, Sven Wischnowsky wrote:
} Subject: Re: PATCH: Assorted parameter stuff
}
} Anyway, about 14008: I only found one problem with a complicated
} subscript in _arguments, line 81, which can be seen by completing after
} `./configure --<TAB>'.  14016 doesn't seem to fix it either and I didn't
} look closely enough to be able to say what exactly the problem is.

Ah ha ha ha.  See the comment on line 79 about (( )) gives a parse error?
Well, parse_subscript() calls dquote_parse(), which is the guts of (( )).

Fortunately, I think I've fixed that (including the parse error in (( )),
though I haven't checked yet).  I encountered a number of other problems
while writing D06subscript.ztst, which I've just finished, and had to fix
several very slippery problems related to interaction of parsestr() with
the (un)tokenization in getindex() and getarg().

} There may be other complicated subscripts in the completion code that
} can serve as good tests

Oh, believe me, there were.  Getting Y01completion.ztst to pass during
"make check" was what got me as far as the original patch in 13992.

However, I'm now up way past my bedtime and can't keep working on this
tonight.  Hopefully I can finish it up sometime tomorrow.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Re: PATCH: Assorted parameter stuff

[Bart Schaefer]

On Apr 19,  1:20pm, Andrej Borsenkow wrote:
} Subject: Re: PATCH: Assorted parameter stuff
}
} There seems to be a problem quoting open bracket:

Yes, I found that while writing the test script, too.  I've fixed it; a
fragment of D06subscript.ztst:

  o='['
  c=']'
  A[\]]=cbrack
  A[\[]=obrack
  A[\\\[]=backobrack
  A[\\\]]=backcbrack
  print -R $A[$o] $A[$c] $A[\[] $A[\]] $A[\\\[] $A[\\\]]
  print -R $A[(i)\[] $A[(i)\]] $A[(i)\\\\\[] $A[(i)\\\\\]]
0:Associative array keys with open and close brackets
>obrack cbrack obrack cbrack backobrack backcbrack
>[ ] \[ \]

This passes with the code I have now, and hope to commit tomorrow night.

} 4.0.1-pre-1 is even more funnier ...
} so I ask myself if it ever worked

No, it never worked before, and quoting of `*' and probably also of `@'
still won't work even after I get done.  assoc[\*]=foo assigns to the
key `\*', not to the key `*', and assoc[*]=bar fails silently (I'm not
sure why it's silent).

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

PATCH: Re: fpath problem on clean install

[Sven Wischnowsky]

Andrej Borsenkow wrote:

> ...
> 
> bor@localhost% grep complete config.modules
> name=zsh/complete modfile=Src/Zle/complete.mdd link=dynamic auto=yes
> load=yes functions=Completion/comp* Completion/*/*/*

I don't get it... I'm absolutely sure that I'd typed in the change below
already.  But somehow it didn't make it into CVS.

Sorry.

Bye
  Sven

Index: Src/Zle/complete.mdd
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/complete.mdd,v
retrieving revision 1.5
diff -u -r1.5 complete.mdd
--- Src/Zle/complete.mdd	2001/04/02 13:04:05	1.5
+++ Src/Zle/complete.mdd	2001/04/19 09:48:44
@@ -1,7 +1,7 @@
 name=zsh/complete
 link=either
 load=yes
-functions='Completion/comp* Completion/*/*/*'
+functions='Completion/comp* Completion/AIX/*/* Completion/BSD/*/* Completion/Base/*/* Completion/Debian/*/* Completion/Redhat/*/* Completion/Unix/*/* Completion/X/*/* Completion/Zsh/*/*'
 
 moddeps="zsh/zle"
 

-- 
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

Re: PATCH: Re: fpath problem on clean install

[Andrej Borsenkow]

On Thu, 19 Apr 2001, Sven Wischnowsky wrote:

> I don't get it... I'm absolutely sure that I'd typed in the change below
> already.  But somehow it didn't make it into CVS.
>
> +functions='Completion/comp* Completion/AIX/*/* Completion/BSD/*/* Completion/Base/*/* Completion/Debian/*/* Completion/Redhat/*/* Completion/Unix/*/* Completion/X/*/* Completion/Zsh/*/*'
>


We need something different as long-term solution. It is way too easy to
forget updating this list when adding new subdirectory.

-andrej

Re: PATCH: Assorted parameter stuff

[Bart Schaefer]

On Apr 19,  9:34am, Bart Schaefer wrote:
}
} On Apr 19,  9:05am, Sven Wischnowsky wrote:
} }
} } Anyway, about 14008: I only found one problem with a complicated
} } subscript in _arguments, line 81, which can be seen by completing after
} } `./configure --<TAB>'.
} 
} Fortunately, I think I've fixed that (including the parse error in (( ))
} though I haven't checked yet).

I've tried replacing the `let' at _arguments line 81 with a (( )), and I
don't get a parse error.

But then, I don't get a parse error in 4.0.1-pre-2, either.

The only problem seems to be that when the entire $tmpargv[...] expression
resolves to the empty string, (( )) gives

    zsh: bad math expression: operand expected at `'

whereas `let' just ignores the empty string and returns false, which is
the desired effect.  Maybe that's what the comment on line 79 meant all
along.

Extremely strange things happen if I try (( "$tmpargv[...]" )) but those
strange things don't seem to have to do with my changes (the same odd
things happen in -pre-2) so I'm not going to worry about that just now.

} D06subscript.ztst, which I've just finished

Here it is, to demonstrate what works after my changes.  I've included
the diff for the last several tweaks that grew out of making the tests
all work; this patch goes on top of 14008, replacing 14016.  I added
several comments about what is going on.

In particular I direct your attention to the test for "Scalar pattern
subscripts that do not match".  This corresponds to some of the changes
that Michal suggested on the Debian bug list, to wit, (i) and (r) on
scalars work more like they do on arrays.  This means that in the
example tested, $s[(r)x,(R)x] yields the empty string and $s[(R)x,(r)x]
yields the entire $s, whereas before the patch both yielded `T'.  This
is the smallest but most potentially controversial item.

The test for "Array pattern subscripts with multiple backslashes" shows
that the subscript syntax is the same whether or not braces or double
quotes are used.

Index: Test/D06subscript.ztst
===================================================================
diff -N D06subscript.ztst
--- /dev/null	Mon Dec 11 17:26:27 2000
+++ Test/D06subscript.ztst	Thu Apr 19 21:50:36 2001
@@ -0,0 +1,130 @@
+# Test parameter subscripting.
+
+%prep
+
+  s='Twinkle, twinkle, little *, [how] I [wonder] what?  You are!'
+  a=('1' ']'  '?' '\2'  '\]' '\?'  '\\3' '\\]'  '\\?' '\\\4'  '\\\]' '\\\?')
+  typeset -g -A A
+  A=($a)
+
+%test
+
+  x=','
+  print $s[(i)winkle] $s[(I)winkle]
+  print ${s[(i)You are]} $#s
+  print ${s[(r)$x,(R)$x]}
+0:Scalar pattern subscripts without wildcards
+>2 11
+>53 60
+>, twinkle, little *,
+
+  x='*'
+  print $s[(i)*] $s[(i)\*] $s[(i)$x*] $s[(i)${(q)x}*] $s[(I)$x\*]
+  print $s[(r)?,(R)\?] $s[(r)\?,(R)?]
+  print $s[(r)\*,(R)*]
+  print $s[(r)\],(R)\[]
+0:Scalar pattern subscripts with wildcards
+>1 26 1 26 26
+>Twinkle, twinkle, little *, [how] I [wonder] what? ?  You are!
+>*, [how] I [wonder] what?  You are!
+>] I [
+
+  # $s[(R)x] actually is $s[0], but zsh treats 0 as 1 for subscripting.
+  print $s[(i)x] : $s[(I)x]
+  print $s[(r)x] : $s[(R)x]
+0:Scalar pattern subscripts that do not match
+>61 : 0
+>: T
+
+  print -R $s[$s[(i)\[]] $s[(i)$s[(r)\*]] $s[(i)${(q)s[(r)\]]}]
+0:Scalar subscripting using a pattern subscript to get the index
+>[ 1 33
+
+  print -R $a[(r)?] $a[(R)?]
+  print $a[(n:2:i)?] $a[(n:2:I)?]
+  print $a[(i)\?] $a[(I)\?]
+  print $a[(i)*] $a[(i)\*]
+0:Array pattern subscripts
+>1 ?
+>2 2
+>3 3
+>1 13
+
+  # It'd be nice to do some of the following with (r), but we run into
+  # limitations of the ztst script parsing of backslashes in the output.
+  print -R $a[(i)\\\\?] $a[(i)\\\\\?]
+  print -R $a[(i)\\\\\\\\?] $a[(i)\\\\\\\\\?]
+  print -R ${a[(i)\\\\\\\\?]} ${a[(i)\\\\\\\\\?]}
+  print -R "$a[(i)\\\\\\\\?] $a[(i)\\\\\\\\\?]"
+  print -R $a[(i)\]] $a[(i)\\\\\]] $a[(i)\\\\\\\\\]] $a[(i)\\\\\\\\\\\\\]]
+  print -R $a[(i)${(q)a[5]}] $a[(i)${(q)a[8]}] $a[(i)${(q)a[11]}]
+  print -R $a[(i)${a[3]}] $a[(i)${a[6]}] $a[(i)${a[9]}] $a[(i)${a[12]}]
+0:Array pattern subscripts with multiple backslashes
+>4 6
+>7 9
+>7 9
+>7 9
+>2 5 8 11
+>5 8 11
+>1 3 4 6
+
+  print -R $A[1] $A[?] $A[\\\\3] $A[\\\]]
+  print -R $A[$a[11]]
+  print -R $A[${(q)a[5]}]
+0:Associative array lookup (direct subscripting)
+>] \2 \\] \?
+>\\\?
+>\\\?
+
+  # The (o) is necessary here for predictable output ordering
+  print -R $A[(I)\?] ${(o)A[(I)?]}
+  print -R $A[(i)\\\\\\\\3]
+  print -R $A[(I)\\\\\\\\\?] ${(o)A[(I)\\\\\\\\?]}
+0:Associative array lookup (pattern subscripting)
+>? 1 ?
+>\\3
+>\\? \\3 \\?
+
+  print -R $A[(R)\?] : ${(o)A[(R)?]}
+  print -R $A[(R)\\\\\?] ${(o)A[(R)\\\\?]} ${(o)A[(R)\\\\\?]}
+  print -R ${(o)A[(R)\\\\\\\\\]]}
+0:Associative array lookup (reverse subscripting)
+>: ]
+>\? \2 \? \?
+>\\]
+
+  x='*'
+  A[$x]=xstar
+  A[${(q)x}]=qxstar
+  print -R ${(k)A[(r)xstar]} $A[$x]
+  print -R ${(k)A[(r)qxstar]} $A[${(q)x}]
+  # A[*] is interpreted specially, assignment to it fails silently (oops)
+  A[*]=star
+  A[\*]=backstar
+  print -R ${(k)A[(r)star]} $A[$x]
+  print -R ${(k)A[(r)backstar]} $A[\*]
+0:Associative array assignment
+>* xstar
+>\* qxstar
+>xstar
+>\* backstar
+
+  o='['
+  c=']'
+  A[\]]=cbrack
+  A[\[]=obrack
+  A[\\\[]=backobrack
+  A[\\\]]=backcbrack
+  print -R $A[$o] $A[$c] $A[\[] $A[\]] $A[\\\[] $A[\\\]]
+  print -R $A[(i)\[] $A[(i)\]] $A[(i)\\\\\[] $A[(i)\\\\\]]
+0:Associative array keys with open and close brackets
+>obrack cbrack obrack cbrack backobrack backcbrack
+>[ ] \[ \]
+
+  print -R $A[$o] $A[$s[(r)\[]]
+  print -R $A[(r)$c] $A[(r)$s[(r)\]]]
+  print -R $A[$A[(i)\\\\\]]]
+0:Associative array lookup using a pattern subscript to get the key
+>obrack obrack
+>] ]
+>backcbrack
Index: Src/lex.c
===================================================================
diff -x CVS -ru ../common/Src/lex.c Src/lex.c
--- ../common/Src/lex.c	Tue Apr 17 19:54:53 2001
+++ Src/lex.c	Thu Apr 19 21:20:35 2001
@@ -1302,10 +1302,13 @@
 	    c = hgetc();
 	    if (c != '\n') {
 		if (c == '$' || c == '\\' || (c == '}' && !intick && bct) ||
-		    c == endchar || c == '`')
+		    c == endchar || c == '`' ||
+		    (math && (c == '[' || c == ']' ||
+			      c == '(' || c == ')' ||
+			      c == '{' || c == '}')))
 		    add(Bnull);
 		else {
-		    /* lexstop is implicitely handled here */
+		    /* lexstop is implicitly handled here */
 		    add('\\');
 		    goto cont;
 		}
Index: Src/params.c
===================================================================
diff -x CVS -ru ../common/Src/params.c Src/params.c
--- ../common/Src/params.c	Wed Apr 18 10:35:30 2001
+++ Src/params.c	Thu Apr 19 21:21:16 2001
@@ -915,26 +915,42 @@
     for (t = s, i = 0;
 	 (c = *t) && ((c != Outbrack &&
 		       (ishash || c != ',')) || i); t++) {
-	if (c == Inbrack)
+	/* Untokenize INULL() except before brackets, for parsestr() */
+	if (INULL(c)) {
+	    if (t[1] == '[' || t[1] == ']') {
+		/* This test handles nested subscripts in hash keys */
+		if (ishash && i)
+		    *t = ztokens[c - Pound];
+		needtok = 1;
+		++t;
+	    } else
+		*t = ztokens[c - Pound];
+	    continue;
+	}
+	/* Inbrack and Outbrack are probably never found here ... */
+	if (c == '[' || c == Inbrack)
 	    i++;
-	else if (c == Outbrack)
+	else if (c == ']' || c == Outbrack)
 	    i--;
-	if (INULL(c)) {
-	    if (!*++t)
-		break;
-	} else if (ispecial(c))
+	if (ispecial(c))
 	    needtok = 1;
     }
     if (!c)
 	return 0;
     s = dupstrpfx(s, t - s);
-    remnulargs(s);
     *str = tt = t;
+    /* If we're NOT reverse subscripting, strip the INULL()s so brackets *
+     * are not backslashed after parsestr().  Otherwise leave them alone *
+     * so that the brackets will be escaped when we patcompile() or when *
+     * subscript arithmetic is performed (for nested subscripts).        */
+    if (ishash && !rev)
+	remnulargs(s);
     if (needtok) {
 	if (parsestr(s))
 	    return 0;
 	singsub(&s);
-    }
+    } else if (rev)
+	remnulargs(s);	/* This is probably always a no-op, but ... */
     if (!rev) {
 	if (ishash) {
 	    HashTable ht = v->pm->gets.hfn(v->pm);
@@ -942,7 +958,6 @@
 		ht = newparamtable(17, v->pm->nam);
 		v->pm->sets.hfn(v->pm, ht);
 	    }
-	    remnulargs(s);
 	    untokenize(s);
 	    if (!(v->pm = (Param) ht->getnode(ht, s))) {
 		HashTable tht = paramtab;
@@ -1157,13 +1172,17 @@
     char *s = *pptr, *tbrack;
 
     *s++ = Inbrack;
-    s = parse_subscript(s); /* Error handled elsewhere */
+    s = parse_subscript(s);	/* Error handled after untokenizing */
+    /* Now we untokenize everthing except INULL() markers so we can check *
+     * for the '*' and '@' special subscripts.  The INULL()s are removed  *
+     * in getarg() after we know whether we're doing reverse indexing.    */
     for (tbrack = *pptr + 1; *tbrack && tbrack != s; tbrack++) {
 	if (INULL(*tbrack) && !*++tbrack)
 	    break;
 	if (itok(*tbrack))
 	    *tbrack = ztokens[*tbrack - Pound];
     }
+    /* If we reached the end of the string (s == NULL) we have an error */
     if (*tbrack)
 	*tbrack = Outbrack;
     else {

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net