assoc array assignment problem.

assoc array assignment problem.

[Tanaka Akira]

Z(2):akr@localhost% Src/zsh -f
localhost% typeset -A arr
localhost% a='$b'
localhost% b='c'
localhost% arr[$a]=d
localhost% print -lr - ${(kv)arr}
c
d
localhost% 

Hm. Variable expansion is performed twice.
I think it should be:

localhost% print -lr - ${(kv)arr}
$b
d
localhost% 
-- 
Tanaka Akira

Re: assoc array assignment problem.

[Bart Schaefer]

On Sep 21, 10:37am, Tanaka Akira wrote:
} Subject: assoc array assignment problem.
}
} Z(2):akr@localhost% Src/zsh -f
} localhost% typeset -A arr
} localhost% a='$b'
} localhost% b='c'
} localhost% arr[$a]=d
} localhost% print -lr - ${(kv)arr}
} c
} d
} localhost% 
} 
} Hm. Variable expansion is performed twice.

This is not an associative array problem; it affects all arrays.  Note:

zagzig<17> q=()
zagzig<18> g=7
zagzig<19> f='$g'
zagzig<20> q[$f]=seven
zagzig<21> print $#q $q
7 seven
zagzig<22>

The first substitution happens at the expected time, during evaluation
of the line.  The second one happens at execute time in addvars() --
assignment is implemented as if `x=y somecommand` where somecommand is
null.  addvars() calls setsparam(), which discovers that there is no
parameter named "q[$g]" and therefore creates "q" and then makes a call
to getvalue() to retrieve a Value structure for the parameter that it
just created.  getvalue() then expands $g (as it would for ${q[$g]} in
which the stuff inside the ${ } has _not_ previously been expanded) and
you get the effect above.

And you can't even protect it with double-quoting, though using single
quotes counterintuitively (but expectedly, given the explaination above)
produces the desired result:

zagzig<26> arr['$a']=d
zagzig<27> print -lr - ${(kv)arr}
$x
d
zagzig<28>

I really have no idea when this got introduced.  It might have something
to do with zsh-workers/4826 and follow-ons, back last year; 3.1.5-pws-4
would be it, but I no longer have anything that old around to try.

The right thing would be for the parser to know that it shouldn't expand
inside the [] on the left-hand side of an assignment, thereby delaying
the one interesting expansion to getvalue(); but that may be a bit too
much to bite off ...

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Re: assoc array assignment problem.

[Peter Stephenson]

"Bart Schaefer" wrote:
> } Hm. Variable expansion is performed twice.
> 
> This is not an associative array problem; it affects all arrays.  Note:
> 
> zagzig<17> q=()
> zagzig<18> g=7
> zagzig<19> f='$g'
> zagzig<20> q[$f]=seven
> zagzig<21> print $#q $q
> 7 seven
> zagzig<22>
> 
> The first substitution happens at the expected time, during evaluation
> of the line.  The second one happens at execute time in addvars() --
> assignment is implemented as if `x=y somecommand` where somecommand is
> null.

Two comments:

- addvars() is performing an initial substitution on the LHS, but the
  parameter expansion code isn't; with $q[$f], nothing gets substituted
  until the look up for q[$f] on hitting the first `$'.  I don't
  see why addvars() needs to do that initial substitution, though:
  in the old days, you could do `foo=bar; $foo=something' to set bar,
  but now you can't.  So I suspect that first substitution may be redundant,
  and that all substitution should be delayed until the parameter code,
  as it is when a parameter is being used rather than asssigned.
  However, this does untokenize the name, which maybe partly explains the
  following behaviour.
- The weirdest thing seems to me to be that getindex() is untokenizing
  the [...] part, only for it to be retokenized again down in getarg().
  This can't be right, can it?  It would be far better if what getindex()
  gets is correctly tokenized according to whether or not you want it
  substituted (including not untokenizing it in addvars()), and if that was
  respected this problem would go away.  That should be OK, because
  the subscript should already be tokenized in just the same way
  as the parameter expansion.  Shouldn't it?
- (three comments, among the comments are): Looking at addvars(),
  it seems that the following (with no existing $foo):
    setopt restricted
    foo=bar cat /dev/null
  could be bad for business, but AIX is unable even to give me the
  satisfaction of a core dump.

I get fed up with this shell sometimes.  Maybe I'll take a look at this,
but I'd prefer to retreat into a hole in the ground.

-- 
Peter Stephenson <pws@ibmth.df.unipi.it>       Tel: +39 050 844536
WWW:  http://www.ifh.de/~pws/
Dipartimento di Fisica, Via Buonarroti 2, 56127 Pisa, Italy

Re: assoc array assignment problem.

[Peter Stephenson]

I wrote:
> Two comments:

This is supposed to take care of the first and the third.  Now the
following works:

typeset -A assoc
foo=random
arg='$foo'
assoc[$arg]=bar
print ${(kv)assoc}
$foo bar

and the restricted mode crash should go, too.  I don't think either of
these should have adverse effects.

The other issue --- tokenization and retokenization of the subscript ---
turns out to be more complicated.  In an expression like "$foo[(r)b*]", you
would indeed need to tokenize what you have already.  Perhaps that could be
done by paramsubst(), which knows if it's in quotes, but as the code that
uses this is buried in fetchvalue() it's hard to be sure.  Alternatively,
it could be given a flag that it was in double quotes and so needed (with
certain flags only) to retokenize the expression.  Alternatively, it could
be left as it is with consequent inefficiency.

I discovered another problem.

foo=
assoc[$foo]=rod
print ${(kv)assoc}
 rod $foo bar
unset assoc[$foo] assoc[] "assoc[$foo]" "assoc[]"
print ${(kv)assoc}
 rod $foo bar

I haven't found anything which gets rid of it.

--- Src/exec.c.subst	Tue Sep 28 13:58:30 1999
+++ Src/exec.c	Tue Sep 28 14:37:08 1999
@@ -1336,9 +1336,6 @@
     for (n = firstnode(l); n; incnode(n)) {
 	v = (Varasg) getdata(n);
 	name = dupstring(v->name);
-	singsub(&name);
-	if (errflag)
-	    return;
 	untokenize(name);
 	if (xtr)
 	    fprintf(stderr, "%s=", name);
@@ -1370,15 +1367,13 @@
 	    }
 	    if (xtr)
 		fprintf(stderr, "%s ", val);
-	    if (export) {
-		if (export < 0) {
-		    /* We are going to fork so do not bother freeing this */
-		    pm = (Param) paramtab->removenode(paramtab, name);
-		    if (isset(RESTRICTED) && (pm->flags & PM_RESTRICTED)) {
-			zerr("%s: restricted", pm->nam, 0);
-			zsfree(val);
-			return;
-		    }
+	    if (export && !strchr(name, '[')) {
+		if (export < 0 && isset(RESTRICTED) &&
+		    (pm = (Param) paramtab->removenode(paramtab, name)) &&
+		    (pm->flags & PM_RESTRICTED)) {
+		    zerr("%s: restricted", pm->nam, 0);
+		    zsfree(val);
+		    return;
 		}
 		allexp = opts[ALLEXPORT];
 		opts[ALLEXPORT] = 1;

-- 
Peter Stephenson <pws@ibmth.df.unipi.it>       Tel: +39 050 844536
WWW:  http://www.ifh.de/~pws/
Dipartimento di Fisica, Via Buonarroti 2, 56127 Pisa, Italy

Re: assoc array assignment problem.

[Bart Schaefer]

On Sep 28,  3:04pm, Peter Stephenson wrote:
} Subject: Re: assoc array assignment problem.
}
} I discovered another problem.
} 
} foo=
} assoc[$foo]=rod
} print ${(kv)assoc}
}  rod $foo bar

The fact that you get a leading space there even when there are no quotes
around the expansion should be telling us something.  If the key were
the empty string, print would drop it and you'd get "rod $foo bar" with
no leading space.  The key for the "rod" value in the $assoc hash table
is "\233" (meta-escape).

Now, the "\233" is coming from singsub(&s) where s = "$foo" at line 863
in getarg().  So perhaps the following will take care of it; I've tried
a few tests and didn't find anything that it breaks, but maybe someone
else will discover a reason this needs to be more complicated.

Index: Src/params.c
===================================================================
@@ -869,6 +869,7 @@
 		ht = newparamtable(17, v->pm->nam);
 		v->pm->sets.hfn(v->pm, ht);
 	    }
+	    untokenize(s);
 	    if (!(v->pm = (Param) ht->getnode(ht, s))) {
 		HashTable tht = paramtab;
 		paramtab = ht;


-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com