RC_EXPAND_PARAM bug

RC_EXPAND_PARAM bug

[Andrew Main]

What do you make of this (observed on 3.0.4 under Linux):

% a=(a b)
% x=(x y)
% echo ${^a}1${^x}
a1y b1x b1y

?

-zefram

Re: RC_EXPAND_PARAM bug

[Bart Schaefer]

On Jul 28,  2:42pm, Andrew Main wrote:
} Subject: RC_EXPAND_PARAM bug
}
} What do you make of this (observed on 3.0.4 under Linux):
} 
} % a=(a b)
} % x=(x y)
} % echo ${^a}1${^x}
} a1y b1x b1y
} 
} ?

It gets weirder:

% echo 1${^a}1${^^x}
1ay 1b1x 1by


-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Re: RC_EXPAND_PARAM bug

[Zoltan Hidvegi]

Bart Schaefer wrote:
> On Jul 28,  2:42pm, Andrew Main wrote:
[...]
> } % a=(a b)
> } % x=(x y)
> } % echo ${^a}1${^x}
> } a1y b1x b1y
> } 
> } ?
> 
> It gets weirder:
> 
> % echo 1${^a}1${^^x}
> 1ay 1b1x 1by

Below is a fix.  The second expansion might still look a bit weird:

% echo 1${^a}1${^^x}
1a1x 1ay 1b1x 1by


The logic is that the string after the rc-param, 1${^^x}, is expanded,
producing two strings, 1x y, which is combined with 1a 1b.  It is true
that this is incompatible with 2.6-beta16 and older, which first expanded
it to 1a1${^^x} 1b1${^^x} and later this was expanded to 1a1x y 1b1x y.
Similarily, let i=0; echo ${^a}$[i++] expanded to a$[i++] b$[i++] and
later to a0 b1, while in zsh-3.0.4 it expands to a0 b0.

Zoltan


*** Src/subst.c	1997/06/17 05:53:19	3.1.3.2
--- Src/subst.c	1997/07/29 05:40:45
***************
*** 1423,1430 ****
  			      (char *) getdata(tn), globsubst);
  		    if (qt && !*y && isarr != 2)
  			y = dupstring(nulstring);
! 		    if (i == 1)
! 			setdata(n, (void *) y);
  		    else
  			insertlinknode(l, n, (void *) y), incnode(n);
  		}
--- 1423,1430 ----
  			      (char *) getdata(tn), globsubst);
  		    if (qt && !*y && isarr != 2)
  			y = dupstring(nulstring);
! 		    if (plan9)
! 			setdata(n, (void *) y), plan9 = 0;
  		    else
  			insertlinknode(l, n, (void *) y), incnode(n);
  		}

Re: RC_EXPAND_PARAM bug

[Bart Schaefer]

On Jul 29,  2:04am, Zoltan Hidvegi wrote:
> 
> % echo 1${^a}1${^^x}
> 1a1x 1ay 1b1x 1by
> 
> The logic is that the string after the rc-param, 1${^^x}, is expanded,
> producing two strings, 1x y, which is combined with 1a 1b.  It is true
> that this is incompatible with 2.6-beta16 and older, which first expanded
> it to 1a1${^^x} 1b1${^^x} and later this was expanded to 1a1x y 1b1x y.
> Similarily, let i=0; echo ${^a}$[i++] expanded to a$[i++] b$[i++] and
> later to a0 b1, while in zsh-3.0.4 it expands to a0 b0.

Can you generalize this rule for us?  E.g.

% echo ${^a}$[i++]$[++j]${^x}....

where .... is some arbitary number of other substitutions?  Is it just that
it now does everything from right to left instead of left to right?  Why?

Re: RC_EXPAND_PARAM bug

[Zoltan Hidvegi]

> On Jul 29,  2:04am, Zoltan Hidvegi wrote:
> > 
> > % echo 1${^a}1${^^x}
> > 1a1x 1ay 1b1x 1by
> > 
> > The logic is that the string after the rc-param, 1${^^x}, is expanded,
> > producing two strings, 1x y, which is combined with 1a 1b.  It is true
> > that this is incompatible with 2.6-beta16 and older, which first expanded
> > it to 1a1${^^x} 1b1${^^x} and later this was expanded to 1a1x y 1b1x y.
> > Similarily, let i=0; echo ${^a}$[i++] expanded to a$[i++] b$[i++] and
> > later to a0 b1, while in zsh-3.0.4 it expands to a0 b0.
> 
> Can you generalize this rule for us?  E.g.
> 
> % echo ${^a}$[i++]$[++j]${^x}....
> 
> where .... is some arbitary number of other substitutions?  Is it just that
> it now does everything from right to left instead of left to right?  Why?

No, it is left to right.  ${^a} is expanded first, then the remaining
part, $[i++]$[++j]${^x} is expanded separately, and the result is
combined with the expansion of ${^a}.  You can see it if you try

let i=0; echo $[i++]${^a}$[i++]

which gives

0a1 0b1

If it were right to left, it would be 1a0 1b0.

Zoltan

Re: RC_EXPAND_PARAM bug

[Geoff Wing]

Zoltan Hidvegi <hzoli@frontiernet.net> typed:
:Bart Schaefer wrote:
:> On Jul 28,  2:42pm, Andrew Main wrote:
:> } % a=(a b)
:> } % x=(x y)
:> } % echo ${^a}1${^x}
:> } a1y b1x b1y
:> It gets weirder:
:> 
:> % echo 1${^a}1${^^x}
:> 1ay 1b1x 1by
:Below is a fix.  The second expansion might still look a bit weird:
:% echo 1${^a}1${^^x}
:1a1x 1ay 1b1x 1by
:
:The logic is that the string after the rc-param, 1${^^x}, is expanded,
:producing two strings, 1x y, which is combined with 1a 1b.  It is true

I don't get this,  shouldn't 1${^^x} produce one string "1x y"?  And then the
expansion of 1${^a}1${^^x} be the string "1a1x y 1b1x y"?  Your logic 
indicates to me that it's producing array like thingies.  Is this dependant
on some other wierd option?  Please explain before I go mental.
-- 
Geoff Wing [mason@primenet.com.au]                   Phone    : +61-3-9818 2977 
 Technical Manager: PrimeNet Computer Consultants    Facsimile: +61-3-9819 3788 
 Web: <URL:http://www.primenet.com.au/>              Mobile   : 0412 162 441

Re: RC_EXPAND_PARAM bug

[Bart Schaefer]

On Jul 29,  3:36am, Zoltan Hidvegi wrote:
} Subject: Re: RC_EXPAND_PARAM bug
}
} > Can you generalize this rule for us?  E.g.
} > 
} > % echo ${^a}$[i++]$[++j]${^x}....
} > 
} > where .... is some arbitary number of other substitutions?  Is it just that
} > it now does everything from right to left instead of left to right?  Why?
} 
} No, it is left to right.  ${^a} is expanded first, then the remaining
} part, $[i++]$[++j]${^x} is expanded separately, and the result is
} combined with the expansion of ${^a}.  You can see it if you try
} 
} let i=0; echo $[i++]${^a}$[i++]
} 
} which gives
} 
} 0a1 0b1

I'm still not comprehending this.

Are you saying that all the variables are expanded first, left to right,
and then all the resulting strings are combined?  Whereas before (2.6 and
earlier) each variable would be expanded and combined with what followed,
and then the process repeated for each new string?

On Jul 29,  7:47am, Geoff Wing wrote:
} Subject: Re: RC_EXPAND_PARAM bug
}
} :The logic is that the string after the rc-param, 1${^^x}, is expanded,
} :producing two strings, 1x y, which is combined with 1a 1b.  It is true
} 
} I don't get this,  shouldn't 1${^^x} produce one string "1x y"?  And then the
} expansion of 1${^a}1${^^x} be the string "1a1x y 1b1x y"?  Your logic 
} indicates to me that it's producing array like thingies.  Is this dependant
} on some other wierd option?  Please explain before I go mental.

The trick there is that x is an array variable, so it produces two strings.
The setting of rcexpandparam doesn't affect that part.

zagzig% a=(a b)
zagzig% x=(x y)
zagzig% echo 1${^a}1${^^x}
1a1x 1ay 1b1x 1by
zagzig% echo 1${^a}1${^x} 
1a1x 1a1y 1b1x 1b1y


-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Re: RC_EXPAND_PARAM bug

[Geoff Wing]

Bart Schaefer wrote about Re: RC_EXPAND_PARAM bug:
:On Jul 29,  7:47am, Geoff Wing wrote:
:} :The logic is that the string after the rc-param, 1${^^x}, is expanded,
:} :producing two strings, 1x y, which is combined with 1a 1b.  It is true
:} I don't get this,  shouldn't 1${^^x} produce one string "1x y"?  And then the
:} expansion of 1${^a}1${^^x} be the string "1a1x y 1b1x y"?  Your logic 
:} indicates to me that it's producing array like thingies.  Is this dependant
:} on some other wierd option?  Please explain before I go mental.
:The trick there is that x is an array variable, so it produces two strings.

Yes, but I still don't get it.  ^^ should prevent expansion, shouldn't it.

  ${^spec}
	Turn on the RC_EXPAND_PARAM option  for  the evaluation  of  spec;
	if  the ^ is doubled, turn it off.  When this option is set, array
	expansions of the form foo${xx}bar, where the parameter xx is set
	to (a b c), are substituted with   fooabar   foobbar  foocbar
        instead of the default fooa b cbar.

So, it seems to me that it should be
  % echo 1${^a}
  1a 1b
  % echo 1${^a}1
  1a1 1b1
  % echo 1${^a}1${^x}
  1a1x 1a1y 1b1x 1b1y           #  in another order?
  % echo 1${^a}1${^^x}
  1a1x y 1b1x y 
-- 
Geoff Wing [mason@primenet.com.au]                   Phone    : +61-3-9818 2977 
 Technical Manager: PrimeNet Computer Consultants    Facsimile: +61-3-9819 3788 
 Web: <URL:http://www.primenet.com.au/>              Mobile   : 0412 162 441
        [ Boulderdash: <URL:http://ciips.ee.uwa.edu.au/~williams/bd/> ]

Re: RC_EXPAND_PARAM bug

[Bart Schaefer]

On Jul 30,  1:04pm, Geoff Wing wrote:
} Subject: Re: RC_EXPAND_PARAM bug
}
} :} ... shouldn't 1${^^x} produce one string "1x y"?  And then the
} :} expansion of 1${^a}1${^^x} be the string "1a1x y 1b1x y"?  Your logic 
} :} indicates to me that it's producing array like thingies.  Is this dependant
} :} on some other wierd option?  Please explain before I go mental.
} :The trick there is that x is an array variable, so it produces two strings.
} 
} Yes, but I still don't get it.  ^^ should prevent expansion, shouldn't it.

It prevents rc_expand_param.  It doesn't prevent array expansion.  That
is, it prevents (x y) from being combined with the rest of the string in
every possible combination; it doesn't make (x y) into a string istead of
into an array.  You need quoting for that.

So either 1a1x 1ay 1b1x 1by is right, or else it should be 1a1x 1b1x y.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Re: RC_EXPAND_PARAM bug

[Zoltan Hidvegi]

> } > % echo ${^a}$[i++]$[++j]${^x}....
> } > 
> } > where .... is some arbitary number of other substitutions?  Is it just that
> } > it now does everything from right to left instead of left to right?  Why?
> } 
> } No, it is left to right.  ${^a} is expanded first, then the remaining
> } part, $[i++]$[++j]${^x} is expanded separately, and the result is
> } combined with the expansion of ${^a}.  You can see it if you try
> } 
> } let i=0; echo $[i++]${^a}$[i++]
> } 
> } which gives
> } 
> } 0a1 0b1
> 
> I'm still not comprehending this.
> 
> Are you saying that all the variables are expanded first, left to right,
> and then all the resulting strings are combined?  Whereas before (2.6 and
> earlier) each variable would be expanded and combined with what followed,
> and then the process repeated for each new string?

OK, I tell you what subst.c does.  You have a string of the form

prefix${^a}suffix

Suppose that prefix does not have anything to expand.  paramsubst is
called to expand this string, which expands ${^a}.  There are three
cases:

1. `a' was an empty array.  In that case the expansion is the empty list,
   and suffix is not evaluated (so even if there is a $[i++] in suffix, i
   will not change).

2. `a' had one element: paransubst replaces ${^a} with its value and
   returns and the calling routine, stringsubst, which continues parsing
   the suffix.  If that suffix contains some other parameter expansions,
   paramsubst will be called again.

3. `a' has more than one elements.  In that case stringsubst is called
   for suffix alone (which may call paramsubst again to expand something
   in suffix), and the result will be be a list.  This list is combined
   with the expansion of ${^a}.  If suffix expands to an empty list, the
   result will be empty, otherwise the result is the first element of `a'
   combined with the each element from the expansion of suffix, followed
   by the second element of `a' combined with list etc.  Note that suffix
   can only expand to an empty list as a result of an rc-expansion in
   suffix (that's because null-argument removal is only done in prefork
   after stringsubst).  With this, in ${^a}1${^^x} the expansion of
   1${^^x} gives two elements, 1x y, which is combined with `a'.  If you
   replace 1${^^x} with 1${^x}, it is expanded to 1x 1y instead of 1x y.

This guarantees left to right evaluation, everything is evaluated at most
once, and everything is evaluated once unless there is an rc-expansion of
an empty array which discards everything following that array.

The 2.6-beta16 and earlier behavior:

${^a} was expanded, and a list was created prepending prefix and
appending suffix for each array element.  The resulting list is parsed
again from the beginning.  For example:

a=('${~^a}' '${~^a}')

prefix${~^a}suffix

expands to

prefix${~^a}suffix prefix${~^a}suffix

and the expansion is restarted, the first element of this list is
expanded again, the result is

prefix${~^a}suffix prefix${~^a}suffix prefix${~^a}suffix

and the expansion is restarted...  For this infinite loop, the ~ or the
globsubst option was necessary.

What would be the preferred evaluation of ${^a}1${^^x}?

Alternatives:

1. a1x ay b1x by  (current)

2. a1x y b1x y    (beta16 and older)

3. a1x b1x y      (just an other logical solution).

The current behaviour is the simplest to code solution, I think the other
two are not very hard to implement, but I'm not sure it is worth the
extra effort.

Zoltan

Re: RC_EXPAND_PARAM bug

[Andrej Borsenkow]

On Wed, 30 Jul 1997, Zoltan Hidvegi wrote:

> 
> What would be the preferred evaluation of ${^a}1${^^x}?
> 
> Alternatives:
> 
> 1. a1x ay b1x by  (current)
> 
> 2. a1x y b1x y    (beta16 and older)
> 
> 3. a1x b1x y      (just an other logical solution).
> 
> The current behaviour is the simplest to code solution, I think the other
> two are not very hard to implement, but I'm not sure it is worth the
> extra effort.
> 

zsh-3.1.2 with RC_EXPAND_PARAM bugfix.

% echo ${^a}1${^^x}
a1x ay b1x by

% echo ${^^a}1${^x}
a b1x b1y

It looks illogical. Either the latter should be 'ax b1x ay b1y' or the
former 'a1x b1x y'. 

-------------------------------------------------------------------------
Andrej Borsenkow 		Fax:   +7 (095) 252 01 05
SNI ITS Moscow			Tel:   +7 (095) 252 13 88

NERV:  borsenkow.msk		E-Mail: borsenkow.msk@sni.de
-------------------------------------------------------------------------

Re: RC_EXPAND_PARAM bug

[Bart Schaefer]

On Jul 30,  1:16am, Zoltan Hidvegi wrote:
} Subject: Re: RC_EXPAND_PARAM bug
}
} > I'm still not comprehending this.
} 
} OK, I tell you what subst.c does.

Thank you!

} prefix${^a}suffix
} 
} Suppose that prefix does not have anything to expand.  paramsubst is
} called to expand this string, which expands ${^a}.  There are three
} cases:
} 
} 1. `a' was an empty array.  In that case the expansion is the empty list,
}    and suffix is not evaluated (so even if there is a $[i++] in suffix, i
}    will not change).

Hrm.  I think I would have expected to get prefixsuffix, with suffix
evaluated.  That's what you get when a=("") or a="", but I can't decide
if that represents an argument in favor, or against.

} 3. `a' has more than one elements.  In that case stringsubst is called
}    for suffix alone (which may call paramsubst again to expand something
}    in suffix), and the result will be be a list.  This list is combined
}    with the expansion of ${^a}.  If suffix expands to an empty list, the
}    result will be empty

Again I find the empty list behavior unexpected, but possibly defensible.

} This guarantees left to right evaluation, everything is evaluated at most
} once,

These are both good things.

} and everything is evaluated once unless there is an rc-expansion of
} an empty array which discards everything following that array.

It's that one that gives me pause.

} What would be the preferred evaluation of ${^a}1${^^x}?

What does the real "rc" shell do?  Duplicating it would be my first choice,
unless it does something wildly different than any previous version of zsh.

} Alternatives:
} 
} 1. a1x ay b1x by  (current)
} 
} 2. a1x y b1x y    (beta16 and older)
} 
} 3. a1x b1x y      (just an other logical solution).

The important question for me is, where are the word breaks in (2)?
That is, if I do:

  z=(${^a}1${^^x})

What is ${#z}?  If z is ("a1x y" "b1x y"), then I think that's wrong.
I presume you mean 4 words, though, in which case I think it's right,
in spite of what I just told Geoff. :-}

On Jul 30,  9:46am, Andrej Borsenkow wrote:
} Subject: Re: RC_EXPAND_PARAM bug
}
} zsh-3.1.2 with RC_EXPAND_PARAM bugfix.
} 
} % echo ${^a}1${^^x}
} a1x ay b1x by
} 
} % echo ${^^a}1${^x}
} a b1x b1y
} 
} It looks illogical. Either the latter should be 'ax b1x ay b1y' or the
} former 'a1x b1x y'. 

I agree with Andrej on that.  If you're going to map over lists, you
have to map over them in the prefix too, not just in the suffix.

However, I don't like the behavior of (3), i.e. that when ${x} is a list
then the word break interrupts the rc-expansion.  I'd prefer that the
expansion proceed across the suffix in the same way regardless of whether
x is an array or a string, and then introduce the word breaks at the end.
That's why I prefer (2) -- but I'd still like to know how "rc" does it.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Re: RC_EXPAND_PARAM bug

[Zoltan Hidvegi]

> } 1. `a' was an empty array.  In that case the expansion is the empty list,
> }    and suffix is not evaluated (so even if there is a $[i++] in suffix, i
> }    will not change).
> 
> Hrm.  I think I would have expected to get prefixsuffix, with suffix
> evaluated.  That's what you get when a=("") or a="", but I can't decide
> if that represents an argument in favor, or against.

No, an empty array is very different from an array with a single empty
element.  For example a is a list of directories, and you wand a list of
files in there directories, you would say

$^a/*

If `a' is empty, you want $^a/* expand to nothing, otherwise you would
get the list of files in the root directory which you do not want.  Such
tricks are used in several zsh scripts I guess, and this is the
preferable behaviour.  And it always worked this way, that's the point
where there is not difference between versions.

> } and everything is evaluated once unless there is an rc-expansion of
> } an empty array which discards everything following that array.
> 
> It's that one that gives me pause.

One might require the evaluation of the suffix even if it is discarded
later.  Do you think it is preferable?

> } What would be the preferred evaluation of ${^a}1${^^x}?
> 
> What does the real "rc" shell do?  Duplicating it would be my first choice,
> unless it does something wildly different than any previous version of zsh.

I do not think this has anything to do with rc.  I always thought that rc
somehow comes from brace, since this kind of array expansion resembles
brace expansion.

> } Alternatives:
> } 
> } 1. a1x ay b1x by  (current)
> } 
> } 2. a1x y b1x y    (beta16 and older)
> } 
> } 3. a1x b1x y      (just an other logical solution).
> 
> The important question for me is, where are the word breaks in (2)?
> That is, if I do:
> 
>   z=(${^a}1${^^x})
> 
> What is ${#z}?  If z is ("a1x y" "b1x y"), then I think that's wrong.

It is 4.

> On Jul 30,  9:46am, Andrej Borsenkow wrote:
> } Subject: Re: RC_EXPAND_PARAM bug
> }
> } zsh-3.1.2 with RC_EXPAND_PARAM bugfix.
> } 
> } % echo ${^a}1${^^x}
> } a1x ay b1x by
> } 
> } % echo ${^^a}1${^x}
> } a b1x b1y
> } 
> } It looks illogical. Either the latter should be 'ax b1x ay b1y' or the
> } former 'a1x b1x y'. 
> 
> I agree with Andrej on that.  If you're going to map over lists, you
> have to map over them in the prefix too, not just in the suffix.
> 
> However, I don't like the behavior of (3), i.e. that when ${x} is a list
> then the word break interrupts the rc-expansion.  I'd prefer that the
> expansion proceed across the suffix in the same way regardless of whether
> x is an array or a string, and then introduce the word breaks at the end.

I do not understand exatly what you mean, but I'm leaning towards option
3.  One possible implementation is to prepend some otherwise impossible
string to suffix before expanding it, and after the expansion, check for
that, and replace it with elements of a.  The impossible string can be {
Meta, '@' } or a new metacharacter.  This seems to be a really simple to
do.  Option 2. can use the same trick, but if the modified ${^^a}1${^x}
behaviour you suggest above is also required, things become more
complicated.

Zoltan

Re: RC_EXPAND_PARAM bug

[Bart Schaefer]

On Jul 30,  2:51am, Zoltan Hidvegi wrote:
} Subject: Re: RC_EXPAND_PARAM bug
}
} If `a' is empty, you want $^a/* expand to nothing, otherwise you would
} get the list of files in the root directory which you do not want.
} Such tricks are used in several zsh scripts I guess, and this is the
} preferable behaviour.  And it always worked this way, that's the point
} where there is not difference between versions.

That's fine, then.

} One might require the evaluation of the suffix even if it is discarded
} later.  Do you think it is preferable?

I think perhaps it is, especially if the suffix contains arithmetic or
command substitutions that might have side effects.

} > What does the real "rc" shell do?
} 
} I do not think this has anything to do with rc.

I'm pretty sure that it does.  All the RC_ options are things that "rc"
does, just like the CSH_ options are things CSH_ does, and KSH_, etc.
I believe Paul originally named all the options that were borrowed from
other shells by that convention.

} > } It looks illogical. Either the latter should be 'ax b1x ay b1y' or the
} > } former 'a1x b1x y'. 
} > 
} > I agree with Andrej on that.  If you're going to map over lists, you
} > have to map over them in the prefix too, not just in the suffix.
} > 
} > However, I don't like the behavior of (3), i.e. that when ${x} is a list
} > then the word break interrupts the rc-expansion.  I'd prefer that the
} > expansion proceed across the suffix in the same way regardless of whether
} > x is an array or a string, and then introduce the word breaks at the end.
} 
} I do not understand exatly what you mean, but I'm leaning towards option
} 3.

Consider:

a=(a b)
c="c d"
e=(e f)

${^a}${c}${^e}	-->	"ac de" "bc de" "ac df" "bc df"

Now:

c=(c d)

Option 2:	${^a}${c}${^e}	-->	ac de bc de ac df bc df
Option 3:	${^a}${c}${^e}	-->	ac bc de df

I like the symmetry between (2) and the case where c is a string.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Re: RC_EXPAND_PARAM bug

[Andrew Main]

Zoltan Hidvegi wrote:
>What would be the preferred evaluation of ${^a}1${^^x}?

Who cares?  Anyone mixing plan9 and non-plan9 array expansions is asking
for trouble.  Perhaps the documentation should warn against it.

-zefram

Re: RC_EXPAND_PARAM bug

[Andrej Borsenkow]

On Wed, 30 Jul 1997, Andrew Main wrote:

> Zoltan Hidvegi wrote:
> >What would be the preferred evaluation of ${^a}1${^^x}?
> 
> Who cares?  Anyone mixing plan9 and non-plan9 array expansions is asking
> for trouble.  Perhaps the documentation should warn against it.
> 

After looking at rc, I agree. Actually, the implementation in ZSH is so
different, that the name RC_EXPAND_PARAM is just misleading.

In rc, there are two legal cases:
 1. concatenation of scalar (or rather list of length 1, as rc doesn't
    have scalars at all); it corresponds to RC_EXPAND_PARAM in ZSH:

    ; a=(a)
    ; x=(x y)
    ; echo $a$x
    ax ay
 2. concatenation of two lists of _equal_ size; it is _pairwise_ 
    and not cartesian product as in ZSH:

    ; a=(a b)
    ; x=(x y)
    ; echo $a$x
    ax by
 Anything else is error:
 
    ; a=(a b)
    ; x=(x y z)
    ; echo $a$x
    bad concatenation

As far as ZSH goes, I am not shure, if any reasonable way to mix arrays
with and without RC_EXPAND_PARAM in one line (word to be shure) exists.
Any way you select will give you very strange results sometimes. Probably,
it is enough, if RC_EXPAND_PARAM works with scalars, and FAQ should
mention, that anything else should better be avoided. The important thing
is to document how selected method really works and most intuitive to me
looks like disallow such mix at all ;)

-------------------------------------------------------------------------
Andrej Borsenkow 		Fax:   +7 (095) 252 01 05
SNI ITS Moscow			Tel:   +7 (095) 252 13 88

NERV:  borsenkow.msk		E-Mail: borsenkow.msk@sni.de
-------------------------------------------------------------------------

Re: RC_EXPAND_PARAM bug

[Bart Schaefer]

On Jul 30,  7:54pm, Andrej Borsenkow wrote:
} Subject: Re: RC_EXPAND_PARAM bug
}
} > Anyone mixing plan9 and non-plan9 array expansions is asking
} > for trouble.  Perhaps the documentation should warn against it.
} 
} After looking at rc, I agree. Actually, the implementation in ZSH is so
} different, that the name RC_EXPAND_PARAM is just misleading.

I think it's a bit late to change now, however.

Also, un(?)fortunately, it's too late to actually match the rc behavior.

} As far as ZSH goes, I am not shure, if any reasonable way to mix arrays
} with and without RC_EXPAND_PARAM in one line (word to be shure) exists.

I think the beta16-and-earlier cartesian product was reasonable and gave
consistent results.  However, I encourage a warning in the doc that the
results are not the same as for rc and that mixing ${^x} and ${^^x} in
the same word may not behave in the way one expects. 

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Re: RC_EXPAND_PARAM bug

[Andrej Borsenkow]

On Wed, 30 Jul 1997, Bart Schaefer wrote:

> 
> I think the beta16-and-earlier cartesian product was reasonable and gave
> consistent results.  However, I encourage a warning in the doc that the
> results are not the same as for rc and that mixing ${^x} and ${^^x} in
> the same word may not behave in the way one expects. 
> 

There is probably one more alternative, but let me first explain how I
understand the problem.

/bin/sh (and POSIX sh) rules are based on textual substitution. They are
very simple and consist of well defined steps:
  1. break line into sequence of substitutions
  2. evaluate each of them
  3. concatenate results
  4. break result in words with field splitting

ZSH coompicates it by adding arrays. They can still fit in above scheme,
if we define what $array means. ZSH defines it as "$array[@]" which is
quite acceptable (I don't speak about implementation; I understand, arrays
are actually implemented as lists internally).

RC_EXAPND_PRAMS is based on absolutely different assumption. Original
implementation (rc) operates on _lists_, and is using _binary operator_
and not plain substitution/concatenation. The '$a$b' in rc is just
shorthand for '$a^$b' where `^' is binary opereator (wether pairwise or
cartesian product doesn't actually matters). rc just cannot have our
problems, because it is driven by usual operator precednce rules.

Both implemetations (pre-beta16 and current) actually do the following:

  - break word into ^- and non-^-parts
  - do substitutions on non-^-parts (and on nested substitions in
    ^-parts) based on traditional (textual) rules
  - convert results into lists (implicit field-splitting)
  - combine the lists

They differ just in how lists are combined:

   pre-beta16 treats `^' as ternary operator:
     xx$^ayy -> xx^$a^yy (in rc notation)
   current treats `^' as binary operator with right precedence:
     xx$^ayy -> xx($a^y)

Both is acceptable as long as it is documented ;)

Correct me, if I am wrong.

Now, if the above rules are explicitly stated in ZSH docs it is
enough. It will enable users to understand what's really going on, which
is very hard to do now ;-} I suggest, that zshexpn(1) explicitly states
this additional step and how it works (FAQ could then give some examples).

The same holds true for brace expansion as well (which is currently done
differently from RC_EXPNAD_PARAM).

Oh, I almost forgot my proposal ...

  - get rid of ^ as subst modifier
  - use explicit operator notation; something like
     ${prefix^suffix}, where prefix and suffix are expanded, converted
     to lists and combined. The result is then stuffed in replaced 
     string in usual manner (as if it were array - which by the way it is
     ;)
    We could include operator precedence so that
     ${xxx^yyy^zzz} could be used.

I am not shure if current grammar permits use of `^' in this case, but I
still like the idea ...

-------------------------------------------------------------------------
Andrej Borsenkow 		Fax:   +7 (095) 252 01 05
SNI ITS Moscow			Tel:   +7 (095) 252 13 88

NERV:  borsenkow.msk		E-Mail: borsenkow.msk@sni.de
-------------------------------------------------------------------------

Re: RC_EXPAND_PARAM bug

[Bart Schaefer]

On Aug 1,  5:17pm, Andrej Borsenkow wrote:
} Subject: Re: RC_EXPAND_PARAM bug
}
}    pre-beta16 treats `^' as ternary operator:
}      xx$^ayy -> xx^$a^yy (in rc notation)
}    current treats `^' as binary operator with right precedence:
}      xx$^ayy -> xx($a^y)
} 
} Correct me, if I am wrong.

Should that read

	xx${^a}yy -> xx^($a^yy)
	
??

The step you're omitting is that what happens when xx and yy represent
arrays differs from what happens when xx and yy represent strings.

xx${a}yy --> xx^${a}^yy --> xx^(ayy byy) --> xxayy xxbyy

b=(p q)
xx${a}${b} --> xx^(a b)^(p q) --> xx^(ap bq bp) --> xxap xxbq xxbp

c=(m n)
${c}${a}${b} --> (m n)^(a b)^(p q) --> (m n)^(ap bq bp) --> mbp nap nbq nbp

What I find weird about this is the way the first element of the left
side gets paired only with the last element of the right side.  One of
the array iterators is not getting started over when it should.  The
more elements there are in each array, the stranger it gets.

} Both is acceptable as long as it is documented ;)

I agree that both are acceptable, because they ought to be completely
equivalent!  The current behavior is not acceptable because it doesn't
obey any such straightforward operator rule.

} Oh, I almost forgot my proposal ...
} 
}   - get rid of ^ as subst modifier
}   - use explicit operator notation

I think it's far too late for this.  Too great an incompatibility with
existing zsh.  (We could add an operator notation, but we can't get rid
of the subst modifier.)

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com