Array expansion interacts with brace expansion in the wrong order

Array expansion interacts with brace expansion in the wrong order

[Anders Kaseorg]

Given a=(1 2 3), I would expect {$^a,4,5,6} to expand to 1 2 3 4 5 6, but 
it doesn’t.  Instead the 4 5 6 is duplicated for every element of $^a:

% echo $ZSH_PATCHLEVEL
zsh-5.3.1-240-ga3b3f0057
% echo {{1,2,3},4,5,6}
1 2 3 4 5 6
% a=(1 2 3)
% echo {$^a,4,5,6}
1 4 5 6 2 4 5 6 3 4 5 6

Anders

Re: Array expansion interacts with brace expansion in the wrong order

[Bart Schaefer]

On Thu, Jul 13, 2017 at 3:45 PM, Anders Kaseorg <andersk@mit.edu> wrote:
> Given a=(1 2 3), I would expect {$^a,4,5,6} to expand to 1 2 3 4 5 6, but
> it doesn’t.  Instead the 4 5 6 is duplicated for every element of $^a

Your expectation is wrong.  Brace expansion happens last of all
(except for globbing), so {$^a,4,5,6} is {1,4,5,6} {2,4,5,6}
{3,4,5,6}.  This is documented.

Even if this happened in the order you expect, the way you've written
it would have given 1 4 1 5 1 6 2 4 2 5 2 6 3 4 3 5 3 6 because of the
meaning of $^a.

What you are after is simply {"$a",4,5,6}.

Re: Array expansion interacts with brace expansion in the wrong order

[Anders Kaseorg]

On Thu, 13 Jul 2017, Bart Schaefer wrote:
> On Thu, Jul 13, 2017 at 3:45 PM, Anders Kaseorg <andersk@mit.edu> wrote:
> > Given a=(1 2 3), I would expect {$^a,4,5,6} to expand to 1 2 3 4 5 6, but
> > it doesn’t.  Instead the 4 5 6 is duplicated for every element of $^a
> 
> Your expectation is wrong.  Brace expansion happens last of all
> (except for globbing), so {$^a,4,5,6} is {1,4,5,6} {2,4,5,6}
> {3,4,5,6}.  This is documented.
> 
> Even if this happened in the order you expect, the way you've written
> it would have given 1 4 1 5 1 6 2 4 2 5 2 6 3 4 3 5 3 6 because of the
> meaning of $^a.

The documentation for ${^spec} says

“Internally, each such expansion is converted into the equivalent list for 
brace expansion. E.g., ${^var} becomes {$var[1],$var[2],...}, and is 
processed as described in Brace Expansion below.”

That would seem to support my expectation that {$^a,4,5,6} should become 
{{1,2,3},4,5,6}, which expands to 1 2 3 4 5 6.

> What you are after is simply {"$a",4,5,6}.

That produces 4 words ("1 2 3" 4 5 6), and doesn’t generalize (e.g. I 
would expect {$^a,4,5,6}{x,y} = {{1,2,3},4,5,6}{x,y} = 1x 1y 2x 2y 3x 3y 
4x 4y 5x 5y 6x 6y).

Anders

Re: Array expansion interacts with brace expansion in the wrong order

[Peter Stephenson]

On Thu, 13 Jul 2017 21:42:50 -0400
Anders Kaseorg <andersk@mit.edu> wrote:
> The documentation for ${^spec} says
> 
> “Internally, each such expansion is converted into the equivalent list for 
> brace expansion. E.g., ${^var} becomes {$var[1],$var[2],...}, and is 
> processed as described in Brace Expansion below.”
> 
> That would seem to support my expectation that {$^a,4,5,6} should become 
> {{1,2,3},4,5,6}, which expands to 1 2 3 4 5 6.

What it means is the effect similar to brace expansion is *performed*
immediately, not simply recorded as happening below with the other brace
expansion.  So by the time the shell even looks to see if there's any
(actual) brace expansion to do, the array has already been expended
using its own similar rules and the shell is looking at:

{1,4,5,6} {2,4,5,6] {3,4,5,6}

There's no likelihood of the big rewrite that would be necessary to
transform the array expansion into such a form that it actually happens
with the brace expansion; indeed the brace expansion code is quite ad
hoc, as I rediscovered only yesterday.

The ordering of diferent sorts of expansion in the shell is always a big
source of confusion and in head scratching about how to get things to
work, but the key is to think of array expansion and brace expansion as
completely separate stages.  I can't think of an easy workaround for
your case except by constructing an array

b=($a 4 5 6)

which is obviously rather more verbose.

pws

Re: Array expansion interacts with brace expansion in the wrong order

[Peter Stephenson]

This might be worth saying explicitly --- though things like this are
implicit throughout zsh's expansion rules.

diff --git a/Doc/Zsh/expn.yo b/Doc/Zsh/expn.yo
index d161b0b..c45f83a 100644
--- a/Doc/Zsh/expn.yo
+++ b/Doc/Zsh/expn.yo
@@ -856,8 +856,9 @@ all arguments to be removed.
 Internally, each such expansion is converted into the
 equivalent list for brace expansion.  E.g., tt(${^var}) becomes
 tt({$var[1],$var[2],)...tt(}), and is processed as described in
-noderef(Brace Expansion) below.
-If word splitting is also in effect the
+noderef(Brace Expansion) below: note, however, the expansion
+happens immediately, with any explicit brace expansion
+happening later.  If word splitting is also in effect the
 tt($var[)var(N)tt(]) may themselves be split into different list
 elements.
 )

pws

Re: Array expansion interacts with brace expansion in the wrong order

[Peter Stephenson]

On Fri, 14 Jul 2017 10:45:55 +0100
Peter Stephenson <p.stephenson@samsung.com> wrote:
> This might be worth saying explicitly --- though things like this are
> implicit throughout zsh's expansion rules.

I wonder if it's also worth expanding the following?  Having reread it
in the light of Anders' query I realised "in one step" might be a bit
misleading.  Other suggestions welcome.

I'm not, to be honest, quite sure what "in one step" really meant.
I think it meant the each command line argument was changed completely
before moving onto the next.  But even if it did, that's not
*quite* right if you consider that you might generate extra arguments
from an earlier step which then need processing themselves in left
to right order.  All in all, I think obscuring the fact that the
expansions happen one after the other is unhelpful, however we tackle
it.

pws

diff --git a/Doc/Zsh/expn.yo b/Doc/Zsh/expn.yo
index d161b0b..5b26f36 100644
--- a/Doc/Zsh/expn.yo
+++ b/Doc/Zsh/expn.yo
@@ -21,9 +21,12 @@ xitem(em(Parameter Expansion))
 xitem(em(Command Substitution))
 xitem(em(Arithmetic Expansion))
 item(em(Brace Expansion))(
-These five are performed in one step in left-to-right fashion.  After
-these expansions, all unquoted occurrences of the characters `tt(\)',
-`tt(')' and `tt(")' are removed.
+These five are performed in left-to-right fashion.  On each argument,
+any of the five steps that are needed are performed one after the other.
+Hence, for example, all the parts of parameter expansion are completed
+before command substitution is started.  After these expansions, all
+unquoted occurrences of the characters `tt(\)',`tt(')' and `tt(")' are
+removed.
 )
 item(em(Filename Expansion))(
 If the tt(SH_FILE_EXPANSION) option is set, the order of expansion is