double quoted expansion question

double quoted expansion question

[Sebastian Stark]

Dear zsh users,

I am trying to make a unique list of tags that is given in a string with a certain format, e.  g.:

  servers="
  madeira linux,gpu,users
  madeira2 linux,gpu,users
  smtp linux,mail,smtp
  isar linux,users
  easygwa linux,web,external
  "

The format is: server name, space, tag, comma, tag, comma, tag ...

What I want from the above list is this:

  external gpu linux mail smtp users web 

and I get it with:

  print  ${(s:,:uo)${(j:,:)${${(f)servers}//* /}}} 

However, if I do this:

  print "${(s:,:uo)${(j:,:)${${(f)servers}//* /}}}"

I get:

  external linux web

This also happens when I use the variable expansion expression in a here document, which, I suppose, is undergoing the same treatment as double quoted strings.

What I would like to understand here is why the output changes the way it does when I add double quotes around my expression. I would expect the same output as without. If anybody could shed some light please, I guess it's just something obvious I cannot see.


Sebastian

Re: double quoted expansion question

[Jérémie Roquet]

Hi,

2012/9/7 Sebastian Stark <seb-zsh@biskalar.de>:
> I am trying to make a unique list of tags that is given in a string with a certain format, e.  g.:
>
>   servers="
>   madeira linux,gpu,users
>   madeira2 linux,gpu,users
>   smtp linux,mail,smtp
>   isar linux,users
>   easygwa linux,web,external
>   "
>
> The format is: server name, space, tag, comma, tag, comma, tag ...
>
> What I want from the above list is this:
>
>   external gpu linux mail smtp users web
>
> and I get it with:
>
>   print  ${(s:,:uo)${(j:,:)${${(f)servers}//* /}}}
>
> However, if I do this:
>
>   print "${(s:,:uo)${(j:,:)${${(f)servers}//* /}}}"
>
> I get:
>
>   external linux web
>
> This also happens when I use the variable expansion expression in a here document, which, I suppose, is undergoing the same treatment as double quoted strings.
>
> What I would like to understand here is why the output changes the way it does when I add double quotes around my expression. I would expect the same output as without. If anybody could shed some light please, I guess it's just something obvious I cannot see.

I'm not 100 % sure, but I suppose the quotes prevents the shell from
splitting your string on $IFS, so you only have *one* string, and the
“//* /" expression suppresses everything before the last space,
leaving only the tags of the last line.

Best regards,

-- 
Jérémie

Re: double quoted expansion question

[Jérémie Roquet]

2012/9/7 Jérémie Roquet <arkanosis@gmail.com>:
> 2012/9/7 Sebastian Stark <seb-zsh@biskalar.de>:
>> I am trying to make a unique list of tags that is given in a string with a certain format, e.  g.:
>>
>>   servers="
>>   madeira linux,gpu,users
>>   madeira2 linux,gpu,users
>>   smtp linux,mail,smtp
>>   isar linux,users
>>   easygwa linux,web,external
>>   "
>>
>> The format is: server name, space, tag, comma, tag, comma, tag ...
>>
>> What I want from the above list is this:
>>
>>   external gpu linux mail smtp users web
>>
>> and I get it with:
>>
>>   print  ${(s:,:uo)${(j:,:)${${(f)servers}//* /}}}
>>
>> However, if I do this:
>>
>>   print "${(s:,:uo)${(j:,:)${${(f)servers}//* /}}}"
>>
>> I get:
>>
>>   external linux web
>>
>> This also happens when I use the variable expansion expression in a here document, which, I suppose, is undergoing the same treatment as double quoted strings.
>>
>> What I would like to understand here is why the output changes the way it does when I add double quotes around my expression. I would expect the same output as without. If anybody could shed some light please, I guess it's just something obvious I cannot see.
>
> I'm not 100 % sure, but I suppose the quotes prevents the shell from
> splitting your string on $IFS, so you only have *one* string, and the
> “//* /" expression suppresses everything before the last space,
> leaving only the tags of the last line.

No, this has nothing to do with $IFS actually, but you still have one string:

$ a=(za zb zc zd)
$ echo ${a/z/v}
va vb vc vd
$ echo "${a/z/v}"
va zb zc zd

-- 
Jérémie

Re: double quoted expansion question

[Sebastian Stark]

Am 07.09.2012 um 14:54 schrieb Jérémie Roquet:
>>>  print  ${(s:,:uo)${(j:,:)${${(f)servers}//* /}}}
>> 
>> I'm not 100 % sure, but I suppose the quotes prevents the shell from
>> splitting your string on $IFS, so you only have *one* string, and the
>> “//* /" expression suppresses everything before the last space,
>> leaving only the tags of the last line.
> 
> No, this has nothing to do with $IFS actually, but you still have one string:
> 
> $ a=(za zb zc zd)
> $ echo ${a/z/v}
> va vb vc vd
> $ echo "${a/z/v}"
> va zb zc zd

Thanks, I just figured that this must be the kind of brain damage they mention in zshexpn(1), and using this:

  $(awk '{print $2}' <<<$servers | tr , '\n' | sort -u | tr '\n' ' ')

seems like a much saner way to achieve the same :) But not very zshy...


Sebastian

Re: double quoted expansion question

[Bart Schaefer]

On Sep 7,  2:31pm, Sebastian Stark wrote:
}
} Dear zsh users,
} 
} What I would like to understand here is why the output changes the way
} it does when I add double quotes around my expression. I would expect
} the same output as without. If anybody could shed some light please, I
} guess it's just something obvious I cannot see.

It may help to read the "Rules" subsection in the "Parameter Expansion"
section of the zsh manual.

Note that "double quoted joining" happens at step 5, before "modifiers"
[including the //* / replacement] at step 7.  Although "forced splitting"
(the s and f flags) doesn't happen until step 16, "nested substitution"
happens at step 1, so your innermost ${(f)servers} still happens before
double quoted joining is done, and it's the result of that split that is
then joined on spaces, causing //* / to do something you don't expect.

So what you need to do is tell zsh not to join that array, which you do
by adding the @ flag at the nesting level where the join would happen:

    print "${(s:,:uo)${(j:,:)${(@)${(f)servers}//* /}}}"

Also note that "forced joining" (the j flag) is applied at step 5 when
the substitution is in double quotes, but not until step 10 when unquoted.
That matters here only to explain why you need only one @ flag.

BTW, you don't need // there; there is only one space in each array
element after ${(f)servers}, so ${(@)${(f)servers}/* /}} is enough.

Re: double quoted expansion question

[Sebastian Stark]

Am 07.09.2012 um 16:41 schrieb Bart Schaefer:

> On Sep 7,  2:31pm, Sebastian Stark wrote:
> }
> } Dear zsh users,
> } 
> } What I would like to understand here is why the output changes the way
> } it does when I add double quotes around my expression. I would expect
> } the same output as without. If anybody could shed some light please, I
> } guess it's just something obvious I cannot see.
> 
> It may help to read the "Rules" subsection in the "Parameter Expansion"
> section of the zsh manual.
> 
> Note that "double quoted joining" happens at step 5, before "modifiers"
> [including the //* / replacement] at step 7.  Although "forced splitting"
> (the s and f flags) doesn't happen until step 16, "nested substitution"
> happens at step 1, so your innermost ${(f)servers} still happens before
> double quoted joining is done, and it's the result of that split that is
> then joined on spaces, causing //* / to do something you don't expect.
> 
> So what you need to do is tell zsh not to join that array, which you do
> by adding the @ flag at the nesting level where the join would happen:
> 
>    print "${(s:,:uo)${(j:,:)${(@)${(f)servers}//* /}}}"

Wow, thanks. My mistake is that I thought that because the quotes are at the outermost level, the double quoted joining was not part of this and applied at the very end. This explains why my assumption was wrong for that case. For here-docs there seems to be yet another twist:

❯ cat <<EOF
${(s:,:uo)${(j:,:)${(@)${(f)servers}/* /}}} 
EOF
linux,gpu,users,linux,gpu,users,linux,mail,smtp,linux,users,linux,web,external,

❯ cat <<EOF
${(s:,:uo)${(j:,:)${${(f)servers}/* /}}}   
EOF
<empty line>


Sebastian

Re: double quoted expansion question

[Bart Schaefer]

On Sep 7,  5:43pm, Sebastian Stark wrote:
}
} Wow, thanks. My mistake is that I thought that because the quotes are
} at the outermost level, the double quoted joining was not part of this
} and applied at the very end.

I can see how you might think that, but actually double-quoted-ness
applies all the way down the nested expansions from whatever level it
first encloses.

} For here-docs there seems to be yet another twist:

Hmm, that's an odd one -- certainly appears to be a bug.  Workaround
is to add an otherwise meaningless level of nesting:

cat <<EOF
${${(s:,:uo)${(j:,:)${(@)${(f)servers}/* /}}}}
EOF

-- 
Barton E. Schaefer