Re: for loop with parameter expansion in zsh vs. bash

Re: for loop with parameter expansion in zsh vs. bash

[Stephane Chazelas]

2015-11-03 07:52:51 +0100, Alexander Skwar:
> Hello
> 
> I've got a variable, where seperate values are limited with a delimiter.
> Let's say PATH with : (but the question is general).
> 
> With bash, I can easily create a for loop which loops over all the
> elements, when I have bash replace the ":" with a " ", like so: ${PATH//:/
> }.
> 
> a@ubuntu-notebook:~$ echo $PATH
> /home/a/Applications/go/bin:/home/a/bin:/opt/bin:/opt/sbin:/usr/local/sbin:/usr/local/bin:/home/a/Applications/copy/x86_64:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/a/Applications/adt-bundle-linux-x86_64/sdk/platform-tools:/home/a/Applications/btsync:/home/a/.rvm/bin
> 
> a@ubuntu-notebook:~$ for e in ${PATH//:/ }; do echo e: $e; done
> e: /home/a/Applications/go/bin
> e: /home/a/bin
> e: /opt/bin
> e: /opt/sbin
> e: /usr/local/sbin
> e: /usr/local/bin
> e: /home/a/Applications/copy/x86_64
> e: /usr/sbin
> e: /usr/bin
> e: /sbin
> e: /bin
> e: /usr/games
> e: /usr/local/games
> e: /home/a/Applications/adt-bundle-linux-x86_64/sdk/platform-tools
> e: /home/a/Applications/btsync
> e: /home/a/.rvm/bin

That approach is wrong. You're replacing : with space and
invoking the split+glob operator (leaving an expansion unquoted
in ksh/bash/sh, not zsh) with the default value of IFS which
contains space, tab and newline.

That means that approach only works if $PATH components don't
contain blanks or wildcards.

Here, you could use the split+glob operator, but you want to
split on :, not blanks, and not invoke the glob part.

So it would be (bash/ksh/sh, not zsh unless in sh/ksh emulation):

IFS=: # split on :
set -f # disabe glob
for e in $PATH

However with bash/ksh/sh (not zsh), it's still wrong, because it
would discard a trailing empty component (like for a $PATH value
of /bin:/usr/bin:)


> 
> In zsh, the same for loop does not work (like it does in bash):

No, because and that's the most FAQ for zsh, leaving a variable
unquoted is not the split+glob operator (like it is in most
other shells and the number one source of bugs and security
vulnerabilities in them,
(http://unix.stackexchange.com/questions/171346/security-implications-of-forgetting-to-quote-a-variable-in-bash-posix-shells)

In zsh, you invoke the split ($=var) or glob ($~var) operators
explicitely, so:

IFS=:
for e in $~PATH

Or use the "s" expansion flag:

for e in "${(s(:))PATH}"

For PATH however, you don't need to as zsh ties the $PATH
variable to the $path array.

So, it's just:

for e in "$path[@]"

-- 
Stephane

Re: for loop with parameter expansion in zsh vs. bash

[Alexander Skwar]

[-- Attachment #1: Type: text/plain, Size: 3704 bytes --]

Hello

2015-11-03 8:35 GMT+01:00 Stephane Chazelas <stephane.chazelas@gmail.com>:

> 2015-11-03 07:52:51 +0100, Alexander Skwar:
> > Hello
> >
> > I've got a variable, where seperate values are limited with a delimiter.
> > Let's say PATH with : (but the question is general).
> >
> > With bash, I can easily create a for loop which loops over all the
> > elements, when I have bash replace the ":" with a " ", like so:
> ${PATH//:/
> > }.
> >
> > a@ubuntu-notebook:~$ echo $PATH
> >
> /home/a/Applications/go/bin:/home/a/bin:/opt/bin:/opt/sbin:/usr/local/sbin:/usr/local/bin:/home/a/Applications/copy/x86_64:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/a/Applications/adt-bundle-linux-x86_64/sdk/platform-tools:/home/a/Applications/btsync:/home/a/.rvm/bin
> >
> > a@ubuntu-notebook:~$ for e in ${PATH//:/ }; do echo e: $e; done
> > e: /home/a/Applications/go/bin
> > e: /home/a/bin
> > e: /opt/bin
> > e: /opt/sbin
> > e: /usr/local/sbin
> > e: /usr/local/bin
> > e: /home/a/Applications/copy/x86_64
> > e: /usr/sbin
> > e: /usr/bin
> > e: /sbin
> > e: /bin
> > e: /usr/games
> > e: /usr/local/games
> > e: /home/a/Applications/adt-bundle-linux-x86_64/sdk/platform-tools
> > e: /home/a/Applications/btsync
> > e: /home/a/.rvm/bin
>
> That approach is wrong. You're replacing : with space and
> invoking the split+glob operator (leaving an expansion unquoted
> in ksh/bash/sh, not zsh) with the default value of IFS which
> contains space, tab and newline.
>
> That means that approach only works if $PATH components don't
> contain blanks or wildcards.
>

​Well, but I do know that my variable does not contain blanks or wildcards.
As I said, PATH was just an example, so that everybody can easily follow.
So, no, ​the approach was not at all wrong. Your assumption was, though.



>
> Here, you could use the split+glob operator, but you want to
> split on :, not blanks, and not invoke the glob part.
>
> So it would be (bash/ksh/sh, not zsh unless in sh/ksh emulation):
>
> IFS=: # split on :
> set -f # disabe glob
> for e in $PATH
>

​Thanks a lot. Works great.​



>
> However with bash/ksh/sh (not zsh), it's still wrong, because it
> would discard a trailing empty component (like for a $PATH value
> of /bin:/usr/bin:)
>


​Good catch ;)​



>
>
> >
> > In zsh, the same for loop does not work (like it does in bash):
>
> No, because and that's the most FAQ for zsh, leaving a variable
> unquoted is not the split+glob operator (like it is in most
> other shells and the number one source of bugs and security
> vulnerabilities in them,
> (
> http://unix.stackexchange.com/questions/171346/security-implications-of-forgetting-to-quote-a-variable-in-bash-posix-shells
> )
>
> In zsh, you invoke the split ($=var) or glob ($~var) operators
> explicitely, so:
>
> IFS=:
> for e in $~PATH
>
> Or use the "s" expansion flag:
>
> for e in "${(s(:))PATH}"
>
>
​Great. Thanks. Appreciated.


Too bad, that there's such a difference between the shells. Makes it hard
to share snippets with co-workers, who (for reasons, that I fail to
understand) don't use zsh.

Because of that, I'm actually using something along the lines of:

  for e in $(echo "$PATH" | tr ':' ' '); do echo e $e; done

This works everywhere.​



> For PATH however, you don't need to as zsh ties the $PATH
> variable to the $path array.
>

​I shouldn't have used PATH as an example… :/​ But thanks for the heads up
for that as well.


Alexander
-- 
=>        *Google+* => http://plus.skwar.me         <==
=> *Chat* (Jabber/Google Talk) => a.skwar@gmail.com <==

Re: for loop with parameter expansion in zsh vs. bash

[Stephane Chazelas]

2015-11-03 08:47:42 +0100, Alexander Skwar:
[...]
> > IFS=:
> > for e in $~PATH

Sorry, meant $=PATH above ($= looks like scissors)

> > Or use the "s" expansion flag:
> >
> > for e in "${(s(:))PATH}"

Note that those don't work if $PATH is empty (which for command
search means one empty element) or unset (which for command
search means commands are looked in a default $PATH (whose value
depends on the shell)).

> ​Great. Thanks. Appreciated.
> 
> 
> Too bad, that there's such a difference between the shells. Makes it hard
> to share snippets with co-workers, who (for reasons, that I fail to
> understand) don't use zsh.
> 
> Because of that, I'm actually using something along the lines of:
> 
>   for e in $(echo "$PATH" | tr ':' ' '); do echo e $e; done
> 
> This works everywhere.​

Note that it is slightly more correct in zsh (that performs only
split upon command substitution) than in other shells that
perform split+glob. The use of echo causes problems with things
that start with - or contain backslashes though. The use of
command substitution means that trailing newlines in the last
element are trimmed.

set -o noglob; IFS=:; for e in $(printf %s "$PATH")

would be better (still a problem for trailing newlines).

Note that if you want to have zsh interpret sh code, you can use
"emulate -L zsh". The -L makes that "emulation" local to the
current function context.

So to share code between zsh and POSIX compliant shells, you
could do:

if [ "$ZSH_VERSION" ]; then
  alias 'START_SH_CODE=function { emulate -L sh'
  alias 'END_SH_CODE=} "$@"'
else
  START_SH_CODE() { :; }
  END_SH_CODE() { :; }
fi

And use as:

START_SH_CODE
IFS=:
set -f
for e in $PATH; do ...; done
END_SH_CODE

(note that it has the side effect of creating a function context
(the code is parsed as a whole, the positional parameters and
variables you declare inside are local to that block, "return"
breaks only out of that block.

(note that doing the same for bash (which by default, unless in
sh emulation is not POSIX compliant either though the
differences are a lot smaller) is a lot trickier as bash doesn't
have local scope for options or anonymous functions like zsh).

-- 
Stephane

Re: for loop with parameter expansion in zsh vs. bash

[Alexander Skwar]

[-- Attachment #1: Type: text/plain, Size: 959 bytes --]

Hello

2015-11-03 8:13 GMT+01:00 lilydjwg <lilydjwg@gmail.com>:


>
> % for e in ${=PATH//:/ }; do echo e: $e; done
>

​That's easy enough to type and remember, even for me ;)​



>
> zsh has sh_word_split option unset by default, so use ${=var} to request
> word splitting explicitely.
>
> I like this behaviour very much, because I don't need to quote nearly
> every variable :-)
>

​Thanks a lot. Might get used to it as well *G*​




>
> PS: there is a coresponding $path array, and you can tie your own
> variables like this:
>





>
> export -TU PYTHONPATH pythonpath 2>/dev/null
>
> Re-tying a variable fails for zsh 4.x so I have to redirect the error
> message to the black hole. And '-U' is for uniq to remove duplicates.
>


​Cool. Thanks ;)​


​Cheers,

Alexander
-- 
=>        *Google+* => http://plus.skwar.me         <==
=> *Chat* (Jabber/Google Talk) => a.skwar@gmail.com <==

Re: for loop with parameter expansion in zsh vs. bash

[lilydjwg]

On Tue, Nov 03, 2015 at 07:52:51AM +0100, Alexander Skwar wrote:
> Hello
> 
> I've got a variable, where seperate values are limited with a delimiter.
> Let's say PATH with : (but the question is general).
> 
> With bash, I can easily create a for loop which loops over all the
> elements, when I have bash replace the ":" with a " ", like so: ${PATH//:/
> }.
> 
> a@ubuntu-notebook:~$ echo $PATH
> /home/a/Applications/go/bin:/home/a/bin:/opt/bin:/opt/sbin:/usr/local/sbin:/usr/local/bin:/home/a/Applications/copy/x86_64:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/a/Applications/adt-bundle-linux-x86_64/sdk/platform-tools:/home/a/Applications/btsync:/home/a/.rvm/bin
> 
> a@ubuntu-notebook:~$ for e in ${PATH//:/ }; do echo e: $e; done
> e: /home/a/Applications/go/bin
> e: /home/a/bin
> e: /opt/bin
> e: /opt/sbin
> e: /usr/local/sbin
> e: /usr/local/bin
> e: /home/a/Applications/copy/x86_64
> e: /usr/sbin
> e: /usr/bin
> e: /sbin
> e: /bin
> e: /usr/games
> e: /usr/local/games
> e: /home/a/Applications/adt-bundle-linux-x86_64/sdk/platform-tools
> e: /home/a/Applications/btsync
> e: /home/a/.rvm/bin
> 
> 
> In zsh, the same for loop does not work (like it does in bash):
> 
> 7:51% for e in ${PATH//:/ }; do echo e: $e; done
> e: /home/a/Applications/go/bin /home/a/bin /opt/bin /opt/sbin
> /usr/local/sbin /usr/local/bin /home/a/Applications/copy/x86_64 /usr/sbin
> /usr/bin /sbin /bin /usr/games /usr/local/games
> /home/a/Applications/adt-bundle-linux-x86_64/sdk/platform-tools
> /home/a/Applications/btsync
> 
> 
> As you can see there, zsh had only 1 iteration of the for loop.
> 
> What would be the zsh way to loop over elements, which are delimited by a
> ":" (or "," or ";" or whatever)?
> 
> Thanks a lot,

% for e in ${=PATH//:/ }; do echo e: $e; done

zsh has sh_word_split option unset by default, so use ${=var} to request
word splitting explicitely.

I like this behaviour very much, because I don't need to quote nearly
every variable :-)

PS: there is a coresponding $path array, and you can tie your own variables like this:

export -TU PYTHONPATH pythonpath 2>/dev/null

Re-tying a variable fails for zsh 4.x so I have to redirect the error
message to the black hole. And '-U' is for uniq to remove duplicates.

-- 
Best regards,
lilydjwg

for loop with parameter expansion in zsh vs. bash

[Alexander Skwar]

[-- Attachment #1: Type: text/plain, Size: 1769 bytes --]

Hello

I've got a variable, where seperate values are limited with a delimiter.
Let's say PATH with : (but the question is general).

With bash, I can easily create a for loop which loops over all the
elements, when I have bash replace the ":" with a " ", like so: ${PATH//:/
}.

a@ubuntu-notebook:~$ echo $PATH
/home/a/Applications/go/bin:/home/a/bin:/opt/bin:/opt/sbin:/usr/local/sbin:/usr/local/bin:/home/a/Applications/copy/x86_64:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/a/Applications/adt-bundle-linux-x86_64/sdk/platform-tools:/home/a/Applications/btsync:/home/a/.rvm/bin

a@ubuntu-notebook:~$ for e in ${PATH//:/ }; do echo e: $e; done
e: /home/a/Applications/go/bin
e: /home/a/bin
e: /opt/bin
e: /opt/sbin
e: /usr/local/sbin
e: /usr/local/bin
e: /home/a/Applications/copy/x86_64
e: /usr/sbin
e: /usr/bin
e: /sbin
e: /bin
e: /usr/games
e: /usr/local/games
e: /home/a/Applications/adt-bundle-linux-x86_64/sdk/platform-tools
e: /home/a/Applications/btsync
e: /home/a/.rvm/bin


In zsh, the same for loop does not work (like it does in bash):

7:51% for e in ${PATH//:/ }; do echo e: $e; done
e: /home/a/Applications/go/bin /home/a/bin /opt/bin /opt/sbin
/usr/local/sbin /usr/local/bin /home/a/Applications/copy/x86_64 /usr/sbin
/usr/bin /sbin /bin /usr/games /usr/local/games
/home/a/Applications/adt-bundle-linux-x86_64/sdk/platform-tools
/home/a/Applications/btsync


As you can see there, zsh had only 1 iteration of the for loop.

What would be the zsh way to loop over elements, which are delimited by a
":" (or "," or ";" or whatever)?

Thanks a lot,



Alexander
-- 
=>        *Google+* => http://plus.skwar.me         <==
=> *Chat* (Jabber/Google Talk) => a.skwar@gmail.com <==