* Re: Fix testsuite errors due to shell quoted parameter expansion issue. [not found] <20100803203204.GG13690@gmx.de> @ 2010-08-03 20:55 ` Eric Blake 2010-08-03 21:21 ` Mikael Magnusson ` (2 more replies) 0 siblings, 3 replies; 8+ messages in thread From: Eric Blake @ 2010-08-03 20:55 UTC (permalink / raw) To: libtool-patches, gary, Bash - Bug, zsh-workers, Autoconf [-- Attachment #1: Type: text/plain, Size: 2156 bytes --] [adding autoconf to document some shell bugs] On 08/03/2010 02:32 PM, Ralf Wildenhues wrote: > Interesting shell unportability: > > $ bash -c 'f=" val" e=; echo "$e"$f' > val > $ ksh -c 'f=" val" e=; echo "$e"$f' > val > > ksh93, dash, zsh all do it like ksh. Is that a bug in bash? Yes; adding bug-bash accordingly. According to POSIX: http://www.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_05 "After parameter expansion ( Parameter Expansion ), command substitution ( Command Substitution ), and arithmetic expansion ( Arithmetic Expansion ), the shell shall scan the results of expansions and substitutions that did not occur in double-quotes for field splitting and multiple fields can result." Since $f is not quoted, its expansion must undergo field splitting. But since "$e" is quoted, it must not be elided even though empty. The result must be _two_ fields, as if you had done "echo '' 'val'". But it is _also_ a bug in zsh; adding zsh-workers accordingly. $ zsh -cvx 'f=" val" e=; echo "$e"$f' +zsh:1> f=' val' e='' +zsh:1> echo ' val' val Oops - zsh only passed one argument to echo, with a leading space, instead of passing an empty argument and letting echo supply the space. ksh93, pdksh, and dash get it right (although dash doesn't use quotes in -vx output, hence my use of n() to force things to tell; n() is another way to expose the bash and zsh bugs). $ ksh -cvx 'n() { echo $#; }; f=" val" e=; n "$e"$f' n() { echo $#; }; f=" val" e=; n ""$f+ f=' val' + e='' + n '' val + echo 2 2 And meanwhile, I found a ksh93 parsing bug (don't know where to report that): $ ksh -c 'a(){ echo hi; }; a' ksh: syntax error at line 1: `}' unexpected $ ksh -c 'a() { echo hi; }; a' hi $ bash -c 'a(){ echo hi; }; a' hi $ /bin/sh -c 'a(){ echo hi; }; a' hi ksh is the only shell that requires a space between the ) and {, even though the ) is a metacharacter and should not need trailing space (even Solaris /bin/sh got that right). -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 619 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fix testsuite errors due to shell quoted parameter expansion issue. 2010-08-03 20:55 ` Fix testsuite errors due to shell quoted parameter expansion issue Eric Blake @ 2010-08-03 21:21 ` Mikael Magnusson 2010-08-03 21:28 ` Eric Blake 2010-08-04 15:18 ` John Lumby 2010-08-03 21:43 ` Peter Stephenson 2010-08-05 22:15 ` Chet Ramey 2 siblings, 2 replies; 8+ messages in thread From: Mikael Magnusson @ 2010-08-03 21:21 UTC (permalink / raw) To: Eric Blake; +Cc: libtool-patches, gary, zsh-workers, Autoconf On 3 August 2010 22:55, Eric Blake <eblake@redhat.com> wrote: > [adding autoconf to document some shell bugs] > > On 08/03/2010 02:32 PM, Ralf Wildenhues wrote: >> Interesting shell unportability: >> >> $ bash -c 'f=" val" e=; echo "$e"$f' >> val >> $ ksh -c 'f=" val" e=; echo "$e"$f' >> val >> >> ksh93, dash, zsh all do it like ksh. Is that a bug in bash? > > Yes; adding bug-bash accordingly. According to POSIX: > > http://www.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_05 > > "After parameter expansion ( Parameter Expansion ), command substitution > ( Command Substitution ), and arithmetic expansion ( Arithmetic > Expansion ), the shell shall scan the results of expansions and > substitutions that did not occur in double-quotes for field splitting > and multiple fields can result." > > Since $f is not quoted, its expansion must undergo field splitting. But > since "$e" is quoted, it must not be elided even though empty. The > result must be _two_ fields, as if you had done "echo '' 'val'". > > But it is _also_ a bug in zsh; adding zsh-workers accordingly. > > $ zsh -cvx 'f=" val" e=; echo "$e"$f' > +zsh:1> f=' val' e='' > +zsh:1> echo ' val' > val > > Oops - zsh only passed one argument to echo, with a leading space, > instead of passing an empty argument and letting echo supply the space. > ksh93, pdksh, and dash get it right (although dash doesn't use quotes > in -vx output, hence my use of n() to force things to tell; n() is > another way to expose the bash and zsh bugs). zsh doesn't do word splitting by default, you can enable it with the = modifier: % zsh -fcvx 'f=" val" e=; echo "$e"$=f' +zsh:1> f=' val' e='' +zsh:1> echo '' val val does what you want Alternatively you can make zsh try to be closer to sh by setting argv[0] to sh when executing it, or running 'emulate sh' as the first command (and possibly other ways I don't know about): % zsh -fcvx 'emulate sh;f=" val" e=; echo "$e"$f' +zsh:1> emulate sh +zsh:1> f=' val' e='' +zsh:1> echo '' val val There's also --shwordsplit for this specific case: % zsh --shwordsplit -fcvx 'f=" val" e=; echo "$e"$f' +zsh:1> f=' val' e='' +zsh:1> echo '' val val (the -f above only avoids loading my .zshenv which would spam my output) -- Mikael Magnusson ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fix testsuite errors due to shell quoted parameter expansion issue. 2010-08-03 21:21 ` Mikael Magnusson @ 2010-08-03 21:28 ` Eric Blake 2010-08-04 15:18 ` John Lumby 1 sibling, 0 replies; 8+ messages in thread From: Eric Blake @ 2010-08-03 21:28 UTC (permalink / raw) To: Mikael Magnusson; +Cc: zsh-workers, Autoconf [-- Attachment #1: Type: text/plain, Size: 1726 bytes --] [trimming the cc a bit] On 08/03/2010 03:21 PM, Mikael Magnusson wrote: >> But it is _also_ a bug in zsh; adding zsh-workers accordingly. >> >> $ zsh -cvx 'f=" val" e=; echo "$e"$f' >> +zsh:1> f=' val' e='' >> +zsh:1> echo ' val' >> val >> >> Oops - zsh only passed one argument to echo, with a leading space, >> instead of passing an empty argument and letting echo supply the space. >> ksh93, pdksh, and dash get it right (although dash doesn't use quotes >> in -vx output, hence my use of n() to force things to tell; n() is >> another way to expose the bash and zsh bugs). > > zsh doesn't do word splitting by default, you can enable it with the = modifier: > % zsh -fcvx 'f=" val" e=; echo "$e"$=f' > +zsh:1> f=' val' e='' > +zsh:1> echo '' val > val > > does what you want Thanks for the information. > > Alternatively you can make zsh try to be closer to sh by setting > argv[0] to sh when executing it, or running 'emulate sh' as the first > command (and possibly other ways I don't know about): > % zsh -fcvx 'emulate sh;f=" val" e=; echo "$e"$f' > +zsh:1> emulate sh > +zsh:1> f=' val' e='' > +zsh:1> echo '' val > val In the context of portable shell programming, and autoconf in particular, we already _always_ force zsh into 'emulate sh' mode, so that we don't trip up on non-POSIX incompatibilities like this behavior of word splitting. So it should not be a problem for autoconf, other than to remind users that portable programming requires the use of 'emulate sh' in scripts that don't pick up autoconf's automatic shell sanitization prelude. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 619 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* RE: Fix testsuite errors due to shell quoted parameter expansion issue. 2010-08-03 21:21 ` Mikael Magnusson 2010-08-03 21:28 ` Eric Blake @ 2010-08-04 15:18 ` John Lumby 2010-08-04 15:27 ` Eric Blake 1 sibling, 1 reply; 8+ messages in thread From: John Lumby @ 2010-08-04 15:18 UTC (permalink / raw) To: eblake; +Cc: zsh-workers, autoconf maillist, libtool-patches, mikachu [-- Attachment #1: Type: text/plain, Size: 3681 bytes --] Re the statement since "$e" is quoted, it must not be elided I don't think that is correct. In fact "$e" *must* be elided - because you juxtaposed it with the next token. eliding and quoting are orthogonal. What you appear to be be expecting is that the effect of quoting one token should somehow have some effect on interpretation of a special character (blank) in a subsequent unquoted token. I don't think that is expected behaviour. The quoting of $e has the effect of quoting any special characters inside *it*, of which there are none. your blank is unquoted. Then word-spilling proceeds. What I find interesting is this bash -c 'declare -i length; f=" val" e=;concat="$e"$f; length=${#concat}; echo "length= $length"' 4 John Lumby > Date: Tue, 3 Aug 2010 23:21:02 +0200 > From: mikachu@gmail.com > To: eblake@redhat.com > CC: zsh-workers@zsh.org; Autoconf@gnu.org; libtool-patches@gnu.org > Subject: Re: Fix testsuite errors due to shell quoted parameter expansion issue. > > On 3 August 2010 22:55, Eric Blake <eblake@redhat.com> wrote: > > [adding autoconf to document some shell bugs] > > > > On 08/03/2010 02:32 PM, Ralf Wildenhues wrote: > >> Interesting shell unportability: > >> > >> $ bash -c 'f=" val" e=; echo "$e"$f' > >> val > >> $ ksh -c 'f=" val" e=; echo "$e"$f' > >> val > >> > >> ksh93, dash, zsh all do it like ksh. Is that a bug in bash? > > > > Yes; adding bug-bash accordingly. According to POSIX: > > > > http://www.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_05 > > > > "After parameter expansion ( Parameter Expansion ), command substitution > > ( Command Substitution ), and arithmetic expansion ( Arithmetic > > Expansion ), the shell shall scan the results of expansions and > > substitutions that did not occur in double-quotes for field splitting > > and multiple fields can result." > > > > Since $f is not quoted, its expansion must undergo field splitting. But > > since "$e" is quoted, it must not be elided even though empty. The > > result must be _two_ fields, as if you had done "echo '' 'val'". > > > > But it is _also_ a bug in zsh; adding zsh-workers accordingly. > > > > $ zsh -cvx 'f=" val" e=; echo "$e"$f' > > +zsh:1> f=' val' e='' > > +zsh:1> echo ' val' > > val > > > > Oops - zsh only passed one argument to echo, with a leading space, > > instead of passing an empty argument and letting echo supply the space. > > ksh93, pdksh, and dash get it right (although dash doesn't use quotes > > in -vx output, hence my use of n() to force things to tell; n() is > > another way to expose the bash and zsh bugs). > > zsh doesn't do word splitting by default, you can enable it with the = modifier: > % zsh -fcvx 'f=" val" e=; echo "$e"$=f' > +zsh:1> f=' val' e='' > +zsh:1> echo '' val > val > > does what you want > > Alternatively you can make zsh try to be closer to sh by setting > argv[0] to sh when executing it, or running 'emulate sh' as the first > command (and possibly other ways I don't know about): > % zsh -fcvx 'emulate sh;f=" val" e=; echo "$e"$f' > +zsh:1> emulate sh > +zsh:1> f=' val' e='' > +zsh:1> echo '' val > val > > There's also --shwordsplit for this specific case: > % zsh --shwordsplit -fcvx 'f=" val" e=; echo "$e"$f' > +zsh:1> f=' val' e='' > +zsh:1> echo '' val > val > > (the -f above only avoids loading my .zshenv which would spam my output) > > -- > Mikael Magnusson > > _______________________________________________ > Autoconf mailing list > Autoconf@gnu.org > http://lists.gnu.org/mailman/listinfo/autoconf ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fix testsuite errors due to shell quoted parameter expansion issue. 2010-08-04 15:18 ` John Lumby @ 2010-08-04 15:27 ` Eric Blake 2010-08-04 17:06 ` John Lumby 0 siblings, 1 reply; 8+ messages in thread From: Eric Blake @ 2010-08-04 15:27 UTC (permalink / raw) To: John Lumby; +Cc: zsh-workers, autoconf maillist, libtool-patches, mikachu [-- Attachment #1: Type: text/plain, Size: 2379 bytes --] [please don't top-post on technical lists] On 08/04/2010 09:18 AM, John Lumby wrote: > > Re the statement > > since "$e" is quoted, it must not be elided > > I don't think that is correct. In fact "$e" *must* be elided - because you juxtaposed it with the next token. eliding and quoting are orthogonal. Your statement is confusing to me. Here, in more detail, is exactly what I meant (and what I though I implied in the first case): The word in question is '"$e"$f' before either parameter substitution, or word splitting, or quote elision. After parameter substitution, you are left with '"" val', where a portion of the string was originally quoted, but you also had a portion of the string that was unquoted. Only the ' val' portion of the result is subject to word splitting. Because "$e" is quoted, the resulting empty string must not be elided during word splitting. Therefore, the result after word splitting MUST be the two words '""' and 'val'; then you apply quote elision and are left with two words: '' and 'val'. > > What you appear to be be expecting is that the effect of quoting one token should somehow have some effect on interpretation of a special character (blank) in a subsequent unquoted token. I don't think that is expected behaviour. The quoting of $e has the effect of quoting any special characters inside *it*, of which there are none. your blank is unquoted. Then word-spilling proceeds. No, what I expect is that within a single word, the quoted portion of the word must not be elided during word splitting, except in the special case of "$@". > > What I find interesting is this > > bash -c 'declare -i length; f=" val" e=;concat="$e"$f; length=${#concat}; echo "length= $length"' > 4 Why do you find that interesting? There is NO word splitting in assignment contexts, so the effect of concat="$e"$f is the same as if you had done concat=$e$f or concat="$e$f" in all three cases, concat is assigned the value ' val', which is indeed length 4. But it is unrelated to the bash bug at hand, which is that when "$e"$f is in a context subject to word splitting, then it must result in the two words '' and 'val', not the single word 'val'. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 619 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* RE: Fix testsuite errors due to shell quoted parameter expansion issue. 2010-08-04 15:27 ` Eric Blake @ 2010-08-04 17:06 ` John Lumby 0 siblings, 0 replies; 8+ messages in thread From: John Lumby @ 2010-08-04 17:06 UTC (permalink / raw) To: Eric Blake; +Cc: zsh-workers, autoconf maillist [-- Attachment #1: Type: text/plain, Size: 1024 bytes --] Eric Blake wrote> On 08/04/2010 09:18 AM, John Lumby wrote: > > > > > Re the statement > > > > since "$e" is quoted, it must not be elided > > > > I don't think that is correct. In fact "$e" *must* be elided - > > because you juxtaposed it with the next token. eliding and quoting are orthogonal. > > Your statement is confusing to me. Here, in more detail, is exactly > what I meant (and what I though I implied in the first case): > > ... > > Because "$e" is quoted, the resulting empty string must not be elided > during word splitting. > I see - thanks. I was not aware of that effect of quoting. But I see you are correct. > > ... > > Why do you find that interesting? There is NO word splitting in > assignment contexts, so the effect of Again I stand corrected, thanks. I thought an unquoted blank would terminate an assignment value. Maybe I was confusing tokenization before substitution with word splitting after substitution. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fix testsuite errors due to shell quoted parameter expansion issue. 2010-08-03 20:55 ` Fix testsuite errors due to shell quoted parameter expansion issue Eric Blake 2010-08-03 21:21 ` Mikael Magnusson @ 2010-08-03 21:43 ` Peter Stephenson 2010-08-05 22:15 ` Chet Ramey 2 siblings, 0 replies; 8+ messages in thread From: Peter Stephenson @ 2010-08-03 21:43 UTC (permalink / raw) To: Eric Blake; +Cc: zsh-workers On Tue, 03 Aug 2010 14:55:36 -0600 Eric Blake <eblake@redhat.com> wrote: > But it is _also_ a bug in zsh; adding zsh-workers accordingly. > > $ zsh -cvx 'f=" val" e=; echo "$e"$f' > +zsh:1> f=' val' e='' > +zsh:1> echo ' val' > val > > Oops - zsh only passed one argument to echo, with a leading space, > instead of passing an empty argument and letting echo supply the space. That's not a bug, you haven't enabled the SH_WORD_SPLIT option. Try in sh compatibility mode. -- Peter Stephenson <p.w.stephenson@ntlworld.com> Web page now at http://homepage.ntlworld.com/p.w.stephenson/ ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Fix testsuite errors due to shell quoted parameter expansion issue. 2010-08-03 20:55 ` Fix testsuite errors due to shell quoted parameter expansion issue Eric Blake 2010-08-03 21:21 ` Mikael Magnusson 2010-08-03 21:43 ` Peter Stephenson @ 2010-08-05 22:15 ` Chet Ramey 2 siblings, 0 replies; 8+ messages in thread From: Chet Ramey @ 2010-08-05 22:15 UTC (permalink / raw) To: Eric Blake Cc: libtool-patches, gary, Bash - Bug, zsh-workers, Autoconf, chet.ramey On 8/3/10 4:55 PM, Eric Blake wrote: > [adding autoconf to document some shell bugs] > > On 08/03/2010 02:32 PM, Ralf Wildenhues wrote: >> Interesting shell unportability: >> >> $ bash -c 'f=" val" e=; echo "$e"$f' >> val >> $ ksh -c 'f=" val" e=; echo "$e"$f' >> val >> >> ksh93, dash, zsh all do it like ksh. Is that a bug in bash? > > Yes; adding bug-bash accordingly. According to POSIX: Thanks; this was an easy fix. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRU chet@case.edu http://cnswww.cns.cwru.edu/~chet/ ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2010-08-05 22:22 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <20100803203204.GG13690@gmx.de> 2010-08-03 20:55 ` Fix testsuite errors due to shell quoted parameter expansion issue Eric Blake 2010-08-03 21:21 ` Mikael Magnusson 2010-08-03 21:28 ` Eric Blake 2010-08-04 15:18 ` John Lumby 2010-08-04 15:27 ` Eric Blake 2010-08-04 17:06 ` John Lumby 2010-08-03 21:43 ` Peter Stephenson 2010-08-05 22:15 ` Chet Ramey
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/zsh/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).