Word splitting with zsh fix

Word splitting with zsh fix

[DervishD]

[-- Attachment #1: Type: text/plain, Size: 1195 bytes --]

    Hi all :)

    I've spotted a problem with libtool that has to do with the word
splitting that Zsh does in the expression '${1+"$@"}' (which, in
turn, is a workaround to fix the "$@" expansion problem when the
variable is empty in old shells). GNU autoconf has already corrected
this problem, and I'm using a similar fix for libtool. The patch is
gzipped and attached to this message.

    Looking at the SUS (POSIX) standard, I'm not sure if field
splitting should be performed in the above expression or not, so I
don't know if the correct behaviour is what Bash does or what Zsh
does. Anyway, I'm not here looking for a culprit but for a solution.
I will be glad to discuss the issue, anyway, just for curiosity's
sake ;)

    I've tested the patch against "littlecms" sources, because that's
the package where the problem was spotted by David Gómez Espinosa
<david@pleyades.net>. The patch was suggested by Andrey Borzenkov
<arvidjaar@newmail.ru>.

    Thanks in advance and happy hacking :)

    Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736 | http://www.dervishd.net
http://www.pleyades.net & http://www.gotesdelluna.net
It's my PC and I'll cry if I want to... RAmen!

[-- Attachment #2: libtool.patch.gz --]
[-- Type: application/x-gzip, Size: 433 bytes --]

Re: Word splitting with zsh fix

[Ralf Wildenhues]

* DervishD wrote on Sat, Feb 11, 2006 at 01:22:26PM CET:
> 
>     I've spotted a problem with libtool that has to do with the word
> splitting that Zsh does in the expression '${1+"$@"}' (which, in
> turn, is a workaround to fix the "$@" expansion problem when the
> variable is empty in old shells). GNU autoconf has already corrected
> this problem, and I'm using a similar fix for libtool. The patch is
> gzipped and attached to this message.

Thanks for noticing this.  In fact, for CVS Libtool, we just call
AS_SHELL_SANITIZE to get Autoconf's initialization blurb (and its
fixes).  There is an open (unrelated) issue about that code being
discussed over at autoconf-patches at the moment; I'll backport it
to branch-1-5 as soon as that has settled.

Thanks again!

Cheers,
Ralf

Re: Word splitting with zsh fix

[DervishD]

    Hi Ralf :)

 * Ralf Wildenhues <Ralf.Wildenhues@gmx.de> dixit:
> * DervishD wrote on Sat, Feb 11, 2006 at 01:22:26PM CET:
> >     I've spotted a problem with libtool that has to do with the word
> > splitting that Zsh does in the expression '${1+"$@"}' (which, in
> > turn, is a workaround to fix the "$@" expansion problem when the
> > variable is empty in old shells). GNU autoconf has already corrected
> > this problem, and I'm using a similar fix for libtool. The patch is
> > gzipped and attached to this message.
> 
> Thanks for noticing this.

    Well, I didn't noticed it, David Gómez Espinosa
<david@pleyades.net> did.

> In fact, for CVS Libtool, we just call AS_SHELL_SANITIZE to get
> Autoconf's initialization blurb (and its fixes).

    Oh, good idea :))

> There is an open (unrelated) issue about that code being discussed
> over at autoconf-patches at the moment; I'll backport it to
> branch-1-5 as soon as that has settled.

    OK, I'll take a look as soon as it is released :)
 
> Thanks again!

    Don't even mention it ;)

    Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736 | http://www.dervishd.net
http://www.pleyades.net & http://www.gotesdelluna.net
It's my PC and I'll cry if I want to... RAmen!

Re: Word splitting with zsh fix

[Ralf Wildenhues]

[-- Attachment #1: Type: text/plain, Size: 2221 bytes --]

[ moving to libtool-patches; sorry for the first-post moderation ]

Hi Raúl,

* DervishD wrote on Sun, Feb 12, 2006 at 11:33:15AM CET:
>  * Ralf Wildenhues <Ralf.Wildenhues@gmx.de> dixit:
> > * DervishD wrote on Sat, Feb 11, 2006 at 01:22:26PM CET:
> > >     I've spotted a problem with libtool that has to do with the word
> > > splitting that Zsh does in the expression '${1+"$@"}' (which, in
> > > turn, is a workaround to fix the "$@" expansion problem when the
> > > variable is empty in old shells). GNU autoconf has already corrected
> > > this problem, and I'm using a similar fix for libtool. The patch is
> > > gzipped and attached to this message.

> > In fact, for CVS Libtool, we just call AS_SHELL_SANITIZE to get
> > Autoconf's initialization blurb (and its fixes).
> 
>     Oh, good idea :))

Yes.

> > There is an open (unrelated) issue about that code being discussed
> > over at autoconf-patches at the moment; I'll backport it to
> > branch-1-5 as soon as that has settled.
> 
>     OK, I'll take a look as soon as it is released :)

That seems to have settled now.  Also, I found that libtool needs to
employ the same workarounds for its shell wrappers, too (we should
eventually add a test for this, I guess, to expose this).

In the shell wrapper, we need the effect of _AS_BOURNE_COMPATIBLE, but
not AS_SHELL_SANITIZE: the executed program should run in the user's
locale.  However, _AS_BOURNE_COMPATIBLE isn't published by Autoconf.
So I'm checking in the following patches to branch-1-5 and HEAD, and
I'll write to autoconf-patches to have a public version of the macro.
When we can rely on a released Autoconf version with the published
macro, we can simplify the shell wrapper head to something like this:
|   cat >>$output <<\_LTEOF
| AS_BOURNE_COMPATIBLE
| _LTEOF

Cheers,
Ralf

HEAD:
	* libltdl/config/ltmain.m4sh (func_mode_link) <shell wrapper>: 
	Add _AS_BOURNE_COMPATIBLE code from Autoconf, to fix issues
	with zsh and other shells.
	Reported by David Gómez Espinosa <david@pleyades.net>.

branch-1-5:
	* ltmain.in <startup, shell wrapper>: Add _AS_BOURNE_COMPATIBLE
	code from Autoconf, to fix issues with zsh and other shells.
	Reported by David Gómez Espinosa <david@pleyades.net>.

[-- Attachment #2: HEAD-bourne-compatible-wrapper.diff --]
[-- Type: text/plain, Size: 1252 bytes --]

	* libltdl/config/ltmain.m4sh (func_mode_link) <shell wrapper>: 
	Add _AS_BOURNE_COMPATIBLE code from Autoconf, to fix issues
	with zsh and other shells.
	Reported by David Gómez Espinosa <david@pleyades.net>.

Index: libltdl/config/ltmain.m4sh
===================================================================
RCS file: /cvsroot/libtool/libtool/libltdl/config/ltmain.m4sh,v
retrieving revision 1.35
diff -u -r1.35 ltmain.m4sh
--- libltdl/config/ltmain.m4sh	5 Feb 2006 11:06:31 -0000	1.35
+++ libltdl/config/ltmain.m4sh	13 Feb 2006 16:35:16 -0000
@@ -6186,6 +6186,20 @@
 Xsed='${SED} -e 1s/^X//'
 sed_quote_subst='$sed_quote_subst'
 
+# Be Bourne compatible
+if test -n \"\${ZSH_VERSION+set}\" && (emulate sh) >/dev/null 2>&1; then
+  emulate sh
+  NULLCMD=:
+  [#] Zsh 3.x and 4.x performs word splitting on \${1+\"\$[@]\"}, which
+  # is contrary to our usage.  Disable this feature.
+  alias -g '\${1+\"\$[@]\"}'='\"\$[@]\"'
+  setopt NO_GLOB_SUBST
+else
+  case \`(set -o) 2>/dev/null\` in *posix*) set -o posix;; esac
+fi
+BIN_SH=xpg4; export BIN_SH # for Tru64
+DUALCASE=1; export DUALCASE # for MKS sh
+
 # The HP-UX ksh and POSIX shell print the target directory to stdout
 # if CDPATH is set.
 (unset CDPATH) >/dev/null 2>&1 && unset CDPATH

[-- Attachment #3: branch-1-5-bourne-compatible.diff --]
[-- Type: text/plain, Size: 2017 bytes --]

	* ltmain.in <startup, shell wrapper>: Add _AS_BOURNE_COMPATIBLE
	code from Autoconf, to fix issues with zsh and other shells.
	Reported by David Gómez Espinosa <david@pleyades.net>.

Index: ltmain.in
===================================================================
RCS file: /cvsroot/libtool/libtool/Attic/ltmain.in,v
retrieving revision 1.334.2.121
diff -u -r1.334.2.121 ltmain.in
--- ltmain.in	5 Feb 2006 11:08:52 -0000	1.334.2.121
+++ ltmain.in	13 Feb 2006 14:35:05 -0000
@@ -46,11 +46,19 @@
 VERSION=@VERSION@
 TIMESTAMP="@TIMESTAMP@"
 
-# See if we are running on zsh, and set the options which allow our
-# commands through without removal of \ escapes.
-if test -n "${ZSH_VERSION+set}" ; then
+# Be Bourne compatible (taken from Autoconf:_AS_BOURNE_COMPATIBLE).
+if test -n "${ZSH_VERSION+set}" && (emulate sh) >/dev/null 2>&1; then
+  emulate sh
+  NULLCMD=:
+  # Zsh 3.x and 4.x performs word splitting on ${1+"$@"}, which
+  # is contrary to our usage.  Disable this feature.
+  alias -g '${1+"$@"}'='"$@"'
   setopt NO_GLOB_SUBST
+else
+  case `(set -o) 2>/dev/null` in *posix*) set -o posix;; esac
 fi
+BIN_SH=xpg4; export BIN_SH # for Tru64
+DUALCASE=1; export DUALCASE # for MKS sh
 
 # Check that we have a working $echo.
 if test "X$1" = X--no-reexec; then
@@ -5271,6 +5279,20 @@
 Xsed='${SED} -e 1s/^X//'
 sed_quote_subst='$sed_quote_subst'
 
+# Be Bourne compatible (taken from Autoconf:_AS_BOURNE_COMPATIBLE).
+if test -n \"\${ZSH_VERSION+set}\" && (emulate sh) >/dev/null 2>&1; then
+  emulate sh
+  NULLCMD=:
+  # Zsh 3.x and 4.x performs word splitting on \${1+\"\$@\"}, which
+  # is contrary to our usage.  Disable this feature.
+  alias -g '\${1+\"\$@\"}'='\"\$@\"'
+  setopt NO_GLOB_SUBST
+else
+  case \`(set -o) 2>/dev/null\` in *posix*) set -o posix;; esac
+fi
+BIN_SH=xpg4; export BIN_SH # for Tru64
+DUALCASE=1; export DUALCASE # for MKS sh
+
 # The HP-UX ksh and POSIX shell print the target directory to stdout
 # if CDPATH is set.
 (unset CDPATH) >/dev/null 2>&1 && unset CDPATH

Re: Word splitting with zsh fix

[Roumen Petrov]

Hi Raúl,

I would like to know if option SH_WORD_SPLIT is set problem is solved too ?

Test case:
===============================
#!/bin/zsh

VAR="v1 v2"

for V in ${VAR}; do
  echo V1=$V
done

setopt SH_WORD_SPLIT
for V in ${VAR}; do
  echo V2=$V
done
===============================
When SH_WORD_SPLIT is set zsh work as bash, ksh, ash etc.


DervishD wrote:
>     Hi all :)
> 
>     I've spotted a problem with libtool that has to do with the word
> splitting that Zsh does in the expression '${1+"$@"}' (which, in
> turn, is a workaround to fix the "$@" expansion problem when the
> variable is empty in old shells). GNU autoconf has already corrected
> this problem, and I'm using a similar fix for libtool. The patch is
> gzipped and attached to this message.
> 
>     Looking at the SUS (POSIX) standard, I'm not sure if field
> splitting should be performed in the above expression or not, so I
> don't know if the correct behaviour is what Bash does or what Zsh
> does. Anyway, I'm not here looking for a culprit but for a solution.
> I will be glad to discuss the issue, anyway, just for curiosity's
> sake ;)
> 
>     I've tested the patch against "littlecms" sources, because that's
> the package where the problem was spotted by David Gómez Espinosa
> <david@pleyades.net>. The patch was suggested by Andrey Borzenkov
> <arvidjaar@newmail.ru>.
> 
>     Thanks in advance and happy hacking :)
> 
>     Raúl Núñez de Arenas Coronado
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Bug-libtool mailing list
> Bug-libtool@gnu.org
> http://lists.gnu.org/mailman/listinfo/bug-libtool

Re: Word splitting with zsh fix

[Andrey Borzenkov]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sunday 12 February 2006 18:05, Roumen Petrov wrote:
> Hi Raúl,
>
> I would like to know if option SH_WORD_SPLIT is set problem is solved too ?
>

No. In this case it splits too much. It was discussed back in 2002 already; 
using global alias seems to be the most straightforward way to fix this 
particular problem.

- -andrey
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)

iD4DBQFD72jhR6LMutpd94wRAprpAJ40U9TouBnaO9NOSPxNJT8m6GJCbACXev4c
5HbH+TWruyxs44u5B522+A==
=8pDI
-----END PGP SIGNATURE-----

Re: Word splitting with zsh fix

[Roumen Petrov]

Andrey Borzenkov wrote:
> On Sunday 12 February 2006 18:05, Roumen Petrov wrote:
> 
>>Hi Raúl,
>>
>>I would like to know if option SH_WORD_SPLIT is set problem is solved too ?
>>
> 
> 
> No. In this case it splits too much. It was discussed back in 2002 already; 
> using global alias seems to be the most straightforward way to fix this 
> particular problem.
> 

Thanks Andrey,
After patch really all libtool tests succeed with zsh. Thanks!
Before the patch "-DPACKAGE_STRING=\"cdemo 0.1\"" was not quoted.

Is word splitting on ${1+"$@"} bug in zsh ?


P.P.: Option SH_WORD_SPLIT is without effect.

Re: Word splitting with zsh fix

[Wayne Davison]

On Sun, Feb 12, 2006 at 05:05:14PM +0200, Roumen Petrov wrote:
> I would like to know if option SH_WORD_SPLIT is set problem is solved
> too?

Can you explain what you mean?  Because your test case works exactly
as it should in zsh:

% ./testcase
V1=v1 v2
V2=v1
V2=v2

(I note that your test case explicitly requested zsh, not bourne-shell
compatibility mode.)

If you wanted the first loop to split VAR on spaces, in zsh you need
to request that (when SH_WORD_SPLIT is not enabled):

for V in ${=VAR}; do
  echo V1=$V
done

..wayne..

Re: Word splitting with zsh fix

[Roumen Petrov]

Wayne Davison wrote:
> On Sun, Feb 12, 2006 at 05:05:14PM +0200, Roumen Petrov wrote:
> 
>>I would like to know if option SH_WORD_SPLIT is set problem is solved
>>too?
> 
> 
> Can you explain what you mean?  Because your test case works exactly
> as it should in zsh:

No objections.

> % ./testcase
> V1=v1 v2
> V2=v1
> V2=v2
> 


I don't know other way to write compatible script without to set SH_WORD_SPLIT for zsh.
As example ash ./testcase will display:
V1=v1
V1=v2
setopt: not found
V2=v1
V2=v2
Output for ksh and bash is same.


Regards,
Roumen

Re: Word splitting with zsh fix

[Wayne Davison]

On Sun, Feb 12, 2006 at 11:52:23PM +0200, Roumen Petrov wrote:
> I don't know other way to write compatible script without to set 
> SH_WORD_SPLIT for zsh.

The right way is to run the script using /bin/sh, and if that happens
to be zsh, it will automatically set SH_WORD_SPLIT when it turns on sh
emulation mode.  When run explicitly as zsh, you can turn it on manually
using something like this:

[ -n "${ZSH_VERSION+set}" ] && emulate sh

> Is word splitting on ${1+"$@"} bug in zsh ?

Yes, it is.  I'm looking into a fix.

..wayne..

Re: Word splitting with zsh fix

[DervishD]

    Hi Roumen :)

    Sorry for not answering before.

 * Roumen Petrov <bugtrack@roumenpetrov.info> dixit:
> I would like to know if option SH_WORD_SPLIT is set problem is solved too ?

    I don't know what you mean, but for the rest of the thread I
reckon you meant something about SH_WORD_SPLIT unset when zsh is
invoked as /bin/zsh and not as /bin/sh.

    If you want to write a "portable" script (which is very
difficult, believe me) you should use "#! /bin/sh" as the first line.
This will set SH_WORD_SPLIT in zsh. By the way, for *really* portable
script, use the first line as shown above, with a space between the
bang and the slash.

    Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736 | http://www.dervishd.net
http://www.pleyades.net & http://www.gotesdelluna.net
It's my PC and I'll cry if I want to... RAmen!