Re: Bug#517008: alias not expanded with zsh -c

Re: Bug#517008: alias not expanded with zsh -c

[Clint Adams]

On Wed, Feb 25, 2009 at 03:28:50AM +0100, Vincent Lefevre wrote:
> Aliases are not expanded with the -c option, as shown below.
> 
> vin% cmd=$(printf "emulate sh\nalias a='echo OK >&2'\na")
> vin% printf "%s\n" "$cmd"
> emulate sh
> alias a='echo OK >&2'
> a
> vin% printf "%s" "$cmd" | zsh -f
> OK
> vin% zsh -fc "$cmd"
> zsh:3: command not found: a
> vin% 
> 
> There's no such problem with ksh93, bash in POSIX mode, and dash
> (pdksh has the same bug).

That does appear to be the behavior.

Re: Bug#517008: alias not expanded with zsh -c

[Peter Stephenson]

Clint Adams wrote:
> On Wed, Feb 25, 2009 at 03:28:50AM +0100, Vincent Lefevre wrote:
> > Aliases are not expanded with the -c option, as shown below.
> > 
> > vin% cmd=$(printf "emulate sh\nalias a='echo OK >&2'\na")
> > vin% printf "%s\n" "$cmd"
> > emulate sh
> > alias a='echo OK >&2'
> > a
> > vin% printf "%s" "$cmd" | zsh -f
> > OK
> > vin% zsh -fc "$cmd"
> > zsh:3: command not found: a
> > vin% 
> > 
> > There's no such problem with ksh93, bash in POSIX mode, and dash
> > (pdksh has the same bug).
> 
> That does appear to be the behavior.

Yes, at least as far as native zsh mode goes this isn't a bug.  The
command passed is treated as a single piece of text that's parsed in one
go, so the alias isn't available until afterwards (by which time the
shell has exited).  This is the same behaviour as other chunks of code
read in one go.  To get script-like behaviour, don't use "-c", use a
script argument.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070

Re: Bug#517008: alias not expanded with zsh -c

[Peter Stephenson]

On Wed, 25 Mar 2009 16:30:08 +0000
Peter Stephenson <pws@csr.com> wrote:
>Clint Adams wrote:
>>On Wed, Feb 25, 2009 at 03:28:50AM +0100, Vincent Lefevre wrote:
>>> Aliases are not expanded with the -c option, as shown below.
>>>
>>> vin% cmd=$(printf "emulate sh\nalias a='echo OK >&2'\na")
>>> vin% printf "%s\n" "$cmd"
>>> emulate sh
>>> alias a='echo OK >&2'
>>> a
>>> vin% printf "%s" "$cmd" | zsh -f
>>> OK
>>> vin% zsh -fc "$cmd"
>>> zsh:3: command not found: a
>>> vin% 
>>>
>>> There's no such problem with ksh93, bash in POSIX mode, and dash
>>> (pdksh has the same bug).
>>
>> That does appear to be the behavior.
>
> Yes, at least as far as native zsh mode goes this isn't a bug.

I should also have pointed out that the "emulate sh" doesn't make any
difference, it only takes effect after the string for -c has already been
parsed---it's basically the same issue as the alias expansion one itself.
You would need to start the shell in the appropriate mode.  I don't think
that's an unreasonable requirement.

Is the fix as simple as the following?  This is not a trick question, I
could very easily have missed something.

Index: Src/init.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/init.c,v
retrieving revision 1.100
diff -u -r1.100 init.c
--- Src/init.c	9 Mar 2009 15:57:59 -0000	1.100
+++ Src/init.c	25 Mar 2009 17:19:57 -0000
@@ -1038,9 +1038,32 @@
     if (cmd) {
 	if (SHIN >= 10)
 	    fclose(bshin);
-	SHIN = movefd(open("/dev/null", O_RDONLY | O_NOCTTY));
-	bshin = fdopen(SHIN, "r");
-	execstring(cmd, 0, 1);
+	if (isset(POSIXALIASES)) {
+	    /*
+	     * We need to process input one line at a time to
+	     * ensure aliases are available.
+	     */
+	    char *fil;
+	    int tempfd = gettempfile(NULL, 1, &fil);
+
+	    if (tempfd < 0) {
+		zerr("fc", "can't open temp file: %e", errno);
+		lastval = 1;
+	    } else {
+		int cmdlen;
+		unmetafy(cmd, &cmdlen);
+		write(tempfd, cmd, cmdlen);
+		lseek(tempfd, 0, SEEK_SET);
+		SHIN = tempfd;
+		bshin = fdopen(SHIN, "r");
+		loop(1, 0);
+	    }
+	    unlink(fil);
+	} else {
+	    SHIN = movefd(open("/dev/null", O_RDONLY | O_NOCTTY));
+	    bshin = fdopen(SHIN, "r");
+	    execstring(cmd, 0, 1);
+	}
 	stopmsg = 1;
 	zexit(lastval, 0);
     }
Index: Doc/Zsh/options.yo
===================================================================
RCS file: /cvsroot/zsh/zsh/Doc/Zsh/options.yo,v
retrieving revision 1.78
diff -u -r1.78 options.yo
--- Doc/Zsh/options.yo	3 Mar 2009 17:35:29 -0000	1.78
+++ Doc/Zsh/options.yo	25 Mar 2009 17:19:57 -0000
@@ -1719,6 +1719,20 @@
 ifnzman(noderef(Reserved Words))\
 ifzman(the section RESERVED WORDS in zmanref(zshmisc)).
 
+Another effect of this option, provided it is set at shell start up,
+is to cause the shell's tt(-c) option to save commands to a temporary file
+and read them line by line to ensure aliases are expanded.  For example,
+
+example(zsh -fc 'alias o="echo Hello"
+o')
+
+reports a `command not found' error, while
+
+example(zsh -o posixaliases -fc 'alias o="echo Hello"
+o')
+
+outputs `Hello'.
+
 Alias expansion takes place while text is being read; hence when this
 option is set it does not take effect until the end of any function or
 other piece of shell code evaluated as one unit.


-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070

Re: Bug#517008: alias not expanded with zsh -c

[Bart Schaefer]

On Mar 25,  5:25pm, Peter Stephenson wrote:
} Subject: Re: Bug#517008: alias not expanded with zsh -c
}
} > Yes, at least as far as native zsh mode goes this isn't a bug.
} 
} I should also have pointed out that the "emulate sh" doesn't make any
} difference, it only takes effect after the string for -c has already been
} parsed---it's basically the same issue as the alias expansion one itself.

It's my recollection that the discussion of this on the austin-group list
led to the conclusion that zsh's native behavior in this case is within
the allowed variation in the POSIX spec.  Vincent, if you're reading this, 
do you disagree, and can you point out why?

My personal feeling is that aliases are an interactive convenience that
should never be used in scripts (including "sh -c") in the first place.

} You would need to start the shell in the appropriate mode.  I don't think
} that's an unreasonable requirement.

Especially considering that now you can do
	emulate sh -c "..."
which is admittedly a little clumsy to use as in
	zsh -c 'emulate sh -c "..."'
but nevertheless is available to get the desired effect.

} Is the fix as simple as the following?  This is not a trick question, I
} could very easily have missed something.

Will this cover "emulate -c"?

I'm a bit leery of even starting down this road, because it still fails
to make any difference if the alias definition and use are on the same
line (separated with ";" or "&&" etc.).

-- 

Re: Bug#517008: alias not expanded with zsh -c

[Peter Stephenson]

Bart Schaefer wrote:
> } Is the fix as simple as the following?  This is not a trick question, I
> } could very easily have missed something.
> 
> Will this cover "emulate -c"?

If you mean cover it in the sense that the option is turned on before
any code is parsed---yes, but that's not changed by this patch.  It
won't cover the case where emulate -c is followed by multiple lines of
code that define aliases that are expected to be available in following
lines.  I'm not sure it needs to: there's no question of that being
standardised and we are at liberty to define the rule as being what it
is by default, i.e. code is parsed and aliases expanded before any code
is executed; although I can see you might expect it to be the same
as the shell's own -c option.

> I'm a bit leery of even starting down this road, because it still fails
> to make any difference if the alias definition and use are on the same
> line (separated with ";" or "&&" etc.).

I think it's accepted that this will not work with any current shell
implementation, not just zsh.

But yes, I share your feeling that this isn't a particularly vital
addition, it just looked easy to do like other shells in compatability
mode, which is the point of that.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070

Re: Bug#517008: alias not expanded with zsh -c

[Vincent Lefevre]

On 2009-03-25 11:11:28 -0700, Bart Schaefer wrote:
> On Mar 25,  5:25pm, Peter Stephenson wrote:
> } Subject: Re: Bug#517008: alias not expanded with zsh -c
> }
> } > Yes, at least as far as native zsh mode goes this isn't a bug.
> } 
> } I should also have pointed out that the "emulate sh" doesn't make any
> } difference, it only takes effect after the string for -c has already been
> } parsed---it's basically the same issue as the alias expansion one itself.
> 
> It's my recollection that the discussion of this on the austin-group list
> led to the conclusion that zsh's native behavior in this case is within
> the allowed variation in the POSIX spec.  Vincent, if you're reading this, 
> do you disagree, and can you point out why?

Yes, this is allowed: David Korn said that the standard doesn't
explain how the shell reads and processes commands. See:

  https://www.opengroup.org/sophocles/show_mail.tpl?CALLER=show_archive.tpl&source=L&listname=austin-group-l&id=12011

All on this subject:

  https://www.opengroup.org/sophocles/show_archive.tpl?source=L&listname=austin-group-l&first=1&pagesize=20&searchstring=alias+substitution+--+shells+differ&zone=G

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)

Re: Bug#517008: alias not expanded with zsh -c

[Bart Schaefer]

On Mar 25,  6:20pm, Peter Stephenson wrote:
}
} But yes, I share your feeling that this isn't a particularly vital
} addition, it just looked easy to do like other shells in compatability
} mode, which is the point of that.

It just seems to me that we're potentially penalizing a script that does
not use aliases in the can't-open-tempfile case in order to support a
script that does not use aliases.

Given potential security problems with using tempfiles and the likelyhood
of a script using an alias vs. the likelyhood of file creation failing,
which case ought we be catering too?

Re: Bug#517008: alias not expanded with zsh -c

[Peter Stephenson]

On Wed, 25 Mar 2009 17:14:31 -0700
Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Mar 25,  6:20pm, Peter Stephenson wrote:
> }
> } But yes, I share your feeling that this isn't a particularly vital
> } addition, it just looked easy to do like other shells in compatability
> } mode, which is the point of that.
> 
> It just seems to me that we're potentially penalizing a script that does
> not use aliases in the can't-open-tempfile case in order to support a
> script that does not use aliases.
> 
> Given potential security problems with using tempfiles and the likelyhood
> of a script using an alias vs. the likelyhood of file creation failing,
> which case ought we be catering too?

From what I've heard, it doesn't sound like it's worth more than a
warning.

Index: Doc/Zsh/options.yo
===================================================================
RCS file: /cvsroot/zsh/zsh/Doc/Zsh/options.yo,v
retrieving revision 1.78
diff -u -r1.78 options.yo
--- Doc/Zsh/options.yo	3 Mar 2009 17:35:29 -0000	1.78
+++ Doc/Zsh/options.yo	26 Mar 2009 09:49:05 -0000
@@ -1721,7 +1721,12 @@
 
 Alias expansion takes place while text is being read; hence when this
 option is set it does not take effect until the end of any function or
-other piece of shell code evaluated as one unit.
+other piece of shell code evaluated as one unit.  Note this may
+cause differences from other shells even when the option is in
+effect.  For example, when running a command with `tt(zsh -c)' the
+entire command argument is parsed in one go, so aliases defined within
+the argument are not available even in later lines.  If in doubt, avoid use
+of aliases in non-interactive code.
 )
 pindex(POSIX_BUILTINS)
 pindex(NO_POSIX_BUILTINS)


-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070

Re: Bug#517008: alias not expanded with zsh -c

[Bart Schaefer]

On Mar 26,  9:50am, Peter Stephenson wrote:
} Subject: Re: Bug#517008: alias not expanded with zsh -c
}
} On Wed, 25 Mar 2009 17:14:31 -0700
} Bart Schaefer <schaefer@brasslantern.com> wrote:
} > It just seems to me that we're potentially penalizing a script that
} > does not use aliases in the can't-open-tempfile case in order to
} > support a script that does not use aliases.

In case it wasn't obvious, there's an extra "not" in that last line.

} > Given potential security problems with using tempfiles and the
} > likelyhood of a script using an alias vs. the likelyhood of file
} > creation failing, which case ought we be catering too?

Good grief, I was having a bad typing day yesterday.  Must be because
I was home sick at the time ...
 
} From what I've heard, it doesn't sound like it's worth more than a
} warning.
} 
} +other piece of shell code evaluated as one unit.  Note this may
} +cause differences from other shells even when the option is in
} +effect.  For example, when running a command with `tt(zsh -c)' the
} +entire command argument is parsed in one go, so aliases defined within

Is it obvious enough that "evaluated as one unit" and "parsed in one go"
mean the same thing?  And should that be tt(sh -c) rather than zsh?

Re: Bug#517008: alias not expanded with zsh -c

[Peter Stephenson]

On Thu, 26 Mar 2009 08:12:59 -0700
Bart Schaefer <schaefer@brasslantern.com> wrote:
> Is it obvious enough that "evaluated as one unit" and "parsed in one go"
> mean the same thing?  And should that be tt(sh -c) rather than zsh?

"parsed as one unit" might be better.

Index: Doc/Zsh/options.yo
===================================================================
RCS file: /cvsroot/zsh/zsh/Doc/Zsh/options.yo,v
retrieving revision 1.79
diff -u -r1.79 options.yo
--- Doc/Zsh/options.yo	26 Mar 2009 09:53:27 -0000	1.79
+++ Doc/Zsh/options.yo	26 Mar 2009 15:17:35 -0000
@@ -1721,12 +1721,13 @@
 
 Alias expansion takes place while text is being read; hence when this
 option is set it does not take effect until the end of any function or
-other piece of shell code evaluated as one unit.  Note this may
+other piece of shell code parsed as one unit.  Note this may
 cause differences from other shells even when the option is in
-effect.  For example, when running a command with `tt(zsh -c)' the
-entire command argument is parsed in one go, so aliases defined within
-the argument are not available even in later lines.  If in doubt, avoid use
-of aliases in non-interactive code.
+effect.  For example, when running a command with `tt(zsh -c)',
+or even `tt(zsh -o posixaliases -c)', the entire command argument is parsed
+as one unit, so aliases defined within the argument are not available even
+in later lines.  If in doubt, avoid use of aliases in non-interactive
+code.
 )
 pindex(POSIX_BUILTINS)
 pindex(NO_POSIX_BUILTINS)

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070