* [BUG] 'exec' runs shell functions and builtins @ 2017-07-25 21:49 ` Martijn Dekker 2017-07-26 9:23 ` Peter Stephenson ` (2 more replies) 0 siblings, 3 replies; 12+ messages in thread From: Martijn Dekker @ 2017-07-25 21:49 UTC (permalink / raw) To: Zsh hackers list In zsh, 'exec' looks up shell functions and builtins before external commands, and if it finds one it appears to do the equivalent of running the function or builtin followed by 'exit'. This is different from most other shells and turns out[1] to be a bug in POSIX terms; 'exec' is supposed to launch a program that overlays the current shell[2], implying the program launched by 'exec' is always external to the shell. This means that (exec commandname arguments ...) is a POSIXly correct way of guaranteeing the execution of an external command without specifying the path (something the "command" command doesn't do in POSIX mode). This was confirmed the other day by Geoff Clare and Chet Ramey on austin-group-l. (I seem to recall Bart asked on this list quite some time ago if POSIX provides for a way to do this.) This behaviour also appears to be contrary to the documentation in zshbuiltins(1) ("Replace the current shell with an external command rather than forking"). Test script: testFn() { exec : } if (PATH=/dev/null exec testFn) 2>/dev/null; then echo "this shell execs both functions and builtins" fi (The only other POSIX-ish shell that zsh has this in common with is pdksh/mksh. In pdksh this is definitely a bug as this behaviour differs from ksh88 which pdksh is supposed to be a clone of. The POSIX spec is also based on ksh88 behaviour.) Confirmed down to 4.3.17 (I didn't check older versions). - M. [1] https://www.mail-archive.com/austin-group-l@opengroup.org/msg01469.html [2] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_20_14 ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [BUG] 'exec' runs shell functions and builtins 2017-07-25 21:49 ` [BUG] 'exec' runs shell functions and builtins Martijn Dekker @ 2017-07-26 9:23 ` Peter Stephenson 2017-07-26 9:27 ` Peter Stephenson 2017-07-26 12:53 ` Peter Stephenson 2017-07-26 17:46 ` Bart Schaefer 2 siblings, 1 reply; 12+ messages in thread From: Peter Stephenson @ 2017-07-26 9:23 UTC (permalink / raw) To: Zsh hackers list On Tue, 25 Jul 2017 22:49:33 +0100 Martijn Dekker <martijn@inlv.org> wrote: > In zsh, 'exec' looks up shell functions and builtins before external > commands, and if it finds one it appears to do the equivalent of running > the function or builtin followed by 'exit'. This is different from most > other shells and turns out[1] to be a bug in POSIX terms; 'exec' is > supposed to launch a program that overlays the current shell[2], > implying the program launched by 'exec' is always external to the shell. Prior art suggests we can get away with adding this behaviour to the POSIX_BUILTINS option. (I'd like to hope people don't actually set the POSIX options separately anyway, but feel free not to tell me.) pws diff --git a/Doc/Zsh/options.yo b/Doc/Zsh/options.yo index cc6ae2a..edf43d4 100644 --- a/Doc/Zsh/options.yo +++ b/Doc/Zsh/options.yo @@ -2140,6 +2140,10 @@ In addition, various error conditions associated with the above builtins or tt(exec) cause a non-interactive shell to exit and an interactive shell to return to its top-level processing. +Furthermore, functions ahd shell builtins are not executed after +an tt(exec) prefix; the command to be executed must be an external +command found in the path. + Furthermore, the tt(getopts) builtin behaves in a POSIX-compatible fashion in that the associated variable tt(OPTIND) is not made local to functions. diff --git a/Src/exec.c b/Src/exec.c index 0a96879..f339dd6 100644 --- a/Src/exec.c +++ b/Src/exec.c @@ -2778,6 +2778,12 @@ execcmd_exec(Estate state, Execcmd_params eparams, * Reserved words take precedence over shell functions. */ checked = 1; + } else if (isset(POSIXBUILTINS) && (cflags & BINF_EXEC)) { + /* + * POSIX doesn't allow "exec" to operate on builtins + * or shell functions. + */ + break; } else { if (!(cflags & (BINF_BUILTIN | BINF_COMMAND)) && (hn = shfunctab->getnode(shfunctab, cmdarg))) { @@ -3123,10 +3129,14 @@ execcmd_exec(Estate state, Execcmd_params eparams, * - we have determined there are options which would * require us to use the "command" builtin); or * - we aren't using POSIX and so BINF_COMMAND indicates a zsh - * precommand modifier is being used in place of the builtin + * precommand modifier is being used in place of the + * builtin + * - we are using POSIX and this is an EXEC, so we can't + * execute a builtin or function. */ if (errflag || checked || is_builtin || - (unset(POSIXBUILTINS) && (cflags & BINF_COMMAND))) + (isset(POSIXBUILTINS) ? + (cflags & BINF_EXEC) : (cflags & BINF_COMMAND))) break; cmdarg = (char *) peekfirst(args); diff --git a/Test/E01options.ztst b/Test/E01options.ztst index dac9430..f01d835 100644 --- a/Test/E01options.ztst +++ b/Test/E01options.ztst @@ -831,6 +831,20 @@ >val2 >val2 + print "Contents of file" >cat_arg + ( + cat() { print Function with argument $1 } + print Without + (exec cat cat_arg; print Not reached) + print With + (setopt posixbuiltins; exec cat cat_arg; print Not reached) + ) +0:POSIX_BUILTINS and exec +>Without +>Function with argument cat_arg +>With +>Contents of file + # PRINTEXITVALUE only works if shell input is coming from standard input. # Goodness only knows why. $ZTST_testdir/../Src/zsh -f <<<' ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [BUG] 'exec' runs shell functions and builtins 2017-07-26 9:23 ` Peter Stephenson @ 2017-07-26 9:27 ` Peter Stephenson 0 siblings, 0 replies; 12+ messages in thread From: Peter Stephenson @ 2017-07-26 9:27 UTC (permalink / raw) To: Zsh hackers list On Wed, 26 Jul 2017 10:23:53 +0100 Peter Stephenson <p.stephenson@samsung.com> wrote: > +Furthermore, functions ahd shell builtins are not executed after ^^^ fixed already, don't write in... pws ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [BUG] 'exec' runs shell functions and builtins 2017-07-25 21:49 ` [BUG] 'exec' runs shell functions and builtins Martijn Dekker 2017-07-26 9:23 ` Peter Stephenson @ 2017-07-26 12:53 ` Peter Stephenson 2017-08-04 15:34 ` Stephane Chazelas 2017-07-26 17:46 ` Bart Schaefer 2 siblings, 1 reply; 12+ messages in thread From: Peter Stephenson @ 2017-07-26 12:53 UTC (permalink / raw) To: Zsh hackers list This clarifies the normal behaviour. diff --git a/Doc/Zsh/builtins.yo b/Doc/Zsh/builtins.yo index 333db1c..cbaf67a 100644 --- a/Doc/Zsh/builtins.yo +++ b/Doc/Zsh/builtins.yo @@ -675,6 +675,10 @@ executed. See ifzman(the section `Precommand Modifiers' in zmanref(zshmisc))\ ifnzman(noderef(Precommand Modifiers)). +If the option tt(POSIX_BUILTINS) is not set, var(command) may be +interpreted as a shell builtin command or shell function: this is run as +normal (as if tt(exec) were not present), then the shell exits. + cindex(redirection, current shell's I/O) If var(command) is omitted but any redirections are specified, then the redirections will take effect in the current shell. pws ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [BUG] 'exec' runs shell functions and builtins 2017-07-26 12:53 ` Peter Stephenson @ 2017-08-04 15:34 ` Stephane Chazelas 0 siblings, 0 replies; 12+ messages in thread From: Stephane Chazelas @ 2017-08-04 15:34 UTC (permalink / raw) To: Peter Stephenson; +Cc: Zsh hackers list 2017-07-26 13:53:38 +0100, Peter Stephenson: > This clarifies the normal behaviour. > > diff --git a/Doc/Zsh/builtins.yo b/Doc/Zsh/builtins.yo > index 333db1c..cbaf67a 100644 > --- a/Doc/Zsh/builtins.yo > +++ b/Doc/Zsh/builtins.yo > @@ -675,6 +675,10 @@ executed. > See ifzman(the section `Precommand Modifiers' in zmanref(zshmisc))\ > ifnzman(noderef(Precommand Modifiers)). > > +If the option tt(POSIX_BUILTINS) is not set, var(command) may be > +interpreted as a shell builtin command or shell function: this is run as > +normal (as if tt(exec) were not present), then the shell exits. > + [...] (you jumped the gun a bit there. The issue is only being discussed on the austin-group mailing list. The standard is currently unclear on the matter, it might very well change to allow the zsh/pdksh behaviour. Then again, though the zsh behaviour makes more sense to me, in sh emulation, what matters more is compatibility and it seems more of the widely used sh implementations got with the ksh88 way). It might be worth adding that after exec a_builtin or exec a_function the EXIT, DEBUG or ERR traps will not be executed, as the text above would suggest they may. Other traps and asynchronous job notifications are still handled though (as opposed to when one execs an external command). Not sure it's worth saying what would (not) happen in ! exec something or time exec something -- Stephane ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [BUG] 'exec' runs shell functions and builtins 2017-07-25 21:49 ` [BUG] 'exec' runs shell functions and builtins Martijn Dekker 2017-07-26 9:23 ` Peter Stephenson 2017-07-26 12:53 ` Peter Stephenson @ 2017-07-26 17:46 ` Bart Schaefer 2017-07-26 17:48 ` Bart Schaefer 2017-07-27 9:02 ` Peter Stephenson 2 siblings, 2 replies; 12+ messages in thread From: Bart Schaefer @ 2017-07-26 17:46 UTC (permalink / raw) To: Martijn Dekker; +Cc: Zsh hackers list AFAIK zsh has always had this behavior. It also allows for example "exec builtin echo" as well as "builtin exec echo" etc. What would POSIX say that exec command echo foo should do? It's relatively easy to stop "exec" from finding functions and builtin commands, but I think fairly painful to get it to ignore other "precommand modifier" tokens. Even after PWS's 41464 the above example runs /bin/echo whereas I think the POSIX expectation would be to report "command: command not found". > This behaviour also appears to be contrary to the documentation in > zshbuiltins(1) ("Replace the current shell with an external command > rather than forking"). The documentation under "precommand modifiers" (which is cross-referenced from zshbuiltins) says: The following command together with any arguments is run in place of the current process, rather than as a sub-process. The shell does not fork and is replaced. I think this is a case of the zshbuiltins doc being ancient and imprecise rather than intentionally indicative of anything, but: ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [BUG] 'exec' runs shell functions and builtins 2017-07-26 17:46 ` Bart Schaefer @ 2017-07-26 17:48 ` Bart Schaefer 2017-07-27 9:02 ` Peter Stephenson 1 sibling, 0 replies; 12+ messages in thread From: Bart Schaefer @ 2017-07-26 17:48 UTC (permalink / raw) To: Zsh hackers list On Wed, Jul 26, 2017 at 10:46 AM, Bart Schaefer <schaefer@brasslantern.com> wrote: > imprecise rather than intentionally indicative of anything, but: Deleted a partial implementation of the patch from 41464 but forgot to remove the last word from that phase. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [BUG] 'exec' runs shell functions and builtins 2017-07-26 17:46 ` Bart Schaefer 2017-07-26 17:48 ` Bart Schaefer @ 2017-07-27 9:02 ` Peter Stephenson 2017-07-28 17:58 ` Bart Schaefer 1 sibling, 1 reply; 12+ messages in thread From: Peter Stephenson @ 2017-07-27 9:02 UTC (permalink / raw) To: Bart Schaefer, Zsh hackers list On Wed, 26 Jul 2017 10:46:06 -0700 Bart Schaefer <schaefer@brasslantern.com> wrote: > AFAIK zsh has always had this behavior. It also allows for example > "exec builtin echo" as well as "builtin exec echo" etc. What would > POSIX say that > exec command echo foo > should do? It's relatively easy to stop "exec" from finding functions > and builtin commands, but I think fairly painful to get it to ignore > other "precommand modifier" tokens. Even after PWS's 41464 the above > example runs /bin/echo whereas I think the POSIX expectation would be > to report "command: command not found". This isn't what I get: % (setopt posixbuiltins; exec command echo foo) zsh: command not found: command % cat ~/bin/builtin #!/bin/sh echo External command named $0 % (setopt posixbuiltins; exec builtin echo) External command named /export/home/pws/bin/builtin There's no further processing of the arguments when we find an exec with posixbuiltins set, so the above is what I'd expect. pws diff --git a/Doc/Zsh/builtins.yo b/Doc/Zsh/builtins.yo index 99b1dd4..0e8580d 100644 --- a/Doc/Zsh/builtins.yo +++ b/Doc/Zsh/builtins.yo @@ -679,7 +679,10 @@ See ifzman(the section `Precommand Modifiers' in zmanref(zshmisc))\ ifnzman(noderef(Precommand Modifiers)). If the option tt(POSIX_BUILTINS) is set, var(command) is never -interpreted as a shell builtin command or shell function. +interpreted as a shell builtin command or shell function. This +means further precommand modifiers such as tt(builtin) and +tt(noglob) are also not interpreted within the shell. Hence +var(command) is always found by searching the command path. cindex(redirection, current shell's I/O) If var(command) is omitted but any redirections are specified, ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [BUG] 'exec' runs shell functions and builtins 2017-07-27 9:02 ` Peter Stephenson @ 2017-07-28 17:58 ` Bart Schaefer 2017-08-11 15:10 ` Kamil Dudka 0 siblings, 1 reply; 12+ messages in thread From: Bart Schaefer @ 2017-07-28 17:58 UTC (permalink / raw) To: Zsh hackers list On Thu, Jul 27, 2017 at 2:02 AM, Peter Stephenson <p.stephenson@samsung.com> wrote: > > % (setopt posixbuiltins; exec command echo foo) > zsh: command not found: command Chet Ramey emailed me: ===== "However, all of the standard utilities, including the regular built-ins in the table, but not the special built-ins described in Special Built-In Utilities, shall be implemented in a manner so that they can be accessed via the exec family of functions as defined in the System Interfaces volume of POSIX.1-2008 and can be invoked directly by those standard utilities that require it (env, find, nice, nohup, time, xargs)." `command' is explicitly listed in the table in http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap01.html#tag_17_06 ===== So "exec command ..." is actually supposed to work, as is "exec read", "exec getopts", etc. Urk. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [BUG] 'exec' runs shell functions and builtins 2017-07-28 17:58 ` Bart Schaefer @ 2017-08-11 15:10 ` Kamil Dudka 2017-08-11 15:25 ` Peter Stephenson 0 siblings, 1 reply; 12+ messages in thread From: Kamil Dudka @ 2017-08-11 15:10 UTC (permalink / raw) To: Bart Schaefer; +Cc: zsh-workers On Friday, July 28, 2017 10:58:59 Bart Schaefer wrote: > On Thu, Jul 27, 2017 at 2:02 AM, Peter Stephenson > > <p.stephenson@samsung.com> wrote: > > % (setopt posixbuiltins; exec command echo foo) > > zsh: command not found: command > > Chet Ramey emailed me: > ===== > "However, all of the standard utilities, including the regular built-ins in > the table, but not the special built-ins described in Special Built-In > Utilities, shall be implemented in a manner so that they can be accessed > via the exec family of functions as defined in the System Interfaces volume > of POSIX.1-2008 and can be invoked directly by those standard utilities > that require it (env, find, nice, nohup, time, xargs)." > > `command' is explicitly listed in the table in > > http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap01.html#tag > _17_06 ===== > > So "exec command ..." is actually supposed to work, as is "exec read", > "exec getopts", etc. Urk. Today I discovered that bash in Fedora installs /usr/bin/cd and the like as external commands to make 'exec' work on those built-ins (primarily in bash): https://bugzilla.redhat.com/820192 I am really not happy about this solution because the external commands only confuse people for no good reason (other than satisfying the POSIX requirement). It is good that zsh does not need this in its native configuration but how is it supposed to work in the POSIX emulation mode now? % (exec cd) % emulate sh % (exec cd) zsh: command not found: cd If I understand http://www.zsh.org/mla/workers/2017/msg01138.html correctly, then the above command should work? Kamil ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [BUG] 'exec' runs shell functions and builtins 2017-08-11 15:10 ` Kamil Dudka @ 2017-08-11 15:25 ` Peter Stephenson 2017-08-14 7:19 ` Kamil Dudka 0 siblings, 1 reply; 12+ messages in thread From: Peter Stephenson @ 2017-08-11 15:25 UTC (permalink / raw) To: zsh-workers On Fri, 11 Aug 2017 17:10:12 +0200 Kamil Dudka <kdudka@redhat.com> wrote: > It is good that zsh does not need this in its native configuration but > how is it supposed to work in the POSIX emulation mode now? > > % (exec cd) > % emulate sh > % (exec cd) > zsh: command not found: cd > > If I understand http://www.zsh.org/mla/workers/2017/msg01138.html correctly, > then the above command should work? I can't actually think of a useful application of the above that would make "work" meaningful. What's the "exec" supposed to be doing? Who would even try to exec a builtin, even if they discover that for some bizarre reason the standard says you can? The one possible use of "exec" in that case might be to ensure you're *not* using a builtin --- but apparently you are? What? pws ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [BUG] 'exec' runs shell functions and builtins 2017-08-11 15:25 ` Peter Stephenson @ 2017-08-14 7:19 ` Kamil Dudka 0 siblings, 0 replies; 12+ messages in thread From: Kamil Dudka @ 2017-08-14 7:19 UTC (permalink / raw) To: Peter Stephenson; +Cc: zsh-workers On Friday, August 11, 2017 16:25:01 Peter Stephenson wrote: > On Fri, 11 Aug 2017 17:10:12 +0200 > > Kamil Dudka <kdudka@redhat.com> wrote: > > It is good that zsh does not need this in its native configuration but > > how is it supposed to work in the POSIX emulation mode now? > > > > % (exec cd) > > % emulate sh > > % (exec cd) > > zsh: command not found: cd > > > > If I understand http://www.zsh.org/mla/workers/2017/msg01138.html > > correctly, then the above command should work? > > I can't actually think of a useful application of the above that would > make "work" meaningful. What's the "exec" supposed to be doing? Who > would even try to exec a builtin, even if they discover that for some > bizarre reason the standard says you can? The one possible use of > "exec" in that case might be to ensure you're *not* using a builtin --- > but apparently you are? What? > > pws Thanks for the reply! It did not make much sense to me either but I am by no means expert in this area. In fact I hoped to hear some explanation on this list because the topic had been raised here recently. Given the lack of information about the reasoning behind the strange POSIX requirement, I agree to keep the behavior of zsh as it is now. Kamil ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2017-08-14 7:19 UTC | newest] Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <CGME20170725221050epcas4p131de822f47289e279c7de12de0d6c127@epcas4p1.samsung.com> 2017-07-25 21:49 ` [BUG] 'exec' runs shell functions and builtins Martijn Dekker 2017-07-26 9:23 ` Peter Stephenson 2017-07-26 9:27 ` Peter Stephenson 2017-07-26 12:53 ` Peter Stephenson 2017-08-04 15:34 ` Stephane Chazelas 2017-07-26 17:46 ` Bart Schaefer 2017-07-26 17:48 ` Bart Schaefer 2017-07-27 9:02 ` Peter Stephenson 2017-07-28 17:58 ` Bart Schaefer 2017-08-11 15:10 ` Kamil Dudka 2017-08-11 15:25 ` Peter Stephenson 2017-08-14 7:19 ` Kamil Dudka
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/zsh/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).