* zsh behavior when fork() failed @ 2012-02-23 10:40 Dipak Gaigole 2012-02-23 16:14 ` Bart Schaefer 0 siblings, 1 reply; 10+ messages in thread From: Dipak Gaigole @ 2012-02-23 10:40 UTC (permalink / raw) To: zsh-workers Hi, Recently I have observed that the behavior of zsh differs from other shells like bash, ksh whenever fork () fails. In order to simulate and make fork () fail with EAGAIN error I have used "ulimit -u <no>" BASH: bash-2.05b$ ulimit -u 11 bash-2.05b$ echo $$ 4699 bash-2.05b$ bash bash: fork: Resource temporarily unavailable bash-2.05b$ echo $$ 4824 bash-2.05b$ date bash: fork: Resource temporarily unavailable bash-2.05b$ echo $? 1 bash-2.05b$ KSH: bash-2.05b$ ksh $ date ksh: cannot fork - try again $ echo $? 1 $ ZSH: bash-2.05b$ zsh /etc/zshrc:21: fork failed: resource temporarily unavailable [dipak@rhas30]/tmp% date zsh: fork failed: resource temporarily unavailable [dipak@rhas30]/tmp% echo $? 0 [dipak@rhas30]/tmp% As we can see that in zsh whenever fork() fails with EAGAIN, the return status is incorrect (i.e. $? is 0) and this causes further failures if this happens in a big script where script's progress depends on execution of previous command i.e $? I have tried this with zsh version 4.0.7, 4.3.4 as well as 4.3.15 (latest as of now I suppose) and found the same behavior. So this looks like a bug to me. Please correct me if I am wrong. Thanks, Dipak ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: zsh behavior when fork() failed 2012-02-23 10:40 zsh behavior when fork() failed Dipak Gaigole @ 2012-02-23 16:14 ` Bart Schaefer 2012-02-24 11:08 ` Dipak Gaigole 0 siblings, 1 reply; 10+ messages in thread From: Bart Schaefer @ 2012-02-23 16:14 UTC (permalink / raw) To: Dipak Gaigole, zsh-workers On Feb 23, 4:10pm, Dipak Gaigole wrote: } } [dipak@rhas30]/tmp% date } zsh: fork failed: resource temporarily unavailable } [dipak@rhas30]/tmp% echo $? } 0 There are several places where zsh forks without obviously propagating the error state, but I only found one case where $? was zero after the command. If the command is part of a pipeline, or is in $(...) or any process substitution, then $? is properly set to 1. However, there are other differences from bash. For example, in zsh, the "not" operator inverts the fork-failed error state: torch% date | read -E zsh: fork failed: resource temporarily unavailable torch% print $? 1 torch% ! date | read -E zsh: fork failed: resource temporarily unavailable torch% print $? 0 In bash, the result is 1 in both cases. The appended patch seems to fix both situations, but in some of the instances of fork failure I haven't traced through how lastval and errflag both get set (except it doesn't happen at the point of the fork) so it is possible that some form of command is still wrong. } As we can see that in zsh whenever fork() fails with EAGAIN, the } return status is incorrect (i.e. $? is 0) and this causes further } failures if this happens in a big script where script's progress } depends on execution of previous command i.e $? Can you provide an example script? A simple one that I put together bails out on fork failure even though $? is zero. Perhaps you mean a case where script A calls script B, then script B fails but script A proceeds because the error wasn't propagated? Index: Src/exec.c =================================================================== --- Src/exec.c 20 Dec 2011 17:13:38 -0000 1.43 +++ Src/exec.c 23 Feb 2012 16:07:50 -0000 @@ -1617,9 +1617,8 @@ (list_pipe || (pline_level && !(jn->stat & STAT_SUBJOB))))) deletejob(jn, 0); thisjob = pj; - } - if (slflags & WC_SUBLIST_NOT) + if ((slflags & WC_SUBLIST_NOT) && !errflag) lastval = !lastval; } if (!pline_level) @@ -2810,6 +2820,7 @@ close(synch[1]); if (oautocont >= 0) opts[AUTOCONTINUE] = oautocont; + lastval = errflag = 1; return; } if (pid) { ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: zsh behavior when fork() failed 2012-02-23 16:14 ` Bart Schaefer @ 2012-02-24 11:08 ` Dipak Gaigole 2012-02-24 18:05 ` Bart Schaefer 0 siblings, 1 reply; 10+ messages in thread From: Dipak Gaigole @ 2012-02-24 11:08 UTC (permalink / raw) To: Bart Schaefer; +Cc: zsh-workers On Thu, Feb 23, 2012 at 9:44 PM, Bart Schaefer <schaefer@brasslantern.com> wrote: > > Perhaps you mean a case where script A calls script B, then script B > fails but script A proceeds because the error wasn't propagated? Yes, you are right. This is what I mean to say. > > Index: Src/exec.c > =================================================================== > --- Src/exec.c 20 Dec 2011 17:13:38 -0000 1.43 > +++ Src/exec.c 23 Feb 2012 16:07:50 -0000 > @@ -1617,9 +1617,8 @@ > (list_pipe || (pline_level && !(jn->stat & STAT_SUBJOB))))) > deletejob(jn, 0); > thisjob = pj; > - > } > - if (slflags & WC_SUBLIST_NOT) > + if ((slflags & WC_SUBLIST_NOT) && !errflag) > lastval = !lastval; > } > if (!pline_level) > @@ -2810,6 +2820,7 @@ > close(synch[1]); > if (oautocont >= 0) > opts[AUTOCONTINUE] = oautocont; > + lastval = errflag = 1; > return; > } > if (pid) { > I have applied this patch and the newly built zsh returns proper $? (i.e. 1) whenever fork fails. So this has fixed the behavior of error propagation, but the script behavior still looks different. Here is a simple script example: bash-2.05b$ cat /tmp/test.sh #!/bin/sh x="My default value" x=`date` echo $? echo "Current Date is:" "$x" date echo $? bash-2.05b$ BASH: bash-2.05b$ bash -x /tmp/test.sh + x=My default value /tmp/test.sh: fork: Resource temporarily unavailable bash-2.05b$ echo $? 128 bash-2.05b$ KSH: bash-2.05b$ ksh -x /tmp/test.sh + x=My default value /tmp/test.sh[3]: cannot fork - try again bash-2.05b$ echo $? 1 bash-2.05b$ ZSH: (Applied above code patch) bash-2.05b$ zsh -x /tmp/test.sh +/tmp/test.sh:2> x='My default value' +/tmp/test.sh:3> x=/tmp/test.sh:3: fork failed: resource temporarily unavailable +/tmp/test.sh:4> echo 1 1 +/tmp/test.sh:5> echo 'Current Date is:' 'My default value' Current Date is: My default value /tmp/test.sh:7: fork failed: resource temporarily unavailable +/tmp/test.sh:8> echo 1 1 bash-2.05b$ echo $? 0 bash-2.05b$ As we can see that zsh continues even if it knows that it has failed in fork and finally the script return status is 0. Also checking for $? after each command is not feasible. So doesn't this zsh behavior looks misleading? Thanks, Dipak ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: zsh behavior when fork() failed 2012-02-24 11:08 ` Dipak Gaigole @ 2012-02-24 18:05 ` Bart Schaefer 2012-02-25 16:33 ` Dipak Gaigole 0 siblings, 1 reply; 10+ messages in thread From: Bart Schaefer @ 2012-02-24 18:05 UTC (permalink / raw) To: Dipak Gaigole; +Cc: zsh-workers On Feb 24, 4:38pm, Dipak Gaigole wrote: } } As we can see that zsh continues even if it knows that it has failed } in fork and finally the script return status is 0. The script return status is zero because the last command it executed was successful. That has nothing to do with the rest of the question. } Also checking for $? after each command is not feasible. So doesn't } this zsh behavior looks misleading? Depends on whether fork failure is supposed to be fatal to the shell. I looked at: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html Section 2.8.1 lists command execution failures where a non-interactive shell "shall exit", but fork failure is not among them. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: zsh behavior when fork() failed 2012-02-24 18:05 ` Bart Schaefer @ 2012-02-25 16:33 ` Dipak Gaigole 2012-02-26 19:52 ` Bart Schaefer 0 siblings, 1 reply; 10+ messages in thread From: Dipak Gaigole @ 2012-02-25 16:33 UTC (permalink / raw) To: Bart Schaefer; +Cc: zsh-workers On Fri, Feb 24, 2012 at 11:35 PM, Bart Schaefer <schaefer@brasslantern.com> wrote: > On Feb 24, 4:38pm, Dipak Gaigole wrote: > } > } As we can see that zsh continues even if it knows that it has failed > } in fork and finally the script return status is 0. > > The script return status is zero because the last command it executed > was successful. That has nothing to do with the rest of the question. > Yes, I understand. > } Also checking for $? after each command is not feasible. So doesn't > } this zsh behavior looks misleading? > > Depends on whether fork failure is supposed to be fatal to the shell. > I looked at: > http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html > > Section 2.8.1 lists command execution failures where a non-interactive > shell "shall exit", but fork failure is not among them. Got it. But I don't want to add a check of $? after each command in the scripts specially in cases where scripts are 1000+ lines. Is there any zsh option to make sure script exit when fork fails? Else can I patch the zsh code to make script exit on fork failure? Thanks, Dipak ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: zsh behavior when fork() failed 2012-02-25 16:33 ` Dipak Gaigole @ 2012-02-26 19:52 ` Bart Schaefer 2012-02-26 21:57 ` Bart Schaefer 2012-02-29 12:57 ` Dipak Gaigole 0 siblings, 2 replies; 10+ messages in thread From: Bart Schaefer @ 2012-02-26 19:52 UTC (permalink / raw) To: Dipak Gaigole, zsh-workers On Feb 25, 10:03pm, Dipak Gaigole wrote: } Subject: Re: zsh behavior when fork() failed } } > Section 2.8.1 lists command execution failures where a non-interactive } > shell "shall exit", but fork failure is not among them. } } Got it. That doesn't mean I didn't miss something. (I was kind of in a hurry on Friday ...) This also raises the question of whether a non-interactive shell should exit on a botched "exec" (see zsh-workers/30111 and subsequent thread, and comment on line 3364 excerpted below). } But I don't want to add a check of $? after each command in } the scripts specially in cases where scripts are 1000+ lines. Would "set -e" (setopt err_exit) do what you need? } Is there any zsh option to make sure script exit when fork fails? Else } can I patch the zsh code to make script exit on fork failure? It's entirely possible that "return;" on lines 2818 and 2824 in exec.c: 2814 if (pipe(synch) < 0) { 2815 zerr("pipe failed: %e", errno); 2816 if (oautocont >= 0) 2817 opts[AUTOCONTINUE] = oautocont; 2818 return; 2819 } else if ((pid = zfork(&bgtime)) == -1) { 2820 close(synch[0]); 2821 close(synch[1]); 2822 if (oautocont >= 0) 2823 opts[AUTOCONTINUE] = oautocont; 2824 return; 2825 } should instead behave the way POSIXBUILTINS does: 3362 /* 3363 * For POSIX-compatible behaviour with special 3364 * builtins (including exec which we don't usually 3365 * classify as a builtin) we treat all errors as fatal. 3366 * The "command" builtin is not special so resets this behaviour. 3367 */ 3368 if (redir_err || errflag) { 3369 if (!isset(INTERACTIVE)) { 3370 if (forked) 3371 _exit(1); 3372 else 3373 exit(1); 3374 } 3375 errflag = 1; 3376 } There are a couple of other cases (e.g., around line 1680) where's also possible the error should be considered unrecoverable. That would make a full patch look something like the following (which still exits with 1, rather than with 128 like bash, but does exit). The oautocont stuff goes away because it's handled below the "fatal" label (outside the visible diff context). The second hunk still might not cause a proper exit, it's in execpline2() from which it's less clear to me how to reach the "fatal" condition but it's pretty obvious that it shouldn't just be falling through in those error states. Index: Src/exec.c --- ../zsh-forge/current/Src/exec.c 2012-02-12 13:31:49.000000000 -0800 +++ Src/exec.c 2012-02-26 11:47:48.000000000 -0800 @@ -1617,9 +1617,8 @@ (list_pipe || (pline_level && !(jn->stat & STAT_SUBJOB))))) deletejob(jn, 0); thisjob = pj; - } - if (slflags & WC_SUBLIST_NOT) + if ((slflags & WC_SUBLIST_NOT) && !errflag) lastval = !lastval; } if (!pline_level) @@ -1679,9 +1678,13 @@ if (pipe(synch) < 0) { zerr("pipe failed: %e", errno); + lastval = errflag = 1; + return; } else if ((pid = zfork(&bgtime)) == -1) { close(synch[0]); close(synch[1]); + lastval = errflag = 1; + return; } else if (pid) { char dummy, *text; @@ -2490,7 +2493,7 @@ if (!firstnode(args)) { zerr("exec requires a command to execute"); errflag = lastval = 1; - return; + goto fatal; } uremnode(args, firstnode(args)); if (!strcmp(next, "--")) @@ -2507,12 +2510,12 @@ if (!firstnode(args)) { zerr("exec requires a command to execute"); errflag = lastval = 1; - return; + goto fatal; } if (!nextnode(firstnode(args))) { zerr("exec flag -a requires a parameter"); errflag = lastval = 1; - return; + goto fatal; } exec_argv0 = (char *) getdata(nextnode(firstnode(args))); @@ -2813,15 +2816,12 @@ if (pipe(synch) < 0) { zerr("pipe failed: %e", errno); - if (oautocont >= 0) - opts[AUTOCONTINUE] = oautocont; - return; + goto fatal; } else if ((pid = zfork(&bgtime)) == -1) { close(synch[0]); close(synch[1]); - if (oautocont >= 0) - opts[AUTOCONTINUE] = oautocont; - return; + lastval = errflag = 1; + goto fatal; } if (pid) { @@ -3365,6 +3365,7 @@ * classify as a builtin) we treat all errors as fatal. * The "command" builtin is not special so resets this behaviour. */ + fatal: if (redir_err || errflag) { if (!isset(INTERACTIVE)) { if (forked) ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: zsh behavior when fork() failed 2012-02-26 19:52 ` Bart Schaefer @ 2012-02-26 21:57 ` Bart Schaefer 2012-02-29 12:57 ` Dipak Gaigole 1 sibling, 0 replies; 10+ messages in thread From: Bart Schaefer @ 2012-02-26 21:57 UTC (permalink / raw) To: zsh-workers On Feb 26, 11:52am, Bart Schaefer wrote: } } This also raises the question of whether a non-interactive shell should } exit on a botched "exec" (see zsh-workers/30111 and subsequent thread, } and comment on line 3364 excerpted below). Interestingly, bash does not exit on an error in exec's argument list. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: zsh behavior when fork() failed 2012-02-26 19:52 ` Bart Schaefer 2012-02-26 21:57 ` Bart Schaefer @ 2012-02-29 12:57 ` Dipak Gaigole 2012-02-29 19:06 ` Bart Schaefer 1 sibling, 1 reply; 10+ messages in thread From: Dipak Gaigole @ 2012-02-29 12:57 UTC (permalink / raw) To: Bart Schaefer; +Cc: zsh-workers > There are a couple of other cases (e.g., around line 1680) where's also > possible the error should be considered unrecoverable. > > That would make a full patch look something like the following (which > still exits with 1, rather than with 128 like bash, but does exit). > The oautocont stuff goes away because it's handled below the "fatal" > label (outside the visible diff context). The second hunk still might > not cause a proper exit, it's in execpline2() from which it's less clear > to me how to reach the "fatal" condition but it's pretty obvious that > it shouldn't just be falling through in those error states. > > > Index: Src/exec.c > --- ../zsh-forge/current/Src/exec.c 2012-02-12 13:31:49.000000000 -0800 > +++ Src/exec.c 2012-02-26 11:47:48.000000000 -0800 > @@ -1617,9 +1617,8 @@ > (list_pipe || (pline_level && !(jn->stat & STAT_SUBJOB))))) > deletejob(jn, 0); > thisjob = pj; > - > } > - if (slflags & WC_SUBLIST_NOT) > + if ((slflags & WC_SUBLIST_NOT) && !errflag) > lastval = !lastval; > } > if (!pline_level) > @@ -1679,9 +1678,13 @@ > > if (pipe(synch) < 0) { > zerr("pipe failed: %e", errno); > + lastval = errflag = 1; > + return; > } else if ((pid = zfork(&bgtime)) == -1) { > close(synch[0]); > close(synch[1]); > + lastval = errflag = 1; > + return; > } else if (pid) { > char dummy, *text; > > @@ -2490,7 +2493,7 @@ > if (!firstnode(args)) { > zerr("exec requires a command to execute"); > errflag = lastval = 1; > - return; > + goto fatal; > } > uremnode(args, firstnode(args)); > if (!strcmp(next, "--")) > @@ -2507,12 +2510,12 @@ > if (!firstnode(args)) { > zerr("exec requires a command to execute"); > errflag = lastval = 1; > - return; > + goto fatal; > } > if (!nextnode(firstnode(args))) { > zerr("exec flag -a requires a parameter"); > errflag = lastval = 1; > - return; > + goto fatal; > } > exec_argv0 = (char *) > getdata(nextnode(firstnode(args))); > @@ -2813,15 +2816,12 @@ > > if (pipe(synch) < 0) { > zerr("pipe failed: %e", errno); > - if (oautocont >= 0) > - opts[AUTOCONTINUE] = oautocont; > - return; > + goto fatal; > } else if ((pid = zfork(&bgtime)) == -1) { > close(synch[0]); > close(synch[1]); > - if (oautocont >= 0) > - opts[AUTOCONTINUE] = oautocont; > - return; > + lastval = errflag = 1; > + goto fatal; > } > if (pid) { > > @@ -3365,6 +3365,7 @@ > * classify as a builtin) we treat all errors as fatal. > * The "command" builtin is not special so resets this behaviour. > */ > + fatal: > if (redir_err || errflag) { > if (!isset(INTERACTIVE)) { > if (forked) I have applied this patch but unfortunately the script still continues even after the fork failures while executing commands from the script. bash-2.05b$ cat /tmp/test1.sh #!/bin/sh x="My default value" x=`date` echo $? echo "Current Date is:" "$x" date echo $? bash-2.05b$ zsh /tmp/test1.sh /tmp/test1.sh:3: fork failed: resource temporarily unavailable 1 Current Date is: My default value /tmp/test1.sh:6: fork failed: resource temporarily unavailable 1 bash-2.05b$ Can you please check this on your end? Thanks, Dipak ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: zsh behavior when fork() failed 2012-02-29 12:57 ` Dipak Gaigole @ 2012-02-29 19:06 ` Bart Schaefer 2012-03-06 9:01 ` Dipak Gaigole 0 siblings, 1 reply; 10+ messages in thread From: Bart Schaefer @ 2012-02-29 19:06 UTC (permalink / raw) To: Dipak Gaigole; +Cc: zsh-workers On Feb 29, 6:27pm, Dipak Gaigole wrote: } } I have applied this patch but unfortunately the script still continues } even after the fork failures while executing commands from the script. } } bash-2.05b$ cat /tmp/test1.sh } #!/bin/sh } x="My default value" } x=`date` } echo $? } echo "Current Date is:" "$x" } date } echo $? } bash-2.05b$ zsh /tmp/test1.sh } /tmp/test1.sh:3: fork failed: resource temporarily unavailable } 1 } Current Date is: My default value } /tmp/test1.sh:6: fork failed: resource temporarily unavailable } 1 } bash-2.05b$ } } Can you please check this on your end? I get (schaefer[N] is my top-level prompt): schaefer[691] Src/zsh -f /tmp/test1.sh /tmp/test1.sh:3: fork failed: resource temporarily unavailable 1 Current Date is: My default value /tmp/test1.sh:6: fork failed: resource temporarily unavailable schaefer[692] echo $? 1 There's no exit after `date` because that was in a subshell, but I do see the exit after the command in the parent shell. Are you sure a path-searched "zsh" is finding the right binary? ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: zsh behavior when fork() failed 2012-02-29 19:06 ` Bart Schaefer @ 2012-03-06 9:01 ` Dipak Gaigole 0 siblings, 0 replies; 10+ messages in thread From: Dipak Gaigole @ 2012-03-06 9:01 UTC (permalink / raw) To: Bart Schaefer; +Cc: zsh-workers On Thu, Mar 1, 2012 at 12:36 AM, Bart Schaefer <schaefer@brasslantern.com> wrote: > On Feb 29, 6:27pm, Dipak Gaigole wrote: > } > } I have applied this patch but unfortunately the script still continues > } even after the fork failures while executing commands from the script. > } > } bash-2.05b$ cat /tmp/test1.sh > } #!/bin/sh > } x="My default value" > } x=`date` > } echo $? > } echo "Current Date is:" "$x" > } date > } echo $? > } bash-2.05b$ zsh /tmp/test1.sh > } /tmp/test1.sh:3: fork failed: resource temporarily unavailable > } 1 > } Current Date is: My default value > } /tmp/test1.sh:6: fork failed: resource temporarily unavailable > } 1 > } bash-2.05b$ > } > } Can you please check this on your end? > > I get (schaefer[N] is my top-level prompt): > > schaefer[691] Src/zsh -f /tmp/test1.sh > /tmp/test1.sh:3: fork failed: resource temporarily unavailable > 1 > Current Date is: My default value > /tmp/test1.sh:6: fork failed: resource temporarily unavailable > schaefer[692] echo $? > 1 > > There's no exit after `date` because that was in a subshell, but I do > see the exit after the command in the parent shell. > > Are you sure a path-searched "zsh" is finding the right binary? Sorry my bad :( I had missed one of the fatal condition. Thanks for your help Bart !!! -Dipak ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2012-03-06 9:01 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2012-02-23 10:40 zsh behavior when fork() failed Dipak Gaigole 2012-02-23 16:14 ` Bart Schaefer 2012-02-24 11:08 ` Dipak Gaigole 2012-02-24 18:05 ` Bart Schaefer 2012-02-25 16:33 ` Dipak Gaigole 2012-02-26 19:52 ` Bart Schaefer 2012-02-26 21:57 ` Bart Schaefer 2012-02-29 12:57 ` Dipak Gaigole 2012-02-29 19:06 ` Bart Schaefer 2012-03-06 9:01 ` Dipak Gaigole
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/zsh/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).