zsh-workers
 help / color / mirror / code / Atom feed
* Inconsistent behavior of ERR_EXIT with conditionals
@ 2022-11-04 16:37 Philippe Altherr
  2022-11-06 20:45 ` Bart Schaefer
  0 siblings, 1 reply; 21+ messages in thread
From: Philippe Altherr @ 2022-11-04 16:37 UTC (permalink / raw)
  To: zsh-workers


[-- Attachment #1.1: Type: text/plain, Size: 1854 bytes --]

In the following code, the only difference between fun1() and fun2() is
that in the former the expression "false && true" is within an extra set of
braces but that's enough to make them behave differently. Calling the
fromer doesn't trigger an exit while calling the latter does.

#!/bin/zsh -e
>
> function fun1() {
>     { false && true }
> }
>
> function fun2() {
>     false && true
> }
>
> echo aaa: $?
> false && true
> echo bbb: $?
> fun1
> echo ccc: $?
> fun2
> echo ddd: $?
>

The output of the script (with Zsh 5.8 on Linux and Zsh 5.8.1 on macOS) is
the following:

aaa: 0
> bbb: 1
> ccc: 1
>

The documentation of ERR_EXIT mentions the special case of conditional
expressions. I understand that the command "false" by itself in the
expression "false && true" shouldn't trigger an exit since it appears in a
position where a condition is expected. However, since the expression
"false && true" as a whole evaluates to a non-zero status and doesn't
appear in a position where a condition is expected, I would assume that it
should trigger an exit. Thus, in my opinion the script should have the
following output:

aaa: 0
>

That's obviously not the case. Apparanty Zsh simply ignores non-zero
statuses of conditionals if they weren't generated by the last command.
However that doesn't explain why the call to fun1 doesn't trigger an exit.
Like the call to fun2, it returns with a non-zero status but for some
reason only the latter triggers an exit.

Note that the following functions behave like fun1 and don't trigger an
exit when they are called.

function fun3() {
>     if true; then
>         false && true
>     fi
> }
>
> function fun4() {
>     case foo in
>         * ) false && true;;
>     esac
> }
>

And the following function behaves like fun2 and triggers an exit.


>  function fun5() {

    ( false && true )
> }
>

Philippe

[-- Attachment #1.2: Type: text/html, Size: 3137 bytes --]

[-- Attachment #2: bug.zsh --]
[-- Type: application/octet-stream, Size: 173 bytes --]

#!/bin/zsh -e

function fun1() {
    { false && true }
}

function fun2() {
    false && true
}

echo aaa: $?
false && true
echo bbb: $?
fun1
echo ccc: $?
fun2
echo ddd: $?

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Inconsistent behavior of ERR_EXIT with conditionals
  2022-11-04 16:37 Inconsistent behavior of ERR_EXIT with conditionals Philippe Altherr
@ 2022-11-06 20:45 ` Bart Schaefer
  2022-11-07  3:50   ` Bart Schaefer
  0 siblings, 1 reply; 21+ messages in thread
From: Bart Schaefer @ 2022-11-06 20:45 UTC (permalink / raw)
  To: Philippe Altherr; +Cc: zsh-workers

On Fri, Nov 4, 2022 at 9:39 AM Philippe Altherr
<philippe.altherr@gmail.com> wrote:
>
>> function fun1() {
>>     { false && true }
>> }
>
> [...] since the expression "false && true" as a whole evaluates to a non-zero status and doesn't appear in a position where a condition is expected, I would assume that it should trigger an exit. [...] Apparanty Zsh simply ignores non-zero statuses of conditionals if they weren't generated by the last command.

In fact, comment in exec.c says:
     * ERREXIT only forces the shell to exit if the last command in a &&
     * or || fails.  This is the case even if an earlier command is a
     * shell function or other current shell structure, so we have to set
     * noerrexit here if the sublist is not of type END.

But execcursh() unconditionally sets this_noerrexit = 1.

Does this act better?  (Hopefully I don't need an attachment for this
one-liner):

diff --git a/Src/exec.c b/Src/exec.c
index c8bcf4ee5..d11f79d90 100644
--- a/Src/exec.c
+++ b/Src/exec.c
@@ -451,7 +451,7 @@ execcursh(Estate state, int do_exec)
     cmdpop();

     state->pc = end;
-    this_noerrexit = 1;
+    this_noerrexit = (WC_SUBLIST_TYPE(*end) != WC_SUBLIST_END);

     return lastval;
 }

All extant tests still pass.  Does not affect how fun3 and fun4
behave, so I suspect this fix may be needed elsewhere ... there are a
bunch of similar cases for multi-line shell constructs in Src/loop.c


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Inconsistent behavior of ERR_EXIT with conditionals
  2022-11-06 20:45 ` Bart Schaefer
@ 2022-11-07  3:50   ` Bart Schaefer
  2022-11-07  5:35     ` [PATCH] " Bart Schaefer
  2022-11-08  4:58     ` Philippe Altherr
  0 siblings, 2 replies; 21+ messages in thread
From: Bart Schaefer @ 2022-11-07  3:50 UTC (permalink / raw)
  To: Philippe Altherr; +Cc: zsh-workers

On Sun, Nov 6, 2022 at 12:45 PM Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> [...] I suspect this fix may be needed elsewhere ... there are a
> bunch of similar cases for multi-line shell constructs in Src/loop.c

Those are:
  for
  select
  while
  repeat
  if
  case

Does anyone disagree that Philippe's examples demonstrate that "if"
and "case" should get this treatment?

More questionable are the looping constructs.  I can't come up with a
way to have the loop end in an error state without the whole shell
ERREXITing before reaching the end of the loop body.  The value of
this_noerrexit matters at all, as far as I can tell, only to suppress
the false-condition that ends a "while" loop, and is meaningless in
for/select/repeat?


^ permalink raw reply	[flat|nested] 21+ messages in thread

* [PATCH] Re: Inconsistent behavior of ERR_EXIT with conditionals
  2022-11-07  3:50   ` Bart Schaefer
@ 2022-11-07  5:35     ` Bart Schaefer
  2022-11-07  9:44       ` Peter Stephenson
  2022-11-08  4:58     ` Philippe Altherr
  1 sibling, 1 reply; 21+ messages in thread
From: Bart Schaefer @ 2022-11-07  5:35 UTC (permalink / raw)
  To: Zsh hackers list

[-- Attachment #1: Type: text/plain, Size: 1075 bytes --]

On Sun, Nov 6, 2022 at 7:50 PM Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> More questionable are the looping constructs.  I can't come up with a
> way to have the loop end in an error state without the whole shell
> ERREXITing before reaching the end of the loop body.

Found the way (and it should have been obvious):

repeat 1; do false && true; done

The statement `false && true` doesn't itself cause an exit, but does
become $? of the loop.

Now the weird bit is that the attached patch DOES NOT cause a slew of
test failures in C03traps.  E.g. if I run this test standalone:

Src/zsh -f =(<<<"(setopt err_exit
  if true; then
    false
  fi
  print OK
  )")

I get no output and $? = 1.  The exact same code in C03traps.ztst
prints OK, which it should not.  The only difference I can see is that
ZTST_execchunk fiddles with localloops, but I've tried doing that too.

There is the question of why ignoring a false status at the end of a
complex command has so far been considered correct for ERR_EXIT,
according to C03.  This is a disagreement with e.g. bash.

[-- Attachment #2: complexerrexit.txt --]
[-- Type: text/plain, Size: 1839 bytes --]

diff --git a/Src/exec.c b/Src/exec.c
index c8bcf4ee5..2422dae91 100644
--- a/Src/exec.c
+++ b/Src/exec.c
@@ -451,7 +451,7 @@ execcursh(Estate state, int do_exec)
     cmdpop();
 
     state->pc = end;
-    this_noerrexit = 1;
+    this_noerrexit = (WC_SUBLIST_TYPE(*end) != WC_SUBLIST_END);
 
     return lastval;
 }
diff --git a/Src/loop.c b/Src/loop.c
index db5b3e097..be5261369 100644
--- a/Src/loop.c
+++ b/Src/loop.c
@@ -208,7 +208,7 @@ execfor(Estate state, int do_exec)
     loops--;
     simple_pline = old_simple_pline;
     state->pc = end;
-    this_noerrexit = 1;
+    this_noerrexit = (WC_SUBLIST_TYPE(*end) != WC_SUBLIST_END);
     return lastval;
 }
 
@@ -336,7 +336,7 @@ execselect(Estate state, UNUSED(int do_exec))
     loops--;
     simple_pline = old_simple_pline;
     state->pc = end;
-    this_noerrexit = 1;
+    this_noerrexit = (WC_SUBLIST_TYPE(*end) != WC_SUBLIST_END);
     return lastval;
 }
 
@@ -478,7 +478,7 @@ execwhile(Estate state, UNUSED(int do_exec))
     popheap();
     loops--;
     state->pc = end;
-    this_noerrexit = 1;
+    this_noerrexit = (WC_SUBLIST_TYPE(*end) != WC_SUBLIST_END);
     return lastval;
 }
 
@@ -532,7 +532,7 @@ execrepeat(Estate state, UNUSED(int do_exec))
     loops--;
     simple_pline = old_simple_pline;
     state->pc = end;
-    this_noerrexit = 1;
+    this_noerrexit = (WC_SUBLIST_TYPE(*end) != WC_SUBLIST_END);
     return lastval;
 }
 
@@ -587,7 +587,7 @@ execif(Estate state, int do_exec)
 	    lastval = 0;
     }
     state->pc = end;
-    this_noerrexit = 1;
+    this_noerrexit = (WC_SUBLIST_TYPE(*end) != WC_SUBLIST_END);
 
     return lastval;
 }
@@ -701,7 +701,7 @@ execcase(Estate state, int do_exec)
 
     if (!anypatok)
 	lastval = 0;
-    this_noerrexit = 1;
+    this_noerrexit = (WC_SUBLIST_TYPE(*end) != WC_SUBLIST_END);
 
     return lastval;
 }

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [PATCH] Re: Inconsistent behavior of ERR_EXIT with conditionals
  2022-11-07  5:35     ` [PATCH] " Bart Schaefer
@ 2022-11-07  9:44       ` Peter Stephenson
  2022-11-08  1:20         ` Bart Schaefer
  0 siblings, 1 reply; 21+ messages in thread
From: Peter Stephenson @ 2022-11-07  9:44 UTC (permalink / raw)
  To: Zsh hackers list

On 07/11/2022 05:35 Bart Schaefer <schaefer@brasslantern.com> wrote:
> There is the question of why ignoring a false status at the end of a
> complex command has so far been considered correct for ERR_EXIT,
> according to C03.  This is a disagreement with e.g. bash.

I can only guess it's gone under the radar before.  I don't think
the change is actually controversial.

Thinking back, the main focus on this stuff has been to make sure
it *doesn't* trigger when it's not supposed to, so that may be
why it was missed (probably by me).

pws


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [PATCH] Re: Inconsistent behavior of ERR_EXIT with conditionals
  2022-11-07  9:44       ` Peter Stephenson
@ 2022-11-08  1:20         ` Bart Schaefer
  0 siblings, 0 replies; 21+ messages in thread
From: Bart Schaefer @ 2022-11-08  1:20 UTC (permalink / raw)
  To: Peter Stephenson; +Cc: Zsh hackers list

On Mon, Nov 7, 2022 at 1:45 AM Peter Stephenson
<p.w.stephenson@ntlworld.com> wrote:
>
> On 07/11/2022 05:35 Bart Schaefer <schaefer@brasslantern.com> wrote:
> > There is the question of why ignoring a false status at the end of a
> > complex command has so far been considered correct for ERR_EXIT,
> > according to C03.  This is a disagreement with e.g. bash.
>
> I can only guess it's gone under the radar before.  I don't think
> the change is actually controversial.

Then we need to figure out why the test harness C03 does the wrong
thing, because it implies there's some situation not covered by the
patch.  It's hard to debug it when I can't find a reproducer outside
the test harness.


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Inconsistent behavior of ERR_EXIT with conditionals
  2022-11-07  3:50   ` Bart Schaefer
  2022-11-07  5:35     ` [PATCH] " Bart Schaefer
@ 2022-11-08  4:58     ` Philippe Altherr
  2022-11-08  5:36       ` Bart Schaefer
  1 sibling, 1 reply; 21+ messages in thread
From: Philippe Altherr @ 2022-11-08  4:58 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: zsh-workers

[-- Attachment #1: Type: text/plain, Size: 2862 bytes --]

Hi Bart,

Thanks for your replies. Great to see that this may get fixed. I feared
that it could be a case where it's too risky to change the existing
behavior, even if it's incorrect. However, before fixing anything, I think
that we should get a clear understanding of what is the expected behavior
of ERR_EXIT. Indeed, there are more cases where I think that Zsh fails to
do the right thing.

My understanding of ERR_EXIT is that it should trigger an exit of the shell
immediately after any command returns with a non-zero exit status, with the
exception of commands whose exit status stands for a condition (for example
in the condition of an if statement or on the left of an || operator). In
other words, when ERR_EXIT is enabled, any command that unexpectedly fails
should trigger an immediate exit of the shell.

Below is a short script that exhibits most of the cases that I know where
Zsh fails to behave as described above. The function EXIT is used
everywhere where I expect Zsh to trigger an exit (sometimes from a
sub-shell). In other words, none of the EXIT function calls should be
executed. Unfortunately, all the calls with an argument "Failure-?" are
executed.

#!/bin/zsh -e
>
> function EXIT() { echo $1 1>&2 }
>
> : $(false; EXIT Success-A); EXIT Failure-a
> local v=$(false; EXIT Success-B); EXIT Failure-b
> if false; EXIT Failure-c; true; then true; fi
> { false; EXIT Failure-d; true } && true
> false && true; EXIT Failure-e
> ! true; EXIT Failure-f
>

The output is the following:

Failure-a
> Failure-b
> Failure-c
> Failure-d
> Failure-e
> Failure-f
>

Regarding your proposed fix, I'll have to take a closer look. A few days
ago, I looked at exec.c
<https://sourceforge.net/p/zsh/code/ci/master/tree/Src/exec.c>, noticed
the noerrexit variables, and figured that was probably the key to some of
the bugs but I would need much more time to understand what exactly is
going on. I'll at least try to build Zsh on my machine so that I can try
your fix.

Philippe


On Mon, Nov 7, 2022 at 4:50 AM Bart Schaefer <schaefer@brasslantern.com>
wrote:

> On Sun, Nov 6, 2022 at 12:45 PM Bart Schaefer <schaefer@brasslantern.com>
> wrote:
> >
> > [...] I suspect this fix may be needed elsewhere ... there are a
> > bunch of similar cases for multi-line shell constructs in Src/loop.c
>
> Those are:
>   for
>   select
>   while
>   repeat
>   if
>   case
>
> Does anyone disagree that Philippe's examples demonstrate that "if"
> and "case" should get this treatment?
>
> More questionable are the looping constructs.  I can't come up with a
> way to have the loop end in an error state without the whole shell
> ERREXITing before reaching the end of the loop body.  The value of
> this_noerrexit matters at all, as far as I can tell, only to suppress
> the false-condition that ends a "while" loop, and is meaningless in
> for/select/repeat?
>

[-- Attachment #2: Type: text/html, Size: 3912 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Inconsistent behavior of ERR_EXIT with conditionals
  2022-11-08  4:58     ` Philippe Altherr
@ 2022-11-08  5:36       ` Bart Schaefer
  2022-11-08  8:04         ` Lawrence Velázquez
  0 siblings, 1 reply; 21+ messages in thread
From: Bart Schaefer @ 2022-11-08  5:36 UTC (permalink / raw)
  To: Philippe Altherr; +Cc: zsh-workers

On Mon, Nov 7, 2022 at 8:58 PM Philippe Altherr
<philippe.altherr@gmail.com> wrote:
>
> My understanding of ERR_EXIT is that it should trigger an exit of the shell immediately after any command returns with a non-zero exit status, with the exception of commands whose exit status stands for a condition (for example in the condition of an if statement or on the left of an || operator). In other words, when ERR_EXIT is enabled, any command that unexpectedly fails should trigger an immediate exit of the shell.

The key word is "unexpectedly".

The output of bash 5.0 for your script (with minor syntax differences tweaked):

Success-A
Failure-a
Success-B
Failure-b
Failure-c
Failure-d
Failure-e
Failure-f

So you can see that zsh agrees with bash on your "Failure" cases.  The
output of dash (as /bin/sh on ubuntu) is the same as zsh with the
exception of Failure-b, which can't be tested because there is no
equivalent of "declare" (or "local").  Bash disagrees with dash and
zsh on your "Success" cases, at least at that version.

The assumption is that anything between the keywords "if" and "then"
(or between "while" and do" by the way) is a compound expression whose
final result needs to be reached.  The entire expression on either
side of an && (or || ) is treated the same way, and the use of ! as
"not" is similar -- in all of those cases, the fact that the
programmer is explicitly testing an exit status is interpreted to mean
that a failure is "expected".  ERR_EXIT kicks in only when the result
is not otherwise checked.


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Inconsistent behavior of ERR_EXIT with conditionals
  2022-11-08  5:36       ` Bart Schaefer
@ 2022-11-08  8:04         ` Lawrence Velázquez
  2022-11-08 18:51           ` Philippe Altherr
  2022-11-08 23:11           ` Bart Schaefer
  0 siblings, 2 replies; 21+ messages in thread
From: Lawrence Velázquez @ 2022-11-08  8:04 UTC (permalink / raw)
  To: Bart Schaefer, Philippe Altherr; +Cc: zsh-workers

On Tue, Nov 8, 2022, at 12:36 AM, Bart Schaefer wrote:
> So you can see that zsh agrees with bash on your "Failure" cases.  The
> output of dash (as /bin/sh on ubuntu) is the same as zsh with the
> exception of Failure-b, which can't be tested because there is no
> equivalent of "declare" (or "local").

Here is a set of tests incorporating most of Philippe's examples
(some slightly modified).  In the output, "no" and "yes" are intended
to indicate whether the shell exited due to "set -e" / ERR_EXIT.
The tl;dr is that zsh agrees with other shells, except on three
constructs from Philippe's original email.

The shells I tested are zsh 5.9, bash 5.1.16(1)-release, ksh AJM
93u+ 2012-08-01, dash 0.5.11.5, and yash 2.52.  For more fun, see
<https://www.in-ulm.de/~mascheck/various/set-e/>.

	% head -n 100 *.sh(n) driver.zsh
	==> 1.sh <==
	f() {
	    { false && true; }
	}
	f
	printf '    no'

	==> 2.sh <==
	f() {
	    false && true
	}
	f
	printf '    no'

	==> 3.sh <==
	f() {
	    if true; then
	        false && true
	    fi
	}
	f
	printf '    no'

	==> 4.sh <==
	f() {
	    case foo in
	        *) false && true ;;
	    esac
	}
	f
	printf '    no'

	==> 5.sh <==
	f() {
	    (false && true)
	}
	f
	printf '    no'

	==> 6.sh <==
	: $(false)
	printf '    no'

	==> 7.sh <==
	if [ "$KSH_VERSION" ]; then
	    f() { typeset v=$(false); }
	else
	    f() { local v=$(false); }
	fi
	f
	printf '    no'

	==> 8.sh <==
	if
	    false
	    printf '    no'
	    true
	then
	    true
	fi

	==> 9.sh <==
	{
	    false
	    printf '    no'
	    true
	} && true

	==> 10.sh <==
	false && true
	printf '    no'

	==> 11.sh <==
	! true
	printf '    no'

	==> driver.zsh <==
	set -- *.sh(n)

	printf '     '
	printf ' %5s' $@
	echo

	for sh in zsh bash ksh dash yash; do
	    printf %4s: $sh
	    for arg; do
	        $sh -e $arg || printf '   yes'
	    done
	    echo
	done
	% zsh driver.zsh
	       1.sh  2.sh  3.sh  4.sh  5.sh  6.sh  7.sh  8.sh  9.sh 10.sh 11.sh
	 zsh:    no   yes    no    no   yes    no    no    no    no    no    no
	bash:   yes   yes   yes   yes   yes    no    no    no    no    no    no
	 ksh:   yes   yes   yes   yes   yes    no    no    no    no    no    no
	dash:   yes   yes   yes   yes   yes    no    no    no    no    no    no
	yash:   yes   yes   yes   yes   yes    no    no    no    no    no    no


> Bash disagrees with dash and
> zsh on your "Success" cases, at least at that version.

Bash disables "set -e" in command substitutions and has done so
since at least 1.14 (the oldest version I can test).  Bash 4.4 added
an option that causes command substitutions to inherit "set -e",
but it remains disabled by default (except in POSIX mode).


-- 
vq


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Inconsistent behavior of ERR_EXIT with conditionals
  2022-11-08  8:04         ` Lawrence Velázquez
@ 2022-11-08 18:51           ` Philippe Altherr
  2022-11-08 19:20             ` Lawrence Velázquez
  2022-11-08 23:28             ` Bart Schaefer
  2022-11-08 23:11           ` Bart Schaefer
  1 sibling, 2 replies; 21+ messages in thread
From: Philippe Altherr @ 2022-11-08 18:51 UTC (permalink / raw)
  To: Lawrence Velázquez; +Cc: Bart Schaefer, zsh-workers

[-- Attachment #1: Type: text/plain, Size: 5072 bytes --]

Thanks Lawrence for the comparison. It's a bit disappointing to see that
Zsh has the worst record :-(

Let me try to illustrate with a more concrete example why I think Zsh's
behavior could/should be improved. Consider that a first developer wrote
the following code:

#!/bin/zsh -e
>
> # ...
>
> function backup() {
>     scp $1 $BACKUP_SERVER:
>     echo $1 >> $BACKUP_LOG
> }
>
> # ...
>
> backup $file; rm $file
>

It's arguably not the greatest code but assuming that it's only ever run
with ERR_EXIT enabled, it behaves correctly. If "scp" fails, ERR_EXIT is
triggered and nothing is logged nor deleted.

Some time later a second developer comes, sees the code "backup $file; rm
$file", and thinks to himself that this looks dangerous; the file should
only be deleted if the backup was successful. Therefore they change the
code to "backup $file && rm $file". Unfortunately, this has exactly the
opposite effect of the intended one. From then on, if "scp" fails, ERR_EXIT
is no longer triggered and the file is logged and deleted.


On Tue, Nov 8, 2022, at 12:36 AM, Bart Schaefer wrote:

> ERR_EXIT kicks in only when the result is not otherwise checked.


Awesome! That's exactly how I would like Zsh to behave. To be slightly more
precise, I would formulate it as follow:

*ERR_EXIT kicks in if and only if the result is not otherwise checked.*


Unfortunately there are several cases where Zsh doesn't behave like that.
For example, consider the following commands:

{ false; true } || true; echo $?
> if false; true; then echo 0; else echo 1; fi


Both commands print "0". In both commands "false" is part of a condition.
However, in both commands, "false" does NOT influence the result of the
condition. In fact, in both commands, the result of "false" is NOT checked;
replacing it with "true" leads to the exact same result. Therefore, given
the specification above, ERR_EXIT should be triggered by "false" in both
commands.

In the commands above the problem is that Zsh never triggers the ERR_EXIT.
Apparently, when Zsh starts evaluating a condition, it no longer
enforces the ERR_EXIT option for the whole evaluation of the condition,
even if it contains commands whose result is not checked, like the "false"
commands in the example above. This is also true if the commands are nested
in called functions. The following code also prints "0" instead of existing
after the "false" command.

function foo() { false; true }
> foo || true; echo $?


There are other cases where ERR_EXIT is triggered but fails to propagate. A
major offender in that regard is the following code:

local var=$(false); echo $?


In this case "false" triggers an ERR_EXIT but it only exits the sub-shell
of the command substitution. For some reason, the local variable
assignment ignores the exit status of the command substitution and always
returns a zero exit status. Therefore the main shell does NOT exit and the
command prints "0".

Interestingly, in the following almost identical code, "false" triggers an
ERR_EXIT that also exits the main shell:

local var; var=$(false); echo $?


However, having to systematically use this style is rather cumbersome.
Furthermore it's not even foolproof. Indeed, if there are multiple command
substitutions, the assignment returns the exit status of the last one.
Thus, the following command does NOT exit and prints "0":

local var; var=$(false)$(true); echo $?


Since I really really want Zsh to behave as described above, I implemented
zabort <https://github.com/paltherr/zabort/blob/main/src/bin/zabort.zsh>,
which configures a ZERR trap to exit the current shell and all parent
shells whenever an ERR_EXIT is triggered. It also prints a nice stack trace
to the command that failed. This fixes the problem for all cases where the
ERR_EXIT isn't propagated, like in the variable assignments above. However,
the problem of the conditional expressions remains because in that case no
ERR_EXIT (and no ZERR trap) is ever triggered. Fixing this seems only
possible by changing the implementation of Zsh.

Here is an example using zabort:

#!/bin/zsh
>
> . zabort.zsh
>
> function log() { echo $@ 1>&2 }
>
> function f1() { false; log f1 }
> function f2() { : $(f1); log f2 }
> function f3() { local v3=$(f2); log f3 }
> function f4() { v4=$(f3)$(true); log f4 }
> function f5() { f4; log f5 }
>
> f5
>

And here is what it prints:

Command unexpectedly exited with the non-zero status 1.
> at abort-example.zsh:7(abort)
> at abort-example.zsh:8(f1)
> at abort-example.zsh:9(f2)
> at abort-example.zsh:10(f3)
> at abort-example.zsh:11(f4)
> at abort-example.zsh:13(f5)


*Is there any chance that Zsh could be changed to more closely follow the
specification above? *

I'm mainly interested in a fix for the conditional expressions but fixes
for the other issues would also be nice. It would be awesome if "zsh -e"
behaved as specified above in all cases.

If needed, I could look into implementing some of the fixes myself.
However, before I invest into this, I would prefer to know whether you
would be open to such changes.

Philippe

[-- Attachment #2: Type: text/html, Size: 7371 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Inconsistent behavior of ERR_EXIT with conditionals
  2022-11-08 18:51           ` Philippe Altherr
@ 2022-11-08 19:20             ` Lawrence Velázquez
  2022-11-08 23:28             ` Bart Schaefer
  1 sibling, 0 replies; 21+ messages in thread
From: Lawrence Velázquez @ 2022-11-08 19:20 UTC (permalink / raw)
  To: Philippe Altherr; +Cc: Bart Schaefer, zsh-workers

On Tue, Nov 8, 2022, at 1:51 PM, Philippe Altherr wrote:
> *Is there any chance that Zsh could be changed to more closely follow 
> the specification above? *

I don't have time right now to provide more tests or otherwise
elaborate, but I am pretty sure that no shell exhibits the behavior
you want, and I am *very* sure that said behavior would violate
POSIX in one way or another.  I know that zsh does not aim for POSIX
compliance, but that doesn't mean it ought to take a feature which
is currently highly compatible with other shells and change it to
be completely incompatible.

I strongly object to any of these proposed changes, except ones
that bring zsh into closer alignment with other shells.  (These are
arguably bugfixes.)

-- 
vq


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Inconsistent behavior of ERR_EXIT with conditionals
  2022-11-08  8:04         ` Lawrence Velázquez
  2022-11-08 18:51           ` Philippe Altherr
@ 2022-11-08 23:11           ` Bart Schaefer
  1 sibling, 0 replies; 21+ messages in thread
From: Bart Schaefer @ 2022-11-08 23:11 UTC (permalink / raw)
  To: Lawrence Velázquez; +Cc: Philippe Altherr, zsh-workers

On Tue, Nov 8, 2022 at 12:04 AM Lawrence Velázquez <larryv@zsh.org> wrote:
>
>                1.sh  2.sh  3.sh  4.sh  5.sh  6.sh  7.sh  8.sh  9.sh 10.sh 11.sh
>          zsh:    no   yes    no    no   yes    no    no    no    no    no    no
>         bash:   yes   yes   yes   yes   yes    no    no    no    no    no    no

All of 1,3,4 are fixed by my patch in workers/50897


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Inconsistent behavior of ERR_EXIT with conditionals
  2022-11-08 18:51           ` Philippe Altherr
  2022-11-08 19:20             ` Lawrence Velázquez
@ 2022-11-08 23:28             ` Bart Schaefer
  2022-11-09  4:11               ` Philippe Altherr
  1 sibling, 1 reply; 21+ messages in thread
From: Bart Schaefer @ 2022-11-08 23:28 UTC (permalink / raw)
  To: Philippe Altherr; +Cc: Lawrence Velázquez, zsh-workers

On Tue, Nov 8, 2022 at 10:51 AM Philippe Altherr
<philippe.altherr@gmail.com> wrote:
>
> Let me try to illustrate with a more concrete example why I think Zsh's behavior could/should be improved. Consider that a first developer wrote the following code:
>
>> #!/bin/zsh -e

The first developer is wrong.  That's not what -e is for.  A script
should be correct WITHOUT the use of -e ... the purpose of -e is to
uncover cases where the developer made a mistake, not to be an
integral part of the script function.

> On Tue, Nov 8, 2022, at 12:36 AM, Bart Schaefer wrote:
>>
>> ERR_EXIT kicks in only when the result is not otherwise checked.
>
> Unfortunately there are several cases where Zsh doesn't behave like that. For example, consider the following commands:
>
>> { false; true } || true; echo $?
>> if false; true; then echo 0; else echo 1; fi

Again, wrong.  "{ false; true }" is a single statement because of the
braces.  When that statement is followed by || the result of the
ENTIRE statement is considered to have been "checked".
Similarly, in "if false; true; then" the conditional part is
considered as a single statement whose result is "checked".

As Lawrence has said, you're discarding the POSIX definition of how
"set -e" is intended to work.

> There are other cases where ERR_EXIT is triggered but fails to propagate. A major offender in that regard is the following code:
>
>> local var=$(false); echo $?
>
> In this case "false" triggers an ERR_EXIT but it only exits the sub-shell of the command substitution.

This is explicitly called out in the documentation:

     Unlike parameter assignment statements, typeset's exit status on an
     assignment that involves a command substitution does not reflect
     the exit status of the command substitution.  Therefore, to test
     for an error in a command substitution, separate the declaration of
     the parameter from its initialization:

          # WRONG
          typeset var1=$(exit 1) || echo "Trouble with var1"

          # RIGHT
          typeset var1 && var1=$(exit 1) || echo "Trouble with var1"

> However, having to systematically use this style is rather cumbersome. Furthermore it's not even foolproof. Indeed, if there are multiple command substitutions, the assignment returns the exit status of the last one. Thus, the following command does NOT exit and prints "0":
>
>> local var; var=$(false)$(true); echo $?

This also follows bash and presumably POSIX behavior.

> Is there any chance that Zsh could be changed to more closely follow the specification above?

No.


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Inconsistent behavior of ERR_EXIT with conditionals
  2022-11-08 23:28             ` Bart Schaefer
@ 2022-11-09  4:11               ` Philippe Altherr
  2022-11-09  6:00                 ` Bart Schaefer
  0 siblings, 1 reply; 21+ messages in thread
From: Philippe Altherr @ 2022-11-09  4:11 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: Lawrence Velázquez, zsh-workers

[-- Attachment #1: Type: text/plain, Size: 4100 bytes --]

>
> All of 1,3,4 are fixed by my patch in workers/50897


That's great! It's a small win but I actually (re-)stumbled on the problem
of conditional expressions because of one of these cases. So it's good to
see that they can be fixed.


My ultimate goal is to be able to run Zsh scripts with the guarantee that
if any command unexpectedly fails (i.e., if any command whose result is not
otherwise checked returns a non-zero exit status), then the whole *script* (not
just some subshells) stops immediately. Wouldn't you agree that this would
be a useful feature?

The question is how can this be achieved. On the surface, it looks like
enabling ERR_EXIT does the trick. However there are several cases where
ERR_EXIT fails to do the job. These cases are of two categories:

   1. *Non-triggering:* In some contexts, commands whose result is not
   otherwise checked don't trigger a shell exit when they return a non-zero
   exit status even when ERR_EXIT is enabled, e.g., the "false" command in
   "{false; true} && true" doesn't trigger a shell exit.
   2. *Non-propagation:* In some contexts, errors in subshells don't
   propagate to the parent shell, e.g., the "false" in "local var=$(false)"
   triggers an exit in the subshell of the command substitution but the
   assignment ignores the result of the command substitution and thus the
   parent shell fails to exit in turn.

I hoped that some of these cases could be "fixed" but I have now checked
the POSIX specification and as you both pointed out, for most of them POSIX
specifies that they have to work as they currently do (this doesn't include
Lawrence's example 1,3,4, which should indeed be fixed).

The first developer is wrong.  That's not what -e is for.  A script
> should be correct WITHOUT the use of -e ... the purpose of -e is to
> uncover cases where the developer made a mistake, not to be an
> integral part of the script function.


I can agree with that but consider that the developer's mistake was to use
a ";" instead of an "&&" in the "backup" function. My broader point was
that the same error (or developer mistake) in a function "foo" triggers an
exit if "foo" is called from a plain statement but not if it's called from
within a condition. Wouldn't you agree that it's unfortunate that the same
error/mistake may or may not trigger an exit depending on whether it's
executed outside or inside a condition?

Again, wrong.  "{ false; true }" is a single statement because of the
> braces.  When that statement is followed by || the result of the
> ENTIRE statement is considered to have been "checked".
> Similarly, in "if false; true; then" the conditional part is
> considered as a single statement whose result is "checked".


Indeed, POSIX states "The -e setting shall be ignored when executing the
compound list following the while, until, if, or elif reserved word, a
pipeline beginning with the ! reserved word, or any command of an AND-OR
list other than the last.", so there is unfortunately no way this can be
changed, at least in the context of ERR_EXIT.


Is all hope lost? Not necessarily. The non-propagation issues can be worked
around. That's what my zabort
<https://github.com/paltherr/zabort/blob/main/src/bin/zabort.zsh> does by
configuring a ZERR trap that forcibly kills all parent shells from within
the subshell where the error occurred. Unfortunately, I don't see how the
non-triggering issues could be worked around. For these some change is
needed in Zsh but I agree that changing the behavior of ERR_EXIT isn't the
way to go as it should remain POSIX compliant. What could work is to
implement a new shell option ERR_EXIT_STRICT, which triggers an exit on any
command that returns a non-zero exit status and whose result isn't checked
otherwise. Only one of ERR_EXIT and ERR_EXIT_STRICT could be enabled at any
given time.

*Would you agree to add a new shell option if it allows to run Zsh scripts
such that if any command unexpectedly fails the script immediately stops
(and its implementation doesn't require too complex changes)?* If yes, I
may look into implementing it.

Philippe

[-- Attachment #2: Type: text/html, Size: 5132 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Inconsistent behavior of ERR_EXIT with conditionals
  2022-11-09  4:11               ` Philippe Altherr
@ 2022-11-09  6:00                 ` Bart Schaefer
  2022-11-09 14:22                   ` Philippe Altherr
  0 siblings, 1 reply; 21+ messages in thread
From: Bart Schaefer @ 2022-11-09  6:00 UTC (permalink / raw)
  To: Philippe Altherr; +Cc: zsh-workers

On Tue, Nov 8, 2022 at 8:11 PM Philippe Altherr
<philippe.altherr@gmail.com> wrote:
>>
>> The first developer is wrong.  That's not what -e is for.  A script
>> should be correct WITHOUT the use of -e ... the purpose of -e is to
>> uncover cases where the developer made a mistake, not to be an
>> integral part of the script function.
>
> I can agree with that but consider that the developer's mistake was to use a ";" instead of an "&&" in the "backup" function.

No, that wasn't his mistake.  His mistake was to not explicitly call
"exit" on failure of scp and instead rely on errexit to bail out of
the backup function.

The second developer's mistake was changing ; to && following the call
to the backup function without actually understanding how the backup
function (didn't) work.  *If* errexit worked the way you want, the &&
test would be spurious anyway, because either the function would have
returned the status of "echo" (always success) or would have died
without getting that far.

Neither of these is a situation where the developer should have been
relying on errexit to save them.

> My broader point was that the same error (or developer mistake) in a function "foo" triggers an exit if "foo" is called from a plain statement but not if it's called from within a condition. Wouldn't you agree that it's unfortunate that the same error/mistake may or may not trigger an exit depending on whether it's executed outside or inside a condition?

I wouldn't necessarily agree that it's the same mistake.  What's
unfortunate is that you're relying on a Deus ex machina to save your
script from disaster.

> Would you agree to add a new shell option if it allows to run Zsh scripts such that if any command unexpectedly fails the script immediately stops (and its implementation doesn't require too complex changes)?

I would abstain from the decision and leave it to others, but I don't
believe it can be done without some rather invasive changes.


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Inconsistent behavior of ERR_EXIT with conditionals
  2022-11-09  6:00                 ` Bart Schaefer
@ 2022-11-09 14:22                   ` Philippe Altherr
  2022-11-10  1:00                     ` Bart Schaefer
  0 siblings, 1 reply; 21+ messages in thread
From: Philippe Altherr @ 2022-11-09 14:22 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: zsh-workers

[-- Attachment #1: Type: text/plain, Size: 3290 bytes --]

>
> Neither of these is a situation where the developer should have been
> relying on errexit to save them.


My ultimate point with that example was to demonstrate that ERR_EXIT
doesn't achieve my goal.

My goal is to be able to run a Zsh script with the guarantee that if
anything goes wrong the script stops immediately where by "anything goes
wrong" I mean any command whose result is not otherwise checked returns a
non-zero exit status. If you don't agree that this would be a useful
feature, then we can stop the discussion.

If you think that ERR_EXIT achieves my goal, then it would be useful if you
could give me an example of a legitimate use of/reliance on ERR_EXIT. I
could then try to build a better example to demonstrate that ERR_EXIT is
not achieving my goal. If I fail to convince you that ERR_EXIT does not
achieve my goal, then too we can stop the discussion.

I would abstain from the decision and leave it to others, but I don't
> believe it can be done without some rather invasive changes.


Indeed, that remains an open question to me and I agree that the new option
shouldn't be added if it requires too much additional complexity.

Philippe


On Wed, Nov 9, 2022 at 7:00 AM Bart Schaefer <schaefer@brasslantern.com>
wrote:

> On Tue, Nov 8, 2022 at 8:11 PM Philippe Altherr
> <philippe.altherr@gmail.com> wrote:
> >>
> >> The first developer is wrong.  That's not what -e is for.  A script
> >> should be correct WITHOUT the use of -e ... the purpose of -e is to
> >> uncover cases where the developer made a mistake, not to be an
> >> integral part of the script function.
> >
> > I can agree with that but consider that the developer's mistake was to
> use a ";" instead of an "&&" in the "backup" function.
>
> No, that wasn't his mistake.  His mistake was to not explicitly call
> "exit" on failure of scp and instead rely on errexit to bail out of
> the backup function.
>
> The second developer's mistake was changing ; to && following the call
> to the backup function without actually understanding how the backup
> function (didn't) work.  *If* errexit worked the way you want, the &&
> test would be spurious anyway, because either the function would have
> returned the status of "echo" (always success) or would have died
> without getting that far.
>
> Neither of these is a situation where the developer should have been
> relying on errexit to save them.
>
> > My broader point was that the same error (or developer mistake) in a
> function "foo" triggers an exit if "foo" is called from a plain statement
> but not if it's called from within a condition. Wouldn't you agree that
> it's unfortunate that the same error/mistake may or may not trigger an exit
> depending on whether it's executed outside or inside a condition?
>
> I wouldn't necessarily agree that it's the same mistake.  What's
> unfortunate is that you're relying on a Deus ex machina to save your
> script from disaster.
>
> > Would you agree to add a new shell option if it allows to run Zsh
> scripts such that if any command unexpectedly fails the script immediately
> stops (and its implementation doesn't require too complex changes)?
>
> I would abstain from the decision and leave it to others, but I don't
> believe it can be done without some rather invasive changes.
>

[-- Attachment #2: Type: text/html, Size: 4377 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Inconsistent behavior of ERR_EXIT with conditionals
  2022-11-09 14:22                   ` Philippe Altherr
@ 2022-11-10  1:00                     ` Bart Schaefer
  2022-11-10  5:09                       ` Bart Schaefer
  0 siblings, 1 reply; 21+ messages in thread
From: Bart Schaefer @ 2022-11-10  1:00 UTC (permalink / raw)
  To: Philippe Altherr; +Cc: zsh-workers

On Wed, Nov 9, 2022 at 6:22 AM Philippe Altherr
<philippe.altherr@gmail.com> wrote:
>>
> If you think that ERR_EXIT achieves my goal, then it would be useful if you could give me an example of a legitimate use of/reliance on ERR_EXIT.

I don't think ERR_EXIT achieves your goal, no, and I don't think it
was intended to.

One example of a legitimate use of ERR_EXIT is to implement (shelling
out from) "make -k" where each command is distinct and you want the
build process to stop if one step is incomplete.

> If I fail to convince you that ERR_EXIT does not achieve my goal, then too we can stop the discussion.

I'm going to stop responding anyway, because I'm not interested in the
stated goal.  If others want to pursue this discussion, please go
ahead.


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Inconsistent behavior of ERR_EXIT with conditionals
  2022-11-10  1:00                     ` Bart Schaefer
@ 2022-11-10  5:09                       ` Bart Schaefer
  2022-11-11  3:04                         ` Philippe Altherr
  0 siblings, 1 reply; 21+ messages in thread
From: Bart Schaefer @ 2022-11-10  5:09 UTC (permalink / raw)
  To: Philippe Altherr; +Cc: zsh-workers

On Wed, Nov 9, 2022 at 5:00 PM Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> I'm going to stop responding anyway, because I'm not interested in the
> stated goal.  If others want to pursue this discussion, please go
> ahead.

One last thought ... have you tried checking the behavior with both
ERR_EXIT and ERR_RETURN ?  Functions behave differently for ERR_RETURN
than ERR_EXIT.


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Inconsistent behavior of ERR_EXIT with conditionals
  2022-11-10  5:09                       ` Bart Schaefer
@ 2022-11-11  3:04                         ` Philippe Altherr
  2022-11-11  4:06                           ` Lawrence Velázquez
  2022-11-11  4:09                           ` Eric Cook
  0 siblings, 2 replies; 21+ messages in thread
From: Philippe Altherr @ 2022-11-11  3:04 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: zsh-workers

[-- Attachment #1: Type: text/plain, Size: 5254 bytes --]

>
> One last thought ... have you tried checking the behavior with both
> ERR_EXIT and ERR_RETURN ?  Functions behave differently for ERR_RETURN
> than ERR_EXIT.


Ha, I had the exact same thought yesterday while studying exec.c and loop.c
but no, ERR_RETURN doesn't help. ERR_EXIT gets disabled on the left of "&&"
and "||" and in the condition of if and loop statements. ERR_RETURN gets
re-enabled on function calls. For my behavior, I too would need re-enabling
but of ERR_EXIT and more importantly it would have to happen on the left of
";" (i.e., for every command that is followed by another one) rather than
on function calls.

I'm still studying the code to figure out whether my behavior is at all
possible and what it would require. At the moment, many parts of the code
still look like black-magic but I slowly start to understand what's going
on. In that regard, the description of word codes in parse.c
<https://sourceforge.net/p/zsh/code/ci/master/tree/Src/parse.c#l100> was
very helpful. I think that I now have a relatively good understanding of
how the parsed code is represented overall but I'm still missing many of
the details. Something that would be of tremendous help is the ability to
see the parsed word codes. Is it possible to print these somehow? And/or is
it possible to print the word codes as they get evaluated? I haven't seen
anything to that effect. Is there anything else you would recommend that I
read and/or use to better understand the codebase?

One example of a legitimate use of ERR_EXIT is to implement (shelling
> out from) "make -k" where each command is distinct and you want the
> build process to stop if one step is incomplete.


Although I must admit that I only very partially understand your example, I
assume that you see ERR_EXIT more like a feature for other tools like
"make" that need to run arbitrary shell commands than something for
standalone shell scripts. I could also imagine that ERR_EXIT was originally
designed for such cases. However, in my experience, ERR_EXIT is often used
as a safeguard for standalone scripts. Even though it's far from perfect,
it's definitely better than nothing. And, in my opinion, it's a legitimate
use of ERR_EXIT, even if that wasn't its original purpose.

I'm mainly a Java developer. In Java many functions may throw runtime
exceptions. You can catch and handle these but in 99% of the cases you just
ignore them because you know (or assume) that they won't be thrown. Doing
otherwise would be very impractical because your code would be littered
with try/catch statements. Every now and then one of these exceptions that
you thought could/should never be thrown is nevertheless thrown for some
reason. In that case the exception goes up the call stack and, if no catch
is encountered, it kills the program and prints a stack trace to the code
that threw the exception. This way it's obvious that something failed. It's
also easy to figure out where it happened. And, it ensures that no extra
harm is done (by keeping running the program).

In the world of shell scripts, there are no runtime exceptions but some
exit statuses play a similar role. Many UNIX commands and shell functions
can potentially return a non-zero exit status but when you use them you
often know (or assume) that they won't return a non-zero exit status. If
they nevertheless do, the default behavior of shell script is to simply
ignore the error. So if "cmd1" fails in "cmd1; cmd2", "cmd2" still gets
executed. It's already bad that "cmd1" failed. Running "cmd2" could
cause extra harm. It looks much safer to exit right after "cmd1". That's
the main reason I run all my scripts with ERR_EXIT enabled.

In addition to that, ERR_EXIT also makes it easier to notice errors and to
figure out where they happened, especially if ERR_EXIT is coupled with code
that prints a stack trace. Without ERR_EXIT you may not even notice that
something went wrong if no error message was printed (or it got swallowed).

My impression is that ERR_EXIT is commonly used for these reasons. In fact,
whenever I have seen it used, it seemed to be for these reasons. But maybe
I should confirm that. I'll ask some of my friends if they use it and with
what expectations.

In my opinion, the only downside of that usage of ERR_EXIT is that it's far
from foolproof. There are plenty of cases where just enabling ERR_EXIT
won't be enough to ensure that the script halts at the first unexpected
error. I would like to improve that. My zabort script already goes a long
way. It works around the non-propagation issues, which are by far the main
reason why just ERR_EXIT isn't enough. That's good but it would be even
better if the non-triggering issues could also be solved, even if these are
less common.

Philippe


On Thu, Nov 10, 2022 at 6:09 AM Bart Schaefer <schaefer@brasslantern.com>
wrote:

> On Wed, Nov 9, 2022 at 5:00 PM Bart Schaefer <schaefer@brasslantern.com>
> wrote:
> >
> > I'm going to stop responding anyway, because I'm not interested in the
> > stated goal.  If others want to pursue this discussion, please go
> > ahead.
>
> One last thought ... have you tried checking the behavior with both
> ERR_EXIT and ERR_RETURN ?  Functions behave differently for ERR_RETURN
> than ERR_EXIT.
>

[-- Attachment #2: Type: text/html, Size: 6385 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Inconsistent behavior of ERR_EXIT with conditionals
  2022-11-11  3:04                         ` Philippe Altherr
@ 2022-11-11  4:06                           ` Lawrence Velázquez
  2022-11-11  4:09                           ` Eric Cook
  1 sibling, 0 replies; 21+ messages in thread
From: Lawrence Velázquez @ 2022-11-11  4:06 UTC (permalink / raw)
  To: Philippe Altherr, Bart Schaefer; +Cc: zsh-workers

On Thu, Nov 10, 2022, at 10:04 PM, Philippe Altherr wrote:
> However, in my 
> experience, ERR_EXIT is often used as a safeguard for standalone 
> scripts. Even though it's far from perfect, it's definitely better than 
> nothing.

Debatable.


> Every now and then one of 
> these exceptions that you thought could/should never be thrown is 
> nevertheless thrown for some reason. In that case the exception goes up 
> the call stack and, if no catch is encountered, it kills the program 
> and prints a stack trace to the code that threw the exception. This way 
> it's obvious that something failed. It's also easy to figure out where 
> it happened. And, it ensures that no extra harm is done (by keeping 
> running the program).

Only at a superficial level.

https://devblogs.microsoft.com/oldnewthing/20040422-00/?p=39683


> In the world of shell scripts, there are no runtime exceptions but some 
> exit statuses play a similar role. Many UNIX commands and shell 
> functions can potentially return a non-zero exit status but when you 
> use them you often know (or assume) that they won't return a non-zero 
> exit status.

Some nonzero exit statuses signify errors that should be fatal.
Some don't.  Some don't really indicate errors at all.  ERR_EXIT
makes no distinction, which is why it can cause confusion when it
kills scripts overzealously.

https://mywiki.wooledge.org/BashFAQ/105


> If they nevertheless do, the default behavior of shell 
> script is to simply ignore the error. So if "cmd1" fails in "cmd1; 
> cmd2", "cmd2" still gets executed. It's already bad that "cmd1" failed. 
> Running "cmd2" could cause extra harm. It looks much safer to exit 
> right after "cmd1". That's the main reason I run all my scripts with 
> ERR_EXIT enabled.

You should handle errors properly instead of relying on ERR_EXIT
to bail you out.


> My impression is that ERR_EXIT is commonly used for these reasons.

It is.  That doesn't mean it's a good idea.


> In my opinion, the only downside of that usage of ERR_EXIT is that it's 
> far from foolproof.

Lulling script writers into a false sense of security is a pretty
big downside.


> There are plenty of cases where just enabling 
> ERR_EXIT won't be enough to ensure that the script halts at the first 
> unexpected error. I would like to improve that. My zabort script 
> already goes a long way.

I do not think this idea is an improvement.  I think it is more of
the same false sense of security, except with rules that are somehow
even more convoluted than those of ERR_EXIT.


> It works around the non-propagation issues, 
> which are by far the main reason why just ERR_EXIT isn't enough.

ERR_EXIT isn't enough because it shoehorns an error handling model
into a context where it doesn't really fit.  Unfortunately that
ship has already sailed, but it's sailed on all shells, so at least
we have company.  I do not think zsh should exacerbate the situation
by shoehorning even harder, and I oppose the addition of an option
for doing so.


-- 
vq


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Inconsistent behavior of ERR_EXIT with conditionals
  2022-11-11  3:04                         ` Philippe Altherr
  2022-11-11  4:06                           ` Lawrence Velázquez
@ 2022-11-11  4:09                           ` Eric Cook
  1 sibling, 0 replies; 21+ messages in thread
From: Eric Cook @ 2022-11-11  4:09 UTC (permalink / raw)
  To: zsh-workers

On 11/10/22 22:04, Philippe Altherr wrote:

> I'm mainly a Java developer. In Java many functions may throw runtime exceptions. You can catch and handle these but in 99% of the cases you just ignore them because you know (or assume) that they won't be thrown. Doing otherwise would be very impractical 
> because your code would be littered with try/catch statements. Every now and then one of these exceptions that you thought could/should never be thrown is nevertheless thrown for some reason. In that case the exception goes up the call stack and, if no 
> catch is encountered, it kills the program and prints a stack trace to the code that threw the exception. This way it's obvious that something failed. It's also easy to figure out where it happened. And, it ensures that no extra harm is done (by keeping 
> running the program).
> 
> In the world of shell scripts, there are no runtime exceptions but some exit statuses play a similar role. Many UNIX commands and shell functions can potentially return a non-zero exit status but when you use them you often know (or assume) that they 
> won't return a non-zero exit status. If they nevertheless do, the default behavior of shell script is to simply ignore the error. So if "cmd1" fails in "cmd1; cmd2", "cmd2" still gets executed. It's already bad that "cmd1" failed. Running "cmd2" could 
> cause extra harm. It looks much safer to exit right after "cmd1". That's the main reason I run all my scripts with ERR_EXIT enabled.
> 
> In addition to that, ERR_EXIT also makes it easier to notice errors and to figure out where they happened, especially if ERR_EXIT is coupled with code that prints a stack trace. Without ERR_EXIT you may not even notice that something went wrong if no 
> error message was printed (or it got swallowed).
> 
> My impression is that ERR_EXIT is commonly used for these reasons. In fact, whenever I have seen it used, it seemed to be for these reasons. But maybe I should confirm that. I'll ask some of my friends if they use it and with what expectations.

Didn't read the entire thread but the first quoted paragraph seems to be the commonly misguided reason to blindly use set -e; attempting to use a paradigm of a language more familiar to the author, to a different language.
As more and more people start their programming careers in languages with exceptions, exception-less languages like unix shells[1] or more recently go-lang[2] looks pretty antiquated.


> In my opinion, the only downside of that usage of ERR_EXIT is that it's far from foolproof. There are plenty of cases where just enabling ERR_EXIT won't be enough to ensure that the script halts at the first unexpected error.
Correct, it is far from foolproof and it can create additional problems when it unexpectedly unexpected errors and terminate the script for reasons that can confuse the author. [1]

[1] http://mywiki.wooledge.org/BashFAQ/105
[2] https://go.dev/doc/faq#exceptions


^ permalink raw reply	[flat|nested] 21+ messages in thread

end of thread, other threads:[~2022-11-11  4:09 UTC | newest]

Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-11-04 16:37 Inconsistent behavior of ERR_EXIT with conditionals Philippe Altherr
2022-11-06 20:45 ` Bart Schaefer
2022-11-07  3:50   ` Bart Schaefer
2022-11-07  5:35     ` [PATCH] " Bart Schaefer
2022-11-07  9:44       ` Peter Stephenson
2022-11-08  1:20         ` Bart Schaefer
2022-11-08  4:58     ` Philippe Altherr
2022-11-08  5:36       ` Bart Schaefer
2022-11-08  8:04         ` Lawrence Velázquez
2022-11-08 18:51           ` Philippe Altherr
2022-11-08 19:20             ` Lawrence Velázquez
2022-11-08 23:28             ` Bart Schaefer
2022-11-09  4:11               ` Philippe Altherr
2022-11-09  6:00                 ` Bart Schaefer
2022-11-09 14:22                   ` Philippe Altherr
2022-11-10  1:00                     ` Bart Schaefer
2022-11-10  5:09                       ` Bart Schaefer
2022-11-11  3:04                         ` Philippe Altherr
2022-11-11  4:06                           ` Lawrence Velázquez
2022-11-11  4:09                           ` Eric Cook
2022-11-08 23:11           ` Bart Schaefer

Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/zsh/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).