From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-3.4 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FROM,HTML_MESSAGE,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 30930 invoked from network); 13 Nov 2022 15:46:34 -0000 Received: from zero.zsh.org (2a02:898:31:0:48:4558:7a:7368) by inbox.vuxu.org with ESMTPUTF8; 13 Nov 2022 15:46:34 -0000 ARC-Seal: i=1; cv=none; a=rsa-sha256; d=zsh.org; s=rsa-20210803; t=1668354394; b=UuJhkdsO90noiKcorkzVcJEvA5xGxX7oMiaIfRe21Q+zssdjoPvUyWQwYK+pO4phK3oUwSs9ry g6EqHan8ht+u9yrXETOXGCDvi0agrhjuoWBVLGdqV9WrFmi64EvVQWcHzZhD4vDjv8yy5LNaef Yhre3ZFHr+s0AEwWjWJ7LvZkn8PNydR6lfACP0tY1y/RLnQ6qH1rbFFMh/ZyUWw+cWEahXvsH5 1T3o5j2mHmF2WpOa+1h56pJKh9Ezw68ufVjFSBW8vT5whcUQVcsr/o76qFIBRHZExFYJ7UzaH3 /YfAU+FYebVUYs2Yc8gwVMCg6wNrkVrRB/vE08CvPUx5xw==; ARC-Authentication-Results: i=1; zsh.org; iprev=pass (mail-vs1-f42.google.com) smtp.remote-ip=209.85.217.42; dkim=pass header.d=gmail.com header.s=20210112 header.a=rsa-sha256; dmarc=pass header.from=gmail.com; arc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed; d=zsh.org; s=rsa-20210803; t=1668354394; bh=hNrHZ4elYZNYte5YJgRXKcbanWy1hgQGds84F9DycOE=; h=List-Archive:List-Owner:List-Post:List-Unsubscribe:List-Subscribe:List-Help: List-Id:Sender:Content-Type:Cc:To:Subject:Message-ID:Date:From:In-Reply-To: References:MIME-Version:DKIM-Signature:DKIM-Signature; b=Yh+fjFlfZz+fVNfw6Ad2+oTdECZau6Qp8Ii9rjVVQlZ9uFDWyz8mUWbqC8lcJhT3dNGH07J2fb WZLXCu7RYYBvfUF/3/PJtv/diUKooeKWMAIouXyws0Z9JGTFV3DRda5Wj2Zt2ogFsgaWzk12hY gqMW51WbpsIJirFKrgLKtP0xOHiACYxuJ5/qBY2obH3/ZGWVk5oAdQ7KKLqtrpx9jFbjV012rv UJaMs8Vgweis2KQSJlqE9T6gKOFHXWHTZdJg71aoKSSreD8d853j48hfTfMYc2SgtbMC/9Opcq 9yVMTSovSDTAp8lMJ45II2tlCP/oNs1ujIXfi65gWcTC/g==; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=zsh.org; s=rsa-20210803; h=List-Archive:List-Owner:List-Post:List-Unsubscribe: List-Subscribe:List-Help:List-Id:Sender:Content-Type:Cc:To:Subject:Message-ID :Date:From:In-Reply-To:References:MIME-Version:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID; bh=6bmv5FrByWsXzK3LhfapToRdJhogdnD/Vwc9LjnhmTs=; b=AO36MweC0jhjv2gwRXYTHjcUtc 5p9MaBSL2ipCoZucp3QwRkWKb7iXpAFggs/f9AEIKyA5B2osy2ienK/o6cBQYYhwf1IHCwj5k/vvu gEYqtiDdxIswu5NJYsnvwdbMdF5SpokgwS2UU2gLcOB2V5LJ6yOfZehwl/0SteuRdji+C2Zau150O PUWSFbKGDOEXD25dsuQ3+JV/G2r66RYE6llyO0S8bUKCCf9GtqWzsuNG235bjN6mHfTGuikkypBvJ OquasM0qqffk+PV8fJbKSdqPqDDHP4ZMhcqIdJWv8VwTnKEbmoQ4HHkD7EwblEF33OEpCp766Hc6n HVzVTdcg==; Received: by zero.zsh.org with local id 1ouFBp-000BDf-UY; Sun, 13 Nov 2022 15:46:34 +0000 Authentication-Results: zsh.org; iprev=pass (mail-vs1-f42.google.com) smtp.remote-ip=209.85.217.42; dkim=pass header.d=gmail.com header.s=20210112 header.a=rsa-sha256; dmarc=pass header.from=gmail.com; arc=none Received: from mail-vs1-f42.google.com ([209.85.217.42]:38790) by zero.zsh.org with esmtps (TLS1.3:TLS_AES_128_GCM_SHA256:128) id 1ouFBU-000AtB-1p; Sun, 13 Nov 2022 15:46:13 +0000 Received: by mail-vs1-f42.google.com with SMTP id a6so5038384vsc.5 for ; Sun, 13 Nov 2022 07:46:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=6bmv5FrByWsXzK3LhfapToRdJhogdnD/Vwc9LjnhmTs=; b=Y86HhZAP2VXwqeYMs2Add30R/1mtndwKg01jXZmu99vtzoVo4Jti9gKE17jMd7vqA0 zLRGUV30BxCVKLtQlryCXT0Mc5WfZ6+T4PB5qmOBSJ1dRpJnOYoT0ft4DWiuytILpvD1 jvDUKyT2jgkwLPakhwv8RzuGIFsI+3M5VJKaGiDGD5//DtYQoIjbT+zvc2UVceIdobIC faRMj3IXqSQjZ8H+rUGECfPDGxHAgFVlDOHYIgi7+86MG5AccXS2HYvduNFEtpgPoiOR 0b8o5AKzYt1co6tQWgTaUrcT0gh4/T4dxXn1QuCjXQEwFfaDRzrA6z45bkntuxCuhLeJ JzJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=6bmv5FrByWsXzK3LhfapToRdJhogdnD/Vwc9LjnhmTs=; b=slsQggkjjOplt/A13LwWAgkXOnyOcraS3Wz2JCd3QSV3+Pp7FXSCjT0YKRWwRNTiON Xc8Z+6HpY/GUgNMDNPJjIh2V9Z4JR4oQHqomfF3v0HrIgLWfIfoJFMdavnwqA3htZZmr XpZjXb0YbgizH5DEGWaTdEcwT3FqE2JwEx2lWjkp7KoWWUWtopxbdJRpHakayrjfge5u qIgyvIxTt1pW3PXU6v/KhsPAx2eP8u5SCZ3WoOs6HdpvnqGoqotKK446mF3fly7wcI4J iQSU6+Tqjukc9BUwfXPhq3NhOZDiCp/baGzgsa6P2ul/9q7YP13T3SHlKEaCM8o0KcvD pb0Q== X-Gm-Message-State: ANoB5pmOc95KPXj8Yq9rgYMcokqo5dGCcCyW9VtzCIEJZetnPsjScM2B H+kYBF2RogZlkbbuaGKOEKQOUIBIWSbsjQ4noGkNT9jfbGE= X-Google-Smtp-Source: AA0mqf5ICQp7/45r11SBsUOESDajxDZu2FeJxU6KhNqk6cTx6JoraMISbsGNU+XQ1N39X5e5lm0nKj9f9nolGDUbuWU= X-Received: by 2002:a67:dd85:0:b0:3ac:bd5b:31e5 with SMTP id i5-20020a67dd85000000b003acbd5b31e5mr4057736vsk.56.1668354370949; Sun, 13 Nov 2022 07:46:10 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Philippe Altherr Date: Sun, 13 Nov 2022 16:45:59 +0100 Message-ID: Subject: Re: [PATCH] More ERR_EXIT (was Re: Tests RE behavior of ERR_EXIT) To: Bart Schaefer Cc: zsh-workers@zsh.org Content-Type: multipart/alternative; boundary="000000000000a3d59005ed5c059f" X-Seq: 50953 Archived-At: X-Loop: zsh-workers@zsh.org Errors-To: zsh-workers-owner@zsh.org Precedence: list Precedence: bulk Sender: zsh-workers-request@zsh.org X-no-archive: yes List-Id: List-Help: , List-Subscribe: , List-Unsubscribe: , List-Post: List-Owner: List-Archive: --000000000000a3d59005ed5c059f Content-Type: text/plain; charset="UTF-8" Maybe this gives you a clue. In the code below the only difference between fooLoop1 and fooLoop2 is that the latter uses "false && true" instead of "false", which should not immediately trigger an ERR_EXIT but instead bubble up to the calling function and trigger an ERR_EXIT there. function fooLoop1() { > init; > v=2; > while [[ $v -ne 0 ]]; do > echo "Loop with v=$v" >&2; > v=$((v-1)); > false; > done > } > function fooLoop2() { > init; > v=2; > while [[ $v -ne 0 ]]; do > echo "Loop with v=$v" >&2; > v=$((v-1)); > false && true; > done > } In the latest Zsh, fooLoop1 triggers an ERR_EXIT at the line of the "false" (expected and correct). fooLoop2 also triggers an ERR_EXIT but after two loop iterations and at the line of the while. Now let's remove the following code from exec.c if (!(oldnoerrexit & NOERREXIT_UNTIL_EXEC)) > noerrexit = oldnoerrexit; With that change the two functions still behave exactly the same but if you replace the while statement with any other statement (if, case, for, ...) then no ERR_EXIT is triggered in foo. Instead the non-zero exit status correctly bubbles up to the caller and triggers an ERR_EXIT in bar. I still don't understand why the change above fixes the problem for if, case, for, ... statements. I understand even less why it doesn't fix it for while statements. Philippe On Sun, Nov 13, 2022 at 3:24 PM Philippe Altherr wrote: > The commenting out also fixes the problem for case statements and braces > (i.e., for "{ ... }"). It works even if loop.c is reverted to the previous > state with "this_noerrexit = 1" statements, which seem more correct to me. > > Apparently execwhile, like execif, needs more complicated > noerrexit resetting logic, even though I still don't understand what it's > doing in execif and why it's needed. > > Philippe > > > On Sun, Nov 13, 2022 at 2:55 PM Philippe Altherr < > philippe.altherr@gmail.com> wrote: > >> You shouldn't even be bothering with 5.8.1, it's been wrong all along; >>> it blindly never errexits at the end of an if/then/fi. >> >> >> I think that this isn't necessarily wrong. My understanding of the code >> so far is that the decision to trigger an ERR_EXIT is pushed down the >> evaluation stack. In "if cmd1; then cmd2; else cmd3; fi", only the >> evaluations of the (word codes representing the) commands "cmd1", "cmd2", >> or "cmd3" can ever trigger an ERR_EXIT. The evaluation of the (word codes >> representing the) if/then/else itself never triggers an ERR_EXIT. In other >> words only (the word codes representing) "basic commands", like function >> calls or UNIX commands, can ever trigger an ERR_EXIT. This strategy has the >> benefit that ERR_EXIT will be triggered exactly at the point where the >> fatal non-zero exit status was produced. >> >> 00 if >> 01 cmd1 >> 02 then >> 03 cmd2 >> 04. else >> 05. cmd3 >> 06 fi >> >> In the example above, with the strategy I described, and with the >> knowledge that the if condition never triggers an ERR_EXIT, it's guaranteed >> that an ERR_EXIT will only ever be thrown at line 03 or 05. If the >> triggering of the ERR_EXIT was sometimes delayed and delegated to the >> if/then/else, then ERR_EXIT could also be triggered at line 02 or 04, or >> worse at line 01 or 06, which wouldn't let you know whether the non-zero >> status originated from "cmd2" or from "cmd3". The delayed/delegated >> triggering looks undesirable because it gives you less information on the >> origin of the error. My understanding is that it's also never needed. >> >> The behavior of ERR_EXIT is controlled by the variables "noerrexit" and >> "local_noerrexit". My understanding of these variables is the following: >> >> - noerrexit: This variable is set to indicate that the triggering of >> ERR_EXIT must be disabled in the evaluation of any word code from the point >> where it's set until it's reset. For example it's set here >> >> in execif before the evaluation of the condition and reset here >> , >> here >> , >> here >> , >> and here >> >> after the evaluation of the condition. I don't really understand why the >> reseting is so complicated. It's much more straightforward in execwhile ( >> here >> >> ). >> >> - local_noerrexit: This variable is set to indicate that the triggering >> of ERR_EXIT must be disabled in the remainder of the evaluation of the >> current word code. For example it's set at the end of each compound >> command, like here >> . >> This used to be a plain "this_noerrexit = 1", which I don't think was wrong. >> >> I think my patches so far have uncovered a different bug that was >>> already present but was masked by the foregoing, which is, that >>> noerrexit is unwound in cases where it should not be. I think this is >>> happening at lines 1530-1531 of exec.c, right under the comment about >>> "hairy code near the end of execif()". That's an area I didn't touch, >>> but I'm pretty sure it's restoring noerrexit to its state before >>> entering the "if" (oldnoerrexit) when it should be preserving the >>> state from the "&&" conditional. In 5.8.1 this gets reversed again >>> via this_noerrexit. >> >> >> I must admit that I don't understand the NOERREXIT_UNTIL_EXEC logic here >> , >> nor the complicated resetting logic of noerrexit at the end of execif. I >> was about to say that this doesn't seem to be the source of the problem >> because if, while, and for statements all behave the same in Zsh. >> >> function fooIf1() { init; cond=true; if $cond; then cond=false; >>> false ; fi ; } >>> function fooIf2() { init; cond=true; if $cond; then cond=false; >>> false && true; fi ; } >> >> >> >> function fooWhile1() { init; cond=true; while $cond; do cond=false; >>> false ; done; } >>> function fooWhile2() { init; cond=true; while $cond; do cond=false; >>> false && true; done; } >> >> >> >> function fooFor1() { init; cond=true; for v in x ; do cond=false; >>> false ; done; } >>> function fooFor2() { init; cond=true; for v in x ; do cond=false; >>> false && true; done; } >> >> >> In the examples above fooIf1, fooWhile1, and fooFor1 all work correctly >> but fooIf2, fooWhile2, and fooFor2 fail to trigger ERR_EXIT in Zsh 5.8 and >> trigger it too early (in foo instead of in bar) in Zsh 5.9. >> >> However, if I comment out the NOERREXIT_UNTIL_EXEC logic in exec.c (or >> remove the negation), then fooIf2 and surprisingly also fooFor2 work >> correctly in Zsh 5.9 but not fooWhile2!?! fooWhile2 still triggers too >> early. >> >> So it looks like this may indeed be the start of the answer. But I'm >> still scratching my head on why that is. >> >> Philippe >> >> --000000000000a3d59005ed5c059f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Maybe this gives you a clue. In the code below the only di= fference between fooLoop1 and fooLoop2 is that the latter uses "false = && true" instead of "false", which should not immedi= ately=C2=A0trigger an ERR_EXIT but instead bubble up to the calling functio= n and trigger an ERR_EXIT there.

function fooLoop1() {=C2=A0 =C2=A0 init;
=C2=A0 =C2=A0 v=3D2;
=C2=A0 =C2=A0 while [[ $v -= ne 0 ]]; do
=C2=A0 =C2=A0 =C2=A0 =C2=A0 echo "Loop with v=3D$v"= ; >&2;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 v=3D$((v-1));
=C2=A0 =C2=A0= =C2=A0 =C2=A0 false;
=C2=A0 =C2=A0 done
}
function fooLoop2() {=C2=A0 =C2=A0 init;
=C2=A0 =C2=A0 v=3D2;
=C2=A0 =C2=A0 while [[ $v = -ne 0 ]]; do
=C2=A0 =C2=A0 =C2=A0 =C2=A0 echo "Loop with v=3D$v&quo= t; >&2;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 v=3D$((v-1));
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 false && true;
=C2=A0 =C2=A0 done
}<= /blockquote>

In the latest Zsh, fooLoop1 triggers an ERR= _EXIT at the line of the "false" (expected and correct). fooLoop2= also triggers an ERR_EXIT but after two loop iterations and at the line of= the while.

Now let's remove the following cod= e from exec.c

=C2=A0 =C2=A0 =C2=A0 =C2=A0if (!(oldnoerrexit & NOERREXIT_UN= TIL_EXEC))
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0noerrexit =3D oldnoe= rrexit;

With that change the two functions = still behave exactly the same but if you replace the while statement with a= ny other statement (if, case, for, ...) then no ERR_EXIT is triggered in fo= o. Instead the non-zero exit status correctly bubbles up to the caller and = triggers an ERR_EXIT in bar.

I still don't und= erstand why the change above fixes the problem for if, case, for, ... state= ments. I understand even less why it doesn't fix it for while statement= s.

Philippe


On Sun, Nov 13, 20= 22 at 3:24 PM Philippe Altherr <philippe.altherr@gmail.com> wrote:
The commenting out also f= ixes the problem for case statements=C2=A0and braces (i.e., for "{ ...= }"). It works even if loop.c is reverted to the previous state with &= quot;this_noerrexit =3D 1" statements, which seem more correct to me.<= div>
Apparently execwhile, like execif, needs more complicate= d noerrexit=C2=A0resetting=C2=A0logic, even though=C2=A0I still don't u= nderstand what it's doing in execif and why it's needed.
=

Philippe


On Sun, Nov 13, 2022= at 2:55 PM Philippe Altherr <philippe.altherr@gmail.com> wrote:
You shouldn't even be bothering= with 5.8.1, it's been wrong all along;
it blindly never errexits at= the end of an if/then/fi.

I think that thi= s isn't necessarily wrong. My understanding of the code so far is that = the decision to trigger an ERR_EXIT is pushed down the evaluation stack. In= "if cmd1; then cmd2; else cmd3; fi", only the evaluations of the= (word codes representing the) commands "cmd1", "cmd2",= or "cmd3" can ever trigger an ERR_EXIT. The evaluation of the (w= ord codes representing the) if/then/else itself never triggers an ERR_EXIT.= In other words only (the word codes representing) "basic commands&quo= t;, like function calls or UNIX commands, can ever trigger an ERR_EXIT. Thi= s strategy has the benefit that ERR_EXIT will be triggered exactly at the p= oint where the fatal non-zero exit status was produced.

00 if
01=C2=A0 =C2=A0cmd1
02=C2=A0 then
03=C2=A0 =C2=A0cmd2
04. else
05.=C2=A0 =C2=A0cmd3
06 fi

In the example above, with the strateg= y=C2=A0I described, and with the knowledge that the if condition never trig= gers an ERR_EXIT, it's guaranteed that an ERR_EXIT will only ever be th= rown at line 03 or 05. If the triggering of the ERR_EXIT was sometimes dela= yed and delegated to the if/then/else, then ERR_EXIT could also be triggere= d at line 02 or 04, or worse at line 01 or 06, which wouldn't let you k= now whether the non-zero status originated=C2=A0from "cmd2" or fr= om "cmd3". The delayed/delegated triggering looks undesirable bec= ause it gives you less information on the origin of the error. My understan= ding is that it's also never needed.

The behav= ior of ERR_EXIT is controlled by the variables "noerrexit" and &q= uot;local_noerrexit". My understanding of these variables is the follo= wing:

- noerrexit: This variable is set to indicat= e that the triggering of ERR_EXIT must be disabled in the evaluation of any= word code from the point where it's set until it's reset. For exam= ple it's set here= in execif before the evaluation of the condition and reset here, here, here, and here after the evaluation of the condition. I don't really understand why= the reseting is so complicated. It's much more straightforward=C2=A0in= execwhile=C2=A0(here= ).

- local_noerrexit: This variable is set to indi= cate that the triggering of ERR_EXIT must be disabled in the remainder of t= he evaluation of the current word code. For example it's set at the end= of each compound command, like here. This used to be a plain "this_noerrexit =3D 1", = which I don't think was wrong.

I think my patches so far have uncovered a di= fferent bug that was
already present but was masked by the foregoing, wh= ich is, that
noerrexit is unwound in cases where it should not be.=C2=A0= I think this is
happening at lines 1530-1531 of exec.c, right under the= comment about
"hairy code near the end of execif()".=C2=A0 Th= at's an area I didn't touch,
but I'm pretty sure it's re= storing noerrexit to its state before
entering the "if" (oldno= errexit) when it should be preserving the
state from the "&&= ;" conditional.=C2=A0 In 5.8.1 this gets reversed again
via this_no= errexit.

I must admit that I don't unde= rstand the=C2=A0NOERREXIT_UNTIL_EXEC logic here, nor the complicated resetting=C2=A0logic of no= errexit=C2=A0at the end of execif. I was about to say that this doesn't= seem to be the source of the problem because if, while, and for statements= all behave the same in Zsh.

function fooIf1() =C2=A0 = =C2=A0{ init; cond=3Dtrue; if =C2=A0 =C2=A0$cond; then cond=3Dfalse; false = =C2=A0 =C2=A0 =C2=A0 =C2=A0; fi =C2=A0; }
function fooIf2() =C2=A0 =C2= =A0{ init; cond=3Dtrue; if =C2=A0 =C2=A0$cond; then cond=3Dfalse; false &am= p;& true; fi =C2=A0; }=C2=A0
=C2=A0
function fooWhile1() { init; cond=3Dtrue; while $cond; do =C2=A0 cond=3D= false; false =C2=A0 =C2=A0 =C2=A0 =C2=A0; done; }
function fooWhile2() {= init; cond=3Dtrue; while $cond; do =C2=A0 cond=3Dfalse; false && t= rue; done; }=C2=A0
=C2=A0
function fo= oFor1() =C2=A0 { init; cond=3Dtrue; for v in x ; do =C2=A0 cond=3Dfalse; fa= lse =C2=A0 =C2=A0 =C2=A0 =C2=A0; done; }
function fooFor2() =C2=A0 { ini= t; cond=3Dtrue; for v in x ; do =C2=A0 cond=3Dfalse; false && true;= done; }

In the examples above fooIf= 1, fooWhile1, and fooFor1 all work correctly but fooIf2, fooWhile2, and foo= For2 fail to trigger ERR_EXIT in Zsh 5.8 and trigger it too early (in foo i= nstead of in bar) in Zsh 5.9.

However, if I commen= t out the=C2=A0NOERREXIT_UNTIL_EXEC logic in exec.c (or remove the negation= ), then fooIf2 and surprisingly=C2=A0also fooFor2 work correctly in Zsh 5.9= but not fooWhile2!?! fooWhile2 still triggers too early.

So it looks like this may indeed be the start of the answer. But I&= #39;m still scratching my head on why that is.

Phi= lippe

--000000000000a3d59005ed5c059f--