From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-3.3 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,T_SCC_BODY_TEXT_LINE, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 28534 invoked from network); 6 Apr 2022 05:32:54 -0000 Received: from zero.zsh.org (2a02:898:31:0:48:4558:7a:7368) by inbox.vuxu.org with ESMTPUTF8; 6 Apr 2022 05:32:54 -0000 ARC-Seal: i=1; cv=none; a=rsa-sha256; d=zsh.org; s=rsa-20210803; t=1649223174; b=PZLqgR3aB96amaxJgm3HRFf7T6JkoHfvjxWDhmWt1QqtGR8X2Y2AE8Me1Cx9wHTuaIdSoWO2ay fSBV/HhF/Jv+sUz5lp6+wrcqAVY2JoLI6F4nH3/a0fFyw+lobmSQHUqXpMh63+qOZAQjG+EZjj cTFFGAYFDjqVwZU3uuY/KWkVgIoCjLH059NH9neV+eAsSFlw9U2b/jKkaOtrcbYZ2rRNDpy12I /Rg+Gk39CXGAkk9rJaahxr72+2i27Svaw3zjIwhgrv3hb/UkDdzURdar5T2mF3egfmSP2seKSL IKvslGB2uQTJaAP6PcJZaQgJz7Ruz/e56p3qXAUJ8N7vfA==; ARC-Authentication-Results: i=1; zsh.org; iprev=pass (snd00007-bg.im.kddi.ne.jp) smtp.remote-ip=27.86.113.7; dmarc=none header.from=kba.biglobe.ne.jp; arc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed; d=zsh.org; s=rsa-20210803; t=1649223174; bh=KbnCo946s8r+ERP9wlfqoALv2vDUy2mSMP41XMdhteQ=; h=List-Archive:List-Owner:List-Post:List-Unsubscribe:List-Subscribe:List-Help: List-Id:Sender:Message-ID:In-Reply-To:To:References:Date:Subject: MIME-Version:Content-Transfer-Encoding:Content-Type:From:DKIM-Signature; b=T3NNDlrqyS1WMbhPU6GBTtHQ6w+jOM+GvWNf6eDLLubwc9f3+aKzJXj2KboR2pKyudBtDKG0BW rmPm+m4l44yyPBWHYeHrloO3Wb36Gny1gHUwB8HOxxnFeNGCiuKzM78gzbjuiSsq5SuqtDfK77 qf9Bis9tBanR4/RDE6lIWxZAnLoipcfaeUIfYggFUyy1FvF1cxQLH7pTOwSeOaTckAo+cfjE/+ l7p0NQ/170wZfPdg59Af/1SaBNKVKg0am9mabRawrJ1SShq42PYYlv7+53y/JPtICRgmyKuCap lf9FdOTnCyYldU6KRSlR2aRMY1SEU+X5oFD3hjcFE8GxSg==; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=zsh.org; s=rsa-20210803; h=List-Archive:List-Owner:List-Post:List-Unsubscribe: List-Subscribe:List-Help:List-Id:Sender:Message-Id:In-Reply-To:To:References: Date:Subject:Mime-Version:Content-Transfer-Encoding:Content-Type:From: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID; bh=+qe4pUANZBq7YWpPz+Lgf9UVo5rixaRz5AkeMd116xk=; b=VOtVJoJtufwm+QrHLTlgstGPMl 4w8Tfj19jfa+Euk0qh1uPpjYjOU5mvdp3Ps2h1w94ioJKqdpK0KUZ/FiPIg1KsKJoASZhlBuJLOXd Hhf8g1z2VlDwGJTbejZPMtHAkBpwV6ibnevaQ98UxEGN9GRFpymr7o5XYiTBuJMo32tyJf7eBEw8m Z8+3pe4gGcVL+uWno1nmGZ3+hQXd3s2e1xIpWuUSscjYRPQOcmm2JgW8khSvFhB4jPOb+QHlXdrGl BHNZSRDJrVU28HJyadDf5Xa/nkCB+R48u1nTZjYNXYL5KZXOvxeWz4byxJqwIDv9kriqdzS80Kfy/ F5PvxHHA==; Received: from authenticated user by zero.zsh.org with local id 1nbyHl-000EyP-TG; Wed, 06 Apr 2022 05:32:53 +0000 Authentication-Results: zsh.org; iprev=pass (snd00007-bg.im.kddi.ne.jp) smtp.remote-ip=27.86.113.7; dmarc=none header.from=kba.biglobe.ne.jp; arc=none Received: from snd00007-bg.im.kddi.ne.jp ([27.86.113.7]:41249 helo=dfmta0003.biglobe.ne.jp) by zero.zsh.org with esmtps (TLS1.3:TLS_AES_256_GCM_SHA384:256) id 1nbyHR-000EeB-UT; Wed, 06 Apr 2022 05:32:36 +0000 Received: from mail.biglobe.ne.jp by omta0003.biglobe.ne.jp with ESMTP id <20220406053228505.IKRW.82237.mail.biglobe.ne.jp@biglobe.ne.jp> for ; Wed, 6 Apr 2022 14:32:28 +0900 From: Jun T Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.21\)) Subject: ZTST_continue (was Re: Lots of test failures when --disable-multibyte) Date: Wed, 6 Apr 2022 14:32:28 +0900 References: <1174195730.504727.1649083398497@mail2.virginmedia.com> To: zsh-workers@zsh.org In-Reply-To: Message-Id: <42130AB1-3B57-4127-9550-1CF2FF0D5857@kba.biglobe.ne.jp> X-Mailer: Apple Mail (2.3445.104.21) X-Biglobe-Sender: takimoto-j@kba.biglobe.ne.jp X-Seq: 49999 Archived-At: X-Loop: zsh-workers@zsh.org Errors-To: zsh-workers-owner@zsh.org Precedence: list Precedence: bulk Sender: zsh-workers-request@zsh.org X-no-archive: yes List-Id: List-Help: List-Subscribe: List-Unsubscribe: List-Post: List-Owner: List-Archive: > 2022/04/06 5:29, Peter Stephenson wrote: >=20 > This might help? I've been preparing basically the same patch. The only differences are: use (( $ZTST_continue )) so that "ZTST_continue=3D0 make check" will = work, rename "failures" --> "ZTST_failures" to avoid possible (future) = conflicts. While reading ztst.zsh I've noticed that the return value of = ZTST_prepclean is not checked. But the comment in B01cd.ztst says: %prep # ... each chunk of code is evaluated # in one go and must return status 0, or the preparation is deemed to = have # failed and the test ends with an appropriate error message.=20 Moreover, I couldn't understand the logic in ZTST_prepclean(): while ZTST_getchunk; do =20 ZTST_execchunk >/dev/null || [[ -n $1 ]] || { =20 [[ -n "$ZTST_unimplemented" ]] || ZTST_testfailed "non-zero status from preparation code: =20 $ZTST_code" && return 0 =20 } =20 done Since ZTST_testfailed() always returns 1, the "return 0" is never = executed? But the "return 0" is not required here; instead, it would be better to "return 1" if a chunk in %prep fails. # ZTST_unimplemented is taken care of in the main while loop (in which # ZTST_prepclean is called) and we need not check it here, I think. With the patch below: (1) if a chunk in %prep fails then the test is aborted immediately. (2) If the "bad placement of 'test' section" happens, we do not exit = immediately but run ZTST_cleanup. Or should we run ZTST_cleanup even in (1)? (it will do no harm...) diff --git a/Test/ztst.zsh b/Test/ztst.zsh index 89fe69b5b..e66db8f91 100755 --- a/Test/ztst.zsh +++ b/Test/ztst.zsh @@ -17,6 +17,9 @@ # Defined in such a way that any value from the environment is used. : ${ZTST_verbose:=3D0} =20 +# If non-zero, continue the tests even after a test fails. +: ${ZTST_continue:=3D0} + # We require all options to be reset, not just emulation options. # Unfortunately, due to the crud which may be in /etc/zshenv this might # still not be good enough. Maybe we should trick it somehow. @@ -143,6 +146,10 @@ ZTST_testfailed() { $ZTST_failmsg" fi ZTST_testfailed=3D1 + # if called from within ZTST_Test() this will increment ZTST_Test's = local + # ZTST_failures. Otherwise global ZTST_failures will be incremented + # (but currently its value is not used). + (( ++ZTST_failures )) return 1 } ZTST_testxpassed() { @@ -156,6 +163,7 @@ ZTST_testxpassed() { $ZTST_failmsg" fi ZTST_testfailed=3D1 + (( ++ZTST_failures )) return 1 } =20 @@ -291,16 +299,17 @@ ZTST_execchunk() { } =20 # Functions for preparation and cleaning. -# When cleaning up (non-zero string argument), we ignore status. -ZTST_prepclean() { +# When cleaning up, we ignore status. +ZTST_prep ZTST_clean () { # Execute indented code chunks. while ZTST_getchunk; do - ZTST_execchunk >/dev/null || [[ -n $1 ]] || { - [[ -n "$ZTST_unimplemented" ]] || + ZTST_execchunk >/dev/null || [[ $0 =3D ZTST_clean ]] || { ZTST_testfailed "non-zero status from preparation code: -$ZTST_code" && return 0 +$ZTST_code" + return 1 } done + return 0 } =20 # diff wrapper @@ -373,12 +382,12 @@ ZTST_diff() { =20 return "$diff_ret" } - =20 + ZTST_test() { local last match mbegin mend found substlines local diff_out diff_err local ZTST_skip - integer expected_to_fail + integer expected_to_fail ZTST_failures =20 while true; do rm -f $ZTST_in $ZTST_out $ZTST_err @@ -492,7 +501,7 @@ $ZTST_curline" $ZTST_code${$(<$ZTST_terr):+ Error output: $(<$ZTST_terr)}" - return 1 + if (( ZTST_continue ));then continue; else return 1; fi fi =20 ZTST_verbose 2 "ZTST_test: test produced standard output: @@ -515,7 +524,7 @@ $(<$ZTST_terr)" $ZTST_code${$(<$ZTST_terr):+ Error output: $(<$ZTST_terr)}" - return 1 + if (( ZTST_continue ));then continue; else return 1; fi fi if [[ $ZTST_flags =3D *q* && -s $ZTST_err ]]; then substlines=3D"$(<$ZTST_err)" @@ -529,21 +538,27 @@ $(<$ZTST_terr)}" fi ZTST_testfailed "error output differs from expected as shown = above for: $ZTST_code" - return 1 + if (( ZTST_continue ));then continue; else return 1; fi fi if (( expected_to_fail )); then ZTST_testxpassed - return 1 + if (( ZTST_continue ));then continue; else return 1; fi fi fi ZTST_verbose 1 "Test successful." [[ -n $last ]] && break done =20 - ZTST_verbose 2 "ZTST_test: all tests successful" + if (( ZTST_failures )); then + ZTST_verbose 1 "ZTST_test: $ZTST_failures tests failed" + else + ZTST_verbose 2 "ZTST_test: all tests successful" + fi =20 # reset message to keep ZTST_testfailed output correct ZTST_message=3D'' + + return ZTST_failures } =20 =20 @@ -565,25 +580,27 @@ while [[ -z "$ZTST_unimplemented" ]] && = ZTST_getsect $ZTST_skipok; do ZTST_testfailed "\`prep' section must come first" exit 1 fi - ZTST_prepclean + ZTST_prep || ZTST_skipok=3D1 ZTST_sects[prep]=3D1 ;; (test) if (( ${ZTST_sects[test]} + ${ZTST_sects[clean]} )); then ZTST_testfailed "bad placement of \`test' section" - exit 1 + break # do not run %clean section, but run ZTST_cleanup fi - # careful here: we can't execute ZTST_test before || or && - # because that affects the behaviour of traps in the tests. - ZTST_test - (( $? )) && ZTST_skipok=3D1 + if [[ -z "$ZTST_skipok" ]]; then # if no error in %prep + # careful here: we can't execute ZTST_test before || or && + # because that affects the behaviour of traps in the tests. + ZTST_test + (( $? )) && ZTST_skipok=3D1 + fi ZTST_sects[test]=3D1 ;; (clean) if (( ${ZTST_sects[test]} =3D=3D 0 || ${ZTST_sects[clean]} = )); then ZTST_testfailed "bad use of \`clean' section" else - ZTST_prepclean 1 + ZTST_clean ZTST_sects[clean]=3D1 fi ZTST_skipok=3D