From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-3.4 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FROM,HTML_MESSAGE,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 26903 invoked from network); 9 Nov 2022 11:13:01 -0000 Received: from zero.zsh.org (2a02:898:31:0:48:4558:7a:7368) by inbox.vuxu.org with ESMTPUTF8; 9 Nov 2022 11:13:01 -0000 ARC-Seal: i=1; cv=none; a=rsa-sha256; d=zsh.org; s=rsa-20210803; t=1667992381; b=Kbfssjipne6mIbWAcOHjgm/FCgUHpx31vylKO/Xr9hn/dB2FNAdcON8VDIOzoOGMrgsLxMlC6i ZZKpntWeIgFOhjKjMKBrMPEtCh8YohhB3z5K4EsuNJ/l/4J9w3fQoiHH2zzrx41Of7hKC5n/z3 twCsEPcmeJsSxL3wYY8WrJXpuJAp5P4v3q2lbCWoei3b7mQ+1/V1VjcH5mjpogSPm/sDi4AfYp yveDWJ97NelII70+AQpiVzIdPhyeQeyG4lQEe/IxtF2XY3sS/I15tHzwz3U/DBi1AwkY6pEAwg qAFk+G1OP/EA4g/9hQEJ5SEtXjeW9yojKT+iWG1VFiYKfA==; ARC-Authentication-Results: i=1; zsh.org; iprev=pass (mail-pf1-f174.google.com) smtp.remote-ip=209.85.210.174; dkim=pass header.d=gmail.com header.s=20210112 header.a=rsa-sha256; dmarc=pass header.from=gmail.com; arc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed; d=zsh.org; s=rsa-20210803; t=1667992381; bh=RkIMaeieri6Ic1bCxITssKW8ZD1JHrMAPJqGrmtmcQ8=; h=List-Archive:List-Owner:List-Post:List-Unsubscribe:List-Subscribe:List-Help: List-Id:Sender:Content-Type:Cc:To:Subject:Message-ID:Date:From:In-Reply-To: References:MIME-Version:DKIM-Signature:DKIM-Signature; b=WgJY2uVzW+6654i3hy5Q3xnSjidVJyxCZioM36X0vTDDtEvdxYLEgGZ54HqVSPYQ9Nwbf6+ody l2LruszVoCOhR9dCr/DjvtvfxPJFTe06wxXbJci7zThdc/Cip4Lz3A8vkkDb59W+qddueWV0Ns SfphFOrepNVJOE1Rgv4qLd8+ZD8joBI7qbU3Z5YzmwWWVJ/+gPTR519vAO7UboDwJPyjCSS+Tc PrC/MzxIRL7A42de8TINhmf2q02kWdeBkuzYgz/T7maF/aNS+9dyy7LRwFn9ZrPEb3yIVDT1b0 v+8Vgv/qkX48/PoPUA3QWw9LbYrqq7U201cZGk5lRf88YQ==; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=zsh.org; s=rsa-20210803; h=List-Archive:List-Owner:List-Post:List-Unsubscribe: List-Subscribe:List-Help:List-Id:Sender:Content-Type:Cc:To:Subject:Message-ID :Date:From:In-Reply-To:References:MIME-Version:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID; bh=GQQtuGKu8Ui6QfJMA/PfTFJ3GsA+Drhic+oo5tknbhU=; b=NgJGqRmQ6p8YC3/rKU4bZ/Tfnm gSkk0apBKyFebYIO3F0uDcSYd1U01cnb3rZpDdfcI155I1EU8JsCJftv1aD944e1vqy9eYTpAFwGM ibL8E33kQqc48z35slLoGULooiKIqIKRCgRjmQK8ORwDBbaCmBcJOoHMh1EtICKDHUsMY3uVpvTl0 GQPQLbtGV9Ml+pFqH/ARb80B/obF/RLimSjmWxLkfpuLH+YcCB5VIv0zk73FP/iNKw/4j6vJI9uQs rSPP/F1shIUIXHzm/DhfonHL8/RebwQ0lsR1S4UYc+surzoACC/m5ixRzVd72CVWuKbwopG9i5l2w 7PkSxWjA==; Received: by zero.zsh.org with local id 1osj0u-000Kka-NK; Wed, 09 Nov 2022 11:13:00 +0000 Authentication-Results: zsh.org; iprev=pass (mail-pf1-f174.google.com) smtp.remote-ip=209.85.210.174; dkim=pass header.d=gmail.com header.s=20210112 header.a=rsa-sha256; dmarc=pass header.from=gmail.com; arc=none Received: from mail-pf1-f174.google.com ([209.85.210.174]:36461) by zero.zsh.org with esmtps (TLS1.3:TLS_AES_128_GCM_SHA256:128) id 1osj09-000K1i-Rj; Wed, 09 Nov 2022 11:12:14 +0000 Received: by mail-pf1-f174.google.com with SMTP id k22so16385846pfd.3 for ; Wed, 09 Nov 2022 03:12:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=GQQtuGKu8Ui6QfJMA/PfTFJ3GsA+Drhic+oo5tknbhU=; b=GGEtYmCd4D9PX/9L1NsWzEHaiubMKx2aD41RfDQUlUQAtYtbGgd050j2zBtQNiniKR vMcmNqvLgL5bF6qe5B/PKO1/Wxkm3gVXwBbiJNBukWBxjJu6WCNhUohvYljrEvRQ4J2E O+8x/4uPiFmCY1+4L18JLgvhJVRMZdtYGlzbhMCfqx6tj+MBN4EX1o7hcycCfO+J0iah 6T69/INU23KRljE/tdQuXH7Xs5lUkPmY2LOJGIA/OS8t+kgalx8sx9B+dQa8sadI3NHM V+bcoQsAEpMKbURSctvapVXBenUq0k8e/J9hfRTSuwHpgePcoGftMJOv/DBOtqYpkROC S+rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=GQQtuGKu8Ui6QfJMA/PfTFJ3GsA+Drhic+oo5tknbhU=; b=I/vSGO/uOm8y9ziwPoMaOvbluHEIxjdKbJl0jMxYxQ64oEiQhI0fHGK1K52eLsnIJV Mz4BOM91G3nTpZSGVMv/Di0Q291DlDgMgSg1WzSlogGs61wV9d36rD+YGN2+KiTpgN7u z0i0flyBFQWn2C4h1SVc3TxLGf8sg/4UFvmNNwu1azhi1WOK9qU6Vi8dKilRV5JoSQEQ RU9aaWKeRKaphSNqv1/xq6o4oFuVPUS01qN2/otGE4bcDPCFjkgzkM9sjuSHLbUIBYu7 o2ZnFl7pWwZnlKQQ+tgOuHSzb/OnPkig6kTuh9IwipUXDNaPeI8LeH8U1bGx8Lq5Vnk2 qrDw== X-Gm-Message-State: ACrzQf0/RPtlUG0z5XZ9mE6sMwdR3AHKJFmjEcV6kM0EQ7abc/gk+6iT piWYKFwvV8Uk2wIvwziu8MlnC3vayvBB2Pq0XXIpD7fm X-Google-Smtp-Source: AMsMyM5jqeBaCkU8A1js5dCOTKBYI3bkmPAz5c2JXH0jZWQvuA8QI9eVcXg8SqjSm0BWgzITRq8b08lUWXpLPwWf0LM= X-Received: by 2002:a05:6a00:234d:b0:561:f0c3:cde1 with SMTP id j13-20020a056a00234d00b00561f0c3cde1mr60416961pfj.34.1667992332022; Wed, 09 Nov 2022 03:12:12 -0800 (PST) MIME-Version: 1.0 References: <1113483502.4632116.1667911158058@mail.virginmedia.com> In-Reply-To: <1113483502.4632116.1667911158058@mail.virginmedia.com> From: Pier Paolo Grassi Date: Wed, 9 Nov 2022 12:11:35 +0100 Message-ID: Subject: Re: while alias To: Peter Stephenson Cc: Zsh-Users List Content-Type: multipart/alternative; boundary="00000000000070398305ed07ba4e" X-Seq: 28350 Archived-At: X-Loop: zsh-users@zsh.org Errors-To: zsh-users-owner@zsh.org Precedence: list Precedence: bulk Sender: zsh-users-request@zsh.org X-no-archive: yes List-Id: List-Help: , List-Subscribe: , List-Unsubscribe: , List-Post: List-Owner: List-Archive: --00000000000070398305ed07ba4e Content-Type: text/plain; charset="UTF-8" thanks the space before the brace indeed solved it. good to know! Pier Paolo Grassi Il giorno mar 8 nov 2022 alle ore 13:40 Peter Stephenson < p.w.stephenson@ntlworld.com> ha scritto: > (This is really for zsh-workers but in case anyone wants to see the > result...) > > > On 08/11/2022 06:13 Bart Schaefer wrote: > > On Mon, Nov 7, 2022 at 5:57 PM Pier Paolo Grassi > wrote: > > > > > > Hello, I have the following alias defined: > > > > > > WI='while {read -r it; ! [[ $? -ne 0 && -z $it ]]}' > > > > > > that works fine except in command substitution: > > > > Put spaces around your braces. > > > > alias WI='while { read -r it; ! [[ $? -ne 0 && -z $it ]] }' > > > > Aliases in command substitutions are parsed by a complicated dance > > requiring that they be expanded and then backtracked over and then > > re-parsed. The character count for the "{" and/or "}" tokens is off > > by one for the backtracking step when they don't have surrounding > > whitespace. (Leaving it to PWS, who choreographed this dance number, > > to fix the footwork if possible.) > > Yes, it's that full combination of a command substitution, an > alias, and expansion text with a lexically significant closing brace > with no preceding space that's doing it, so it's very specific. > > This will come as a shock, but there's a hack in the lexical analyser! > (Counselling will be available by the usual channels.) It seems > that the lexer really doesn't like closing braces without a space > before them and sometimes we have to fix up the text after the event. > > It seems the extra special code for command substitution needs a > slightly modified version of the hack. The new test in D08 checks > the case Pier Paolo hit; it turns out the existing test just above > checks the case where we do after all need to unget an extra closing > brace, i.e. lex_add_raw ends up 1 in the code added by the first hunk. > > I am not going to attempt to get my mind round this any further > and will be going out for some fresh air. > > pws > > diff --git a/Src/lex.c b/Src/lex.c > index ece02659e..e2f8bcfb1 100644 > --- a/Src/lex.c > +++ b/Src/lex.c > @@ -1429,10 +1429,18 @@ gettokstr(int c, int sub) > peek == STRING && lexbuf.ptr[-1] == '}' && > lexbuf.ptr[-2] != Bnull) { > /* hack to get {foo} command syntax work */ > + /* > + * Alias expansion when parsing command substitution means that > + * the case for raw lexical analysis may not be the same. > + * (Just go with it, OK?) > + */ > + int lar = lex_add_raw; > + lex_add_raw = lexbuf_raw.len > 0 && lexbuf_raw.ptr[-1] == '}'; > lexbuf.ptr--; > lexbuf.len--; > lexstop = 0; > hungetc('}'); > + lex_add_raw = lar; > } > *lexbuf.ptr = '\0'; > DPUTS(cmdsp != ocmdsp, "BUG: gettok: cmdstack changed."); > diff --git a/Test/D08cmdsubst.ztst b/Test/D08cmdsubst.ztst > index 04bf698aa..8b6bcb469 100644 > --- a/Test/D08cmdsubst.ztst > +++ b/Test/D08cmdsubst.ztst > @@ -177,3 +177,11 @@ > 0:Alias expansion needed in parsing substitutions > >hi > >bye > + > +# This should silently print a blank line; the original problem was > +# a parse error as the last character of the unexpanded alias > +# was erased, symptom: "command not found: W" > + alias WI='while {false}' > + eval 'echo $(WI blah)' > +0:Aliases with braces in command substitution can cause havoc > +> > > --00000000000070398305ed07ba4e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
thanks the space before the brace indeed solved it. good t= o know!

Pier Paolo Grassi


Il giorno mar 8 nov 2022 alle ore 13:40 Peter Stephe= nson <p.w.stephenson@ntlw= orld.com> ha scritto:
(This is really for zsh-workers but in case anyone wants to se= e the result...)

> On 08/11/2022 06:13 Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Mon, Nov 7, 2022 at 5:57 PM Pier Paolo Grassi <pierpaolog@gmail.com> wrote= :
> >
> > Hello, I have the following alias defined:
> >
> > WI=3D'while {read -r it; ! [[ $? -ne 0 && -z $it ]]}&= #39;
> >
> > that works fine except in command substitution:
>
> Put spaces around your braces.
>
> alias WI=3D'while { read -r it; ! [[ $? -ne 0 && -z $it ]]= }'
>
> Aliases in command substitutions are parsed by a complicated dance
> requiring that they be expanded and then backtracked over and then
> re-parsed.=C2=A0 The character count for the "{" and/or &quo= t;}" tokens is off
> by one for the backtracking step when they don't have surrounding<= br> > whitespace.=C2=A0 (Leaving it to PWS, who choreographed this dance num= ber,
> to fix the footwork if possible.)

Yes, it's that full combination of a command substitution, an
alias, and expansion text with a lexically significant closing brace
with no preceding space that's doing it, so it's very specific.

This will come as a shock, but there's a hack in the lexical analyser!<= br> (Counselling will be available by the usual channels.)=C2=A0 It seems
that the lexer really doesn't like closing braces without a space
before them and sometimes we have to fix up the text after the event.

It seems the extra special code for command substitution needs a
slightly modified version of the hack.=C2=A0 The new test in D08 checks
the case Pier Paolo hit; it turns out the existing test just above
checks the case where we do after all need to unget an extra closing
brace, i.e. lex_add_raw ends up 1 in the code added by the first hunk.

I am not going to attempt to get my mind round this any further
and will be going out for some fresh air.

pws

diff --git a/Src/lex.c b/Src/lex.c
index ece02659e..e2f8bcfb1 100644
--- a/Src/lex.c
+++ b/Src/lex.c
@@ -1429,10 +1429,18 @@ gettokstr(int c, int sub)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0peek =3D=3D STRING &= amp;& lexbuf.ptr[-1] =3D=3D '}' &&
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0lexbuf.ptr[-2] !=3D = Bnull) {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 /* hack to get {foo} command syntax work */
+=C2=A0 =C2=A0 =C2=A0 =C2=A0/*
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 * Alias expansion when parsing command substit= ution means that
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 * the case for raw lexical analysis may not be= the same.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 * (Just go with it, OK?)
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 */
+=C2=A0 =C2=A0 =C2=A0 =C2=A0int lar =3D lex_add_raw;
+=C2=A0 =C2=A0 =C2=A0 =C2=A0lex_add_raw =3D lexbuf_raw.len > 0 &&= ; lexbuf_raw.ptr[-1] =3D=3D '}';
=C2=A0 =C2=A0 =C2=A0 =C2=A0 lexbuf.ptr--;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 lexbuf.len--;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 lexstop =3D 0;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 hungetc('}');
+=C2=A0 =C2=A0 =C2=A0 =C2=A0lex_add_raw =3D lar;
=C2=A0 =C2=A0 =C2=A0}
=C2=A0 =C2=A0 =C2=A0*lexbuf.ptr =3D '\0';
=C2=A0 =C2=A0 =C2=A0DPUTS(cmdsp !=3D ocmdsp, "BUG: gettok: cmdstack ch= anged.");
diff --git a/Test/D08cmdsubst.ztst b/Test/D08cmdsubst.ztst
index 04bf698aa..8b6bcb469 100644
--- a/Test/D08cmdsubst.ztst
+++ b/Test/D08cmdsubst.ztst
@@ -177,3 +177,11 @@
=C2=A00:Alias expansion needed in parsing substitutions
=C2=A0>hi
=C2=A0>bye
+
+# This should silently print a blank line; the original problem was
+# a parse error as the last character of the unexpanded alias
+# was erased, symptom: "command not found: W"
+=C2=A0 alias WI=3D'while {false}'
+=C2=A0 eval 'echo $(WI blah)'
+0:Aliases with braces in command substitution can cause havoc
+>

--00000000000070398305ed07ba4e--