while alias

while alias

[Pier Paolo Grassi]

[-- Attachment #1: Type: text/plain, Size: 504 bytes --]

Hello, I have the following alias defined:

WI='while {read -r it; ! [[ $? -ne 0 && -z $it ]]}'

that works fine except in command substitution:

a=$( echo 1 | WI { echo $it } )
zsh: parse error near `}'
zsh: parse error in command substitution

if I substitute the alias with the aliased code all works fine:

a=$( echo 1 | while {read -r it; ! [[ $? -ne 0 && -z $it ]]} { echo $it }
); echo $a
1

alias are normally correctly substituted inside command substitution, what
can it be?

Pier Paolo Grassi

[-- Attachment #2: Type: text/html, Size: 928 bytes --]

Re: while alias

[Bart Schaefer]

On Mon, Nov 7, 2022 at 5:57 PM Pier Paolo Grassi <pierpaolog@gmail.com> wrote:
>
> Hello, I have the following alias defined:
>
> WI='while {read -r it; ! [[ $? -ne 0 && -z $it ]]}'
>
> that works fine except in command substitution:

Put spaces around your braces.

alias WI='while { read -r it; ! [[ $? -ne 0 && -z $it ]] }'

Aliases in command substitutions are parsed by a complicated dance
requiring that they be expanded and then backtracked over and then
re-parsed.  The character count for the "{" and/or "}" tokens is off
by one for the backtracking step when they don't have surrounding
whitespace.  (Leaving it to PWS, who choreographed this dance number,
to fix the footwork if possible.)

Re: while alias

[Peter Stephenson]

(This is really for zsh-workers but in case anyone wants to see the result...)

> On 08/11/2022 06:13 Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Mon, Nov 7, 2022 at 5:57 PM Pier Paolo Grassi <pierpaolog@gmail.com> wrote:
> >
> > Hello, I have the following alias defined:
> >
> > WI='while {read -r it; ! [[ $? -ne 0 && -z $it ]]}'
> >
> > that works fine except in command substitution:
> 
> Put spaces around your braces.
> 
> alias WI='while { read -r it; ! [[ $? -ne 0 && -z $it ]] }'
> 
> Aliases in command substitutions are parsed by a complicated dance
> requiring that they be expanded and then backtracked over and then
> re-parsed.  The character count for the "{" and/or "}" tokens is off
> by one for the backtracking step when they don't have surrounding
> whitespace.  (Leaving it to PWS, who choreographed this dance number,
> to fix the footwork if possible.)

Yes, it's that full combination of a command substitution, an
alias, and expansion text with a lexically significant closing brace
with no preceding space that's doing it, so it's very specific.

This will come as a shock, but there's a hack in the lexical analyser!
(Counselling will be available by the usual channels.)  It seems
that the lexer really doesn't like closing braces without a space
before them and sometimes we have to fix up the text after the event.

It seems the extra special code for command substitution needs a
slightly modified version of the hack.  The new test in D08 checks
the case Pier Paolo hit; it turns out the existing test just above
checks the case where we do after all need to unget an extra closing
brace, i.e. lex_add_raw ends up 1 in the code added by the first hunk.

I am not going to attempt to get my mind round this any further
and will be going out for some fresh air.

pws

diff --git a/Src/lex.c b/Src/lex.c
index ece02659e..e2f8bcfb1 100644
--- a/Src/lex.c
+++ b/Src/lex.c
@@ -1429,10 +1429,18 @@ gettokstr(int c, int sub)
 	       peek == STRING && lexbuf.ptr[-1] == '}' &&
 	       lexbuf.ptr[-2] != Bnull) {
 	/* hack to get {foo} command syntax work */
+	/*
+	 * Alias expansion when parsing command substitution means that
+	 * the case for raw lexical analysis may not be the same.
+	 * (Just go with it, OK?)
+	 */
+	int lar = lex_add_raw;
+	lex_add_raw = lexbuf_raw.len > 0 && lexbuf_raw.ptr[-1] == '}';
 	lexbuf.ptr--;
 	lexbuf.len--;
 	lexstop = 0;
 	hungetc('}');
+	lex_add_raw = lar;
     }
     *lexbuf.ptr = '\0';
     DPUTS(cmdsp != ocmdsp, "BUG: gettok: cmdstack changed.");
diff --git a/Test/D08cmdsubst.ztst b/Test/D08cmdsubst.ztst
index 04bf698aa..8b6bcb469 100644
--- a/Test/D08cmdsubst.ztst
+++ b/Test/D08cmdsubst.ztst
@@ -177,3 +177,11 @@
 0:Alias expansion needed in parsing substitutions
 >hi
 >bye
+
+# This should silently print a blank line; the original problem was
+# a parse error as the last character of the unexpanded alias
+# was erased, symptom: "command not found: W"
+  alias WI='while {false}'
+  eval 'echo $(WI blah)'
+0:Aliases with braces in command substitution can cause havoc
+>

Re: while alias

[Pier Paolo Grassi]

[-- Attachment #1: Type: text/plain, Size: 3500 bytes --]

thanks the space before the brace indeed solved it. good to know!

Pier Paolo Grassi


Il giorno mar 8 nov 2022 alle ore 13:40 Peter Stephenson <
p.w.stephenson@ntlworld.com> ha scritto:

> (This is really for zsh-workers but in case anyone wants to see the
> result...)
>
> > On 08/11/2022 06:13 Bart Schaefer <schaefer@brasslantern.com> wrote:
> > On Mon, Nov 7, 2022 at 5:57 PM Pier Paolo Grassi <pierpaolog@gmail.com>
> wrote:
> > >
> > > Hello, I have the following alias defined:
> > >
> > > WI='while {read -r it; ! [[ $? -ne 0 && -z $it ]]}'
> > >
> > > that works fine except in command substitution:
> >
> > Put spaces around your braces.
> >
> > alias WI='while { read -r it; ! [[ $? -ne 0 && -z $it ]] }'
> >
> > Aliases in command substitutions are parsed by a complicated dance
> > requiring that they be expanded and then backtracked over and then
> > re-parsed.  The character count for the "{" and/or "}" tokens is off
> > by one for the backtracking step when they don't have surrounding
> > whitespace.  (Leaving it to PWS, who choreographed this dance number,
> > to fix the footwork if possible.)
>
> Yes, it's that full combination of a command substitution, an
> alias, and expansion text with a lexically significant closing brace
> with no preceding space that's doing it, so it's very specific.
>
> This will come as a shock, but there's a hack in the lexical analyser!
> (Counselling will be available by the usual channels.)  It seems
> that the lexer really doesn't like closing braces without a space
> before them and sometimes we have to fix up the text after the event.
>
> It seems the extra special code for command substitution needs a
> slightly modified version of the hack.  The new test in D08 checks
> the case Pier Paolo hit; it turns out the existing test just above
> checks the case where we do after all need to unget an extra closing
> brace, i.e. lex_add_raw ends up 1 in the code added by the first hunk.
>
> I am not going to attempt to get my mind round this any further
> and will be going out for some fresh air.
>
> pws
>
> diff --git a/Src/lex.c b/Src/lex.c
> index ece02659e..e2f8bcfb1 100644
> --- a/Src/lex.c
> +++ b/Src/lex.c
> @@ -1429,10 +1429,18 @@ gettokstr(int c, int sub)
>                peek == STRING && lexbuf.ptr[-1] == '}' &&
>                lexbuf.ptr[-2] != Bnull) {
>         /* hack to get {foo} command syntax work */
> +       /*
> +        * Alias expansion when parsing command substitution means that
> +        * the case for raw lexical analysis may not be the same.
> +        * (Just go with it, OK?)
> +        */
> +       int lar = lex_add_raw;
> +       lex_add_raw = lexbuf_raw.len > 0 && lexbuf_raw.ptr[-1] == '}';
>         lexbuf.ptr--;
>         lexbuf.len--;
>         lexstop = 0;
>         hungetc('}');
> +       lex_add_raw = lar;
>      }
>      *lexbuf.ptr = '\0';
>      DPUTS(cmdsp != ocmdsp, "BUG: gettok: cmdstack changed.");
> diff --git a/Test/D08cmdsubst.ztst b/Test/D08cmdsubst.ztst
> index 04bf698aa..8b6bcb469 100644
> --- a/Test/D08cmdsubst.ztst
> +++ b/Test/D08cmdsubst.ztst
> @@ -177,3 +177,11 @@
>  0:Alias expansion needed in parsing substitutions
>  >hi
>  >bye
> +
> +# This should silently print a blank line; the original problem was
> +# a parse error as the last character of the unexpanded alias
> +# was erased, symptom: "command not found: W"
> +  alias WI='while {false}'
> +  eval 'echo $(WI blah)'
> +0:Aliases with braces in command substitution can cause havoc
> +>
>
>

[-- Attachment #2: Type: text/html, Size: 4716 bytes --]