[[ is being treated as a pattern in the command/reserved word position.

[[ is being treated as a pattern in the command/reserved word position.

[Eric Cook]

With a recent build, this happens to me. Does it happen for anyone else?

% type [[
zsh: bad pattern: [[
% type '[['
[[ is a reserved word
% '[[' a == a ]]
zsh: = not found
% '[[' a '==' a ]]
zsh: command not found: [[
% print $ZSH_PATCHLEVEL
zsh-5.0.7-0-g208bded

Re: [[ is being treated as a pattern in the command/reserved word position.

[Eric Cook]

On 03/28/2015 05:08 PM, Eric Cook wrote:
> With a recent build, this happens to me. Does it happen for anyone else?
>
> % type [[
> zsh: bad pattern: [[
> % type '[['
> [[ is a reserved word
> % '[[' a == a ]]
> zsh: = not found
> % '[[' a '==' a ]]
> zsh: command not found: [[
> % print $ZSH_PATCHLEVEL
> zsh-5.0.7-0-g208bded
>
forgot to paste the line of trying [[ normally, but that also results in
a bad pattern error.

Re: [[ is being treated as a pattern in the command/reserved word position.

[Eric Cook]

Triple post :D
the patchlevel is actually zsh-5.0.7-362-gab40656 not zsh-5.0.7-0-g208bded

Re: [[ is being treated as a pattern in the command/reserved word position.

[ZyX]

29.03.2015, 00:14, "Eric Cook" <llua@gmx.com>:
> With a recent build, this happens to me. Does it happen for anyone else?
>
> % type [[
> zsh: bad pattern: [[

This is expected: you open a collection, but do not close it.

> % type '[['
> [[ is a reserved word
> % '[[' a == a ]]
> zsh: = not found

`[[` is *not* a command. Neither it is a shell builtin. It is an extension to the shell syntax. So this is expected as well. You see exactly this error message because `=tty` expands to `/usr/bin/tty`.

> % '[[' a '==' a ]]
> zsh: command not found: [[

Same, but you got rid of `==` error.

> % print $ZSH_PATCHLEVEL
> zsh-5.0.7-0-g208bded

`[[` is a *syntax extension* which works *only* if `[[` (literally) occurs in command position. In first commented case you violate “in command position condition”, in other commented cases you violate “literally”.

Re: [[ is being treated as a pattern in the command/reserved word position.

[Bart Schaefer]

On Mar 28,  5:08pm, Eric Cook wrote:
}
} With a recent build, this happens to me. Does it happen for anyone else?

"A recent build"?  This happens to me with zsh-3.0.5.  Can you find ANY
build of zsh where this DOESN'T happen?

} % type [[
} zsh: bad pattern: [[

torch% setopt nobadpattern
torch% type [[
[[ is a reserved word

} % '[[' a == a ]]
} zsh: = not found

torch% setopt noequals
torch% '[[' a == a ]]
zsh: command not found: [[

Re: [[ is being treated as a pattern in the command/reserved word position.

[Eric Cook]

On 03/28/2015 05:44 PM, Bart Schaefer wrote:
> On Mar 28,  5:08pm, Eric Cook wrote:
> }
> } With a recent build, this happens to me. Does it happen for anyone else?
>
Yea, those wasn't the best choices of examples. But i did narrow it down
a bit.

% print $ZSH_PATCHLEVEL; emulate sh -c '[[ a == a ]]'
zsh-5.0.7-362-gab40656
zsh: command not found: [[

Did that behavior change?

I ask because with 5.0.7 the following happens:

% print $ZSH_PATCHLEVEL; emulate sh -c '[[ a == a ]]'
zsh-5.0.7-0-g208bded

Re: [[ is being treated as a pattern in the command/reserved word position.

[Bart Schaefer]

On Mar 28,  5:55pm, Eric Cook wrote:
}
} % print $ZSH_PATCHLEVEL; emulate sh -c '[[ a == a ]]'
} zsh-5.0.7-362-gab40656
} zsh: command not found: [[
} 
} Did that behavior change?

Aha.  That makes more sense.

The '[[' reserved word is handled a special kind of built-in alias [*]
and the change to POSIX_ALIASES handling caused that to be disabled.

This will take a bit of thought.


[*] Not literally, but the alias expansion code is where '[[' is noted
and the lexer changed into "parsing a conditional" state.

Re: [[ is being treated as a pattern in the command/reserved word position.

[ZyX]

29.03.2015, 01:25, "Bart Schaefer" <schaefer@brasslantern.com>:
> On Mar 28,  5:55pm, Eric Cook wrote:
> }
> } % print $ZSH_PATCHLEVEL; emulate sh -c '[[ a == a ]]'
> } zsh-5.0.7-362-gab40656
> } zsh: command not found: [[
> }
> } Did that behavior change?
>
> Aha.  That makes more sense.
>
> The '[[' reserved word is handled a special kind of built-in alias [*]
> and the change to POSIX_ALIASES handling caused that to be disabled.
>
> This will take a bit of thought.
>
> [*] Not literally, but the alias expansion code is where '[[' is noted
> and the lexer changed into "parsing a conditional" state.

I would say that this is actually an expected behaviour: `posh -c '[[ a == a ]]'` will show `posh: [[: not found` because `[[` is not in POSIX. Similar error will be shown by dash.

Re: [[ is being treated as a pattern in the command/reserved word position.

[Bart Schaefer]

On Mar 29,  1:33am, ZyX wrote:
} Subject: Re: [[ is being treated as a pattern in the command/reserved word
}
} 29.03.2015, 01:25, "Bart Schaefer" <schaefer@brasslantern.com>:
} >
} > The '[[' reserved word is handled a special kind of built-in alias [*]
} > and the change to POSIX_ALIASES handling caused that to be disabled.
} 
} I would say that this is actually an expected behaviour:
} `posh -c '[[ a == a ]]'` will show `posh: [[: not found`
} because `[[` is not in POSIX. Similar error will be shown by dash.

That could be, but POSIX_ALIASES is the wrong way to handle that.  If we
wanted to remove the '[[' extension for posix emulation, we should do
`disable -r "[["` instead.


diff --git a/Src/lex.c b/Src/lex.c
index 4d8355b..2e61170 100644
--- a/Src/lex.c
+++ b/Src/lex.c
@@ -1794,9 +1794,6 @@ exalias(void)
 	if (has_token(tokstr)) {
 	    char *p, *t;
 
-	    if (isset(POSIXALIASES))
-		return 0;
-
 	    zshlextext = p = copy;
 	    for (t = tokstr;
 		 (*p++ = itok(*t) ? ztokens[*t++ - Pound] : *t++););
@@ -1816,7 +1813,7 @@ exalias(void)
 
 	if (tok == STRING) {
 	    /* Check for an alias */
-	    if (checkalias()) {
+	    if ((zshlextext != copy || !isset(POSIXALIASES)) && checkalias()) {
 		if (zshlextext == copy)
 		    zshlextext = tokstr;
 		return 1;
diff --git a/Test/B07emulate.ztst b/Test/B07emulate.ztst
index 315206a..2de097e 100644
--- a/Test/B07emulate.ztst
+++ b/Test/B07emulate.ztst
@@ -247,3 +247,7 @@
 >extendedglob is initially off
 >bareglobqual is still on
 >extendedglob is on, too
+
+ emulate sh -c '[[ a == a ]]'
+0:regression test for POSIX_ALIASES reserved words
+F:Some reserved tokens are handled in alias expansion

Re: [[ is being treated as a pattern in the command/reserved word position.

[Eric Cook]

On 03/28/2015 06:33 PM, ZyX wrote:
> 29.03.2015, 01:25, "Bart Schaefer" <schaefer@brasslantern.com>:
>> On Mar 28,  5:55pm, Eric Cook wrote:
>> }
>> } % print $ZSH_PATCHLEVEL; emulate sh -c '[[ a == a ]]'
>> } zsh-5.0.7-362-gab40656
>> } zsh: command not found: [[
>> }
>> } Did that behavior change?
>>
>> Aha.  That makes more sense.
>>
>> The '[[' reserved word is handled a special kind of built-in alias [*]
>> and the change to POSIX_ALIASES handling caused that to be disabled.
>>
>> This will take a bit of thought.
>>
>> [*] Not literally, but the alias expansion code is where '[[' is noted
>> and the lexer changed into "parsing a conditional" state.
> I would say that this is actually an expected behaviour: `posh -c '[[ a == a ]]'` will show `posh: [[: not found` because `[[` is not in POSIX. Similar error will be shown by dash.
Nor is it disallowed by POSIXand it used to work, the bug report is
still valid.

Re: [[ is being treated as a pattern in the command/reserved word position.

[ZyX]

29.03.2015, 02:07, "Eric Cook" <llua@gmx.com>:
> On 03/28/2015 06:33 PM, ZyX wrote:
>>  29.03.2015, 01:25, "Bart Schaefer" <schaefer@brasslantern.com>:
>>>  On Mar 28,  5:55pm, Eric Cook wrote:
>>>  }
>>>  } % print $ZSH_PATCHLEVEL; emulate sh -c '[[ a == a ]]'
>>>  } zsh-5.0.7-362-gab40656
>>>  } zsh: command not found: [[
>>>  }
>>>  } Did that behavior change?
>>>
>>>  Aha.  That makes more sense.
>>>
>>>  The '[[' reserved word is handled a special kind of built-in alias [*]
>>>  and the change to POSIX_ALIASES handling caused that to be disabled.
>>>
>>>  This will take a bit of thought.
>>>
>>>  [*] Not literally, but the alias expansion code is where '[[' is noted
>>>  and the lexer changed into "parsing a conditional" state.
>>  I would say that this is actually an expected behaviour: `posh -c '[[ a == a ]]'` will show `posh: [[: not found` because `[[` is not in POSIX. Similar error will be shown by dash.
>
> Nor is it disallowed by POSIXand it used to work, the bug report is
> still valid.

`[[` is a *syntax extension*. This *is* going against POSIX. If `[[` in POSIX emulation mode was implemented as a shell built-in in a manner that allows implementing it as a script your concern would be valid. But implementing `[[` as a shell built-in is breaking certain expectations about how `[[` is supposed to work.

Specifically, with

    setopt SH_WORD_SPLIT
    w="a = b"
    test $w # False: a ≠ b, expected
    [[ a = b ]] # False: a ≠ b, expected
    [[ $w ]] # True: w is not empty, should be false if shell is POSIX

. It is easier to disable `[[` then keep two implementations of it.

Re: [[ is being treated as a pattern in the command/reserved word position.

[Eric Cook]

On 03/28/2015 07:19 PM, ZyX wrote:
> 29.03.2015, 02:07, "Eric Cook" <llua@gmx.com>
> `[[` is a *syntax extension*. This *is* going against POSIX. If `[[` in POSIX emulation mode was implemented as a shell built-in in a manner that allows implementing it as a script your concern would be valid. But implementing `[[` as a shell built-in is breaking certain expectations about how `[[` is supposed to work.
>
> Specifically, with
>
>     setopt SH_WORD_SPLIT
>     w="a = b"
>     test $w # False: a ≠ b, expected
>     [[ a = b ]] # False: a ≠ b, expected
>     [[ $w ]] # True: w is not empty, should be false if shell is POSIX
>
> . It is easier to disable `[[` then keep two implementations of it.
See 2.4 of:
http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html

> The following words may be recognized as reserved words on some
implementations (when none of the characters are quoted), causing
unspecified results:
>
> [[ ]]functionselect

Re: [[ is being treated as a pattern in the command/reserved word position.

[ZyX]

29.03.2015, 02:27, "Eric Cook" <llua@gmx.com>:
> On 03/28/2015 07:19 PM, ZyX wrote:
>>  29.03.2015, 02:07, "Eric Cook" <llua@gmx.com>
>>  `[[` is a *syntax extension*. This *is* going against POSIX. If `[[` in POSIX emulation mode was implemented as a shell built-in in a manner that allows implementing it as a script your concern would be valid. But implementing `[[` as a shell built-in is breaking certain expectations about how `[[` is supposed to work.
>>
>>  Specifically, with
>>
>>      setopt SH_WORD_SPLIT
>>      w="a = b"
>>      test $w # False: a ≠ b, expected
>>      [[ a = b ]] # False: a ≠ b, expected
>>      [[ $w ]] # True: w is not empty, should be false if shell is POSIX
>>
>>  . It is easier to disable `[[` then keep two implementations of it.
>
> See 2.4 of:
> http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html
>>  The following words may be recognized as reserved words on some
>
> implementations (when none of the characters are quoted), causing
> unspecified results:
>>  [[ ]]functionselect

There is a bit of problem: http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_06_05 (Field Splitting) section does not talk about any exceptions for the field splitting. There is only one official exception to this rule: function definition. I cannot say whether “uspecified result” makes current behaviour of `[[` (non-)conformant.

There is also another consideration: POSIX does not define `[[`. So when you run an emulation mode *what exactly* should `[[` do? When emulating `bash` (not present for some reason) or `ksh` this is defined by those shells, but `sh` is clearly not a bash emulation (reasoning: BASH_REMATCH is not set) and not ksh emulation (reasoning: `emulate ksh` is a distinct emulation mode and `sh` emulation mode is not referenced as an alias of `ksh` emulation mode). I have failed to find what zsh developers meant by `sh` emulation exactly though: it is not explicitly said that it emulates POSIX shell. But since it not bash or ksh emulation mode `[[ $w ]]` in the above case should be `false` (busybox emulation) or a command-not-found error (posh or dash emulation).

Re: [[ is being treated as a pattern in the command/reserved word position.

[Eric Cook]

On 03/28/2015 07:57 PM, ZyX wrote:
> There is also another consideration: POSIX does not define `[[`. So
> when you run an emulation mode *what exactly* should `[[` do? When
> emulating `bash` (not present for some reason) or `ksh` this is
> defined by those shells, but `sh` is clearly not a bash emulation
> (reasoning: BASH_REMATCH is not set) and not ksh emulation (reasoning:
> `emulate ksh` is a distinct emulation mode and `sh` emulation mode is
> not referenced as an alias of `ksh` emulation mode). I have failed to
> find what zsh developers meant by `sh` emulation exactly though: it is
> not explicitly said that it emulates POSIX shell. But since it not
> bash or ksh emulation mode `[[ $w ]]` in the above case should be
> `false` (busybox emulation) or a command-not-found error (posh or dash
> emulation). 
>From my understanding of emulate, it works similar to bash's posix mode.
Changing anything that conflicts with the mode, but doesn't disable
other non-conflicting extensions.

Distros like Arch Linux relies on it to source their /etc/profile from
the global zprofile is more sh like environment.
(with the added perk of stuff like [[ working.)
I've also been using it for stuff like that for a long time and would
appreciate it staying the way it currently is.

Re: [[ is being treated as a pattern in the command/reserved word position.

[Peter Stephenson]

On Sat, 28 Mar 2015 15:24:30 -0700
Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Mar 28,  5:55pm, Eric Cook wrote:
> }
> } % print $ZSH_PATCHLEVEL; emulate sh -c '[[ a == a ]]'
> } zsh-5.0.7-362-gab40656
> } zsh: command not found: [[
> } 
> } Did that behavior change?
> 
> Aha.  That makes more sense.
> 
> The '[[' reserved word is handled a special kind of built-in alias [*]
> and the change to POSIX_ALIASES handling caused that to be disabled.
> 
> This will take a bit of thought.

Is this good enough?

Looks like we need some more emulation tests for POSIX_ALIASES.

pws

diff --git a/Src/lex.c b/Src/lex.c
index 6d0079c..4758354 100644
--- a/Src/lex.c
+++ b/Src/lex.c
@@ -1789,13 +1789,14 @@ exalias(void)
 	    return 0;
 	return checkalias();
     } else {
+	int noalias = 0;
 	VARARR(char, copy, (strlen(tokstr) + 1));
 
 	if (has_token(tokstr)) {
 	    char *p, *t;
 
 	    if (isset(POSIXALIASES))
-		return 0;
+		noalias = 1;
 
 	    zshlextext = p = copy;
 	    for (t = tokstr;
@@ -1816,7 +1817,7 @@ exalias(void)
 
 	if (tok == STRING) {
 	    /* Check for an alias */
-	    if (checkalias()) {
+	    if (!noalias && checkalias()) {
 		if (zshlextext == copy)
 		    zshlextext = tokstr;
 		return 1;

Re: [[ is being treated as a pattern in the command/reserved word position.

[Bart Schaefer]

On Mar 28, 10:44pm, Peter Stephenson wrote:
}
} Is this good enough?

Equivalent to what I sent immediately after, but using one more local.
Whichever.