Builtin test and parsing of conditionals

Builtin test and parsing of conditionals

[Bart Schaefer]

According to some discussion on the austin-group (POSIX) mailing list,
the following:

	test ! -a !
	test ! -o !
	test ! = !

should all be parsed as comparing the string "!" to the string "!", but
zsh gets this right only in the last case.

This is especially annoying because

	test ! -f !

should be interpreted as the negation of the unary file-exists operator.

This is probably complicated by the different meanings of -a and -o in
[[ ... ]] as opposed to "test" because the same parser is used.

Furthermore:

	test ! = -o a

should be parsed as the negation of the comparison between the strings "="
and "a", implying that zsh also gets this wrong:

	test ! = = =

which should return false (negation of (equal is the same string as equal)),
rather than give a parse error as zsh does.  (Zsh does get

	test = = =

right.)

Apparently the only way to do this correctly is to examine both argv[1] and
the number of arguments, and skip to argv[2] for recursive descent parsing
when argv[1] is "!" and there are four arguments.  (As already mentioned in
the zsh manual, what to do with more than four arguments in the absence of
explicit parens is unclear [unspecified?].)

Re: Builtin test and parsing of conditionals

[Peter Stephenson]

On Wed, 04 Sep 2013 09:15:03 -0700
Bart Schaefer <schaefer@brasslantern.com> wrote:
> According to some discussion on the austin-group (POSIX) mailing list,
> the following:
> 
> 	test ! -a !
> 	test ! -o !
> 	test ! = !
> 
> should all be parsed as comparing the string "!" to the string "!", but
> zsh gets this right only in the last case.

I don't understand that.  -a means "and" and -o means "or", unless
they're being interpreted as strings, but I don't see how that could be
if ! is supposed to be interpreted as a string.   So where does the
implicit comparison come from?  If -a were taken as high precedence you
might read it as

test -n ! -a n !
test -n ! -a n !

which is how

test foo -a bar
test foo -o bar

are interpreted, which you can see by

test foo -a '' # false
test foo -o '' # true

... same if you omit the ''.

(Outside our control, but I imagine people aren't daft enough to rely on
this sort of behaviour in new scripts...?)

pws

Re: Builtin test and parsing of conditionals

[Bart Schaefer]

On Sep 4,  5:31pm, Peter Stephenson wrote:
} Subject: Re: Builtin test and parsing of conditionals
}
} On Wed, 04 Sep 2013 09:15:03 -0700
} Bart Schaefer <schaefer@brasslantern.com> wrote:
} > According to some discussion on the austin-group (POSIX) mailing list,
} > the following:
} > 
} > 	test ! -a !
} > 	test ! -o !
} > 	test ! = !
} > 
} > should all be parsed as comparing the string "!" to the string "!", but
} > zsh gets this right only in the last case.
} 
} I don't understand that.  -a means "and" and -o means "or", unless
} they're being interpreted as strings, but I don't see how that could be
} if ! is supposed to be interpreted as a string.

Sorry, perhaps I should have been clearer:  "! -a !" means to compare the
truth value "!" to the truth value of "!".  But "truth value" in "test"
is the same as "non-empty string" (true) and "empty string" (false) [not
0 or 1 as numeric values], so the -a operator is still comparing two
strings.

The point being that "!" should be parsed as a string, not as a negation
operator, in the three-argument case.

} So where does the
} implicit comparison come from?  If -a were taken as high precedence you
} might read it as
} 
} test -n ! -a n !
} test -n ! -a n !

Presuming you mean -a -n there, this is exactly how the austin-group
thread claims it should be interpreted, and is reportedly how ksh and
bash and GNU /usr/bin/test all interpret it.

} (Outside our control, but I imagine people aren't daft enough to rely on
} this sort of behaviour in new scripts...?)

I would hope so, but ...

Re: Builtin test and parsing of conditionals

[Peter Stephenson]

On Wed, 04 Sep 2013 10:39:40 -0700
Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Sep 4,  5:31pm, Peter Stephenson wrote:
> } On Wed, 04 Sep 2013 09:15:03 -0700
> } Bart Schaefer <schaefer@brasslantern.com> wrote:
> } > According to some discussion on the austin-group (POSIX) mailing list,
> } > the following:
> } > 
> } > 	test ! -a !
> } > 	test ! -o !
> } > 	test ! = !
> } > 
> } > should all be parsed as comparing the string "!" to the string "!", but
> } > zsh gets this right only in the last case.
> 
> Sorry, perhaps I should have been clearer:  "! -a !" means to compare the
> truth value "!" to the truth value of "!".  But "truth value" in "test"
> is the same as "non-empty string" (true) and "empty string" (false) [not
> 0 or 1 as numeric values], so the -a operator is still comparing two
> strings.

OK, this looks straightforward enough.

diff --git a/Src/parse.c b/Src/parse.c
index 0c2a458..f0d0855 100644
--- a/Src/parse.c
+++ b/Src/parse.c
@@ -2088,9 +2088,17 @@ par_cond_2(void)
 	}
     }
     if (tok == BANG) {
-	condlex();
-	ecadd(WCB_COND(COND_NOT, 0));
-	return par_cond_2();
+	/*
+	 * In "test" compatibility mode, "! -a ..." and "! -o ..."
+	 * are treated as "[string] [and] ..." and "[string] [or] ...".
+	 */
+	if (!(condlex == testlex && *testargs && 
+	      (!strcmp(*testargs, "-a") || !strcmp(*testargs, "-o"))))
+	{
+	    condlex();
+	    ecadd(WCB_COND(COND_NOT, 0));
+	    return par_cond_2();
+	}
     }
     if (tok == INPAR) {
 	int r;
diff --git a/Test/C02cond.ztst b/Test/C02cond.ztst
index 494261e..8562519 100644
--- a/Test/C02cond.ztst
+++ b/Test/C02cond.ztst
@@ -324,6 +324,27 @@ F:Failures in these cases do not indicate a problem in the shell.
 >	fi
 >}
 
+  weirdies=(
+    '! -a !'
+    '! -o !'
+    '! -a'
+    '! -o'
+    '! -a ! -a !'
+    '! = !'
+    '! !')
+  for w in $weirdies; do
+     eval test $w
+     print $?
+  done
+0:test compatability weirdness: treat ! as a string sometimes
+>0
+>0
+>1
+>0
+>0
+>0
+>1
+
 %clean
   # This works around a bug in rm -f in some versions of Cygwin
   chmod 644 unmodish

-- 
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/

Re: Builtin test and parsing of conditionals

[Chet Ramey]

On 9/4/13 1:39 PM, Bart Schaefer wrote:

> } So where does the
> } implicit comparison come from?  If -a were taken as high precedence you
> } might read it as
> } 
> } test -n ! -a n !
> } test -n ! -a n !
> 
> Presuming you mean -a -n there, this is exactly how the austin-group
> thread claims it should be interpreted, and is reportedly how ksh and
> bash and GNU /usr/bin/test all interpret it.

The rules, at least through four arguments, are well-defined.

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/test.html#tag_20_128

-a and -o are binary primaries.

Chet

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
		 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRU    chet@case.edu    http://cnswww.cns.cwru.edu/~chet/

Re: Builtin test and parsing of conditionals

[Bart Schaefer]

On Sep 4,  8:09pm, Peter Stephenson wrote:
}
} On Wed, 04 Sep 2013 10:39:40 -0700
} Bart Schaefer <schaefer@brasslantern.com> wrote:
} > Sorry, perhaps I should have been clearer:  "! -a !" means to compare the
} > truth value "!" to the truth value of "!".
} 
} OK, this looks straightforward enough.

That's good, but it still doesn't fix the other case mentioned in my
original email:

torch% /usr/bin/test ! = -a o ; print $?
1
torch% test ! = -a o ; print $? 
test: too many arguments
1

Do you want to try to handle that in parse.c, or am I correct that peeling
off the "!" in builtin.c:bin_test [the way parens are peeled off] is the
better way to go?

Either way, the comment in builtin.c should be updated with the new URL
path to test.html that Chet was kind enough to provide.  The one that's
there now is from 2007.

Re: Builtin test and parsing of conditionals

[Peter Stephenson]

On Thu, 05 Sep 2013 07:25:45 -0700
Bart Schaefer <schaefer@brasslantern.com> wrote:
> That's good, but it still doesn't fix the other case mentioned in my
> original email:
> 
> torch% /usr/bin/test ! = -a o ; print $?
> 1
> torch% test ! = -a o ; print $? 
> test: too many arguments
> 1
> 
> Do you want to try to handle that in parse.c, or am I correct that peeling
> off the "!" in builtin.c:bin_test [the way parens are peeled off] is the
> better way to go?

That seems sensible.

diff --git a/Src/builtin.c b/Src/builtin.c
index f8be4ac..c3f0169 100644
--- a/Src/builtin.c
+++ b/Src/builtin.c
@@ -5995,7 +5995,7 @@ bin_test(char *name, char **argv, UNUSED(Options ops), int func)
     char **s;
     Eprog prog;
     struct estate state;
-    int nargs;
+    int nargs, sense = 0, ret;
 
     /* if "test" was invoked as "[", it needs a matching "]" *
      * which is subsequently ignored                         */
@@ -6014,7 +6014,7 @@ bin_test(char *name, char **argv, UNUSED(Options ops), int func)
     /*
      * Implement some XSI extensions to POSIX here.
      * See
-     * http://www.opengroup.org/onlinepubs/009695399/utilities/test.html.
+     * http://pubs.opengroup.org/onlinepubs/9699919799/utilities/test.html
      */
     nargs = arrlen(argv);
     if (nargs == 3 || nargs == 4)
@@ -6023,6 +6023,10 @@ bin_test(char *name, char **argv, UNUSED(Options ops), int func)
 	    argv[nargs-1] = NULL;
 	    argv++;
 	}
+	if (nargs == 4 && !strcmp("!", argv[0])) {
+	    sense = 1;
+	    argv++;
+	}
     }
 
     lexsave();
@@ -6057,8 +6061,11 @@ bin_test(char *name, char **argv, UNUSED(Options ops), int func)
     state.pc = prog->prog;
     state.strs = prog->strs;
 
+    ret = evalcond(&state, name);
+    if (ret < 2 && sense)
+	ret = ! ret;
 
-    return evalcond(&state, name);
+    return ret;
 }
 
 /* display a time, provided in units of 1/60s, as minutes and seconds */
diff --git a/Test/C02cond.ztst b/Test/C02cond.ztst
index 8562519..94fca8b 100644
--- a/Test/C02cond.ztst
+++ b/Test/C02cond.ztst
@@ -331,7 +331,9 @@ F:Failures in these cases do not indicate a problem in the shell.
     '! -o'
     '! -a ! -a !'
     '! = !'
-    '! !')
+    '! !'
+    '= -a o'
+    '! = -a o')
   for w in $weirdies; do
      eval test $w
      print $?
@@ -344,6 +346,8 @@ F:Failures in these cases do not indicate a problem in the shell.
 >0
 >0
 >1
+>0
+>1
 
 %clean
   # This works around a bug in rm -f in some versions of Cygwin