arithmetic operator precedence

arithmetic operator precedence

[Stephane Chazelas]

$ zsh -c 'echo $(( 1 & 2 == 2 ))'
0

"==" is meant to have precedence over &, so the above should
give 1 as in C I think. All other shells do.

-- 
Stéphane

Re: arithmetic operator precedence

[Mikael Magnusson]

2008/6/12 Stephane Chazelas <Stephane_Chazelas@yahoo.fr>:
> $ zsh -c 'echo $(( 1 & 2 == 2 ))'
> 0
>
> "==" is meant to have precedence over &, so the above should
> give 1 as in C I think. All other shells do.

>From the manual:

       An arithmetic expression uses nearly the same syntax,
precedence, and associativ-
       ity of expressions in C.   The  following  operators  are
supported  (listed  in
       decreasing order of precedence):

       + - ! ~ ++ --
              unary plus/minus, logical NOT, complement,
{pre,post}{in,de}crement
       << >>  bitwise shift left, right
       &      bitwise AND
       ^      bitwise XOR
       |      bitwise OR
       **     exponentiation
       * / %  multiplication, division, modulus (remainder)
       + -    addition, subtraction
       < > <= >=
              comparison
       == !=  equality and inequality
       &&     logical AND
       || ^^  logical OR, XOR
       ? :    ternary operator
       = += -= *= /= %= &= ^= |= <<= >>= &&= ||= ^^= **=
              assignment
       ,      comma operator

       The  operators `&&', `||', `&&=', and `||=' are
short-circuiting, and only one of
       the latter two expressions in a ternary operator is evaluated.
Note  the  prece-
       dence of the bitwise AND, OR, and XOR operators.


-- 
Mikael Magnusson

Re: arithmetic operator precedence

[Peter Stephenson]

On Thu, 12 Jun 2008 10:57:23 +0100
Stephane Chazelas <Stephane_Chazelas@yahoo.fr> wrote:
> $ zsh -c 'echo $(( 1 & 2 == 2 ))'
> 0
> 
> "==" is meant to have precedence over &, so the above should
> give 1 as in C I think. All other shells do.

This is a fairly icky incompatibility, given that the manual page claims
"an arithmetic expression uses nearly the same syntax, precedence, and
associativity of expressions [as] in C".

This adds the option C_PRECEDENCES.  I'd prefer to be able just to change
them, but it's been documented the other way for too long.  I imagine we
need to set the option for emulating any other shell.

(The two long bits of code differene are mostly due to moving mathevall()
so it could use a local enum in its argument list without screwing up
prototyping.)

Index: Doc/Zsh/arith.yo
===================================================================
RCS file: /cvsroot/zsh/zsh/Doc/Zsh/arith.yo,v
retrieving revision 1.12
diff -u -r1.12 arith.yo
--- Doc/Zsh/arith.yo	12 Nov 2007 16:55:12 -0000	1.12
+++ Doc/Zsh/arith.yo	12 Jun 2008 13:35:40 -0000
@@ -91,8 +91,8 @@
 
 cindex(arithmetic operators)
 cindex(operators, arithmetic)
-An arithmetic expression uses nearly the same syntax, precedence, and
-associativity of expressions in C.
+An arithmetic expression uses nearly the same syntax and
+associativity of expressions as in C.
 The following operators are supported (listed in decreasing order
 of precedence):
 
@@ -119,6 +119,29 @@
 operator is evaluated.  Note the precedence of the bitwise AND, OR,
 and XOR operators.
 
+With the option tt(C_PRECEDENCES) the precedences (but no other
+properties) of the operators are altered to be the same as those in
+most other languages that support the relevant operators:
+
+startsitem()
+sitem(tt(PLUS() - ! ~ PLUS()PLUS() --))(unary plus/minus, logical NOT, complement, {pre,post}{in,de}crement)
+sitem(tt(**))(exponentiation)
+sitem(tt(* / %))(multiplication, division, modulus (remainder))
+sitem(tt(PLUS() -))(addition, subtraction)
+sitem(tt(<< >>))(bitwise shift left, right)
+sitem(tt(< > <= >=))(comparison)
+sitem(tt(== !=))(equality and inequality)
+sitem(tt(&))(bitwise AND)
+sitem(tt(^))(bitwise XOR)
+sitem(tt(|))(bitwise OR)
+sitem(tt(&&))(logical AND)
+sitem(tt(^^))(logical XOR)
+sitem(tt(||))(logical OR)
+sitem(tt(? :))(ternary operator)
+sitem(tt(= PLUS()= -= *= /= %= &= ^= |= <<= >>= &&= ||= ^^= **=))(assignment)
+sitem(tt(,))(comma operator)
+endsitem()
+
 cindex(mathematical functions, use of)
 cindex(functions, math, use of)
 Mathematical functions can be called with the syntax
Index: Doc/Zsh/options.yo
===================================================================
RCS file: /cvsroot/zsh/zsh/Doc/Zsh/options.yo,v
retrieving revision 1.60
diff -u -r1.60 options.yo
--- Doc/Zsh/options.yo	10 Jun 2008 08:50:48 -0000	1.60
+++ Doc/Zsh/options.yo	12 Jun 2008 13:35:40 -0000
@@ -1036,6 +1036,16 @@
 hexadecimal and octal.  Note that these formats will be understood on input
 irrespective of the setting of tt(C_BASES).
 )
+pindex(C_PRECEDENCES)
+cindex(precedence, operator)
+cindex(operator precedence)
+item(tt(C_PRECEDENCES))(
+This alters the precedence of arithemtic operators to be more
+like C and other programming languages;
+ifnzman(Arithmetic Evaluation)\
+ifzman(the section ARITHMETIC EVALUATION in zmanref(zshmisc))
+has an explicit list.
+)
 pindex(DEBUG_BEFORE_CMD)
 cindex(traps, DEBUG, before or after command)
 cindex(DEBUG trap, before or after command)
Index: Src/math.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/math.c,v
retrieving revision 1.32
diff -u -r1.32 math.c
--- Src/math.c	10 Jun 2008 08:50:52 -0000	1.32
+++ Src/math.c	12 Jun 2008 13:35:40 -0000
@@ -159,25 +159,128 @@
 #define FUNC 52
 #define TOKCOUNT 53
 
-/* precedences */
+/*
+ * Opeator recedences: in reverse order, i.e. lower number, high precedence.
+ * These are the C precedences.
+ *
+ * 0   Non-operators: NUM (numeric constant), ID (identifier),
+ *                    CID (identifier with '#'), FUNC (math function)
+ * 1   Opening parenthesis: M_INPAR '('  (for convenience, not an operator)
+ * 2   Unary operators: PREPLUS/POSTPLUS '++', PREMINUS/POSTMINUS '--',
+ *                      NOT '!', COMP '~', UPLUS '+', UMINUS '-'
+ * 3   POWER '**' (not in C but at high precedence in Perl)
+ * 4   MUL '*', DIV '/', MOD '%'
+ * 5   PLUS '+', MINUS '-'
+ * 6   SHLEFT '<<', SHRIGHT '>>'
+ * 7   GRE '>', 'GEQ' '>=', LES '<', LEQ '<='
+ * 8   DEQ '==', NEQ '!='
+ * 9   AND '&'
+ * 10  XOR '^'
+ * 11  OR  '|'
+ * 12  DAND '&&'
+ * 13  DXOR '^^' (not in C)
+ * 14  DOR '||'
+ * 15  QUEST '?'
+ * 16  COLON ':'
+ * 17  EQ '=', PLUSEQ '+=', MINUSEQ '-=', MULEQ '*=', DIVEQ '/=',
+ *     MODEQ '%=', ANDEQ '&=', XOREQ '^=', OREQ '|=',
+ *     SHFLEFTEQ '<<=', SHRIGHTEQ '>>=', DANDEQ '&&=', DOREQ '||=',
+ *     DXOREQ '^^='
+ * 18 COMMA ','
+ * 137 M_OUTPAR ')' (for convenience, not an operator)
+ * 200 EOI (end of input:  for convenience, not an operator)
+ */
+static int c_prec[TOKCOUNT] =
+{
+/*        M_INPAR   M_OUTPAR     NOT       COMP     POSTPLUS */
+/*  0 */     1,       137,        2,        2,         2,
+/*        POSTMINUS   UPLUS     UMINUS     AND        XOR    */
+/*  5 */     2,         2,        2,        9,        10,
+/*          OR         MUL       DIV       MOD       PLUS    */
+/* 10 */    11,         4,        4,        4,         5,
+/*         MINUS      SHLEFT   SHRIGHT     LES        LEQ    */
+/* 15 */     5,         6,        6,        7,         7,
+/*          GRE        GEQ       DEQ       NEQ       DAND    */
+/* 20 */     7,         7,        8,        8,        12,
+/*          DOR        DXOR     QUEST     COLON       EQ     */
+/* 25 */    14,        13,       15,       16,        17,
+/*         PLUSEQ    MINUSEQ    MULEQ     DIVEQ      MODEQ   */
+/* 30 */    17,        17,       17,       17,        17,
+/*         ANDEQ      XOREQ     OREQ    SHLEFTEQ   SHRIGHTEQ */
+/* 35 */    17,        17,       17,       17,        17,
+/*        DANDEQ      DOREQ    DXOREQ    COMMA       EOI     */
+/* 40 */    17,        17,       17,       18,       200,
+/*       PREPLUS    PREMINUS     NUM        ID       POWER   */
+/* 45 */     2,         2,        0,        0,         3,
+/*          CID      POWEREQ     FUNC  */
+/* 50 */     0,        17,        0
+};
 
-static int prec[TOKCOUNT] =
+/*
+ * Opeator recedences: in reverse order, i.e. lower number, high precedence.
+ * These are the default zsh precedences.
+ *
+ * 0   Non-operators: NUM (numeric constant), ID (identifier),
+ *                    CID (identifier with '#'), FUNC (math function)
+ * 1   Opening parenthesis: M_INPAR '('  (for convenience, not an operator)
+ * 2   Unary operators: PREPLUS/POSTPLUS '++', PREMINUS/POSTMINUS '--',
+ *                      NOT '!', COMP '~', UPLUS '+', UMINUS '-' 
+ * 3   SHLEFT '<<', SHRIGHT '>>'
+ * 4   AND '&'
+ * 5   XOR '^'
+ * 6   OR  '|'
+ * 7   POWER '**' (not in C but at high precedence in Perl)
+ * 8   MUL '*', DIV '/', MOD '%'
+ * 9   PLUS '+', MINUS '-'
+ * 10  GRE '>', 'GEQ' '>=', LES '<', LEQ '<='
+ * 11  DEQ '==', NEQ '!='
+ * 12  DAND '&&'
+ * 13  DOR '||', DXOR '^^' (not in C)
+ * 14  QUEST '?'
+ * 15  COLON ':'
+ * 16  EQ '=', PLUSEQ '+=', MINUSEQ '-=', MULEQ '*=', DIVEQ '/=',
+ *     MODEQ '%=', ANDEQ '&=', XOREQ '^=', OREQ '|=',
+ *     SHFLEFTEQ '<<=', SHRIGHTEQ '>>=', DANDEQ '&&=', DOREQ '||=',
+ *     DXOREQ '^^='
+ * 17 COMMA ','
+ * 137 M_OUTPAR ')' (for convenience, not an operator)
+ * 200 EOI (end of input:  for convenience, not an operator)
+ */
+static int z_prec[TOKCOUNT] =
 {
-     1, 137,  2,  2,   2,
-     2,   2,  2,  4,   5,
-     6,   8,  8,  8,   9,
-     9,   3,  3, 10,  10,
-    10,  10, 11, 11,  12,
-    13,  13, 14, 15,  16,
-    16,  16, 16, 16,  16,
-    16,  16, 16, 16,  16,
-    16,  16, 16, 17, 200,
-     2,   2,  0,  0,   7,
-     0,  16, 0
+/*        M_INPAR   M_OUTPAR     NOT       COMP     POSTPLUS */
+/*  0 */     1,       137,        2,        2,         2,
+/*        POSTMINUS   UPLUS     UMINUS     AND        XOR    */
+/*  5 */     2,         2,        2,        4,         5,
+/*          OR         MUL       DIV       MOD       PLUS    */
+/* 10 */     6,         8,        8,        8,         9,
+/*         MINUS      SHLEFT   SHRIGHT     LES        LEQ    */
+/* 15 */     9,         3,        3,       10,        10,
+/*          GRE        GEQ       DEQ       NEQ       DAND    */
+/* 20 */    10,        10,       11,       11,        12,
+/*          DOR        DXOR     QUEST     COLON       EQ     */
+/* 25 */    13,        13,       14,       15,        16,
+/*         PLUSEQ    MINUSEQ    MULEQ     DIVEQ      MODEQ   */
+/* 30 */    16,        16,       16,       16,        16,
+/*         ANDEQ      XOREQ     OREQ    SHLEFTEQ   SHRIGHTEQ */
+/* 35 */    16,        16,       16,       16,        16,
+/*        DANDEQ      DOREQ    DXOREQ    COMMA       EOI     */
+/* 40 */    16,        16,       16,       17,       200,
+/*       PREPLUS    PREMINUS     NUM        ID       POWER   */
+/* 45 */     2,         2,        0,        0,         7,
+/*          CID      POWEREQ     FUNC  */
+/* 50 */     0,        16,        0
 };
 
-#define TOPPREC 18
-#define ARGPREC 16
+/* Option-selectable preference table */
+static int *prec;
+
+/*
+ * Precedences for top and argument evaluation.  Careful:
+ * prec needs to be set before we use these.
+ */
+#define TOPPREC (prec[COMMA]+1)
+#define ARGPREC (prec[COMMA]-1)
 
 static int type[TOKCOUNT] =
 {
@@ -194,6 +297,113 @@
 /* 50 */  LR|OP_OPF, RL|OP_E2, LR|OP_OPF
 };
 
+/* the value stack */
+
+#define STACKSZ 100
+static int mtok;			/* last token */
+static int sp = -1;			/* stack pointer */
+
+struct mathvalue {
+    char *lval;
+    mnumber val;
+};
+
+static struct mathvalue *stack;
+
+enum prec_type {
+    /* Evaluating a top-level expression */
+    MPREC_TOP,
+    /* Evaluating a function argument */
+    MPREC_ARG
+};
+
+static mnumber
+mathevall(char *s, enum prec_type prec_tp, char **ep)
+{
+    int xlastbase, xnoeval, xunary, *xprec;
+    char *xptr;
+    mnumber xyyval;
+    char *xyylval;
+    int xsp;
+    struct mathvalue *xstack = 0, nstack[STACKSZ];
+    mnumber ret;
+
+    if (mlevel >= MAX_MLEVEL) {
+	xyyval.type = MN_INTEGER;
+	xyyval.u.l = 0;
+
+	zerr("math recursion limit exceeded");
+
+	return xyyval;
+    }
+    if (mlevel++) {
+	xlastbase = lastbase;
+	xnoeval = noeval;
+	xunary = unary;
+	xptr = ptr;
+	xyyval = yyval;
+	xyylval = yylval;
+
+	xsp = sp;
+	xstack = stack;
+	xprec = prec;
+    } else {
+	xlastbase = xnoeval = xunary = xsp = 0;
+	xyyval.type = MN_INTEGER;
+	xyyval.u.l = 0;
+	xyylval = NULL;
+	xptr = NULL;
+	xprec = NULL;
+    }
+    prec = isset(CPRECEDENCES) ? c_prec : z_prec;
+    stack = nstack;
+    lastbase = -1;
+    ptr = s;
+    sp = -1;
+    unary = 1;
+    stack[0].val.type = MN_INTEGER;
+    stack[0].val.u.l = 0;
+    mathparse(prec_tp == MPREC_TOP ? TOPPREC : ARGPREC);
+    *ep = ptr;
+    DPUTS(!errflag && sp > 0,
+	  "BUG: math: wallabies roaming too freely in outback");
+
+    if (errflag) {
+	/*
+	 * This used to set the return value to errflag.
+	 * I don't understand how that could be useful; the
+	 * caller doesn't know that's what's happened and
+	 * may not get a value at all.
+	 * Worse, we reset errflag in execarith() and setting
+	 * this explicitly non-zero means a (( ... )) returns
+	 * status 0 if there's an error.  That surely can't
+	 * be right.  execarith() now detects an error and returns
+	 * status 2.
+	 */
+	ret.type = MN_INTEGER;
+	ret.u.l = 0;
+    } else {
+	if (stack[0].val.type == MN_UNSET)
+	    ret = getnparam(stack[0].lval);
+	else
+	    ret = stack[0].val;
+    }
+
+    if (--mlevel) {
+	lastbase = xlastbase;
+	noeval = xnoeval;
+	unary = xunary;
+	ptr = xptr;
+	yyval = xyyval;
+	yylval = xyylval;
+
+	sp = xsp;
+	stack = xstack;
+	prec = xprec;
+    }
+    return lastmathval = ret;
+}
+
 static int
 lexconstant(void)
 {
@@ -521,19 +731,6 @@
 	}
 }
 
-/* the value stack */
-
-#define STACKSZ 100
-static int mtok;			/* last token */
-static int sp = -1;			/* stack pointer */
-
-struct mathvalue {
-    char *lval;
-    mnumber val;
-};
-
-static struct mathvalue *stack;
-
 /**/
 static void
 push(mnumber val, char *lval, int getme)
@@ -645,7 +842,7 @@
 		    if (f->flags & MFF_USERFUNC) {
 			/* need to pass strings */
 			char *str;
-			marg = mathevall(a, ARGPREC, &a);
+			marg = mathevall(a, MPREC_ARG, &a);
 			if (marg.type & MN_FLOAT) {
 			    /* convfloat is off the heap */
 			    str = convfloat(marg.u.d, 0, 0, NULL);
@@ -657,7 +854,7 @@
 			addlinknode(l, str);
 		    } else {
 			q = (mnumber *) zhalloc(sizeof(mnumber));
-			*q = mathevall(a, ARGPREC, &a);
+			*q = mathevall(a, MPREC_ARG, &a);
 			addlinknode(l, q);
 		    }
 		    if (errflag || mtok != COMMA)
@@ -1017,91 +1214,6 @@
 
 
 /**/
-static mnumber
-mathevall(char *s, int prek, char **ep)
-{
-    int xlastbase, xnoeval, xunary;
-    char *xptr;
-    mnumber xyyval;
-    char *xyylval;
-    int xsp;
-    struct mathvalue *xstack = 0, nstack[STACKSZ];
-    mnumber ret;
-
-    if (mlevel >= MAX_MLEVEL) {
-	xyyval.type = MN_INTEGER;
-	xyyval.u.l = 0;
-
-	zerr("math recursion limit exceeded");
-
-	return xyyval;
-    }
-    if (mlevel++) {
-	xlastbase = lastbase;
-	xnoeval = noeval;
-	xunary = unary;
-	xptr = ptr;
-	xyyval = yyval;
-	xyylval = yylval;
-
-	xsp = sp;
-	xstack = stack;
-    } else {
-	xlastbase = xnoeval = xunary = xsp = 0;
-	xyyval.type = MN_INTEGER;
-	xyyval.u.l = 0;
-	xyylval = NULL;
-	xptr = NULL;
-    }
-    stack = nstack;
-    lastbase = -1;
-    ptr = s;
-    sp = -1;
-    unary = 1;
-    stack[0].val.type = MN_INTEGER;
-    stack[0].val.u.l = 0;
-    mathparse(prek);
-    *ep = ptr;
-    DPUTS(!errflag && sp > 0,
-	  "BUG: math: wallabies roaming too freely in outback");
-
-    if (errflag) {
-	/*
-	 * This used to set the return value to errflag.
-	 * I don't understand how that could be useful; the
-	 * caller doesn't know that's what's happened and
-	 * may not get a value at all.
-	 * Worse, we reset errflag in execarith() and setting
-	 * this explicitly non-zero means a (( ... )) returns
-	 * status 0 if there's an error.  That surely can't
-	 * be right.  execarith() now detects an error and returns
-	 * status 2.
-	 */
-	ret.type = MN_INTEGER;
-	ret.u.l = 0;
-    } else {
-	if (stack[0].val.type == MN_UNSET)
-	    ret = getnparam(stack[0].lval);
-	else
-	    ret = stack[0].val;
-    }
-
-    if (--mlevel) {
-	lastbase = xlastbase;
-	noeval = xnoeval;
-	unary = xunary;
-	ptr = xptr;
-	yyval = xyyval;
-	yylval = xyylval;
-
-	sp = xsp;
-	stack = xstack;
-    }
-    return lastmathval = ret;
-}
-
-
-/**/
 mod_export mnumber
 matheval(char *s)
 {
@@ -1117,7 +1229,7 @@
 	x.u.l = 0;
 	return x;
     }
-    x = mathevall(s, TOPPREC, &junk);
+    x = mathevall(s, MPREC_TOP, &junk);
     mtok = xmtok;
     if (*junk)
 	zerr("bad math expression: illegal character: %c", *junk);
@@ -1140,7 +1252,7 @@
     mnumber x;
     int xmtok = mtok;
 
-    x = mathevall(s, ARGPREC, ss);
+    x = mathevall(s, MPREC_ARG, ss);
     if (mtok == COMMA)
 	(*ss)--;
     mtok = xmtok;
Index: Src/options.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/options.c,v
retrieving revision 1.41
diff -u -r1.41 options.c
--- Src/options.c	17 Apr 2008 10:23:53 -0000	1.41
+++ Src/options.c	12 Jun 2008 13:35:40 -0000
@@ -96,6 +96,7 @@
 {{NULL, "caseglob",	      OPT_ALL},			 CASEGLOB},
 {{NULL, "casematch",	      OPT_ALL},			 CASEMATCH},
 {{NULL, "cbases",	      0},			 CBASES},
+{{NULL, "cprecedences",	      OPT_EMULATE|OPT_NONZSH},	 CPRECEDENCES},
 {{NULL, "cdablevars",	      OPT_EMULATE},		 CDABLEVARS},
 {{NULL, "chasedots",	      OPT_EMULATE},		 CHASEDOTS},
 {{NULL, "chaselinks",	      OPT_EMULATE},		 CHASELINKS},
Index: Src/zsh.h
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/zsh.h,v
retrieving revision 1.137
diff -u -r1.137 zsh.h
--- Src/zsh.h	8 Jun 2008 17:53:55 -0000	1.137
+++ Src/zsh.h	12 Jun 2008 13:35:40 -0000
@@ -1797,6 +1797,7 @@
     COMPLETEINWORD,
     CORRECT,
     CORRECTALL,
+    CPRECEDENCES,
     CSHJUNKIEHISTORY,
     CSHJUNKIELOOPS,
     CSHJUNKIEQUOTES,
Index: Test/C01arith.ztst
===================================================================
RCS file: /cvsroot/zsh/zsh/Test/C01arith.ztst,v
retrieving revision 1.15
diff -u -r1.15 C01arith.ztst
--- Test/C01arith.ztst	10 Jun 2008 09:13:01 -0000	1.15
+++ Test/C01arith.ztst	12 Jun 2008 13:35:40 -0000
@@ -43,6 +43,16 @@
 0:precedence (arithmetic)
 >1591
 
+  fn() {
+    setopt localoptions c_precedences
+    integer i
+    (( i = 4 - - 3 * 7 << 1 & 7 ^ 1 | 16 ** 2 ))
+    print $i
+  }
+  fn
+0:precedence (arithmetic, with C_PRECEDENCES)
+>259
+
   print $(( 1 < 2 || 2 < 2 && 3 > 4 ))
 0:precedence (logical)
 >1


-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070

Re: arithmetic operator precedence

[Bart Schaefer]

On Jun 12,  2:40pm, Peter Stephenson wrote:
} Subject: Re: arithmetic operator precedence
}
} On Thu, 12 Jun 2008 10:57:23 +0100
} Stephane Chazelas <Stephane_Chazelas@yahoo.fr> wrote:
} > "==" is meant to have precedence over &, so the above should
} > give 1 as in C I think. All other shells do.
} 
} This is a fairly icky incompatibility

What I'm struggling (although not very hard) to remember is why Paul
would have chosen a non-standard precedence in the first place.

Re: arithmetic operator precedence

[Stephane Chazelas]

On Thu, Jun 12, 2008 at 07:47:51AM -0700, Bart Schaefer wrote:
> On Jun 12,  2:40pm, Peter Stephenson wrote:
> } Subject: Re: arithmetic operator precedence
> }
> } On Thu, 12 Jun 2008 10:57:23 +0100
> } Stephane Chazelas <Stephane_Chazelas@yahoo.fr> wrote:
> } > "==" is meant to have precedence over &, so the above should
> } > give 1 as in C I think. All other shells do.
> } 
> } This is a fairly icky incompatibility
> 
> What I'm struggling (although not very hard) to remember is why Paul
> would have chosen a non-standard precedence in the first place.

I find the zsh precedence more intuitive but that's not of much
help given that I get to use those binary operators mostly only
in C/perl, so I have to know the C precedence anyway.

And having zsh behaving differently doesn't help at all.

Actually, I reported that bug because I wanted to double check
what the C precedence was for & vs ==. I thought that was one of
those unintuitive cases where "==" had precedence, but in doubt,
I checked with zsh's $((...)) that told me I was wrong.
Fortunately, I did reread the C precedence chart and tried with
another shell, otherwise I'd have been fooled by zsh...

But I now know that zsh's precedence is not the same as
elsewhere so I hopefully won't be fooled again.

-- 
Stéphane

Re: arithmetic operator precedence

[Vincent Lefevre]

On 2008-06-12 16:01:26 +0100, Stephane Chazelas wrote:
> I find the zsh precedence more intuitive but that's not of much

ditto

> help given that I get to use those binary operators mostly only
> in C/perl, so I have to know the C precedence anyway.
> 
> And having zsh behaving differently doesn't help at all.

I agree. POSIX even requires C-like behavior:

http://www.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_06_04

  2.6.4 Arithmetic Expansion

  Arithmetic expansion provides a mechanism for evaluating an
  arithmetic expression and substituting its value. The format for
  arithmetic expansion shall be as follows:

    $((expression))

  The expression shall be treated as if it were in double-quotes,
  except that a double-quote inside the expression is not treated
  specially. The shell shall expand all tokens in the expression for
  parameter expansion, command substitution, and quote removal.

  Next, the shell shall treat this as an arithmetic expression and
  substitute the value of the expression. The arithmetic expression
  shall be processed according to the rules given in Arithmetic
                                                     ^^^^^^^^^^
  Precision and Operations , with the following exceptions:
  ^^^^^^^^^^^^^^^^^^^^^^^^

(the exceptions are not about precedence), where the link to
Arithmetic Precision and Operations is:

http://www.opengroup.org/onlinepubs/009695399/utilities/xcu_chap01.html#tag_01_07_02_01

which corresponds to the ISO C operators.

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)

Re: arithmetic operator precedence

[Vincent Lefevre]

Speaking of precedence, the following one is nasty:

vin% zsh -c 'echo $((-3**2))'
9
vin% bash -c 'echo $((-3**2))'
9
vin% ksh93 -c 'echo $((-3**2))'
9

IMHO these shells should be fixed to give -9, i.e. ** should have
the precedence over the unary -, like conventional math writing.
Nowadays most modern software does it right, the main exception
being MS Excel and other spreadsheet software that mimics its
behavior (current versions of OpenOffice and Gnumeric, despite
the users' complaints).

See http://www.macnauchtan.com/pub/precedence.html for the behavior
of various software on that. This page is incomplete. One can add:
Qalculate/qalc, calc, wcalc, GP/PARI and MuPAD as doing it right,
ditto for the future version of Abakus (that will be released for
KDE 4).

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)

Re: arithmetic operator precedence

[Peter Stephenson]

On Mon, 16 Jun 2008 10:07:26 +0200
Vincent Lefevre <vincent@vinc17.org> wrote:
> Speaking of precedence, the following one is nasty:
> 
> vin% zsh -c 'echo $((-3**2))'
> 9 
> vin% bash -c 'echo $((-3**2))'
> 9
> vin% ksh93 -c 'echo $((-3**2))'
> 9
> 
> IMHO these shells should be fixed to give -9, i.e. ** should have
> the precedence over the unary -, like conventional math writing.

That's an interesting point for C_PRECEDENCES since I was trying to get
it behave as much as possible like Perl.  What does anyone else think?

Index: Doc/Zsh/arith.yo
===================================================================
RCS file: /cvsroot/zsh/zsh/Doc/Zsh/arith.yo,v
retrieving revision 1.13
diff -u -r1.13 arith.yo
--- Doc/Zsh/arith.yo	12 Jun 2008 13:45:05 -0000	1.13
+++ Doc/Zsh/arith.yo	16 Jun 2008 13:38:41 -0000
@@ -121,11 +121,13 @@
 
 With the option tt(C_PRECEDENCES) the precedences (but no other
 properties) of the operators are altered to be the same as those in
-most other languages that support the relevant operators:
+most other languages that support the relevant operators, in particular
+Perl (note that exponentiation, not present in C, has a higher
+precedence than unary operators):
 
 startsitem()
-sitem(tt(PLUS() - ! ~ PLUS()PLUS() --))(unary plus/minus, logical NOT, complement, {pre,post}{in,de}crement)
 sitem(tt(**))(exponentiation)
+sitem(tt(PLUS() - ! ~ PLUS()PLUS() --))(unary plus/minus, logical NOT, complement, {pre,post}{in,de}crement)
 sitem(tt(* / %))(multiplication, division, modulus (remainder))
 sitem(tt(PLUS() -))(addition, subtraction)
 sitem(tt(<< >>))(bitwise shift left, right)
Index: Src/math.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/math.c,v
retrieving revision 1.33
diff -u -r1.33 math.c
--- Src/math.c	12 Jun 2008 13:45:06 -0000	1.33
+++ Src/math.c	16 Jun 2008 13:38:42 -0000
@@ -193,9 +193,9 @@
 static int c_prec[TOKCOUNT] =
 {
 /*        M_INPAR   M_OUTPAR     NOT       COMP     POSTPLUS */
-/*  0 */     1,       137,        2,        2,         2,
+/*  0 */     1,       137,        3,        3,         3,
 /*        POSTMINUS   UPLUS     UMINUS     AND        XOR    */
-/*  5 */     2,         2,        2,        9,        10,
+/*  5 */     3,         3,        3,        9,        10,
 /*          OR         MUL       DIV       MOD       PLUS    */
 /* 10 */    11,         4,        4,        4,         5,
 /*         MINUS      SHLEFT   SHRIGHT     LES        LEQ    */
@@ -211,7 +211,7 @@
 /*        DANDEQ      DOREQ    DXOREQ    COMMA       EOI     */
 /* 40 */    17,        17,       17,       18,       200,
 /*       PREPLUS    PREMINUS     NUM        ID       POWER   */
-/* 45 */     2,         2,        0,        0,         3,
+/* 45 */     3,         3,        0,        0,         2,
 /*          CID      POWEREQ     FUNC  */
 /* 50 */     0,        17,        0
 };
@@ -514,13 +514,9 @@
 		ptr++;
 		return MINUSEQ;
 	    }
-	    if (unary) {
-		if (idigit(*ptr) || *ptr == '.') {
-		    ptr--;
-		    return lexconstant();
-		} else
-		    return UMINUS;
-	    } else
+	    if (unary)
+		return UMINUS;
+	    else
 		return MINUS;
 	case '(':
 	    return M_INPAR;


-- 
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/

Re: arithmetic operator precedence

[Stephane Chazelas]

On Mon, Jun 16, 2008 at 02:42:11PM +0100, Peter Stephenson wrote:
> On Mon, 16 Jun 2008 10:07:26 +0200
> Vincent Lefevre <vincent@vinc17.org> wrote:
> > Speaking of precedence, the following one is nasty:
> > 
> > vin% zsh -c 'echo $((-3**2))'
> > 9 
> > vin% bash -c 'echo $((-3**2))'
> > 9
> > vin% ksh93 -c 'echo $((-3**2))'
> > 9
> > 
> > IMHO these shells should be fixed to give -9, i.e. ** should have
> > the precedence over the unary -, like conventional math writing.
> 
> That's an interesting point for C_PRECEDENCES since I was trying to get
> it behave as much as possible like Perl.  What does anyone else think?
[...]

It's at least consistent with other shells and bc at the moment:

~$ bash -c 'echo $((-2**2))'
4
~$ zsh -c 'echo $((-2**2))'
4
~$ ksh93 -c 'echo $((-2**2))'
4
~$ echo '-2 ^ 2' | bc
4

bc is one place where POSIX specifies the precedence of
unary-minus vs power:
http://www.opengroup.org/onlinepubs/009695399/utilities/bc.html
minus has higher precedence than ^ which I have to say is more
intuitive to me.

Having said that:

~$ perl -le 'print -2 ** 2'
-4
~$ python -c 'print -2 ** 2'
-4
~$ ruby -e 'print -2 ** 2'
-4%
~$ gawk 'BEGIN {print -2 ** 2}'
-4

TCL has pow(x, y) instead of an operator.

-- 
Stéphane

Re: arithmetic operator precedence

[Vincent Lefevre]

On 2008-06-16 14:59:34 +0100, Stephane Chazelas wrote:
> It's at least consistent with other shells and bc at the moment:
> 
> ~$ bash -c 'echo $((-2**2))'
> 4
> ~$ zsh -c 'echo $((-2**2))'
> 4
> ~$ ksh93 -c 'echo $((-2**2))'
> 4
> ~$ echo '-2 ^ 2' | bc
> 4

That are almost the only ones.

> bc is one place where POSIX specifies the precedence of
> unary-minus vs power:
> http://www.opengroup.org/onlinepubs/009695399/utilities/bc.html
> minus has higher precedence than ^ which I have to say is more
> intuitive to me.

Most users find the other way more intuitive, as with conventional
math writing

    2
  -2  = -4

(and that's also the reason why ** is right-associative, whereas
most operations are left-associative).

> Having said that:
> 
> ~$ perl -le 'print -2 ** 2'
> -4
> ~$ python -c 'print -2 ** 2'
> -4
> ~$ ruby -e 'print -2 ** 2'
> -4%
> ~$ gawk 'BEGIN {print -2 ** 2}'
> -4

and:

vin:~> calc -- '-2 ^ 2'
        -4
vin:~> qalc '-2 ^ 2'
-(2^2) = -4
vin:~> wcalc '-2 ^ 2'
 = -4
vin:~> echo '-2 ^ 2' | gp -q
-4
vin:~> echo '-2 ^ 2;' | maple -q
                                      -4
vin:~> echo '-2 ^ 2; quit' | mupad -S
>> -2 ^ 2; quit

                                    -4

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)

Re: arithmetic operator precedence

[Richard Hartmann]

On Mon, Jun 16, 2008 at 15:42, Peter Stephenson
<p.w.stephenson@ntlworld.com> wrote:


> That's an interesting point for C_PRECEDENCES since I was trying to get
> it behave as much as possible like Perl.  What does anyone else think?

Being unsure myself, I did some somewhat extensive research. The only
mathematically correct way of doing this is

  -3**2   = -9
  (-3)**2 =  9

On the other hand, compability with the other shells is A Good Thing,
especially as this is the kind of syntax change that can break a script
in a very, very evil and hard-to-debug way.


Still, all things considered, I would say the only possible way to go is to
use the mathematically correct syntax. Both options are bad, in a way,
but this is the better one.


Richard

Re: arithmetic operator precedence

[Stephane Chazelas]

On Tue, Jun 17, 2008 at 11:19:40AM +0200, Richard Hartmann wrote:
[...]
> > That's an interesting point for C_PRECEDENCES since I was trying to get
> > it behave as much as possible like Perl.  What does anyone else think?
> 
> Being unsure myself, I did some somewhat extensive research. The only
> mathematically correct way of doing this is
> 
>   -3**2   = -9
>   (-3)**2 =  9
[...]

In which way is it more "mathematically" correct?

Is that because -3² is -9? 

But ** is not ^, it's a binary operator whose shape reminds that
of multiply, like a multiply++. And even then, POSIX's ^ in bc
is handled as -3^2 = 9.

It could be a good idea to ask ksh, POSIX/bc and perl authors for
the rationale behind their choices.

-- 
Stéphane

Re: arithmetic operator precedence

[Richard Hartmann]

On Tue, Jun 17, 2008 at 11:45, Stephane Chazelas
<Stephane_Chazelas@yahoo.fr> wrote:


> In which way is it more "mathematically" correct?
>
> Is that because -3² is -9?
>
> But ** is not ^, it's a binary operator whose shape reminds that
> of multiply, like a multiply++. And even then, POSIX's ^ in bc
> is handled as -3^2 = 9.

As far as I know, ^ and ** are fuly equivalent. If that is not the case, then
sorry. Do you have a link/manpage/whatever on this topic?


> It could be a good idea to ask ksh, POSIX/bc and perl authors for
> the rationale behind their choices.


Richard

Re: arithmetic operator precedence

[Richard Hartmann]

Sorry, forgot to answer that part..

On Tue, Jun 17, 2008 at 12:24, Richard Hartmann
<richih.mailinglist@gmail.com> wrote:
> On Tue, Jun 17, 2008 at 11:45, Stephane Chazelas

>> It could be a good idea to ask ksh, POSIX/bc and perl authors for
>> the rationale behind their choices.

Agreed.


Richard

Re: arithmetic operator precedence

[Stephane Chazelas]

On Tue, Jun 17, 2008 at 12:24:12PM +0200, Richard Hartmann wrote:
> On Tue, Jun 17, 2008 at 11:45, Stephane Chazelas
> <Stephane_Chazelas@yahoo.fr> wrote:
> 
> 
> > In which way is it more "mathematically" correct?
> >
> > Is that because -3² is -9?
> >
> > But ** is not ^, it's a binary operator whose shape reminds that
> > of multiply, like a multiply++. And even then, POSIX's ^ in bc
> > is handled as -3^2 = 9.
> 
> As far as I know, ^ and ** are fuly equivalent. If that is not the case, then
> sorry. Do you have a link/manpage/whatever on this topic?
[...]

Not sure what you mean. ^ in bc is the power operator and
there's no ** there. In shells, ** is the power operator and ^
is the XOR operator.

What I meant is that ^ reminds of the /human/ (as opposed to
/computer/) representation as it indicates that follows must be
raised up as in 3². So, one can understand that it should follow
the same rules (that is -3^2 should be the same as -3², even
though that's not what POSIX decided for bc). But given that **
has more the shape of the * operator, I'm not sure we can tell
the same thing.

Anyway, I was just curious about Peter's statement. I'm most
probably not as versed in maths as he is, so was curious about
the rationale behind his statement about -3**2 = 9 not being
mathematically correct.

Looking at bc history, it seems to have appeared in Unix V6
(1975), and it was a wrapper written in yacc around dc (a
reverse polish calculator: _3 2 ^ is less ambiguous there). I'm
not versed enough in yacc or grammars to tell whether the
precedence was the same, but you can have a look at:

http://minnie.tuhs.org/UnixTree/V6/usr/source/s1/bc.y.html

 %right '='
 %left '+' '-'
 %left '*' '/' '%'
 %right '^'
 %left UMINUS
[...]
 e       :  e '+' e
                 = bundle( $1, $3, "+" );
         |  e '-' e
                 = bundle( $1, $3, "-" );
         | '-' e         %prec UMINUS
                 = bundle( " 0", $2, "-" );
         |  e '*' e
                 = bundle( $1, $3, "*" );
         |  e '/' e
                 = bundle( $1, $3, "/" );
         |  e '%' e
                 = bundle( $1, $3, "%%" );
         |  e '^' e
                 = bundle( $1, $3, "^" );
[...]

-- 
Stéphane

Re: arithmetic operator precedence

[Peter Stephenson]

On Tue, 17 Jun 2008 11:38:29 +0100
Stephane Chazelas <Stephane_Chazelas@yahoo.fr> wrote:
> Anyway, I was just curious about Peter's statement. I'm most
> probably not as versed in maths as he is, so was curious about
> the rationale behind his statement about -3**2 = 9 not being
> mathematically correct.

That was Vincent.  I'm inclined to leave the shell the way it is since
there seems to be no widespread agreement on having C_PRECEDENCES (which
probably few people will set explicitly) different from other shells (for
which it will automatically be used in the appropriate emulation).

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070

Re: arithmetic operator precedence

[Vincent Lefevre]

On 2008-06-17 11:43:40 +0100, Peter Stephenson wrote:
> On Tue, 17 Jun 2008 11:38:29 +0100
> Stephane Chazelas <Stephane_Chazelas@yahoo.fr> wrote:
> > Anyway, I was just curious about Peter's statement. I'm most
> > probably not as versed in maths as he is, so was curious about
> > the rationale behind his statement about -3**2 = 9 not being
> > mathematically correct.
> 
> That was Vincent.  I'm inclined to leave the shell the way it is since
> there seems to be no widespread agreement on having C_PRECEDENCES (which
> probably few people will set explicitly) different from other shells (for
> which it will automatically be used in the appropriate emulation).

Well, the C language doesn't have a power operator (there's a power
function, with prefix notation, so no precedence problem with it).

Concerning the precedence of & and |, I'd say that zsh should do what
POSIX requires in sh emulation mode.

Also, when people don't agree about the precedence of operators, a
solution could be to require parentheses (but I don't know if this
is easy to implement such rules). At least ambiguity is avoided.

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)

Re: arithmetic operator precedence

[Peter Stephenson]

Vincent Lefevre wrote:
> Also, when people don't agree about the precedence of operators, a
> solution could be to require parentheses (but I don't know if this
> is easy to implement such rules). At least ambiguity is avoided.

That would be quite sensible, but the problem is we already have two
cases: traditional zsh behaviour, which we don't want to change now, and
"some sort of" compatilibity mode.  As it looks like "some sort of"
compatibility might as well be with other shells (so the option could
really be called "SH_PRECEDENCES", although it's perhaps less obvious that
means "something approximating to the precedences you probably expect if
you are a regular programmer") this doesn't give us room for manoeuvre
without yet another option.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070

Re: arithmetic operator precedence

[Vincent Lefevre]

On 2008-06-17 12:46:43 +0100, Peter Stephenson wrote:
> That would be quite sensible, but the problem is we already have two
> cases: traditional zsh behaviour, which we don't want to change now, and
> "some sort of" compatilibity mode.  As it looks like "some sort of"
> compatibility might as well be with other shells (so the option could
> really be called "SH_PRECEDENCES", although it's perhaps less obvious that
> means "something approximating to the precedences you probably expect if
> you are a regular programmer") this doesn't give us room for manoeuvre
> without yet another option.

Yes, at least two options are needed. You have the precedences defined
by POSIX (e.g. & vs ==). But concerning the power operator **, this
isn't defined by POSIX, and even the other shells that implement it
may change its behavior and/or make the behavior controlled by an
option.

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)

Re: arithmetic operator precedence

[Jun T.]

At 0:46 PM +0100 08.6.17, Peter Stephenson wrote:
>As it looks like "some sort of"
>compatibility might as well be with other shells (so the option could
>really be called "SH_PRECEDENCES", although it's perhaps less obvious that
>means "something approximating to the precedences you probably expect if
>you are a regular programmer") this doesn't give us room for manoeuvre
>without yet another option.

-3**2 = 9 is quite counter-intuitive and "mathematically wrong",
but If the aim of the new option C_PRECECNECES (of SH_PRECEDENCES) is to
maximize the compatibility with other shells (not with perl etc.),
then I think we are forced to stick to -3**2 = 9.

Even if yet another new option is introduced which changes -3**2 to -9,
a user whould not notice it unless (s)he reads the long zsh man page
carefully.

I think it would be sufficient to add to the man page something like
"-3**2 gives 9, not -9, in zsh and many other shells.
Be sure to use parentheses -(3**2) if you want -9."

Of course I have no objection to adding yet another option.

Re: arithmetic operator precedence

[Stephane Chazelas]

On Thu, Jun 19, 2008 at 06:37:17PM +0900, Jun T. wrote:
> 
> At 0:46 PM +0100 08.6.17, Peter Stephenson wrote:
> >As it looks like "some sort of"
> >compatibility might as well be with other shells (so the option could
> >really be called "SH_PRECEDENCES", although it's perhaps less obvious that
> >means "something approximating to the precedences you probably expect if
> >you are a regular programmer") this doesn't give us room for manoeuvre
> >without yet another option.
> 
> -3**2 = 9 is quite counter-intuitive and "mathematically wrong",
[...]

I don't find it counter-intuitive nor mathematically wrong as
there's no "**" operator of that shape in "maths".

But to rephrase what I was trying to say rather awkwardly
before.

With all the existing operators, one can do

x=$(( some-expression ))
y=$(( some-other-expression ))
z=$(( $x <op> $y ))

and it's OK whatever some-expression and some-other-expression
and <op> as long as they are POSIX.

The syntax above is the only one you can write in a POSIX
script, and POSIX guarantees it to work at the moment.

If you change ** so that it has  higher precedence than the
unary minus, you break that harmony.

Typically, that means you'll have to write:

z=$(( ($x) <op> ($y) ))

because you now have operators that have higher precedence than
the unary "-" and unary "+".

Again ** is not a POSIX operator, so it's not really a concern
at this time, as you can't use ** in a POSIX script anyway. But
that's to show how it could become one if POSIX was to include
"**" in a future version of the standard and set its precedence
as you suggest.

-- 
Stéphane

Re: arithmetic operator precedence

[Vincent Lefevre]

On 2008-06-19 10:54:54 +0100, Stephane Chazelas wrote:
> With all the existing operators, one can do
> 
> x=$(( some-expression ))
> y=$(( some-other-expression ))
> z=$(( $x <op> $y ))
> 
> and it's OK whatever some-expression and some-other-expression
> and <op> as long as they are POSIX.

This is not guaranteed to work. Indeed POSIX allows constants other
than the "canonical" ones to be recognized. With such constants,
you can get wrong results. Whether such constants can occur for some
reason is another matter. In *practice*, it is safer to write:

  z=$(( ($x) <op> ($y) ))

and even safer to write:

  z=$(( x <op> y ))

though it may not work in non-POSIX shells.

> The syntax above is the only one you can write in a POSIX
> script, and POSIX guarantees it to work at the moment.
> 
> If you change ** so that it has  higher precedence than the
> unary minus, you break that harmony.

That's a completely stupid reason. In practice, many shells recognize
other constants, such as "1+1" (this is allowed by POSIX), even more
than those which support **. For instance:

$ a='1+1'; echo $((a * 2)); echo $((3**2))
4
pdksh: 3**2: unexpected `*'

and:

$ a='1+1'; echo $((a * 2)); echo $((3**2))
4
posh: 3**2: unexpected `*'

So, writing z=$(( $x <op> $y )) will not work anyway.

Shells should be designed to work *in practice* (and do what the
user expects to get), not to work specifically on theoretical
implementations that will never exist.

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)

Re: arithmetic operator precedence

[Stephane Chazelas]

On Thu, Jun 19, 2008 at 06:00:24PM +0200, Vincent Lefevre wrote:
> On 2008-06-19 10:54:54 +0100, Stephane Chazelas wrote:
> > With all the existing operators, one can do
> > 
> > x=$(( some-expression ))
> > y=$(( some-other-expression ))
> > z=$(( $x <op> $y ))
> > 
> > and it's OK whatever some-expression and some-other-expression
> > and <op> as long as they are POSIX.
> 
> This is not guaranteed to work. Indeed POSIX allows constants other
> than the "canonical" ones to be recognized. With such constants,
> you can get wrong results. Whether such constants can occur for some
> reason is another matter. In *practice*, it is safer to write:
> 
>   z=$(( ($x) <op> ($y) ))
> 
> and even safer to write:
> 
>   z=$(( x <op> y ))
> 
> though it may not work in non-POSIX shells.

I'm under the impression your misunderstand the purpose of
POSIX. POSIX is a tool to help people write portable
applications (when it comes to shells, that means _scripts_).

You cannot write z=$(( x <op> y ))

in a POSIX script, because the result is unspecified.

z=$(($x <op> $y))

is specified (with the conditions I detailed).

If in your script, you make sure $x and $y are integer constants
(as in [-+]?(0[xX][a-fA-F0-9]+|0[0-7]+|[1-9][0-9]*)), then it
will work as expected.

If $x and $y are valid arithmetic expressions that may not be
integer constants as described above, then you need:

z=$((($x) <op> ($y)))

But I wouldn't do such things in a script.

My example was about $x and $y being the result of arithmetic
expansions.


[...]
> Shells should be designed to work *in practice* (and do what the
> user expects to get), not to work specifically on theoretical
> implementations that will never exist.
[...]

POSIX is not about a theoretical implementation. It's the
specification of an API or a language if you like.

Just like the C spec doesn't specify a theoretical compiler, but
a language. Of course, someone who wants to design a C compiler
must refer to that spec, but the main target of that spec is
people writing C code.

-- 
Stéphane

Re: arithmetic operator precedence

[Vincent Lefevre]

On 2008-06-19 17:20:36 +0100, Stephane Chazelas wrote:
> I'm under the impression your misunderstand the purpose of
> POSIX. POSIX is a tool to help people write portable
> applications (when it comes to shells, that means _scripts_).
> 
> You cannot write z=$(( x <op> y ))
> 
> in a POSIX script, because the result is unspecified.
> 
> z=$(($x <op> $y))
> 
> is specified (with the conditions I detailed).
> 
> If in your script, you make sure $x and $y are integer constants
> (as in [-+]?(0[xX][a-fA-F0-9]+|0[0-7]+|[1-9][0-9]*)), then it
> will work as expected.

I think it is useless to continue the discussion with someone that
hasn't read what POSIX says.

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)

Re: arithmetic operator precedence

[Peter Stephenson]

"Jun T." wrote:
> I think it would be sufficient to add to the man page something like
> "-3**2 gives 9, not -9, in zsh and many other shells.
> Be sure to use parentheses -(3**2) if you want -9."

That seems a good idea in any case.

Index: Doc/Zsh/arith.yo
===================================================================
RCS file: /cvsroot/zsh/zsh/Doc/Zsh/arith.yo,v
retrieving revision 1.13
diff -u -r1.13 arith.yo
--- Doc/Zsh/arith.yo	12 Jun 2008 13:45:05 -0000	1.13
+++ Doc/Zsh/arith.yo	19 Jun 2008 09:56:13 -0000
@@ -93,8 +93,9 @@
 cindex(operators, arithmetic)
 An arithmetic expression uses nearly the same syntax and
 associativity of expressions as in C.
-The following operators are supported (listed in decreasing order
-of precedence):
+
+In the native mode of operation, the following operators are supported
+(listed in decreasing order of precedence):
 
 startsitem()
 sitem(tt(PLUS() - ! ~ PLUS()PLUS() --))(unary plus/minus, logical NOT, complement, {pre,post}{in,de}crement)
@@ -142,6 +143,10 @@
 sitem(tt(,))(comma operator)
 endsitem()
 
+Note the precedence of exponentiation in both cases is below
+that of unary operators, hence `tt(-3**2)' evaluates as `tt(9)', not
+tt(-9).  Use parentheses where necessary: `tt(-(3**2))'.
+
 cindex(mathematical functions, use of)
 cindex(functions, math, use of)
 Mathematical functions can be called with the syntax

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070

Re: arithmetic operator precedence

[Richard Hartmann]

On Thu, Jun 19, 2008 at 11:58, Peter Stephenson <pws@csr.com> wrote:


> That seems a good idea in any case.

Can you add that this is done to remain both backwards and shell-
compatible and that this is not what a 'real' language/calculator
will do? Perhaps even include a reference to the POSIX argument.

Personally, I still think it is better to be compatbile to accepted
math syntax than to the older/other implementations, but.. :)


Richard

Re: arithmetic operator precedence

[Vincent Lefevre]

On 2008-06-19 14:29:03 +0200, Richard Hartmann wrote:
> On Thu, Jun 19, 2008 at 11:58, Peter Stephenson <pws@csr.com> wrote:
> > That seems a good idea in any case.
> 
> Can you add that this is done to remain both backwards and shell-
> compatible and that this is not what a 'real' language/calculator
> will do?

This isn't shell-compatible. AFAIK, only bash and ksh93 support **.

> Perhaps even include a reference to the POSIX argument.

There's no POSIX argument. ** is just an extension.

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)

Re: arithmetic operator precedence

[Mikael Magnusson]

2008/6/19 Vincent Lefevre <vincent@vinc17.org>:
> On 2008-06-19 14:29:03 +0200, Richard Hartmann wrote:
>> On Thu, Jun 19, 2008 at 11:58, Peter Stephenson <pws@csr.com> wrote:
>> > That seems a good idea in any case.
>>
>> Can you add that this is done to remain both backwards and shell-
>> compatible and that this is not what a 'real' language/calculator
>> will do?
>
> This isn't shell-compatible. AFAIK, only bash and ksh93 support **.
>
>> Perhaps even include a reference to the POSIX argument.
>
> There's no POSIX argument. ** is just an extension.

I've followed this discussion, and I'm left wondering only one thing.
If it's decided that $(( -1 ** 2 )) == 1, then what's the point? Why
would anyone ever write that? Why require everyone to write -(1**2) when
the expression makes no sense at all without the parentheses? Ie you would
just write 1**2 without the - at all, nobody would ever write (-1)**2.

-- 
Mikael Magnusson

Re: arithmetic operator precedence

[Stephane Chazelas]

On Thu, Jun 19, 2008 at 06:10:56PM +0200, Mikael Magnusson wrote:
> 2008/6/19 Vincent Lefevre <vincent@vinc17.org>:
> > On 2008-06-19 14:29:03 +0200, Richard Hartmann wrote:
> >> On Thu, Jun 19, 2008 at 11:58, Peter Stephenson <pws@csr.com> wrote:
> >> > That seems a good idea in any case.
> >>
> >> Can you add that this is done to remain both backwards and shell-
> >> compatible and that this is not what a 'real' language/calculator
> >> will do?
> >
> > This isn't shell-compatible. AFAIK, only bash and ksh93 support **.
> >
> >> Perhaps even include a reference to the POSIX argument.
> >
> > There's no POSIX argument. ** is just an extension.
> 
> I've followed this discussion, and I'm left wondering only one thing.
> If it's decided that $(( -1 ** 2 )) == 1, then what's the point? Why
> would anyone ever write that? Why require everyone to write -(1**2) when
> the expression makes no sense at all without the parentheses? Ie you would
> just write 1**2 without the - at all, nobody would ever write (-1)**2.
[...]

Think of:

double() {
  REPLY=$(($1 * 2))
}

square() {
  REPLY=$(($1 ** 2))
}

Where those functions are meant to be passed a number, positive
or negative, not any random arithmetic expression.

If you make ** have higher precedence than unary "-", then
square -1

returns -1

while double -1 returns -2 as expected.

-- 
Stéphane

Re: arithmetic operator precedence

[Vincent Lefevre]

On 2008-06-19 17:27:18 +0100, Stephane Chazelas wrote:
> Think of:
> 
> double() {
>   REPLY=$(($1 * 2))
> }
> 
> square() {
>   REPLY=$(($1 ** 2))
> }

Why are you so stubborn?

double() {
  REPLY=$((($1) * 2))
}

is the correct way to do and works with 'double 1+1' and so on.

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)

Re: arithmetic operator precedence

[Vincent Lefevre]

On 2008-06-19 18:10:56 +0200, Mikael Magnusson wrote:
> I've followed this discussion, and I'm left wondering only one thing.
> If it's decided that $(( -1 ** 2 )) == 1, then what's the point? Why
> would anyone ever write that?

Because this is something valid in most software, and it is too easy
to make a mistake. It would be much better to return an error in that
case.

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)

Re: arithmetic operator precedence

[Richard Hartmann]

On Tue, Jun 17, 2008 at 12:38, Stephane Chazelas
<Stephane_Chazelas@yahoo.fr> wrote:


> Not sure what you mean. ^ in bc is the power operator and
> there's no ** there. In shells, ** is the power operator and ^
> is the XOR operator.

I did not refer to the XOR interpretation of ^, of course.
What I thought you said is that ^ and ** as in

  3^2  = 9
  3**2 = 9

are not the same. Did not make sense to me, but did not want
to discard the idea entirely without checking.


> What I meant is that ^ reminds of the /human/ (as opposed to
> /computer/) representation as it indicates that follows must be
> raised up as in 3². So, one can understand that it should follow
> the same rules (that is -3^2 should be the same as -3², even
> though that's not what POSIX decided for bc). But given that **
> has more the shape of the * operator, I'm not sure we can tell
> the same thing.

I can see what you mean, but the operator, not its representation,
must be must stay the same. Do I understand you correctly that
you think this is the way to go:

  -3^2  = -9
  -3**2 =  9

If yes, that looks _really_ harmful to me.


> Anyway, I was just curious about Peter's statement. I'm most
> probably not as versed in maths as he is, so was curious about
> the rationale behind his statement about -3**2 = 9 not being
> mathematically correct.

See above for my question, requesting clarification.


Richard

Re: arithmetic operator precedence

[Vincent Lefevre]

On 2008-06-17 11:38:29 +0100, Stephane Chazelas wrote:
> What I meant is that ^ reminds of the /human/ (as opposed to
> /computer/) representation as it indicates that follows must be
> raised up as in 3². So, one can understand that it should follow
> the same rules (that is -3^2 should be the same as -3², even
> though that's not what POSIX decided for bc). But given that **
> has more the shape of the * operator, I'm not sure we can tell
> the same thing.

Note that in math, -(a * b) and (-a) * b are equivalent (even in
the general case of a ring), and I'd say that most people regard
- a * b more like -(a * b) than (-a) * b. So, you can't really
compare to * without confusion.

Also ** is merely a replacement for the more conventional (and
shorter) ^ when ^ can't be used (e.g. because it already has
another meaning).

Math writing asside, the fact that -3**2 and 0-3**2 return different
results is quite confusing too.

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)

Re: arithmetic operator precedence

[Vincent Lefevre]

On 2008-06-17 10:45:09 +0100, Stephane Chazelas wrote:
> But ** is not ^, it's a binary operator whose shape reminds that
> of multiply, like a multiply++.

So, why is ** right-associative while * is left-associative?

> And even then, POSIX's ^ in bc is handled as -3^2 = 9.

But note that bc is the only calculator with such an unfortunate
choice. And I doubt that bc has been designed by end users. Also,
perhaps those who wrote bc in the first place didn't think about
this problem and just wanted to privilege the precedence of
unary operators as it is often the case.

> It could be a good idea to ask ksh, POSIX/bc and perl authors for
> the rationale behind their choices.

I think that Perl authors would say something like conventional math
writing (that's what some of authors of calculators say and what users
often demand).

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)

Re: arithmetic operator precedence

[Stephane Chazelas]

On Tue, Jun 17, 2008 at 01:19:34PM +0200, Vincent Lefevre wrote:
> On 2008-06-17 10:45:09 +0100, Stephane Chazelas wrote:
> > But ** is not ^, it's a binary operator whose shape reminds that
> > of multiply, like a multiply++.
> 
> So, why is ** right-associative while * is left-associative?

Don't know. I can't think of a good reason why 2**3**4 shoud be
2**(3**4) rather than (2**3)**4

Or
  4    / 4\                4
 3     \3 / rather than / 3\
2     2                 \2 /


> > And even then, POSIX's ^ in bc is handled as -3^2 = 9.
> 
> But note that bc is the only calculator with such an unfortunate
> choice. And I doubt that bc has been designed by end users. Also,
> perhaps those who wrote bc in the first place didn't think about
> this problem and just wanted to privilege the precedence of
> unary operators as it is often the case.

Which makes sense to me. I was serious when I said that -3 ** 2
was more intuitive to me. Because -3 for me looks like a single
number constant.

BTW, I think I've got a rationale for -3 ** 2 == 9:

I think POSIX allows $((a * 3)) only as far as $a contains a
constant. If it contains "1 + 1", you won't be guaranteed to
have either 6 or 4.

SUSv3> If the shell variable x contains a value that forms a valid
SUSv3> integer constant, then the arithmetic expansions "$((x))" and
SUSv3> "$(($x))" shall return the same value.

So, if you agree that -3 is an integer constant, and agree that
as POSIX says variable expansion should be performed before the
arithmetic expression is evaluated, so that x=3; echo $(($x *
2)) is the same as echo $((-3 ** 2)), that'd mean tha

x=-3; $((x ** 2)) would either have to expand to -9, or $((-3 **
2)) should expand to 9.


> > It could be a good idea to ask ksh, POSIX/bc and perl authors for
> > the rationale behind their choices.
> 
> I think that Perl authors would say something like conventional math
> writing (that's what some of authors of calculators say and what users
> often demand).
[...]

But I don't write 2**3 in conventional math writing, nor do I
write (1+2)/3.

I can understand there be different rules, because of the
linear, one character right to the previous one contraint of
computer strings while we have 2D freedom in hand writing.

1 + 2
-----
  3

has to be translated to (1+2)/3 because of that constraint.

-- 
Stéphane

Re: arithmetic operator precedence

[Vincent Lefevre]

On 2008-06-17 12:57:42 +0100, Stephane Chazelas wrote:
> Which makes sense to me. I was serious when I said that -3 ** 2
> was more intuitive to me. Because -3 for me looks like a single
> number constant.

But it behaves in the same way as - 3, which doesn't look like a
single number constant. And in math, when writing -3^2, the -3 isn't
a single number constant; said otherwise, you need to take the whole
expression into account.

BTW, note that in C at least, -3 isn't a single number constant.
And this fact is visible in practice, e.g. on a machine with 32-bit
int's:

#include <stdio.h>
int main (void)
{
  printf ("%d %d\n",
          (int) sizeof(-2147483648),
          (int) sizeof(-2147483647));
  return 0;
}

outputs "8 4", though the value -2147483648 fits in an int (if
-2147483648 were regarded as a single number constant, one would
have got "4 4").

> BTW, I think I've got a rationale for -3 ** 2 == 9:
> 
> I think POSIX allows $((a * 3)) only as far as $a contains a
> constant. If it contains "1 + 1", you won't be guaranteed to
> have either 6 or 4.

But this rationale doesn't apply if the shell chooses to return 6
(which is IMHO the right extension, and what both bash and zsh do),
i.e. you can see $((a * 3)) as equivalent to $(((a) * 3)). Indeed,
with this choice, -3**2 == -9 is consistent with:

  $ a=-3
  $ echo $((a ** 2))
  9

> So, if you agree that -3 is an integer constant, and agree that
> as POSIX says variable expansion should be performed before the
> arithmetic expression is evaluated, so that x=3; echo $(($x *
> 2)) is the same as echo $((-3 ** 2)), that'd mean tha

But POSIX does not say that variable expansion should be performed
before the arithmetic expression (at least in th way you think).
If POSIX said so, then with a="1 + 1", $((a * 3)) would give
$((1 + 1 * 3)) after variable expansion (without being able to make
the difference with $((1 + 1 * 3)) written directly), then 5 after
arithmetic expansion. But neither zsh nor bash behaves in that way.

> > I think that Perl authors would say something like conventional math
> > writing (that's what some of authors of calculators say and what users
> > often demand).
> [...]
> 
> But I don't write 2**3 in conventional math writing,

But there is a direct equivalence (the only ambiguity is where the
part in exponent ends, and there's an rule in languages for that).

Let's take another example: "-3!" which is conventional math writing.
In math, this means -(3!), i.e. -6. As you can see, -3 isn't regarded
as a constant. And if a shell (or some other language) wants to
support the factorial, it should regard -3! as -(3!).

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)

Re: arithmetic operator precedence

[Stephane Chazelas]

On Tue, Jun 17, 2008 at 02:35:51PM +0200, Vincent Lefevre wrote:
[...]
> But POSIX does not say that variable expansion should be performed
> before the arithmetic expression (at least in th way you think).
> If POSIX said so, then with a="1 + 1", $((a * 3)) would give
> $((1 + 1 * 3)) after variable expansion (without being able to make
> the difference with $((1 + 1 * 3)) written directly), then 5 after
> arithmetic expansion. But neither zsh nor bash behaves in that way.
[...]

$((a * 3)) is unspecified as per POSIX if $a doesn't contain an
integer constant. That was my point.

If $a contains an integer constant such as -3, then as per
POSIX, $((a * 3)) should be the same as $(($a * 3)), that is
$((-3 * 3)). If we extend that to the non-POSIX **, that would
be:

a=-3; $((a ** 2)) should be the same as $((-3 ** 2)).

So, if a future version of POSIX adds a **, it is bound to
specify it as $((-3 ** 2)) == 9 if it wants to keep consistant.

That would even be in accordance with all the existing shell
implementations that have **.

So, I think that with SH_PRECEDENCE, $((-3 ** 2)) should give 9.

-- 
Stéphane

Re: arithmetic operator precedence

[Vincent Lefevre]

On 2008-06-17 13:46:07 +0100, Stephane Chazelas wrote:
> On Tue, Jun 17, 2008 at 02:35:51PM +0200, Vincent Lefevre wrote:
> [...]
> > But POSIX does not say that variable expansion should be performed
> > before the arithmetic expression (at least in th way you think).
> > If POSIX said so, then with a="1 + 1", $((a * 3)) would give
> > $((1 + 1 * 3)) after variable expansion (without being able to make
> > the difference with $((1 + 1 * 3)) written directly), then 5 after
> > arithmetic expansion. But neither zsh nor bash behaves in that way.
> [...]
> 
> $((a * 3)) is unspecified as per POSIX if $a doesn't contain an
> integer constant. That was my point.

It is unspecified. So, the shell is right to choose how it sees it.

> If $a contains an integer constant such as -3, then as per
> POSIX, $((a * 3)) should be the same as $(($a * 3)), that is
> $((-3 * 3)).

No, POSIX does not say that. It happens to be the same thing here
just because of the properties of *, but you can't deduce anything
for extensions. And in practice, shells don't treat $((a * 3)) and
$(($a * 3)) in the same way:

vin:~> a="1 + 1"
vin:~> echo $((a * 3))
6
vin:~> echo $(($a * 3))
4

> If we extend that to the non-POSIX **, that would be:
> 
> a=-3; $((a ** 2)) should be the same as $((-3 ** 2)).

No, if we extend POSIX in an intuitive way (see above), $((a ** 2))
should be the same as $(((-3) ** 2)), hence 9. So, no problem with
$((-3**2)) being -9.

> So, if a future version of POSIX adds a **, it is bound to
> specify it as $((-3 ** 2)) == 9 if it wants to keep consistant.

Choosing -9 is also consistent (see above).

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)

Re: arithmetic operator precedence

[Stephane Chazelas]

On Tue, Jun 17, 2008 at 03:02:46PM +0200, Vincent Lefevre wrote:
[...]
> It is unspecified. So, the shell is right to choose how it sees it.
> 
> > If $a contains an integer constant such as -3, then as per
> > POSIX, $((a * 3)) should be the same as $(($a * 3)), that is
> > $((-3 * 3)).
> 
> No, POSIX does not say that. It happens to be the same thing here
> just because of the properties of *, but you can't deduce anything
> for extensions.

It says $((x)) is meant to be the same as $(($x)) which I
understand as any occurrance of a variable name (other than $-,
$?, $0... obviously) in $((...)) should be the same as if the $
was not ommited (when $x contains an integer constant).

That's a recent addition to the text. Only recent versions of
ash (BSD shs) support that for instance, which is why it has
been recommanded for a while to write it $(($x + 1)) instead of
$((x + 1)) in POSIX scripts. And I still use x=$(($x + 1)).

For newer ashes, $((x + 1)) is now accepted and note that you do
get an error in ash if $x is not an integer constant.

> And in practice, shells don't treat $((a * 3)) and
> $(($a * 3)) in the same way:
> 
> vin:~> a="1 + 1"
> vin:~> echo $((a * 3))
> 6
> vin:~> echo $(($a * 3))
> 4

But here, $a doesn't contain an integer constant, that's out of
the scope of POSIX.

> 
> > If we extend that to the non-POSIX **, that would be:
> > 
> > a=-3; $((a ** 2)) should be the same as $((-3 ** 2)).
> 
> No, if we extend POSIX in an intuitive way (see above), $((a ** 2))
> should be the same as $(((-3) ** 2)), hence 9. So, no problem with
> $((-3**2)) being -9.
[...]

No, POSIX does say that $((a ** 2)) is the same as $(($a ** 2))
because $a contains an integer constant, and that's $((-3**2)).

-- 
Stéphane

Re: arithmetic operator precedence

[Vincent Lefevre]

On 2008-06-17 14:20:39 +0100, Stephane Chazelas wrote:
> On Tue, Jun 17, 2008 at 03:02:46PM +0200, Vincent Lefevre wrote:
> [...]
> > It is unspecified. So, the shell is right to choose how it sees it.
> > 
> > > If $a contains an integer constant such as -3, then as per
> > > POSIX, $((a * 3)) should be the same as $(($a * 3)), that is
> > > $((-3 * 3)).
> > 
> > No, POSIX does not say that. It happens to be the same thing here
> > just because of the properties of *, but you can't deduce anything
> > for extensions.
> 
> It says $((x)) is meant to be the same as $(($x))

Yes, but not more, and only if x is a valid integer constant.
Note that recognizing negative constants is not required by POSIX;
so, they are extensions, just like "1+1".

BTW, I've just sent a mail to the Austin group about that and other
things related to arithmetic expansion.

> which I understand as any occurrance of a variable name (other than
> $-, $?, $0... obviously) in $((...)) should be the same as if the $
> was not ommited (when $x contains an integer constant).

No, POSIX doesn't say that. This sentence is a mean to define the
value of x from the contents of $x. But note that parsing has already
been done and you have something like a C expression ("The arithmetic
expression shall be processed according to the rules given in
Arithmetic Precision and Operations"); variables are just replaced
by their values, like in C. Without any extension, both interpretations
are equivalent anyway.

> That's a recent addition to the text. Only recent versions of
> ash (BSD shs) support that for instance, which is why it has
> been recommanded for a while to write it $(($x + 1)) instead of
> $((x + 1)) in POSIX scripts. And I still use x=$(($x + 1)).

But if you want to use extensions and/or more complex expressions,
you should add parentheses, e.g. x=$((($x) + 1)). This will also
work.

> > And in practice, shells don't treat $((a * 3)) and
> > $(($a * 3)) in the same way:
> > 
> > vin:~> a="1 + 1"
> > vin:~> echo $((a * 3))
> > 6
> > vin:~> echo $(($a * 3))
> > 4
> 
> But here, $a doesn't contain an integer constant, that's out of
> the scope of POSIX.

Not completely. The shell may recognize "1 + 1" as an integer constant
as an extension, in which case 6 would be the only valid result. Note
that negative numbers are in the same case.

Also ** is out of the scope of POSIX too.

> No, POSIX does say that $((a ** 2)) is the same as $(($a ** 2))
> because $a contains an integer constant, and that's $((-3**2)).

No (see above). You are 3 times wrong. POSIX doesn't say anything
about **, POSIX doesn't say that non-trivial expressions $((a ...))
and $(($a ...)) are equivalent (and in pratice, they aren't), POSIX
doesn't say anything about negative constants (probably a bug, but
the whole section needs to be improved anyway). Perhaps instead of
saying falsehood, you should read the standards.

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)

Re: arithmetic operator precedence

[Stephane Chazelas]

On Tue, Jun 17, 2008 at 04:33:57PM +0200, Vincent Lefevre wrote:
[...]
> > which I understand as any occurrance of a variable name (other than
> > $-, $?, $0... obviously) in $((...)) should be the same as if the $
> > was not ommited (when $x contains an integer constant).
> 
> No, POSIX doesn't say that. This sentence is a mean to define the
> value of x from the contents of $x. But note that parsing has already
> been done and you have something like a C expression ("The arithmetic
> expression shall be processed according to the rules given in
> Arithmetic Precision and Operations"); variables are just replaced
> by their values, like in C. Without any extension, both interpretations
> are equivalent anyway.

That's one interpretation.

[...]
> Also ** is out of the scope of POSIX too.

Yes, as I said I think clearly enough, if POSIX were to specify
** _in a future version of the standard_, it is more likely that
it implements it as -3 ** 2 == 9, to avoid confusion.

- because it doesn't at the moment clearly state how bare
  variable names are to be interpreted as you point out (it's true
  that I was interpreting maybe a bit too much)

- because it doesn't clearly state whether -3 is an integer
  constant or not. ash allows a=-3; echo $((a * 3)) while it
  doesn't allow a=0-3; echo $((a * 3)) for instance. But then,
  given that a negative number can be the result of an
  arithmetic expansion, I can't see how it can be interpreted in
  any other way than -3 is an integer constant.

- because all existing shell implementations do it that way at
  the moment.

> > No, POSIX does say that $((a ** 2)) is the same as $(($a ** 2))
> > because $a contains an integer constant, and that's $((-3**2)).
> 
> No (see above). You are 3 times wrong. POSIX doesn't say anything
> about **, POSIX doesn't say that non-trivial expressions $((a ...))
> and $(($a ...)) are equivalent (and in pratice, they aren't), POSIX
> doesn't say anything about negative constants (probably a bug, but
> the whole section needs to be improved anyway). Perhaps instead of
> saying falsehood, you should read the standards.
[...]

No need to be rude.


Now, I think this has probably been discussed enough by now.

What is certain is that I will keep writing x=$(($x + 1)) and
make sure all the variables I use in arithmetic expansions
contain integer constants (negative or not), and if for some
obscure reason, I want to write a non-POSIX script meant to be
portable to ksh93, bash and zsh, I'll write

x=$((($y)**3))

as some shell maintainer might want to follow your advice in the
future and change the behavior so that

x=$(($y**3)) doesn't work anymore as I would expect it to.

I'll stop digressing on this on the list, please feel free to
follow up in private if you're keen.

-- 
Stéphane

Re: arithmetic operator precedence

[Vincent Lefevre]

On 2008-06-17 15:53:38 +0100, Stephane Chazelas wrote:
> On Tue, Jun 17, 2008 at 04:33:57PM +0200, Vincent Lefevre wrote:
> [...]
> > > which I understand as any occurrance of a variable name (other than
> > > $-, $?, $0... obviously) in $((...)) should be the same as if the $
> > > was not ommited (when $x contains an integer constant).
> > 
> > No, POSIX doesn't say that. This sentence is a mean to define the
> > value of x from the contents of $x. But note that parsing has already
> > been done and you have something like a C expression ("The arithmetic
> > expression shall be processed according to the rules given in
> > Arithmetic Precision and Operations"); variables are just replaced
> > by their values, like in C. Without any extension, both interpretations
> > are equivalent anyway.
> 
> That's one interpretation.

And that's the most intuitive one I can see. And that's why both bash
and zsh follow this interpretation.

> [...]
> > Also ** is out of the scope of POSIX too.
> 
> Yes, as I said I think clearly enough, if POSIX were to specify
> ** _in a future version of the standard_,

No, you said: "POSIX does say that $((a ** 2)) is the same as $(($a ** 2))
because $a contains an integer constant, and that's $((-3**2))."

> it is more likely that it implements it as -3 ** 2 == 9, to avoid
> confusion.

That's your opinion. I'd say it is very unlikely to regard
$((a ** 2)) as $(($a ** 2)) because
  1. It is very confusing (more precisely, the $(($a ** 2)) form is
     confusing as far arithmetic evaluation is concerned, and that's
     probably why the form with variables in arithmetic expressions
     have been added).
  2. Several shells (bash, zsh) do it in another way, and AFAIK,
     no-one has complained on their behavior.

> - because it doesn't at the moment clearly state how bare
>   variable names are to be interpreted as you point out (it's true
>   that I was interpreting maybe a bit too much)

As it does a reference to C expressions, I'd say that it should do
like C. Note that with your interpretation, two passes are necessary
if the shell regards advanced forms of integer constants (such as
"1 + 1"): one to identify the variable names, and one to reparse the
expression after replacing the variable names.

> - because it doesn't clearly state whether -3 is an integer
>   constant or not. ash allows a=-3; echo $((a * 3)) while it
>   doesn't allow a=0-3; echo $((a * 3)) for instance. But then,
>   given that a negative number can be the result of an
>   arithmetic expansion, I can't see how it can be interpreted in
>   any other way than -3 is an integer constant.

Yes, that's why it's probably a bug (but POSIX doesn't even say
that the shell should be able to re-read the output value).

> - because all existing shell implementations do it that way at
>   the moment.

Only for precedence reasons (probably without much reflection and
because the unnecessary precedence of the unary - over * was already
there), certainly not because $((a ** 2)) and $(($a ** 2)) are
regarded as the same: they are handled differently.

BTW, when using directed rounding modes, this is very confusing in C.

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)

Re: arithmetic operator precedence

[Stephane Chazelas]

On Tue, Jun 17, 2008 at 02:20:39PM +0100, Stephane Chazelas wrote:
> On Tue, Jun 17, 2008 at 03:02:46PM +0200, Vincent Lefevre wrote:
> [...]
> > It is unspecified. So, the shell is right to choose how it sees it.
> > 
> > > If $a contains an integer constant such as -3, then as per
> > > POSIX, $((a * 3)) should be the same as $(($a * 3)), that is
> > > $((-3 * 3)).
> > 
> > No, POSIX does not say that. It happens to be the same thing here
> > just because of the properties of *, but you can't deduce anything
> > for extensions.
> 
> It says $((x)) is meant to be the same as $(($x)) which I
> understand as any occurrance of a variable name (other than $-,
> $?, $0... obviously) in $((...)) should be the same as if the $
> was not ommited (when $x contains an integer constant).
[...]

It can't be that simple though, as in

x=-1; $((1 x))

which would be POSIX but fails in all shells.

Or

x=1; $((x = 1 + x))

which obviously is not meant to be the same as $(($x = 1 + $x))

So that

  If the shell variable x contains a value that forms a valid
  integer constant, then the arithmetic expansions "$((x))" and
  "$(($x))" shall return the same value.

says either too little or too much to be useful.

-- 
Stéphane

Re: arithmetic operator precedence

[Vincent Lefevre]

On 2008-06-17 15:35:59 +0100, Stephane Chazelas wrote:
> It can't be that simple though, as in
> 
> x=-1; $((1 x))
> 
> which would be POSIX but fails in all shells.

No, $((1 x)) is an invalid expression (unless the shell sees this
as an extension), so POSIX requires it to fail.

This is different from $((1 $x)), where the expansion of $x occurs
first.

> Or
> 
> x=1; $((x = 1 + x))
> 
> which obviously is not meant to be the same as $(($x = 1 + $x))

So? What's the problem with them not being the same?

Note that POSIX says:

  The shell shall expand all tokens in the expression for parameter
                                                          ^^^^^^^^^
  expansion, command substitution, and quote removal.
  ^^^^^^^^^

  Next, the shell shall treat this as an arithmetic expression
  ^^^^

In the first case, the expression is x = 1 + x. In the second case,
it is 1 = 1 + 1.

> So that
> 
>   If the shell variable x contains a value that forms a valid
>   integer constant, then the arithmetic expansions "$((x))" and
>   "$(($x))" shall return the same value.
> 
> says either too little or too much to be useful.

I agree that this isn't perfect (BTW, see my mail in the Austin group
when it appears -- I haven't received a copy yet).

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)

Re: arithmetic operator precedence

[Vincent Lefevre]

On 2008-06-17 11:19:40 +0200, Richard Hartmann wrote:
> Being unsure myself, I did some somewhat extensive research. The only
> mathematically correct way of doing this is
> 
>   -3**2   = -9
>   (-3)**2 =  9
> 
> On the other hand, compability with the other shells is A Good Thing,
> especially as this is the kind of syntax change that can break a script
> in a very, very evil and hard-to-debug way.

** is a non-portable feature. Portable scripts should not be based on
it. And it is an even more idea to base a script on the precedence of
the unary - over **, since this rule doesn't follow conventional math
writing. For interoperability with other software, it is advised to
put parentheses in both cases anyway.

Such writings without explicit parentheses are useful mainly for the
end user, in particular when typing expressions interactively, and
conventional math writing is important here.

> Still, all things considered, I would say the only possible way to
> go is to use the mathematically correct syntax. Both options are
> bad, in a way, but this is the better one.

Yes. There would also be the option to force parentheses. For instance,
this is the case with Maple and the associativity of ^: the user cannot
write 2^3^2 (Maple returns an error), he must use parentheses: 2^(3^2)
or (2^3)^2.

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)