[PATCH] math recursion and array overflow

[Bart Schaefer <schaefer@brasslantern.com>]

[-- Attachment #1: Type: text/plain, Size: 790 bytes --]

Math operations normally abort if recursion gets deeper than MAX_MLEVEL (256).

However, if the recursion is happening in an array subscript, it can
rewind and try again until eventually we try to calculate the length
of an empty array expansion, and kaboom.

This happens because parse_subscript() clears the error flag while
using the lexer, so instead of stopping at the recursion limit we just
try again on the next subscript until something goes wrong.

The easiest way to reproduce is to use a nameref:

n=(1)
typeset -n i='n[++i]'
print $i

The following fixes it by aborting math operators upon operand error,
but there might be other more elaborate ways to set it off.  I tried
some combinations using user-defined math functions but didn't find a
failing recursive call strategy.

[-- Attachment #2: math-array-overflow.txt --]
[-- Type: text/plain, Size: 647 bytes --]

diff --git a/Src/math.c b/Src/math.c
index a060181ed..50b69d6a1 100644
--- a/Src/math.c
+++ b/Src/math.c
@@ -352,6 +352,8 @@ getmathparam(struct mathvalue *mptr)
 	    }
 	    return zero_mnumber;
 	}
+	if (errflag)
+	    return zero_mnumber;
     }
     result = getnumvalue(mptr->pval);
     if (isset(FORCEFLOAT) && result.type == MN_INTEGER) {
@@ -1367,8 +1369,11 @@ op(int what)
     }
 
     spval = &stack[sp].val;
-    if (stack[sp].val.type == MN_UNSET)
+    if (stack[sp].val.type == MN_UNSET) {
 	*spval = getmathparam(stack + sp);
+	if (errflag)
+	    return;
+    }
     switch (what) {
     case NOT:
 	if (spval->type & MN_FLOAT) {