[PATCH] math: fix signed int left shift ub in sqrt

[Szabolcs Nagy <nsz@port70.net>]

[-- Attachment #1: Type: text/plain, Size: 50 bytes --]

old fdlibm bug, only affects soft float targets.


[-- Attachment #2: 0001-math-fix-signed-int-left-shift-ub-in-sqrt.patch --]
[-- Type: text/x-diff, Size: 1230 bytes --]

From b877237b506e03fd3c92ad7b39adbac5e1b74441 Mon Sep 17 00:00:00 2001
From: Szabolcs Nagy <nsz@port70.net>
Date: Sun, 13 Oct 2019 14:54:31 +0000
Subject: [PATCH] math: fix signed int left shift ub in sqrt

Both sqrt and sqrtf shifted the signed exponent as signed int to adjust
the bit representation of the result. There are signed right shifts too
in the code but those are implementation defined and are expected to
compile to arithmetic shift on supported compilers and targets.
---
 src/math/sqrt.c  | 3 +--
 src/math/sqrtf.c | 3 +--
 2 files changed, 2 insertions(+), 4 deletions(-)

diff --git a/src/math/sqrt.c b/src/math/sqrt.c
index b2775673..f1f6d76c 100644
--- a/src/math/sqrt.c
+++ b/src/math/sqrt.c
@@ -179,7 +179,6 @@ double sqrt(double x)
 	ix1 = q1>>1;
 	if (q&1)
 		ix1 |= sign;
-	ix0 += m << 20;
-	INSERT_WORDS(z, ix0, ix1);
+	INSERT_WORDS(z, ix0 + ((uint32_t)m << 20), ix1);
 	return z;
 }
diff --git a/src/math/sqrtf.c b/src/math/sqrtf.c
index 28cb4ad3..d6ace38a 100644
--- a/src/math/sqrtf.c
+++ b/src/math/sqrtf.c
@@ -78,7 +78,6 @@ float sqrtf(float x)
 		}
 	}
 	ix = (q>>1) + 0x3f000000;
-	ix += m << 23;
-	SET_FLOAT_WORD(z, ix);
+	SET_FLOAT_WORD(z, ix + ((uint32_t)m << 23));
 	return z;
 }
-- 
2.23.0