[PATCH v2] fix a bug in the rand48 family of prng

[Jens Gustedt <Jens.Gustedt@inria.fr>]

[-- Attachment #1: Type: text/plain, Size: 914 bytes --]


This fixes a bug found by Nadav Har'El, who observed that musl was giving
different prn sequences than other systems, even if seeded with the same
value.

The problem with something like

a = lc[0] | lc[1]<<16 | lc[2]+0ULL<<32;

where lc[1] is an unsigned short and int is 32bit is the following

(1) lc[1] is promoted to int
(2) the left shift 16 is performed on int

this is UB if bit 15 is set in lc[1], since it moves a 1 into the sign
bit.

In particular, bit 15 *is* 1 for the default multplicator A as defined by POSIX.

(On systems with 16 bit int all of this has UB anyhow.)

If the platform defines the behavior (most do) we are still not
saved. Then we have an bit-or between an int and an ullong. If the int is
negative sign extension sets all the high bits of the result, messing up
the computation completely.
---
 src/prng/__rand48_step.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)


[-- Attachment #2: 0001-fix-a-bug-in-the-rand48-family-of-prng.patch --]
[-- Type: text/x-patch, Size: 480 bytes --]

diff --git a/src/prng/__rand48_step.c b/src/prng/__rand48_step.c
index ccaffc3..961d30f 100644
--- a/src/prng/__rand48_step.c
+++ b/src/prng/__rand48_step.c
@@ -3,8 +3,8 @@
 uint64_t __rand48_step(unsigned short *xi, unsigned short *lc)
 {
 	uint64_t a, x;
-	x = xi[0] | xi[1]<<16 | xi[2]+0ULL<<32;
-	a = lc[0] | lc[1]<<16 | lc[2]+0ULL<<32;
+	x = xi[0] | xi[1]+0U<<16 | xi[2]+0ULL<<32;
+	a = lc[0] | lc[1]+0U<<16 | lc[2]+0ULL<<32;
 	x = a*x + lc[3];
 	xi[0] = x;
 	xi[1] = x>>16;