[musl] bug: integer overflow in memmem()

[Alfred Agrell <alfred@agrell.info>]

To reproduce: Compile src/string/memmem.c with -fsanitize=undefined, then

int main()
{
char a[4] = { -1,-1,-1,-1 };
memmem(a, 4, a, 3);
memmem(a, 4, a, 4);
}

Expected result: No output

Actual (Ubuntu 18.04 x86_64, gcc 7.5.0, ):

memmem.c:15:20: runtime error: left shift of 255 by 24 places cannot be 
represented in type 'int'
memmem.c:16:20: runtime error: left shift of 255 by 24 places cannot be 
represented in type 'int'
memmem.c:24:20: runtime error: left shift of 255 by 24 places cannot be 
represented in type 'int'
memmem.c:25:20: runtime error: left shift of 255 by 24 places cannot be 
represented in type 'int'

C's integer promotion rules are fairly unintuitive for <<; it promotes 
unsigned small LHS to signed. To fix, change the two n[0]<<24 to 
(uint32_t)n[0]<<24, and similar for h[0]<<24.

I'm not aware of any compiler on any platform where it'll actually 
break, so your choice whether this is a real bug. I didn't check if 
similar issues exist elsewhere across musl.

I'm not subscribed to the list; I'll read the archives, but if you want 
a timely response, please cc me.

-- Alfred Agrell