musl segfaulting when compiled with -O2

musl segfaulting when compiled with -O2

[Breno Leitao]

Musl package on Debian on ppc64le is broken.

When running any software with it, it segfaults. Doing a little bit of
debugging I found that libc.so is broken.

I got the upstream code, and found that the problme is also
reproducible.

I found that the problem only happen when compiling with -O2 and -O3. If
I compile musl with -O1 or -O0, the problm does not happen.

This is the bt of the code that crashes:

(gdb) bt
#0  0x0000000148b84dc0 in ?? ()
#1  0x0000000048bdb8dc in _dlstart_c (sp=0x3fffc33294b0, dynv=<optimized out>) at ldso/dlstart.c:147
#2  0x0000000048bdebe0 in _dlstart ()

(gdb) up
#1  0x0000000048bdb8dc in _dlstart_c (sp=0x3fffc33294b0, dynv=<optimized out>) at ldso/dlstart.c:147
147		dls2((void *)base, sp);

$ gcc --version
gcc (Debian 6.3.0-5) 6.3.0 20170124

Not sure if this is a GCC issue or a musl issue now.

Re: musl segfaulting when compiled with -O2

[Rich Felker]

On Tue, Mar 07, 2017 at 05:32:35PM -0300, Breno Leitao wrote:
> Musl package on Debian on ppc64le is broken.
> 
> When running any software with it, it segfaults. Doing a little bit of
> debugging I found that libc.so is broken.
> 
> I got the upstream code, and found that the problme is also
> reproducible.
> 
> I found that the problem only happen when compiling with -O2 and -O3. If
> I compile musl with -O1 or -O0, the problm does not happen.
> 
> This is the bt of the code that crashes:
> 
> (gdb) bt
> #0  0x0000000148b84dc0 in ?? ()
> #1  0x0000000048bdb8dc in _dlstart_c (sp=0x3fffc33294b0, dynv=<optimized out>) at ldso/dlstart.c:147
> #2  0x0000000048bdebe0 in _dlstart ()
> 
> (gdb) up
> #1  0x0000000048bdb8dc in _dlstart_c (sp=0x3fffc33294b0, dynv=<optimized out>) at ldso/dlstart.c:147
> 147		dls2((void *)base, sp);
> 
> $ gcc --version
> gcc (Debian 6.3.0-5) 6.3.0 20170124
> 
> Not sure if this is a GCC issue or a musl issue now.

Can you post the crashing libc.so somewhere? Depending on how they're
building it, the problem could be in various different places. A
glibc-targeted powerpc* gcc is not suitable for compiling musl, since
it has various ABI mismatches, but that's probably not what you're
seeing here even if one was used.

Rich

Re: musl segfaulting when compiled with -O2

[Breno Leitao]

Hi Rich,

On Tue, Mar 07, 2017 at 03:39:05PM -0500, Rich Felker wrote:
> On Tue, Mar 07, 2017 at 05:32:35PM -0300, Breno Leitao wrote:
> > Musl package on Debian on ppc64le is broken.
> > 
> > When running any software with it, it segfaults. Doing a little bit of
> > debugging I found that libc.so is broken.
> > 
> > I got the upstream code, and found that the problme is also
> > reproducible.
> > 
> > I found that the problem only happen when compiling with -O2 and -O3. If
> > I compile musl with -O1 or -O0, the problm does not happen.
> > 
> > This is the bt of the code that crashes:
> > 
> > (gdb) bt
> > #0  0x0000000148b84dc0 in ?? ()
> > #1  0x0000000048bdb8dc in _dlstart_c (sp=0x3fffc33294b0, dynv=<optimized out>) at ldso/dlstart.c:147
> > #2  0x0000000048bdebe0 in _dlstart ()
> > 
> > (gdb) up
> > #1  0x0000000048bdb8dc in _dlstart_c (sp=0x3fffc33294b0, dynv=<optimized out>) at ldso/dlstart.c:147
> > 147		dls2((void *)base, sp);
> > 
> > $ gcc --version
> > gcc (Debian 6.3.0-5) 6.3.0 20170124
> > 
> > Not sure if this is a GCC issue or a musl issue now.
> 
> Can you post the crashing libc.so somewhere? Depending on how they're
> building it, the problem could be in various different places. A
> glibc-targeted powerpc* gcc is not suitable for compiling musl, since
> it has various ABI mismatches, but that's probably not what you're
> seeing here even if one was used.

Sure. You can grab it directly from Debian:

  $ wget http://cdn-fastly.deb.debian.org/debian/pool/main/m/musl/musl_1.1.16-2_ppc64el.deb

  $ dpkg-deb -x musl_1.1.16-2_ppc64el.deb .

  $ lib/powerpc64le-linux-musl/libc.so 
[2]    25672 segmentation fault (core dumped) lib/powerpc64le-linux-musl/libc.so

Re: musl segfaulting when compiled with -O2

[Rich Felker]

On Tue, Mar 07, 2017 at 05:55:29PM -0300, Breno Leitao wrote:
> Hi Rich,
> 
> On Tue, Mar 07, 2017 at 03:39:05PM -0500, Rich Felker wrote:
> > On Tue, Mar 07, 2017 at 05:32:35PM -0300, Breno Leitao wrote:
> > > Musl package on Debian on ppc64le is broken.
> > > 
> > > When running any software with it, it segfaults. Doing a little bit of
> > > debugging I found that libc.so is broken.
> > > 
> > > I got the upstream code, and found that the problme is also
> > > reproducible.
> > > 
> > > I found that the problem only happen when compiling with -O2 and -O3. If
> > > I compile musl with -O1 or -O0, the problm does not happen.
> > > 
> > > This is the bt of the code that crashes:
> > > 
> > > (gdb) bt
> > > #0  0x0000000148b84dc0 in ?? ()
> > > #1  0x0000000048bdb8dc in _dlstart_c (sp=0x3fffc33294b0, dynv=<optimized out>) at ldso/dlstart.c:147
> > > #2  0x0000000048bdebe0 in _dlstart ()
> > > 
> > > (gdb) up
> > > #1  0x0000000048bdb8dc in _dlstart_c (sp=0x3fffc33294b0, dynv=<optimized out>) at ldso/dlstart.c:147
> > > 147		dls2((void *)base, sp);
> > > 
> > > $ gcc --version
> > > gcc (Debian 6.3.0-5) 6.3.0 20170124
> > > 
> > > Not sure if this is a GCC issue or a musl issue now.
> > 
> > Can you post the crashing libc.so somewhere? Depending on how they're
> > building it, the problem could be in various different places. A
> > glibc-targeted powerpc* gcc is not suitable for compiling musl, since
> > it has various ABI mismatches, but that's probably not what you're
> > seeing here even if one was used.
> 
> Sure. You can grab it directly from Debian:
> 
>   $ wget http://cdn-fastly.deb.debian.org/debian/pool/main/m/musl/musl_1.1.16-2_ppc64el.deb
> 
>   $ dpkg-deb -x musl_1.1.16-2_ppc64el.deb .
> 
>   $ lib/powerpc64le-linux-musl/libc.so 
> [2]    25672 segmentation fault (core dumped) lib/powerpc64le-linux-musl/libc.so

Can you do a register dump and a dissassembly dump and around the
point that crashes? The Debian package is fully stripped (lacks debug
symbols) so it's hard to follow what it's doing, but I didn't see
anything obviously wrong.

Rich

Re: musl segfaulting when compiled with -O2

[Szabolcs Nagy]

* Rich Felker <dalias@libc.org> [2017-03-07 16:45:40 -0500]:
> On Tue, Mar 07, 2017 at 05:55:29PM -0300, Breno Leitao wrote:
> >   $ wget http://cdn-fastly.deb.debian.org/debian/pool/main/m/musl/musl_1.1.16-2_ppc64el.deb
> > 
> >   $ dpkg-deb -x musl_1.1.16-2_ppc64el.deb .
> > 
> >   $ lib/powerpc64le-linux-musl/libc.so 
> > [2]    25672 segmentation fault (core dumped) lib/powerpc64le-linux-musl/libc.so
> 
> Can you do a register dump and a dissassembly dump and around the
> point that crashes? The Debian package is fully stripped (lacks debug
> symbols) so it's hard to follow what it's doing, but I didn't see
> anything obviously wrong.
> 
> Rich

i used qemu-ppc64le -singlestep -d in_asm,cpu to debug this

the relevant code is the end of _dlstart_c:
...
   8c574:       09 00 00 48     bl      8c57c <writev+0x28c>
   8c578:       28 98 fa ff     fsub    f31,f26,f19
   8c57c:       a6 02 48 7d     mflr    r10
   8c580:       00 00 2a 81     lwz     r9,0(r10)
   8c584:       14 52 29 7d     add     r9,r9,r10
   8c588:       a6 03 29 7d     mtctr   r9
   8c58c:       18 00 41 f8     std     r2,24(r1)
   8c590:       78 3b e4 7c     mr      r4,r7
   8c594:       78 4b 2c 7d     mr      r12,r9
   8c598:       21 04 80 4e     bctrl

it's a pc relative address load and jump (to call __dls2):

   bl 1f
   data          // data == 0xfffa9828
1: mflr r10      // r10 = &data == 0x8c57c
   lwz r9,0(r10) // r9 = *r10 (== data)
   add r9,r9,r10 // r9 += r10 (== 0xfffa9828+0x8c57c = 0x100035da0)
   mtctr r9      // ctr = r9 (== 0x100035da0)
   std r2,24(r1)
   mr r4,r7
   mr r12,r9
   bctrl         // ctr(r12,r4)

it seems __dls2 is at 0x35da0 so the pc relative data
should have been signextended.

Re: musl segfaulting when compiled with -O2

[Rich Felker]

[-- Attachment #1: Type: text/plain, Size: 1909 bytes --]

On Wed, Mar 08, 2017 at 02:18:43AM +0100, Szabolcs Nagy wrote:
> * Rich Felker <dalias@libc.org> [2017-03-07 16:45:40 -0500]:
> > On Tue, Mar 07, 2017 at 05:55:29PM -0300, Breno Leitao wrote:
> > >   $ wget http://cdn-fastly.deb.debian.org/debian/pool/main/m/musl/musl_1.1.16-2_ppc64el.deb
> > > 
> > >   $ dpkg-deb -x musl_1.1.16-2_ppc64el.deb .
> > > 
> > >   $ lib/powerpc64le-linux-musl/libc.so 
> > > [2]    25672 segmentation fault (core dumped) lib/powerpc64le-linux-musl/libc.so
> > 
> > Can you do a register dump and a dissassembly dump and around the
> > point that crashes? The Debian package is fully stripped (lacks debug
> > symbols) so it's hard to follow what it's doing, but I didn't see
> > anything obviously wrong.
> > 
> > Rich
> 
> i used qemu-ppc64le -singlestep -d in_asm,cpu to debug this
> 
> the relevant code is the end of _dlstart_c:
> ....
>    8c574:       09 00 00 48     bl      8c57c <writev+0x28c>
>    8c578:       28 98 fa ff     fsub    f31,f26,f19
>    8c57c:       a6 02 48 7d     mflr    r10
>    8c580:       00 00 2a 81     lwz     r9,0(r10)
>    8c584:       14 52 29 7d     add     r9,r9,r10
>    8c588:       a6 03 29 7d     mtctr   r9
>    8c58c:       18 00 41 f8     std     r2,24(r1)
>    8c590:       78 3b e4 7c     mr      r4,r7
>    8c594:       78 4b 2c 7d     mr      r12,r9
>    8c598:       21 04 80 4e     bctrl
> 
> it's a pc relative address load and jump (to call __dls2):
> 
>    bl 1f
>    data          // data == 0xfffa9828
> 1: mflr r10      // r10 = &data == 0x8c57c
>    lwz r9,0(r10) // r9 = *r10 (== data)
>    add r9,r9,r10 // r9 += r10 (== 0xfffa9828+0x8c57c = 0x100035da0)
>    mtctr r9      // ctr = r9 (== 0x100035da0)
>    std r2,24(r1)
>    mr r4,r7
>    mr r12,r9
>    bctrl         // ctr(r12,r4)
> 
> it seems __dls2 is at 0x35da0 so the pc relative data
> should have been signextended.

Does the attached patch fix it?

Rich

[-- Attachment #2: ppc64_crash.diff --]
[-- Type: text/plain, Size: 357 bytes --]

diff --git a/arch/powerpc64/reloc.h b/arch/powerpc64/reloc.h
index e1bad00..faf70ac 100644
--- a/arch/powerpc64/reloc.h
+++ b/arch/powerpc64/reloc.h
@@ -27,6 +27,6 @@
 	"	bl 1f \n" \
 	"	.long " #sym "-. \n" \
 	"1:	mflr %1 \n" \
-	"	lwz %0, 0(%1) \n" \
+	"	lwa %0, 0(%1) \n" \
 	"	add %0, %0, %1 \n" \
 	: "=r"(*(fp)), "=r"((long){0}) : : "memory", "lr" )

Re: musl segfaulting when compiled with -O2

[Breno Leitao]

Hi Rich,

On Tue, Mar 07, 2017 at 09:28:27PM -0500, Rich Felker wrote:
> Does the attached patch fix it?

Yes. I just tested on my machine, and the problem is not reproducible
anymore.

Thanks!

Re: musl segfaulting when compiled with -O2

[Rich Felker]

On Wed, Mar 08, 2017 at 11:19:10AM -0300, Breno Leitao wrote:
> Hi Rich,
> 
> On Tue, Mar 07, 2017 at 09:28:27PM -0500, Rich Felker wrote:
> > Does the attached patch fix it?
> 
> Yes. I just tested on my machine, and the problem is not reproducible
> anymore.
> 
> Thanks!

Thanks for testing/feedback. Committing.

Rich