[musl] Support for -static-pie relocations

[musl] Support for -static-pie relocations

[Dmitry Vyukov]

Hello,

This simple program crashes when compiled with -static-pie:

#include <stdio.h>
int main() { fprintf(stderr, "Hello\n"); }

Program received signal SIGSEGV, Segmentation fault.
0x0000000000001170 in ?? ()
(gdb) bt
#0  0x0000000000001170 in ?? ()
#1  0x00007ffff7ffb3b8 in libc_start_init () at src/env/__libc_start_main.c:64
#2  0x00007ffff7ffb3e8 in libc_start_main_stage2 (main=0x7ffff7ffb180
<main>, argc=1, argv=0x7fffffffdc98)
    at src/env/__libc_start_main.c:92
#3  0x00007ffff7ffb0b1 in _start ()

(gdb) up
#1  0x00007ffff7ffb3b8 in libc_start_init () at src/env/__libc_start_main.c:64
64 (*(void (**)(void))a)();

(gdb) disass
Dump of assembler code for function libc_start_init:
   0x00007ffff7ffb39b <+0>: push   %rbp
   0x00007ffff7ffb39c <+1>: push   %rbx
   0x00007ffff7ffb39d <+2>: sub    $0x8,%rsp
   0x00007ffff7ffb3a1 <+6>: call   0x7ffff7ffb000 <_init>
   0x00007ffff7ffb3a6 <+11>: lea    0x2a9b(%rip),%rbx        # 0x7ffff7ffde48
   0x00007ffff7ffb3ad <+18>: lea    0x2a9c(%rip),%rbp        # 0x7ffff7ffde50
   0x00007ffff7ffb3b4 <+25>: jmp    0x7ffff7ffb3bc <libc_start_init+33>
   0x00007ffff7ffb3b6 <+27>: call   *(%rbx)
=> 0x00007ffff7ffb3b8 <+29>: add    $0x8,%rbx
   0x00007ffff7ffb3bc <+33>: cmp    %rbp,%rbx
   0x00007ffff7ffb3bf <+36>: jb     0x7ffff7ffb3b6 <libc_start_init+27>
   0x00007ffff7ffb3c1 <+38>: add    $0x8,%rsp
   0x00007ffff7ffb3c5 <+42>: pop    %rbx
   0x00007ffff7ffb3c6 <+43>: pop    %rbp
   0x00007ffff7ffb3c7 <+44>: ret
End of assembler dump.

(gdb) p /x $rbx
$1 = 0x7ffff7ffde48
(gdb) p /x *(void**)$rbx
$2 = 0x1170

This 0x1170 is probably a ctor pointer offset that wasn't relocated.

A fix would probably be calling _dl_relocate_object() somewhere on the
__libc_start_main() path.

Is there a reason this is not supported? Or merely not implemented yet?
Can't find an issue tracker nor searchable archives for previous
discussions on this...

Thanks in advance

Re: [musl] Support for -static-pie relocations

[Rich Felker]

On Sat, Mar 01, 2025 at 04:08:52PM +0100, Dmitry Vyukov wrote:
> Hello,
> 
> This simple program crashes when compiled with -static-pie:
> 
> #include <stdio.h>
> int main() { fprintf(stderr, "Hello\n"); }

Not reproducible here. Could you include details on how the toolchain
was built and what compile/link command lines were used?

> Program received signal SIGSEGV, Segmentation fault.
> 0x0000000000001170 in ?? ()
> (gdb) bt
> #0  0x0000000000001170 in ?? ()
> #1  0x00007ffff7ffb3b8 in libc_start_init () at src/env/__libc_start_main.c:64
> #2  0x00007ffff7ffb3e8 in libc_start_main_stage2 (main=0x7ffff7ffb180
> <main>, argc=1, argv=0x7fffffffdc98)
>     at src/env/__libc_start_main.c:92
> #3  0x00007ffff7ffb0b1 in _start ()
> 
> (gdb) up
> #1  0x00007ffff7ffb3b8 in libc_start_init () at src/env/__libc_start_main.c:64
> 64 (*(void (**)(void))a)();
> 
> (gdb) disass
> Dump of assembler code for function libc_start_init:
>    0x00007ffff7ffb39b <+0>: push   %rbp
>    0x00007ffff7ffb39c <+1>: push   %rbx
>    0x00007ffff7ffb39d <+2>: sub    $0x8,%rsp
>    0x00007ffff7ffb3a1 <+6>: call   0x7ffff7ffb000 <_init>
>    0x00007ffff7ffb3a6 <+11>: lea    0x2a9b(%rip),%rbx        # 0x7ffff7ffde48
>    0x00007ffff7ffb3ad <+18>: lea    0x2a9c(%rip),%rbp        # 0x7ffff7ffde50
>    0x00007ffff7ffb3b4 <+25>: jmp    0x7ffff7ffb3bc <libc_start_init+33>
>    0x00007ffff7ffb3b6 <+27>: call   *(%rbx)
> => 0x00007ffff7ffb3b8 <+29>: add    $0x8,%rbx
>    0x00007ffff7ffb3bc <+33>: cmp    %rbp,%rbx
>    0x00007ffff7ffb3bf <+36>: jb     0x7ffff7ffb3b6 <libc_start_init+27>
>    0x00007ffff7ffb3c1 <+38>: add    $0x8,%rsp
>    0x00007ffff7ffb3c5 <+42>: pop    %rbx
>    0x00007ffff7ffb3c6 <+43>: pop    %rbp
>    0x00007ffff7ffb3c7 <+44>: ret
> End of assembler dump.
> 
> (gdb) p /x $rbx
> $1 = 0x7ffff7ffde48
> (gdb) p /x *(void**)$rbx
> $2 = 0x1170
> 
> This 0x1170 is probably a ctor pointer offset that wasn't relocated.

Yes that looks plausible.

> A fix would probably be calling _dl_relocate_object() somewhere on the
> __libc_start_main() path.

Relocations are processed in the crt entry point before
__libc_start_main is reached. The relevant code is ldso/dlstart.c.

Perhaps you're using newer tooling that's encoding some or all of the
relocations in some unsupported manner? For example I think using musl
1.2.3 (pre RELR addition) rcrt1.o along with a newer binutils that's
been configured to default to RELR encoding would produce the failure
you're seeing.

Rich

Re: [musl] Support for -static-pie relocations

[Szabolcs Nagy]

* Dmitry Vyukov <dvyukov@google.com> [2025-03-01 16:08:52 +0100]:

> Hello,
> 
> This simple program crashes when compiled with -static-pie:
> 
> #include <stdio.h>
> int main() { fprintf(stderr, "Hello\n"); }
> 
> Program received signal SIGSEGV, Segmentation fault.
> 0x0000000000001170 in ?? ()
> (gdb) bt
> #0  0x0000000000001170 in ?? ()
> #1  0x00007ffff7ffb3b8 in libc_start_init () at src/env/__libc_start_main.c:64
> #2  0x00007ffff7ffb3e8 in libc_start_main_stage2 (main=0x7ffff7ffb180
> <main>, argc=1, argv=0x7fffffffdc98)
>     at src/env/__libc_start_main.c:92
> #3  0x00007ffff7ffb0b1 in _start ()
> 
> (gdb) up
> #1  0x00007ffff7ffb3b8 in libc_start_init () at src/env/__libc_start_main.c:64
> 64 (*(void (**)(void))a)();
> 
> (gdb) disass
> Dump of assembler code for function libc_start_init:
>    0x00007ffff7ffb39b <+0>: push   %rbp
>    0x00007ffff7ffb39c <+1>: push   %rbx
>    0x00007ffff7ffb39d <+2>: sub    $0x8,%rsp
>    0x00007ffff7ffb3a1 <+6>: call   0x7ffff7ffb000 <_init>
>    0x00007ffff7ffb3a6 <+11>: lea    0x2a9b(%rip),%rbx        # 0x7ffff7ffde48
>    0x00007ffff7ffb3ad <+18>: lea    0x2a9c(%rip),%rbp        # 0x7ffff7ffde50
>    0x00007ffff7ffb3b4 <+25>: jmp    0x7ffff7ffb3bc <libc_start_init+33>
>    0x00007ffff7ffb3b6 <+27>: call   *(%rbx)
> => 0x00007ffff7ffb3b8 <+29>: add    $0x8,%rbx
>    0x00007ffff7ffb3bc <+33>: cmp    %rbp,%rbx
>    0x00007ffff7ffb3bf <+36>: jb     0x7ffff7ffb3b6 <libc_start_init+27>
>    0x00007ffff7ffb3c1 <+38>: add    $0x8,%rsp
>    0x00007ffff7ffb3c5 <+42>: pop    %rbx
>    0x00007ffff7ffb3c6 <+43>: pop    %rbp
>    0x00007ffff7ffb3c7 <+44>: ret
> End of assembler dump.
> 
> (gdb) p /x $rbx
> $1 = 0x7ffff7ffde48
> (gdb) p /x *(void**)$rbx
> $2 = 0x1170
> 
> This 0x1170 is probably a ctor pointer offset that wasn't relocated.
> 
> A fix would probably be calling _dl_relocate_object() somewhere on the
> __libc_start_main() path.
> 
> Is there a reason this is not supported? Or merely not implemented yet?
> Can't find an issue tracker nor searchable archives for previous
> discussions on this...
> 
> Thanks in advance

i think this is a disagreement about what dynrelocs may appear in
static pie between musl and binutils.

it is a linker bug if static pie has non-relative relocs.
in this case likely there is a symbolic reloc for the init array
entry even though we know the symbol value at link time.
check the readelf output. (alternative theory: the relocs are
missing: bfd ld had bugs like that before)

binutils ld is sloppy on some targets but because glibc handles
symbolic relocs it is just an unnecessary runtime symbol lookup
there. musl considers this unacceptable linker behaviour: it
would require half of the dynlinker static linked into every
static pie unnecessarily.

Re: [musl] Support for -static-pie relocations

[Rich Felker]

On Sat, Mar 01, 2025 at 05:10:39PM +0100, Szabolcs Nagy wrote:
> * Dmitry Vyukov <dvyukov@google.com> [2025-03-01 16:08:52 +0100]:
> 
> > Hello,
> > 
> > This simple program crashes when compiled with -static-pie:
> > 
> > #include <stdio.h>
> > int main() { fprintf(stderr, "Hello\n"); }
> > 
> > Program received signal SIGSEGV, Segmentation fault.
> > 0x0000000000001170 in ?? ()
> > (gdb) bt
> > #0  0x0000000000001170 in ?? ()
> > #1  0x00007ffff7ffb3b8 in libc_start_init () at src/env/__libc_start_main.c:64
> > #2  0x00007ffff7ffb3e8 in libc_start_main_stage2 (main=0x7ffff7ffb180
> > <main>, argc=1, argv=0x7fffffffdc98)
> >     at src/env/__libc_start_main.c:92
> > #3  0x00007ffff7ffb0b1 in _start ()
> > 
> > (gdb) up
> > #1  0x00007ffff7ffb3b8 in libc_start_init () at src/env/__libc_start_main.c:64
> > 64 (*(void (**)(void))a)();
> > 
> > (gdb) disass
> > Dump of assembler code for function libc_start_init:
> >    0x00007ffff7ffb39b <+0>: push   %rbp
> >    0x00007ffff7ffb39c <+1>: push   %rbx
> >    0x00007ffff7ffb39d <+2>: sub    $0x8,%rsp
> >    0x00007ffff7ffb3a1 <+6>: call   0x7ffff7ffb000 <_init>
> >    0x00007ffff7ffb3a6 <+11>: lea    0x2a9b(%rip),%rbx        # 0x7ffff7ffde48
> >    0x00007ffff7ffb3ad <+18>: lea    0x2a9c(%rip),%rbp        # 0x7ffff7ffde50
> >    0x00007ffff7ffb3b4 <+25>: jmp    0x7ffff7ffb3bc <libc_start_init+33>
> >    0x00007ffff7ffb3b6 <+27>: call   *(%rbx)
> > => 0x00007ffff7ffb3b8 <+29>: add    $0x8,%rbx
> >    0x00007ffff7ffb3bc <+33>: cmp    %rbp,%rbx
> >    0x00007ffff7ffb3bf <+36>: jb     0x7ffff7ffb3b6 <libc_start_init+27>
> >    0x00007ffff7ffb3c1 <+38>: add    $0x8,%rsp
> >    0x00007ffff7ffb3c5 <+42>: pop    %rbx
> >    0x00007ffff7ffb3c6 <+43>: pop    %rbp
> >    0x00007ffff7ffb3c7 <+44>: ret
> > End of assembler dump.
> > 
> > (gdb) p /x $rbx
> > $1 = 0x7ffff7ffde48
> > (gdb) p /x *(void**)$rbx
> > $2 = 0x1170
> > 
> > This 0x1170 is probably a ctor pointer offset that wasn't relocated.
> > 
> > A fix would probably be calling _dl_relocate_object() somewhere on the
> > __libc_start_main() path.
> > 
> > Is there a reason this is not supported? Or merely not implemented yet?
> > Can't find an issue tracker nor searchable archives for previous
> > discussions on this...
> > 
> > Thanks in advance
> 
> i think this is a disagreement about what dynrelocs may appear in
> static pie between musl and binutils.
> 
> it is a linker bug if static pie has non-relative relocs.
> in this case likely there is a symbolic reloc for the init array
> entry even though we know the symbol value at link time.
> check the readelf output. (alternative theory: the relocs are
> missing: bfd ld had bugs like that before)
> 
> binutils ld is sloppy on some targets but because glibc handles
> symbolic relocs it is just an unnecessary runtime symbol lookup
> there. musl considers this unacceptable linker behaviour: it
> would require half of the dynlinker static linked into every
> static pie unnecessarily.

I don't think we've hit that on x86_64, and it was long ago fixed on
the archs we did hit it on, so I don't think that's what's going on.

But a full readelf -a of the failing binary would quickly reveal if
that's what happened, and would probaby shed light on whatever else if
wrong if not that.

Rich

Re: [musl] Support for -static-pie relocations

[Sertonix]

On Sat Mar 1, 2025 at 5:22 PM CET, Rich Felker wrote:
> On Sat, Mar 01, 2025 at 05:10:39PM +0100, Szabolcs Nagy wrote:
>> * Dmitry Vyukov <dvyukov@google.com> [2025-03-01 16:08:52 +0100]:
>> 
>> > Hello,
>> > 
>> > This simple program crashes when compiled with -static-pie:
>> > 
>> > #include <stdio.h>
>> > int main() { fprintf(stderr, "Hello\n"); }
>> > 
>> > Program received signal SIGSEGV, Segmentation fault.
>> > 0x0000000000001170 in ?? ()
>> > (gdb) bt
>> > #0  0x0000000000001170 in ?? ()
>> > #1  0x00007ffff7ffb3b8 in libc_start_init () at src/env/__libc_start_main.c:64
>> > #2  0x00007ffff7ffb3e8 in libc_start_main_stage2 (main=0x7ffff7ffb180
>> > <main>, argc=1, argv=0x7fffffffdc98)
>> >     at src/env/__libc_start_main.c:92
>> > #3  0x00007ffff7ffb0b1 in _start ()
>> > 
>> > (gdb) up
>> > #1  0x00007ffff7ffb3b8 in libc_start_init () at src/env/__libc_start_main.c:64
>> > 64 (*(void (**)(void))a)();
>> > 
>> > (gdb) disass
>> > Dump of assembler code for function libc_start_init:
>> >    0x00007ffff7ffb39b <+0>: push   %rbp
>> >    0x00007ffff7ffb39c <+1>: push   %rbx
>> >    0x00007ffff7ffb39d <+2>: sub    $0x8,%rsp
>> >    0x00007ffff7ffb3a1 <+6>: call   0x7ffff7ffb000 <_init>
>> >    0x00007ffff7ffb3a6 <+11>: lea    0x2a9b(%rip),%rbx        # 0x7ffff7ffde48
>> >    0x00007ffff7ffb3ad <+18>: lea    0x2a9c(%rip),%rbp        # 0x7ffff7ffde50
>> >    0x00007ffff7ffb3b4 <+25>: jmp    0x7ffff7ffb3bc <libc_start_init+33>
>> >    0x00007ffff7ffb3b6 <+27>: call   *(%rbx)
>> > => 0x00007ffff7ffb3b8 <+29>: add    $0x8,%rbx
>> >    0x00007ffff7ffb3bc <+33>: cmp    %rbp,%rbx
>> >    0x00007ffff7ffb3bf <+36>: jb     0x7ffff7ffb3b6 <libc_start_init+27>
>> >    0x00007ffff7ffb3c1 <+38>: add    $0x8,%rsp
>> >    0x00007ffff7ffb3c5 <+42>: pop    %rbx
>> >    0x00007ffff7ffb3c6 <+43>: pop    %rbp
>> >    0x00007ffff7ffb3c7 <+44>: ret
>> > End of assembler dump.
>> > 
>> > (gdb) p /x $rbx
>> > $1 = 0x7ffff7ffde48
>> > (gdb) p /x *(void**)$rbx
>> > $2 = 0x1170
>> > 
>> > This 0x1170 is probably a ctor pointer offset that wasn't relocated.
>> > 
>> > A fix would probably be calling _dl_relocate_object() somewhere on the
>> > __libc_start_main() path.
>> > 
>> > Is there a reason this is not supported? Or merely not implemented yet?
>> > Can't find an issue tracker nor searchable archives for previous
>> > discussions on this...
>> > 
>> > Thanks in advance
>> 
>> i think this is a disagreement about what dynrelocs may appear in
>> static pie between musl and binutils.
>> 
>> it is a linker bug if static pie has non-relative relocs.
>> in this case likely there is a symbolic reloc for the init array
>> entry even though we know the symbol value at link time.
>> check the readelf output. (alternative theory: the relocs are
>> missing: bfd ld had bugs like that before)
>> 
>> binutils ld is sloppy on some targets but because glibc handles
>> symbolic relocs it is just an unnecessary runtime symbol lookup
>> there. musl considers this unacceptable linker behaviour: it
>> would require half of the dynlinker static linked into every
>> static pie unnecessarily.
>
> I don't think we've hit that on x86_64, and it was long ago fixed on
> the archs we did hit it on, so I don't think that's what's going on.
>
> But a full readelf -a of the failing binary would quickly reveal if
> that's what happened, and would probaby shed light on whatever else if
> wrong if not that.
>
> Rich


I wasn't able to reproduce the exact same issue but I have seen -static-pie
binaries being broken for arm 32 targets. Which arch are you testing on?

Ref https://gitlab.alpinelinux.org/alpine/aports/-/issues/16942#note_484996

Re: [musl] Support for -static-pie relocations

[Dmitry Vyukov]

RIght, sorry.

I checked out the latest HEAD c47ad25ea3b484e10326f933e927c0bc8cded3da.
Standard build: ./configure --enable-debug && make
arch x86_64

Reproducible with both (standard Debian builds):
$ clang --version
Debian clang version 16.0.6 (27+build3)
$ ld -v
GNU ld (GNU Binutils for Debian) 2.43.1
$ clang /tmp/test.c lib/libc.a -O2 -g -static-pie && ./a.out
Segmentation fault (core dumped)

and
$ gcc -v
gcc version 14.2.0 (Debian 14.2.0-3+build4)
$ gcc /tmp/test.c lib/libc.a -O2 -g -static-pie && ./a.out
Segmentation fault (core dumped)

Relocations are mostly rip-relative except for global vars that
contain pointers:

$ readelf -r ./a.out
Relocation section '.rela.dyn' at offset 0x350 contains 11 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
000000003e48  000000000008 R_X86_64_RELATIVE                    11a0
000000003e50  000000000008 R_X86_64_RELATIVE                    1160
000000003e58  000000000008 R_X86_64_RELATIVE                    4020
000000003e60  000000000008 R_X86_64_RELATIVE                    4380
000000003fd8  000000000008 R_X86_64_RELATIVE                    3e68
000000004008  000000000008 R_X86_64_RELATIVE                    4008
000000004010  000000000008 R_X86_64_RELATIVE                    4020
000000004038  000000000008 R_X86_64_RELATIVE                    19ca
000000004068  000000000008 R_X86_64_RELATIVE                    19f5
000000004070  000000000008 R_X86_64_RELATIVE                    19ed
000000004078  000000000008 R_X86_64_RELATIVE                    41d0

> Relocations are processed in the crt entry point before
> __libc_start_main is reached. The relevant code is ldso/dlstart.c.

Am I linking musl somehow incorrectly (need to do something with crt)?

My entry function calls __libc_start_main:

Dump of assembler code for function _start:
   0x00007ffff7ffb0c0 <+0>: xor    %ebp,%ebp
   0x00007ffff7ffb0c2 <+2>: mov    %rdx,%r9
   0x00007ffff7ffb0c5 <+5>: pop    %rsi
   0x00007ffff7ffb0c6 <+6>: mov    %rsp,%rdx
   0x00007ffff7ffb0c9 <+9>: and    $0xfffffffffffffff0,%rsp
   0x00007ffff7ffb0cd <+13>: push   %rax
   0x00007ffff7ffb0ce <+14>: push   %rsp
   0x00007ffff7ffb0cf <+15>: xor    %r8d,%r8d
   0x00007ffff7ffb0d2 <+18>: xor    %ecx,%ecx
   0x00007ffff7ffb0d4 <+20>: lea    -0x4b(%rip),%rdi        #
0x7ffff7ffb090 <main>
   0x00007ffff7ffb0db <+27>: addr32 call 0x7ffff7ffb3fa <__libc_start_main>
=> 0x00007ffff7ffb0e1 <+33>: hlt


I don't see any static libs in musl build that include _start symbol...






On Sat, 1 Mar 2025 at 17:34, Sertonix <sertonix@posteo.net> wrote:
>
> On Sat Mar 1, 2025 at 5:22 PM CET, Rich Felker wrote:
> > On Sat, Mar 01, 2025 at 05:10:39PM +0100, Szabolcs Nagy wrote:
> >> * Dmitry Vyukov <dvyukov@google.com> [2025-03-01 16:08:52 +0100]:
> >>
> >> > Hello,
> >> >
> >> > This simple program crashes when compiled with -static-pie:
> >> >
> >> > #include <stdio.h>
> >> > int main() { fprintf(stderr, "Hello\n"); }
> >> >
> >> > Program received signal SIGSEGV, Segmentation fault.
> >> > 0x0000000000001170 in ?? ()
> >> > (gdb) bt
> >> > #0  0x0000000000001170 in ?? ()
> >> > #1  0x00007ffff7ffb3b8 in libc_start_init () at src/env/__libc_start_main.c:64
> >> > #2  0x00007ffff7ffb3e8 in libc_start_main_stage2 (main=0x7ffff7ffb180
> >> > <main>, argc=1, argv=0x7fffffffdc98)
> >> >     at src/env/__libc_start_main.c:92
> >> > #3  0x00007ffff7ffb0b1 in _start ()
> >> >
> >> > (gdb) up
> >> > #1  0x00007ffff7ffb3b8 in libc_start_init () at src/env/__libc_start_main.c:64
> >> > 64 (*(void (**)(void))a)();
> >> >
> >> > (gdb) disass
> >> > Dump of assembler code for function libc_start_init:
> >> >    0x00007ffff7ffb39b <+0>: push   %rbp
> >> >    0x00007ffff7ffb39c <+1>: push   %rbx
> >> >    0x00007ffff7ffb39d <+2>: sub    $0x8,%rsp
> >> >    0x00007ffff7ffb3a1 <+6>: call   0x7ffff7ffb000 <_init>
> >> >    0x00007ffff7ffb3a6 <+11>: lea    0x2a9b(%rip),%rbx        # 0x7ffff7ffde48
> >> >    0x00007ffff7ffb3ad <+18>: lea    0x2a9c(%rip),%rbp        # 0x7ffff7ffde50
> >> >    0x00007ffff7ffb3b4 <+25>: jmp    0x7ffff7ffb3bc <libc_start_init+33>
> >> >    0x00007ffff7ffb3b6 <+27>: call   *(%rbx)
> >> > => 0x00007ffff7ffb3b8 <+29>: add    $0x8,%rbx
> >> >    0x00007ffff7ffb3bc <+33>: cmp    %rbp,%rbx
> >> >    0x00007ffff7ffb3bf <+36>: jb     0x7ffff7ffb3b6 <libc_start_init+27>
> >> >    0x00007ffff7ffb3c1 <+38>: add    $0x8,%rsp
> >> >    0x00007ffff7ffb3c5 <+42>: pop    %rbx
> >> >    0x00007ffff7ffb3c6 <+43>: pop    %rbp
> >> >    0x00007ffff7ffb3c7 <+44>: ret
> >> > End of assembler dump.
> >> >
> >> > (gdb) p /x $rbx
> >> > $1 = 0x7ffff7ffde48
> >> > (gdb) p /x *(void**)$rbx
> >> > $2 = 0x1170
> >> >
> >> > This 0x1170 is probably a ctor pointer offset that wasn't relocated.
> >> >
> >> > A fix would probably be calling _dl_relocate_object() somewhere on the
> >> > __libc_start_main() path.
> >> >
> >> > Is there a reason this is not supported? Or merely not implemented yet?
> >> > Can't find an issue tracker nor searchable archives for previous
> >> > discussions on this...
> >> >
> >> > Thanks in advance
> >>
> >> i think this is a disagreement about what dynrelocs may appear in
> >> static pie between musl and binutils.
> >>
> >> it is a linker bug if static pie has non-relative relocs.
> >> in this case likely there is a symbolic reloc for the init array
> >> entry even though we know the symbol value at link time.
> >> check the readelf output. (alternative theory: the relocs are
> >> missing: bfd ld had bugs like that before)
> >>
> >> binutils ld is sloppy on some targets but because glibc handles
> >> symbolic relocs it is just an unnecessary runtime symbol lookup
> >> there. musl considers this unacceptable linker behaviour: it
> >> would require half of the dynlinker static linked into every
> >> static pie unnecessarily.
> >
> > I don't think we've hit that on x86_64, and it was long ago fixed on
> > the archs we did hit it on, so I don't think that's what's going on.
> >
> > But a full readelf -a of the failing binary would quickly reveal if
> > that's what happened, and would probaby shed light on whatever else if
> > wrong if not that.
> >
> > Rich
>
>
> I wasn't able to reproduce the exact same issue but I have seen -static-pie
> binaries being broken for arm 32 targets. Which arch are you testing on?
>
> Ref https://gitlab.alpinelinux.org/alpine/aports/-/issues/16942#note_484996

Re: [musl] Support for -static-pie relocations

[Rich Felker]

On Sat, Mar 01, 2025 at 06:03:24PM +0100, Dmitry Vyukov wrote:
> RIght, sorry.
> 
> I checked out the latest HEAD c47ad25ea3b484e10326f933e927c0bc8cded3da.
> Standard build: ./configure --enable-debug && make
> arch x86_64
> 
> Reproducible with both (standard Debian builds):
> $ clang --version
> Debian clang version 16.0.6 (27+build3)
> $ ld -v
> GNU ld (GNU Binutils for Debian) 2.43.1
> $ clang /tmp/test.c lib/libc.a -O2 -g -static-pie && ./a.out
> Segmentation fault (core dumped)
> 
> and
> $ gcc -v
> gcc version 14.2.0 (Debian 14.2.0-3+build4)
> $ gcc /tmp/test.c lib/libc.a -O2 -g -static-pie && ./a.out
> Segmentation fault (core dumped)

If you're going to link as PIE, all source files need to be compiled
as PIE-compatible. You're missing -fPIE unless that's already default
on your toolchains.

> Relocations are mostly rip-relative except for global vars that
> contain pointers:
> 
> $ readelf -r ./a.out
> Relocation section '.rela.dyn' at offset 0x350 contains 11 entries:
>   Offset          Info           Type           Sym. Value    Sym. Name + Addend
> 000000003e48  000000000008 R_X86_64_RELATIVE                    11a0
> 000000003e50  000000000008 R_X86_64_RELATIVE                    1160
> 000000003e58  000000000008 R_X86_64_RELATIVE                    4020
> 000000003e60  000000000008 R_X86_64_RELATIVE                    4380
> 000000003fd8  000000000008 R_X86_64_RELATIVE                    3e68
> 000000004008  000000000008 R_X86_64_RELATIVE                    4008
> 000000004010  000000000008 R_X86_64_RELATIVE                    4020
> 000000004038  000000000008 R_X86_64_RELATIVE                    19ca
> 000000004068  000000000008 R_X86_64_RELATIVE                    19f5
> 000000004070  000000000008 R_X86_64_RELATIVE                    19ed
> 000000004078  000000000008 R_X86_64_RELATIVE                    41d0

Those are all fine.

> > Relocations are processed in the crt entry point before
> > __libc_start_main is reached. The relevant code is ldso/dlstart.c.
> 
> Am I linking musl somehow incorrectly (need to do something with crt)?
> 
> My entry function calls __libc_start_main:
> 
> Dump of assembler code for function _start:
>    0x00007ffff7ffb0c0 <+0>: xor    %ebp,%ebp
>    0x00007ffff7ffb0c2 <+2>: mov    %rdx,%r9
>    0x00007ffff7ffb0c5 <+5>: pop    %rsi
>    0x00007ffff7ffb0c6 <+6>: mov    %rsp,%rdx
>    0x00007ffff7ffb0c9 <+9>: and    $0xfffffffffffffff0,%rsp
>    0x00007ffff7ffb0cd <+13>: push   %rax
>    0x00007ffff7ffb0ce <+14>: push   %rsp
>    0x00007ffff7ffb0cf <+15>: xor    %r8d,%r8d
>    0x00007ffff7ffb0d2 <+18>: xor    %ecx,%ecx
>    0x00007ffff7ffb0d4 <+20>: lea    -0x4b(%rip),%rdi        #
> 0x7ffff7ffb090 <main>
>    0x00007ffff7ffb0db <+27>: addr32 call 0x7ffff7ffb3fa <__libc_start_main>
> => 0x00007ffff7ffb0e1 <+33>: hlt
> 
> 
> I don't see any static libs in musl build that include _start symbol...

_start comes from one of the *crt1.o files in the library directory
(/usr/lib on a normal system wide install; your cross lib dir for
cross compiles). In the case of static pie, the compiler driver (clang
or gcc command) should be pulling rcrt1.o.

The above asm dump does not look like any variant of the musl crt
start files. Are you perhaps doing this on a glibc-based host and just
trying to link musl as a static library? That's not expected to work.
You need to either use a musl-targeting cross toolchain or the
musl-gcc wrapper (but the wrapper is only for minimal/evaluation type
uses and might not support static pie.

Rich

Re: [musl] Support for -static-pie relocations

[Dmitry Vyukov]

On Sat, 1 Mar 2025 at 18:40, Rich Felker <dalias@libc.org> wrote:
>
> On Sat, Mar 01, 2025 at 06:03:24PM +0100, Dmitry Vyukov wrote:
> > RIght, sorry.
> >
> > I checked out the latest HEAD c47ad25ea3b484e10326f933e927c0bc8cded3da.
> > Standard build: ./configure --enable-debug && make
> > arch x86_64
> >
> > Reproducible with both (standard Debian builds):
> > $ clang --version
> > Debian clang version 16.0.6 (27+build3)
> > $ ld -v
> > GNU ld (GNU Binutils for Debian) 2.43.1
> > $ clang /tmp/test.c lib/libc.a -O2 -g -static-pie && ./a.out
> > Segmentation fault (core dumped)
> >
> > and
> > $ gcc -v
> > gcc version 14.2.0 (Debian 14.2.0-3+build4)
> > $ gcc /tmp/test.c lib/libc.a -O2 -g -static-pie && ./a.out
> > Segmentation fault (core dumped)
>
> If you're going to link as PIE, all source files need to be compiled
> as PIE-compatible. You're missing -fPIE unless that's already default
> on your toolchains.
>
> > Relocations are mostly rip-relative except for global vars that
> > contain pointers:
> >
> > $ readelf -r ./a.out
> > Relocation section '.rela.dyn' at offset 0x350 contains 11 entries:
> >   Offset          Info           Type           Sym. Value    Sym. Name + Addend
> > 000000003e48  000000000008 R_X86_64_RELATIVE                    11a0
> > 000000003e50  000000000008 R_X86_64_RELATIVE                    1160
> > 000000003e58  000000000008 R_X86_64_RELATIVE                    4020
> > 000000003e60  000000000008 R_X86_64_RELATIVE                    4380
> > 000000003fd8  000000000008 R_X86_64_RELATIVE                    3e68
> > 000000004008  000000000008 R_X86_64_RELATIVE                    4008
> > 000000004010  000000000008 R_X86_64_RELATIVE                    4020
> > 000000004038  000000000008 R_X86_64_RELATIVE                    19ca
> > 000000004068  000000000008 R_X86_64_RELATIVE                    19f5
> > 000000004070  000000000008 R_X86_64_RELATIVE                    19ed
> > 000000004078  000000000008 R_X86_64_RELATIVE                    41d0
>
> Those are all fine.
>
> > > Relocations are processed in the crt entry point before
> > > __libc_start_main is reached. The relevant code is ldso/dlstart.c.
> >
> > Am I linking musl somehow incorrectly (need to do something with crt)?
> >
> > My entry function calls __libc_start_main:
> >
> > Dump of assembler code for function _start:
> >    0x00007ffff7ffb0c0 <+0>: xor    %ebp,%ebp
> >    0x00007ffff7ffb0c2 <+2>: mov    %rdx,%r9
> >    0x00007ffff7ffb0c5 <+5>: pop    %rsi
> >    0x00007ffff7ffb0c6 <+6>: mov    %rsp,%rdx
> >    0x00007ffff7ffb0c9 <+9>: and    $0xfffffffffffffff0,%rsp
> >    0x00007ffff7ffb0cd <+13>: push   %rax
> >    0x00007ffff7ffb0ce <+14>: push   %rsp
> >    0x00007ffff7ffb0cf <+15>: xor    %r8d,%r8d
> >    0x00007ffff7ffb0d2 <+18>: xor    %ecx,%ecx
> >    0x00007ffff7ffb0d4 <+20>: lea    -0x4b(%rip),%rdi        #
> > 0x7ffff7ffb090 <main>
> >    0x00007ffff7ffb0db <+27>: addr32 call 0x7ffff7ffb3fa <__libc_start_main>
> > => 0x00007ffff7ffb0e1 <+33>: hlt
> >
> >
> > I don't see any static libs in musl build that include _start symbol...
>
> _start comes from one of the *crt1.o files in the library directory
> (/usr/lib on a normal system wide install; your cross lib dir for
> cross compiles). In the case of static pie, the compiler driver (clang
> or gcc command) should be pulling rcrt1.o.
>
> The above asm dump does not look like any variant of the musl crt
> start files. Are you perhaps doing this on a glibc-based host and just
> trying to link musl as a static library? That's not expected to work.
> You need to either use a musl-targeting cross toolchain or the
> musl-gcc wrapper (but the wrapper is only for minimal/evaluation type
> uses and might not support static pie.
>
> Rich


Guilty as charged. This explains everything. I just extracted this
repro from an environment where I assumed people already did the right
thing.

For the record, doing this makes it work:

$ clang /tmp/test.c lib/rcrt1.o lib/libc.a -O2 -g -static-pie -nostartfiles
$ ./a.out
Hello

I understand this is hacky-ish.

Thanks, Rich

Re: [musl] Support for -static-pie relocations

[Thorsten Glaser]

On Sat, 1 Mar 2025, Rich Felker wrote:

>I don't think we've hit that on x86_64, and it was long ago fixed on
>the archs we did hit it on, so I don't think that's what's going on.

I use static-pie for mksh/musl builds on Debian, so I can report
what works for me and what doesn’t.

Using musl 1.2.5 on: amd64, arm64, armel, armhf, i386, loong64,
m68k, mips64el, ppc64el, riscv64, s390x, sh4

(For mips64el I’m still patching the specs file as elaborated in
the earlier thread on this list or Debian #1050429.)

Test program:

	#include <stdio.h>
	int main(void) {
		printf("main = 0x%lX\n", (unsigned long)main);
		return (0);
	}

Test compilation:

	musl-gcc -fPIE -static -static-pie -fno-lto -Wl,-z,text \
	    -o t.exe t.c && test -x t.exe

For some reason, all three of -fPIE -static -static-pie are needed
(I put -fPIE into CFLAGS and -static -static-pie -Wl,-z,text into
LDFLAGS if the test succeeds).

Then, the following is tested:

	x1=$(./t.exe)
	x2=$(./t.exe)
	x3=$(./t.exe)
	x4=$(./t.exe)
	if test x"$x1" = x"$x2" && test x"$x2" = x"$x3" && \
	   test x"$x3" = x"$x4"; then
		x5=0
	elif test -z "$x1"; then
		x5=0
	else
		x5=1
	fi
	x6=$(env LC_ALL=C file t.exe)
	case $x6 in
	(*', static-pie linked'*)
		x7=1 ;;
	(*'pie executable'*)
		echo >&2 "W: file(1) does not report static-pie, guessing"
		x7=1 ;;
	(*executable*)
		x7=0 ;;
	(*)
		x7=2 ;;
	esac

(Basically, whether a test program can be built, run, reports
different locations for main in each run, and whether file(1)
says it is indeed static-pie.)

Afterwards, the results:

• amd64: works
• arm64: works
• armel: works
• armhf: works
• i386: works
• loong64: works
• m68k: fails, see below
• mips64el: works
• ppc64el: works
• riscv64: musl static-pie blacklisted due to #1068350
• s390x: musl static-pie blacklisted due to #1068350
• sh4: fails to ASLR, see below

‣ m68k:

| qemu: uncaught target signal 11 (Segmentation fault) - core dumped
| Segmentation fault

Compilation seemed to succeed though:

| t.exe: ELF 32-bit MSB pie executable, Motorola m68k, 68020, version 1 (SYSV), static-pie linked, with debug_info, not stripped

‣ sh4:

| main = 0x40000674
| main = 0x40000674
| main = 0x40000674
| main = 0x40000674

| t.exe: ELF 32-bit LSB pie executable, Renesas SH, version 1 (SYSV), static-pie linked, with debug_info, not stripped

‣ riscv64, s390x

The last messages on the bugreport were also sent to this list,
<Pine.BSM.4.64L.2404061513110.25918@herc.mirbsd.org> being the
last one with a question I posed nobody seems to have had an idea
how to answer.

Good however is: all architectures marked as “works” above have
all tests passing, so things seem to work.

bye,
//mirabilos
-- 
11:56⎜«liwakura:#!/bin/mksh» also, i wanted to add mksh to my own distro │
i was disappointed that there is no makefile │ but somehow the Build.sh is
the least painful built system i've ever seen │ honours CC, {CPP,C,LD}FLAGS
properly │ looks cleary like done by someone who knows what they are doing