Re: [musl] Segmentation fault musl 1.2.4

[Rich Felker <dalias@libc.org>]

On Tue, Jan 30, 2024 at 11:43:38AM +0100, Szabolcs Nagy wrote:
> * Rich Felker <dalias@libc.org> [2024-01-16 15:45:52 -0500]:
> 
> > On Tue, Jan 16, 2024 at 07:29:18PM +0100, Szabolcs Nagy wrote:
> > > * Cody Wetzel <codyawetzel@gmail.com> [2024-01-16 09:21:05 -0600]:
> > > > Here is the output for the old
> > > > ....
> > > > >
> > > > > / # /tmp/ld-musl-armhf.so.1 /usr/bin/readelf -lW /tmp/ld-musl-armhf.so.1
> > > > >
> > > > > Elf file type is DYN (Shared object file)
> > > > > Entry point 0x359cd
> > > > > There are 6 program headers, starting at offset 52
> > > > >
> > > > > Program Headers:
> > > > >   Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
> > > > >   EXIDX          0x07acec 0x0007acec 0x0007acec 0x00008 0x00008 R   0x4
> > > > >   LOAD           0x000000 0x00000000 0x00000000 0x7acf4 0x7acf4 R E 0x10000
> > > > >   LOAD           0x07fd6c 0x0008fd6c 0x0008fd6c 0x0054a 0x02258 RW  0x10000
> > > 
> > > this load segment is 64k aligned.
> > > 
> > > > >   DYNAMIC        0x07febc 0x0008febc 0x0008febc 0x000c0 0x000c0 RW  0x4
> > > > >   GNU_STACK      0x000000 0x00000000 0x00000000 0x00000 0x00000 RW  0x10
> > > > >   GNU_RELRO      0x07fd6c 0x0008fd6c 0x0008fd6c 0x00294 0x00294 R   0x1
> > > > >
> > > > >  Section to Segment mapping:
> > > > >   Segment Sections...
> > > > >    00     .ARM.exidx
> > > > >    01     .hash .gnu.hash .dynsym .dynstr .rel.dyn .rel.plt .plt .text
> > > > > .rodata .ARM.exidx
> > > > >    02     .data.rel.ro .dynamic .got .data .bss
> > > > >    03     .dynamic
> > > > >    04
> > > > >    05     .data.rel.ro .dynamic .got
> > > > >
> > > > 
> > > > And the new...
> > > > 
> > > > / # /tmp/ld-musl-armhf.so.1 /usr/bin/readelf -lW /lib/ld-musl-armhf.so.1
> > > > >
> > > > > Elf file type is DYN (Shared object file)
> > > > > Entry point 0x362f1
> > > > > There are 6 program headers, starting at offset 52
> > > > >
> > > > > Program Headers:
> > > > >   Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
> > > > >   EXIDX          0x07b81c 0x0007b81c 0x0007b81c 0x00008 0x00008 R   0x4
> > > > >   LOAD           0x000000 0x00000000 0x00000000 0x7b824 0x7b824 R E 0x1000
> > > > >   LOAD           0x07bd74 0x0007cd74 0x0007cd74 0x0054a 0x0225c RW  0x1000
> > > 
> > > this load segment is 4k aligned and offset vs addr is not congruent
> > > modulo 64k, or 32k, so won't work on systems with such page size.
> > > 
> > > > >   DYNAMIC        0x07bebc 0x0007cebc 0x0007cebc 0x000c0 0x000c0 RW  0x4
> > > > >   GNU_STACK      0x000000 0x00000000 0x00000000 0x00000 0x00000 RW  0x10
> > > > >   GNU_RELRO      0x07bd74 0x0007cd74 0x0007cd74 0x0028c 0x0028c R   0x1
> > > > >
> > > > >  Section to Segment mapping:
> > > > >   Segment Sections...
> > > > >    00     .ARM.exidx
> > > > >    01     .hash .gnu.hash .dynsym .dynstr .rel.dyn .rel.plt .plt .text
> > > > > .rodata .ARM.exidx
> > > > >    02     .data.rel.ro .dynamic .got .data .bss
> > > > >    03     .dynamic
> > > > >    04
> > > > >    05     .data.rel.ro .dynamic .got
> > > > 
> > > > 
> > > > I hope that helps.
> > > 
> > > yes, this is a linking issue, not musl libc.
> > > 
> > > alpine linux links binaries for 4k pagesize only.
> > > 
> > > arm linkers were updated at some point to create binaries supporting
> > > up to 64k pagesize.  i suspect some ppl ran into issues in practice
> > > and decided the larger binaries are not worth it, if they dont work
> > > reliably and forced 4k page size at link time.
> > > 
> > > you have to raise an issue with alpine linux, if you think 32k
> > > oage size is useful and reliably supportable.
> > 
> > Are they using -Wl,-z,separate-code? That incurs a large
> > binary-size-on-disk penalty when supporting oversized pages, and IIRC
> > something was done to make the linker default to not supporting
> > oversized pages when that's used. It might be the reason, if arm
> > linking is normally expected to use a larger max pagesize.
> 
> i looked at this now, turns out they just changed the
> pagesize back to 4k (i missed this change):
> 
> https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=1a26a53a0dee39106ba58fcb15496c5f13074652

This doesn't help immediately, but a major ingredient to fix this
situation would be getting the kernel to stop doing the wrong thing.
Right now, it's ignoring the fact that the ELF program header
constraints are incompatible with mmap given the oversized system
pagesize, and just incorrectly mapping the executable and trying to
run it anyway, whereby it blows up.

The right thing to do would be either to fail with ENOEXEC in this
case, or when mmap with the required offset constraint fails, falling
back to making an anonymous map and copying the whole content of the
loadable segment into that (no COW sharing). The latter is really not
all that bad for got/data/etc. mappings which you expect will be dirty
(modified) anyway.

BTW the former choice (ENOEXEC) would allow doing the latter in
userspace with a binfmt_misc loader.

Rich