broken shared executables on armeb (illegal instruction)

["Jason A. Donenfeld" <Jason@zx2c4.com>]

Hello,

There appears to be a problem with shared linking on big-endian
ARM (armeb).

First I'll show that static linking works correctly:

$ armeb-pc-linux-gnueabi-gcc -v
Using built-in specs.
COLLECT_GCC=armeb-pc-linux-gnueabi-gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/armeb-pc-linux-gnueabi/8.2.0/lto-wrapper
Target: armeb-pc-linux-gnueabi
Configured with: /var/tmp/portage/cross-armeb-pc-linux-gnueabi/gcc-8.2.0-r3/work/gcc-8.2.0/configure --host=x86_64-pc-linux-gnu --target=armeb-pc-linux-gnueabi --build=x86_64-pc-linux-gnu --prefix=/usr --bindir=/usr/x86_64-pc-linux-gnu/armeb-pc-linux-gnueabi/gcc-bin/8.2.0 --includedir=/usr/lib/gcc/armeb-pc-linux-gnueabi/8.2.0/include --datadir=/usr/share/gcc-data/armeb-pc-linux-gnueabi/8.2.0 --mandir=/usr/share/gcc-data/armeb-pc-linux-gnueabi/8.2.0/man --infodir=/usr/share/gcc-data/armeb-pc-linux-gnueabi/8.2.0/info --with-gxx-include-dir=/usr/lib/gcc/armeb-pc-linux-gnueabi/8.2.0/include/g++-v8 --with-python-dir=/share/gcc-data/armeb-pc-linux-gnueabi/8.2.0/python --enable-languages=c,c++,fortran --enable-obsolete --enable-secureplt --disable-werror --with-system-zlib --enable-nls --without-included-gettext --enable-checking=release --with-bugurl=https://bugs.gentoo.org/ --with-pkgversion='Gentoo 8.2.0-r3 p1.3' --disable-esp --enable-libstdcxx-time --enable-poison-system-directories --with-sysroot=/usr/armeb-pc-linux-gnueabi --disable-bootstrap --enable-__cxa_atexit --enable-clocale=gnu --disable-multilib --disable-altivec --disable-fixed-point --with-float=soft --enable-libgomp --disable-libmudflap --disable-libssp --disable-libmpx --disable-systemtap --enable-vtable-verify --enable-libvtv --enable-lto --without-isl --enable-libsanitizer --enable-default-pie --enable-default-ssp
Thread model: posix
gcc version 8.2.0 (Gentoo 8.2.0-r3 p1.3) 
$ tar xf musl-1.1.20.tar.gz 
$ cd musl-1.1.20/
$ export CFLAGS="-O2 -march=armv7-a -mtune=cortex-a15 -mabi=aapcs-linux"
$ CC=armeb-pc-linux-gnueabi-gcc ./configure --prefix=$PWD/prefix --enable-static --disable-shared --build=armeb-pc-linux-gnueabi
[...]
$ make -j$(nproc)
[...]
$ make install
[...]
$ cd prefix/
$ printf '#include <stdio.h>\nint main(){puts("hello world");}' | bin/musl-gcc -xc -o helloworld $CFLAGS -
/usr/libexec/gcc/armeb-pc-linux-gnueabi/ld: /usr/lib/gcc/armeb-pc-linux-gnueabi/8.2.0/libgcc.a(_dvmd_lnx.o): in function `__aeabi_idiv0':
/var/tmp/portage/cross-armeb-pc-linux-gnueabi/gcc-8.2.0-r3/work/gcc-8.2.0/libgcc/config/arm/lib1funcs.S:1545: undefined reference to `raise'
collect2: error: ld returned 1 exit status
[This appears to be a well-known bug in some other mailing list post. Working around with the next command:]
$ printf '#include <stdio.h>\nint main(){puts("hello world");}' | bin/musl-gcc -xc -o helloworld $CFLAGS -static -
$ cp /usr/bin/qemu-armeb .
$ sudo chroot $(readlink -f .) /qemu-armeb /helloworld
hello world

Now let's try with shared linking, and you'll see it generates a
broken binary:

$ tar xf musl-1.1.20.tar.gz 
$ cd musl-1.1.20/
$ export CFLAGS="-O2 -march=armv7-a -mtune=cortex-a15 -mabi=aapcs-linux"
$ CC=armeb-pc-linux-gnueabi-gcc ./configure --prefix=$PWD/prefix --disable-static --enable-shared --build=armeb-pc-linux-gnueabi
[...]
$ make -j$(nproc)
[...]
$ make install
[...]
$ cd prefix/
$ printf '#include <stdio.h>\nint main(){puts("hello world");}' | bin/musl-gcc -xc -o helloworld $CFLAGS -
$ cd lib/
$ ln -s libc.so ld-musl-armeb.so.1
$ cd ..
$ cp /usr/bin/qemu-armeb .
$ sudo chroot $(readlink -f .) /qemu-armeb /helloworld
Illegal instruction

I've experienced similar failures when trying to boot with armeb
executables as init with a real kernel that are generated this way.
I've also experienced this with both my own toolchain (above) and
with linaro's toolchain.

I expect the commands above should result in an easily reproducable bug.

Any idea what's up?

Thanks,
Jason