From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/14188 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Rich Felker Newsgroups: gmane.linux.lib.musl.general Subject: Re: Illegal instruction in __copy_tls() Date: Wed, 5 Jun 2019 20:24:31 -0400 Message-ID: <20190606002431.GM1506@brightrain.aerifal.cx> References: <20190605183742.GK1506@brightrain.aerifal.cx> <20190605200747.GL1506@brightrain.aerifal.cx> <1ae0f7f0-ed7e-570b-5c65-5e6bc48b3306@iee.org> Reply-To: musl@lists.openwall.com Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="135840"; mail-complaints-to="usenet@blaine.gmane.org" User-Agent: Mutt/1.5.21 (2010-09-15) Cc: musl@lists.openwall.com To: "tenspd137 ." Original-X-From: musl-return-14204-gllmg-musl=m.gmane.org@lists.openwall.com Thu Jun 06 02:24:50 2019 Return-path: Envelope-to: gllmg-musl@m.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by blaine.gmane.org with smtp (Exim 4.89) (envelope-from ) id 1hYgD4-000ZBO-DO for gllmg-musl@m.gmane.org; Thu, 06 Jun 2019 02:24:50 +0200 Original-Received: (qmail 8178 invoked by uid 550); 6 Jun 2019 00:24:47 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Original-Received: (qmail 8154 invoked from network); 6 Jun 2019 00:24:46 -0000 Content-Disposition: inline In-Reply-To: Original-Sender: Rich Felker Xref: news.gmane.org gmane.linux.lib.musl.general:14188 Archived-At: On Wed, Jun 05, 2019 at 05:37:49PM -0600, tenspd137 . wrote: > bummer.... Hmm - I wonder if an i686 cross compiler would work? Yes, you can do that. It's just that gcc's multilib dir layout is not compatible with musl -- it assumes library headers can be shared between 32- and 64-bit versions of "the same" arch rather than treating them as completely separate things. Rich > On Wed, Jun 5, 2019 at 5:34 PM Michael Everitt wrote: > > > > Musl doesn't support multilib ... :] fyi > > > > On 06/06/19 00:32, tenspd137 . wrote: > > > Quick question - anyone know how to make a crossdev crosscompiler > > > multilib so I can use -m32 when I compile? > > > > > > Thanks! > > > > > > On Wed, Jun 5, 2019 at 5:09 PM tenspd137 . wrote: > > >> I probably don't fully understand how it works. In > > >> /etc/x86_64-pc-linux-musl/etc.portage/make.conf, march is not > > >> specified. It is specified in /etc/portage/make.conf. When I removed > > >> it from there, things worked. Then I just put it back after using > > >> crossdev to make x86_64-pc-linux-musl-*. > > >> > > >> Either way, loads of fun :) > > >> > > >> Thanks! > > >> > > >> -C > > >> > > >> On Wed, Jun 5, 2019 at 3:56 PM Michael Everitt wrote: > > >>> In your /usr/${CTARGET}/etc/portage/make.conf you can set appropriate > > >>> target flags there. But using 'march=native' is going to catch you out a > > >>> lot cross-compiling anything ... :) At least if you've got it set to > > >>> something, you're going to see errors that point immediately to your CFLAGS > > >>> when/if it goes wrong! > > >>> > > >>> Michael. > > >>> > > >>> On 05/06/19 22:27, tenspd137 . wrote: > > >>>> Thanks! That was indeed the problem. I had march=native for all > > >>>> portage related stuff set, and my guess is that crossdev read that to > > >>>> start. I removed that, reran the crossdev tool to buiild the cross > > >>>> toolchain, and can now build static binaries that run on different > > >>>> systems. > > >>>> > > >>>> Much appreciated! > > >>>> > > >>>> -C > > >>>> > > >>>> On Wed, Jun 5, 2019 at 2:07 PM Rich Felker wrote: > > >>>>> On Wed, Jun 05, 2019 at 01:53:58PM -0600, tenspd137 . wrote: > > >>>>>> Sure - > > >>>>>> > > >>>>>> strace -d -v ./a.out: > > >>>>>> strace: ptrace_setoptions = 0x51 > > >>>>>> strace: new tcb for pid 26302, active tcbs:1 > > >>>>>> strace: [wait(0x80137f) = 26302] WIFSTOPPED,sig=SIGSTOP,EVENT_STOP (128) > > >>>>>> strace: pid 26302 has TCB_STARTUP, initializing it > > >>>>>> strace: [wait(0x80057f) = 26302] WIFSTOPPED,sig=SIGTRAP,EVENT_STOP (128) > > >>>>>> strace: [wait(0x00127f) = 26302] WIFSTOPPED,sig=SIGCONT > > >>>>>> strace: [wait(0x00857f) = 26302] WIFSTOPPED,sig=133 > > >>>>>> execve("./a.out", ["./a.out"], ["SSH_CONNECTION=15.86.136.27 5196"..., > > >>>>>> "LANG=en_US.UTF-8", "XDG_SESSION_ID=41", "USER=dayd", > > >>>>>> "PWD=/home/dayd/projects/musl-tes"..., "HOME=/home/dayd", > > >>>>>> "SSH_CLIENT=xx.xxx.xxx.xx 51960 22", "CCACHE_DIR=/ssd/dayd/.ccache", > > >>>>>> "VMWARE_USE_SHIPPED_GTK=yes", "XDG_DATA_DIRS=/usr/local/share:/"..., > > >>>>>> "USECCACHE=ON", "SSH_TTY=/dev/pts/2", "MAIL=/var/mail/dayd", > > >>>>>> "TERM=rxvt-unicode-256color", "SHELL=/bin/bash", "SHLVL=1", > > >>>>>> "PYTHONPATH=:/home/dayd/projects/"..., "LOGNAME=dayd", > > >>>>>> "DBUS_SESSION_BUS_ADDRESS=unix:pa"..., > > >>>>>> "XDG_RUNTIME_DIR=/run/user/6686", > > >>>>>> "PATH=/usr/lib/ccache/bin/:/usr/l"..., "USEDISTCC=OFF", > > >>>>>> "_=/usr/bin/strace", "OLDPWD=/home/dayd/projects"]strace: > > >>>>>> [wait(0x04057f) = 26302] WIFSTOPPED,sig=SIGTRAP,EVENT_EXEC (4) > > >>>>>> strace: [wait(0x00857f) = 26302] WIFSTOPPED,sig=133 > > >>>>>> ) = 0 > > >>>>>> strace: [wait(0x00047f) = 26302] WIFSTOPPED,sig=SIGILL > > >>>>>> --- SIGILL {si_signo=SIGILL, si_code=ILL_ILLOPN, si_addr=0x40169e} --- > > >>>>>> strace: [wait(0x06057f) = 26302] WIFSTOPPED,sig=SIGTRAP,EVENT_EXIT (6) > > >>>>>> strace: [wait(0x000084) = 26302] WIFSIGNALED,core,sig=SIGILL > > >>>>>> +++ killed by SIGILL (core dumped) +++ > > >>>>>> strace: dropped tcb for pid 26302, 0 remain > > >>>>>> Illegal instruction (core dumped) > > >>>>>> ****************************************************************************************** > > >>>>>> (gdb) bt > > >>>>>> #0 0x00000000004016ae in __copy_tls () > > >>>>>> #1 0x0000000000000000 in ?? () > > >>>>>> ***************************************************************************************** > > >>>>>> > > >>>>>> (gdb) disassemble > > >>>>>> Dump of assembler code for function __copy_tls: > > >>>>>> 0x0000000000401680 <+0>: push %r13 > > >>>>>> 0x0000000000401682 <+2>: mov %rdi,%r13 > > >>>>>> 0x0000000000401685 <+5>: push %r12 > > >>>>>> 0x0000000000401687 <+7>: push %rbp > > >>>>>> 0x0000000000401688 <+8>: push %rbx > > >>>>>> 0x0000000000401689 <+9>: sub $0x8,%rsp > > >>>>>> 0x000000000040168d <+13>: mov 0x410c(%rip),%rax # > > >>>>>> 0x4057a0 <__libc+32> > > >>>>>> 0x0000000000401694 <+20>: mov 0x410d(%rip),%rcx # > > >>>>>> 0x4057a8 <__libc+40> > > >>>>>> 0x000000000040169b <+27>: mov 0x40f6(%rip),%rbx # > > >>>>>> 0x405798 <__libc+24> > > >>>>>> 0x00000000004016a2 <+34>: lea -0xe0(%rdi,%rax,1),%rax > > >>>>>> 0x00000000004016aa <+42>: lea -0x1(%rcx),%r12 > > >>>>>> => 0x00000000004016ae <+46>: andn %rax,%r12,%r12 > > >>>>> This instruction, andn, is not in the base x86_64 ISA. It's part of > > >>>>> the BMI1 extension: > > >>>>> > > >>>>> https://en.wikipedia.org/wiki/Bit_Manipulation_Instruction_Sets#BMI1_(Bit_Manipulation_Instruction_Set_1) > > >>>>> > > >>>>> Apparently Gentoo is providing cross compilers that default to > > >>>>> producing "native ISA level" code for the host, rather than baseline > > >>>>> ISA for the target. But: > > >>>>> > > >>>>>> as a bonus (I thought this might also help): > > >>>>>> (%:~/projects/musl-test)┌- x86_64-pc-linux-musl-gcc -v -static -O0 -g > > >>>>>> hello.c > > >>>>>> Using built-in specs. > > >>>>>> COLLECT_GCC=x86_64-pc-linux-musl-gcc > > >>>>>> COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-pc-linux-musl/9.1.0/lto-wrapper > > >>>>>> Target: x86_64-pc-linux-musl > > >>>>>> Configured with: > > >>>>>> /var/tmp/portage/cross-x86_64-pc-linux-musl/gcc-9.1.0/work/gcc-9.1..0/configure > > >>>>>> --host=x86_64-pc-linux-gnu --target=x86_64-pc-linux-musl > > >>>>>> --build=x86_64-pc-linux-gnu --prefix=/usr > > >>>>>> --bindir=/usr/x86_64-pc-linux-gnu/x86_64-pc-linux-musl/gcc-bin/9..1.0 > > >>>>>> --includedir=/usr/lib/gcc/x86_64-pc-linux-musl/9.1.0/include > > >>>>>> --datadir=/usr/share/gcc-data/x86_64-pc-linux-musl/9.1.0 > > >>>>>> --mandir=/usr/share/gcc-data/x86_64-pc-linux-musl/9.1.0/man > > >>>>>> --infodir=/usr/share/gcc-data/x86_64-pc-linux-musl/9.1.0/info > > >>>>>> --with-gxx-include-dir=/usr/lib/gcc/x86_64-pc-linux-musl/9.1.0/include/g++-v9 > > >>>>>> --with-python-dir=/share/gcc-data/x86_64-pc-linux-musl/9.1.0/python > > >>>>>> --enable-languages=c,c++,fortran --enable-obsolete --enable-secureplt > > >>>>>> --disable-werror --with-system-zlib --enable-nls > > >>>>>> --without-included-gettext --enable-checking=release > > >>>>>> --with-bugurl=https://bugs.gentoo.org/ --with-pkgversion='Gentoo 9.1.0 > > >>>>>> p1.0' --disable-esp --enable-libstdcxx-time > > >>>>>> --enable-poison-system-directories > > >>>>>> --with-sysroot=/usr/x86_64-pc-linux-musl --disable-bootstrap > > >>>>>> --enable-__cxa_atexit --disable-multilib --with-multilib-list=m64 > > >>>>>> --disable-altivec --disable-fixed-point --enable-targets=all > > >>>>>> --enable-libgomp --disable-libmudflap --disable-libssp > > >>>>>> --disable-systemtap --disable-vtable-verify --disable-libvtv > > >>>>>> --enable-lto --without-isl --disable-libsanitizer --enable-default-pie > > >>>>>> --enable-default-ssp > > >>>>>> Thread model: posix > > >>>>>> gcc version 9.1.0 (Gentoo 9.1.0 p1.0) > > >>>>>> COLLECT_GCC_OPTIONS='-v' '-static' '-O0' '-g' '-mtune=generic' '-march=x86-64' > > >>>>>> /usr/libexec/gcc/x86_64-pc-linux-musl/9.1.0/cc1 -quiet -v hello.c > > >>>>>> -quiet -dumpbase hello.c -mtune=generic -march=x86-64 -auxbase hello > > >>>>>> -g -O0 -version -o /tmp/ccoTkh38.s > > >>>>>> GNU C17 (Gentoo 9.1.0 p1.0) version 9.1.0 (x86_64-pc-linux-musl) > > >>>>>> compiled by GNU C version 8.3.0, GMP version 6.1.2, MPFR version > > >>>>>> 4.0.2, MPC version 1.1.0, isl version none > > >>>>>> GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072 > > >>>>>> ignoring nonexistent directory "/usr/x86_64-pc-linux-musl/usr/local/include" > > >>>>>> ignoring nonexistent directory > > >>>>>> "/usr/lib/gcc/x86_64-pc-linux-musl/9.1.0/../../../../x86_64-pc-linux-musl/include" > > >>>>>> #include "..." search starts here: > > >>>>>> #include <...> search starts here: > > >>>>>> /usr/x86_64-pc-linux-musl/usr/include > > >>>>>> /usr/lib/gcc/x86_64-pc-linux-musl/9.1.0/include > > >>>>>> End of search list. > > >>>>>> GNU C17 (Gentoo 9.1.0 p1.0) version 9.1.0 (x86_64-pc-linux-musl) > > >>>>>> compiled by GNU C version 8.3.0, GMP version 6.1.2, MPFR version > > >>>>>> 4.0.2, MPC version 1.1.0, isl version none > > >>>>>> GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072 > > >>>>>> Compiler executable checksum: a3183e5f435d1938198d432af6db159b > > >>>>>> COLLECT_GCC_OPTIONS='-v' '-static' '-O0' '-g' '-mtune=generic' '-march=x86-64' > > >>>>>> /usr/libexec/gcc/x86_64-pc-linux-musl/as -v --64 -o /tmp/ccHq7CL6..o > > >>>>>> /tmp/ccoTkh38.s > > >>>>>> GNU assembler version 2.32.0 (x86_64-pc-linux-musl) using BFD version > > >>>>>> (Gentoo 2.32 p2) 2.32.0 > > >>>>>> COMPILER_PATH=/usr/libexec/gcc/x86_64-pc-linux-musl/9.1.0/:/usr/libexec/gcc/x86_64-pc-linux-musl/9.1.0/:/usr/libexec/gcc/x86_64-pc-linux-musl/:/usr/lib/gcc/x86_64-pc-linux-musl/9.1.0/:/usr/lib/gcc/x86_64-pc-linux-musl/ > > >>>>>> LIBRARY_PATH=/usr/lib/gcc/x86_64-pc-linux-musl/9.1.0/:/usr/lib/gcc/x86_64-pc-linux-musl/9.1.0/../../../../x86_64-pc-linux-musl/lib/:/usr/x86_64-pc-linux-musl/lib/:/usr/x86_64-pc-linux-musl/usr/lib/ > > >>>>>> COLLECT_GCC_OPTIONS='-v' '-static' '-O0' '-g' '-mtune=generic' '-march=x86-64' > > >>>>>> /usr/libexec/gcc/x86_64-pc-linux-musl/9.1.0/collect2 -plugin > > >>>>>> /usr/libexec/gcc/x86_64-pc-linux-musl/9.1.0/liblto_plugin.so > > >>>>>> -plugin-opt=/usr/libexec/gcc/x86_64-pc-linux-musl/9.1.0/lto-wrapper > > >>>>>> -plugin-opt=-fresolution=/tmp/ccWcifv4.res > > >>>>>> -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_eh > > >>>>>> -plugin-opt=-pass-through=-lc --sysroot=/usr/x86_64-pc-linux-musl -m > > >>>>>> elf_x86_64 -static /usr/x86_64-pc-linux-musl/usr/lib/crt1.o > > >>>>>> /usr/x86_64-pc-linux-musl/usr/lib/crti.o > > >>>>>> /usr/lib/gcc/x86_64-pc-linux-musl/9.1.0/crtbeginT.o > > >>>>>> -L/usr/lib/gcc/x86_64-pc-linux-musl/9.1.0 > > >>>>>> -L/usr/lib/gcc/x86_64-pc-linux-musl/9.1.0/../../../../x86_64-pc-linux-musl/lib > > >>>>>> -L/usr/x86_64-pc-linux-musl/lib -L/usr/x86_64-pc-linux-musl/usr/lib > > >>>>>> /tmp/ccHq7CL6.o --start-group -lgcc -lgcc_eh -lc --end-group > > >>>>>> /usr/lib/gcc/x86_64-pc-linux-musl/9.1.0/crtend.o > > >>>>>> /usr/x86_64-pc-linux-musl/usr/lib/crtn.o > > >>>>>> COLLECT_GCC_OPTIONS='-v' '-static' '-O0' '-g' '-mtune=generic' '-march=x86-64' > > >>>>>> > > >>>>>> mtune=generic and march=x86-64 suggests it isn't trying to do > > >>>>>> something weird on the compile computer. > > >>>>> Indeed, it looks like the default is right. So you probably need to > > >>>>> check how musl was built. Is it also a package provided by Gentoo, and > > >>>>> did they perhaps use -march=native even though it's being cross > > >>>>> compiled? > > >>>>> > > >>>>> Rich > > >>> > > > >