From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/13333 Path: news.gmane.org!.POSTED!not-for-mail From: "Jason A. Donenfeld" Newsgroups: gmane.linux.lib.musl.general Subject: Re: broken shared executables on armeb (illegal instruction) Date: Mon, 1 Oct 2018 01:11:15 +0200 Message-ID: References: <20180930215317.GA11313@zx2c4.com> <20180930221754.GW17995@brightrain.aerifal.cx> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Trace: blaine.gmane.org 1538348979 20040 195.159.176.226 (30 Sep 2018 23:09:39 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sun, 30 Sep 2018 23:09:39 +0000 (UTC) To: musl@lists.openwall.com Original-X-From: musl-return-13349-gllmg-musl=m.gmane.org@lists.openwall.com Mon Oct 01 01:09:35 2018 Return-path: Envelope-to: gllmg-musl@m.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by blaine.gmane.org with smtp (Exim 4.84_2) (envelope-from ) id 1g6kqB-00056O-Px for gllmg-musl@m.gmane.org; Mon, 01 Oct 2018 01:09:33 +0200 Original-Received: (qmail 5125 invoked by uid 550); 30 Sep 2018 23:11:40 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Original-Received: (qmail 4080 invoked from network); 30 Sep 2018 23:11:40 -0000 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=zx2c4.com; h=mime-version :references:in-reply-to:from:date:message-id:subject:to :content-type; s=mail; bh=U9Ektaud7DCcO3jBZ8w2QS4ekow=; b=Nopqzu Fk98IKZhFVIF6HE96T0lretMU4fnlRt2ZOJffi8P9eSFvqjcyaxfk+c6dlkxvmHT UxXbzyFz3i4FtDvJOrPfdFKmaDKoTzJtIwtfN5SOrFCSm0qhLoGN78s0CuVzeu3l c8zkkGXI8PxV1cM1ubKfole0L9n1V0/aCpvnS1vvWR9gNnD7bXdlKPIWxJpoddkU AWRpkDM6OO37Uc9lpV8G0sl5cl9xlAlCp6P9qL2ElmA9dZvN10WvwhfZ4971ziCq ph0iBh+AtJx4SNL21l4ld4QU2UYcXzAf294DNbYrDthyZFu0BBSz3gj08wp4fvLj XOU6XH/44mTCkAkQ== X-Gm-Message-State: ABuFfogDGna5DWh0y0seeRuWJqEi5qSZdf6aLLyhzH5c0jgUGEiM8Dvm 73cHL2abCOUbR67ROzbzVxqwwYf6MCeCJ2145xw= X-Google-Smtp-Source: ACcGV62eyS0yK28ziFjjURR4vl0EGOQz9mPp/TcdCzfRnADuxplwdHeH+jS9/Pm6krtQQCcLmf+XqkYe7FjfOijWm8o= X-Received: by 2002:a9d:61d0:: with SMTP id h16-v6mr5331452otk.225.1538349087173; Sun, 30 Sep 2018 16:11:27 -0700 (PDT) In-Reply-To: <20180930221754.GW17995@brightrain.aerifal.cx> X-Gmail-Original-Message-ID: Xref: news.gmane.org gmane.linux.lib.musl.general:13333 Archived-At: Hey Rich, Thanks for the insight. On Mon, Oct 1, 2018 at 12:24 AM Rich Felker wrote: > > $ printf '#include \nint main(){puts("hello world");}' | bin/musl-gcc -xc -o helloworld $CFLAGS - > > /usr/libexec/gcc/armeb-pc-linux-gnueabi/ld: /usr/lib/gcc/armeb-pc-linux-gnueabi/8.2.0/libgcc.a(_dvmd_lnx.o): in function `__aeabi_idiv0': > > /var/tmp/portage/cross-armeb-pc-linux-gnueabi/gcc-8.2.0-r3/work/gcc-8.2.0/libgcc/config/arm/lib1funcs.S:1545: undefined reference to `raise' > > collect2: error: ld returned 1 exit status > > [This appears to be a well-known bug in some other mailing list post. Working around with the next command:] > > This looks like you're trying to dynamic-link anyway...? Yes. It's this old "bug", fwiw: https://www.openwall.com/lists/musl/2018/05/09/1 > > Overriding the ABI seems like a really bad idea. What ABI is your > toolchain defaulting to? Good thinking: armv5. Though notably I don't have the same issue with little endian. And passing '-cpu cortex-a15' or the like to qemu-user doesn't fix that. Maybe I'll ask around over on the qemu mailing list, though. However: > Have you fun with -singlestep -d in_asm,nochain so you can see what > instruction it faults on? Interestingly, when in armv5 mode, it works fine and the disassembly looks correct. But when in armv7 mode, it seems to be interpreting all of the instructions in the wrong endian, right up to the failing one: ---------------- IN: 0xff79561c: 00b0a0e3 adcseq sl, r0, r3, ror #1 ---------------- IN: 0xff795620: 00e0a0e3 rsceq sl, r0, r3, ror #1 ---------------- IN: 0xff795624: 10109fe5 andsne r9, r0, r5, ror #31 ---------------- IN: 0xff795628: 01108fe0 tsteq r0, r0, ror #31 ---------------- IN: 0xff79562c: 0d00a0e1 stceq 0, cr10, [r0, #-900] So this looks like what's actually happening is gcc goes into -mbe8 mode with armv7-a, which is to be expected. But QEMU is always in BE32 mode. Passing -mbe32 to the cflags "fixes" the issue, though it's still unclear how to run BE8 code in qemu. But anyway, it's clear this is probably not a musl issue anymore at this point. So thanks for the pointer. The real issue I'm facing is not being able to start a userland in the kernel in big endian mode, and this persists even using the above tricks (-mbe8 and -march=armv5 and so forth). I'll keep plugging away, but indeed this probably isn't musl related. Thanks again, Jason