[musl] musl: unusable on mipsel, mips64el: mipsel-linux-gnu-gcc: unrecognised command-line option '-EL'

[musl] musl: unusable on mipsel, mips64el: mipsel-linux-gnu-gcc: unrecognised command-line option '-EL'

[Thorsten Glaser]

Hi musl maintainers,

waldi indeed provided a fix for this bug forgot to Cc me, so I missed
it until now. I tested this:



(sid_mips64el-dchroot)tg@eberlin:~$ sh -x $(which musl-gcc) hello.c
+ exec mips64el-linux-gnuabi64-gcc hello.c -specs /usr/lib/mips64el-linux-musl/musl-gcc.specs
mips64el-linux-gnuabi64-gcc: error: unrecognized command-line option '-EL'
(sid_mips64el-dchroot)tg@eberlin:~$ mips64el-linux-gnuabi64-gcc hello.c -specs ~/musl-gcc.specs
(sid_mips64el-dchroot)tg@eberlin:~$ ./a.out
hi
(sid_mips64el-dchroot)tg@eberlin:~$ diff -u /usr/lib/mips64el-linux-musl/musl-gcc.specs musl-gcc.specs
--- /usr/lib/mips64el-linux-musl/musl-gcc.specs 2023-11-10 19:30:40.000000000 +0000
+++ musl-gcc.specs      2024-02-03 08:07:01.309465472 +0000
@@ -1,10 +1,11 @@
 %rename cpp_options old_cpp_options
+%rename cc1 old_cc1

 *cpp_options:
 -nostdinc -isystem /usr/include/mips64el-linux-musl -isystem include%s %(old_cpp_options)

 *cc1:
-%(cc1_cpu) -nostdinc -isystem /usr/include/mips64el-linux-musl -isystem include%s
+%(cc1_cpu) -nostdinc -isystem /usr/include/mips64el-linux-musl -isystem include%s %(old_cc1)

 *link_libgcc:
 -L/usr/lib/mips64el-linux-musl -L .%s



This change (to tools/musl-gcc.specs.sh in the source tree) probably
makes sense on all architectures, so perhaps do that even. Upstream
should also consider including this and check which of the original
specs need not be removed and can be kept like this.

bye,
//mirabilos
-- 
22:20⎜<asarch> The crazy that persists in his craziness becomes a master
22:21⎜<asarch> And the distance between the craziness and geniality is
only measured by the success 18:35⎜<asarch> "Psychotics are consistently
inconsistent. The essence of sanity is to be inconsistently inconsistent

Re: [musl] musl: unusable on mipsel, mips64el: mipsel-linux-gnu-gcc: unrecognised command-line option '-EL'

[Thorsten Glaser]

>-%(cc1_cpu) -nostdinc -isystem /usr/include/mips64el-linux-musl -isystem include%s
>+%(cc1_cpu) -nostdinc -isystem /usr/include/mips64el-linux-musl -isystem include%s %(old_cc1)

Since old_cc1 includes cc1_cpu already, it can probably even be dropped.

bye,
//mirabilos
-- 
<ch> you introduced a merge commit        │<mika> % g rebase -i HEAD^^
<mika> sorry, no idea and rebasing just fscked │<mika> Segmentation
<ch> should have cloned into a clean repo      │  fault (core dumped)
<ch> if I rebase that now, it's really ugh     │<mika:#grml> wuahhhhhh

Re: [musl] musl: unusable on mipsel, mips64el: mipsel-linux-gnu-gcc: unrecognised command-line option '-EL'

[Rich Felker]

On Sat, Feb 03, 2024 at 08:20:28AM +0000, Thorsten Glaser wrote:
> Hi musl maintainers,
> 
> waldi indeed provided a fix for this bug forgot to Cc me, so I missed
> it until now. I tested this:
> 
> 
> 
> (sid_mips64el-dchroot)tg@eberlin:~$ sh -x $(which musl-gcc) hello.c
> + exec mips64el-linux-gnuabi64-gcc hello.c -specs /usr/lib/mips64el-linux-musl/musl-gcc.specs
> mips64el-linux-gnuabi64-gcc: error: unrecognized command-line option '-EL'
> (sid_mips64el-dchroot)tg@eberlin:~$ mips64el-linux-gnuabi64-gcc hello.c -specs ~/musl-gcc.specs
> (sid_mips64el-dchroot)tg@eberlin:~$ ./a.out
> hi
> (sid_mips64el-dchroot)tg@eberlin:~$ diff -u /usr/lib/mips64el-linux-musl/musl-gcc.specs musl-gcc.specs
> --- /usr/lib/mips64el-linux-musl/musl-gcc.specs 2023-11-10 19:30:40.000000000 +0000
> +++ musl-gcc.specs      2024-02-03 08:07:01.309465472 +0000
> @@ -1,10 +1,11 @@
>  %rename cpp_options old_cpp_options
> +%rename cc1 old_cc1
> 
>  *cpp_options:
>  -nostdinc -isystem /usr/include/mips64el-linux-musl -isystem include%s %(old_cpp_options)
> 
>  *cc1:
> -%(cc1_cpu) -nostdinc -isystem /usr/include/mips64el-linux-musl -isystem include%s
> +%(cc1_cpu) -nostdinc -isystem /usr/include/mips64el-linux-musl -isystem include%s %(old_cc1)
> 
>  *link_libgcc:
>  -L/usr/lib/mips64el-linux-musl -L .%s
> 
> 
> 
> This change (to tools/musl-gcc.specs.sh in the source tree) probably
> makes sense on all architectures, so perhaps do that even. Upstream
> should also consider including this and check which of the original
> specs need not be removed and can be kept like this.
> 
> bye,
> //mirabilos

OK, it's not musl that's unusable, but the gcc wrapper. This is not
the recommended way to use musl except as minimal evaluation setup or
for compiling very simple programs, and as you've found, it seems it's
almost entirely untested except on a couple mainstream archs.

This is something we should fix, but there are two issues:

1. On some archs, which I think includes mips, it can work in
   principle, but has gratuitous bugs keeping it from working. These
   should be easy to fix.

2. On some archs such as powerpc, the ABI is almost surely mismatched
   with the existing toolchain on a non-musl system. This should be
   caught at configure time, since the existing gcc is not able to
   build musl anyway, but it's possible that with sufficient additions
   to CFLAGS/CC, it might be possible to get musl to build, but then
   have a broken wrapper still (or to compile, but fail at link time
   or produce a broken libc.so due to mismatched libgcc.a). This
   probably needs attention too.

I'll try to take a look at this soon and see if the proposed wrapper
fix seems right for the mips situation, but the wrapper is generally
low-priority, and there's other stuff I'm trying to get to/finish
first.

Rich

Re: [musl] musl-gcc: unusable on mipsel, mips64el: mipsel-linux-gnu-gcc: unrecognised command-line option '-EL'

[Thorsten Glaser]

Rich Felker dixit:

>OK, it's not musl that's unusable

That referred to the Debian package musl-tools originally.

>, but the gcc wrapper. This is not
>the recommended way to use musl except as minimal evaluation setup or
>for compiling very simple programs

Huh? Where does it say that, and how else is one supposed to do this?
I have been building and testing mksh with the diet, klcc and musl-gcc
wrappers on all Debian architectures they are available for years, and
this is the first time I remember running into problems with it.

>, and as you've found, it seems it's
>almost entirely untested except on a couple mainstream archs.
>
>This is something we should fix

That would be welcome, thanks.

>1. On some archs, which I think includes mips, it can work in
>   principle, but has gratuitous bugs keeping it from working. These
>   should be easy to fix.

I’m testing those semi-automatically for all arches in Debian that
have a musl-tools package. That is currently:

release arches: amd64 arm64 armel armhf i386 mips64el ppc64el riscv64 s390x
ports arches: m68k sh4
removed arches: mips mipsel

>2. On some archs such as powerpc, the ABI is almost surely mismatched
>   with the existing toolchain on a non-musl system. This should be
>   caught at configure time, since the existing gcc is not able to
>   build musl anyway

Ah, this may be the reason we have no musl on powerpc in Debian.

> , but it's possible that with sufficient additions
>   to CFLAGS/CC, it might be possible to get musl to build, but then
>   have a broken wrapper still (or to compile, but fail at link time
>   or produce a broken libc.so due to mismatched libgcc.a). This
>   probably needs attention too.

This sounds to me like configure should take care of adding the
needed flags both to CFLAGS and the wrapper on powerpc?

>I'll try to take a look at this soon and see if the proposed wrapper
>fix seems right for the mips situation

At least for MIPS, it looks to me so. The original specs are:

Debian sid, gcc 13.2.0-13/mips64el:

*cc1:
%{G*} %{EB:-meb} %{EL:-mel} %{EB:%{EL:%emay not use both -EB and -EL}} %(subtarget_cc1_spec)

*subtarget_cc1_spec:
%{!mandroid|tno-android-cc:%{profile:-p};:%{profile:-p} %{!mglibc:%{!muclibc:%{!mbionic: -mbionic}}} %{!fno-pic:%{!fno-PIC:%{!fpic:%{!fPIC: -fPIC}}}}}


We lost mipsel in unstable recently, but bookworm still has it
with gcc 12.2.0-14:

*cc1:
%{G*} %{EB:-meb} %{EL:-mel} %{EB:%{EL:%emay not use both -EB and -EL}} %(subtarget_cc1_spec)

*subtarget_cc1_spec:
%{!mandroid|tno-android-cc:%{profile:-p};:%{profile:-p} %{!mglibc:%{!muclibc:%{!mbionic: -mbionic}}} %{!fno-pic:%{!fno-PIC:%{!fpic:%{!fPIC: -fPIC}}}}}


On amd64, cc1 does include the defaulting to PIC part:

[…] %{!fno-pic:%{!fno-PIC:%{!fpic:%{!fPIC: -fPIC}}}}}

So maybe this does need specific looking at in general, to avoid
surprises.

One idea would be to output gcc -dumpspecs during the build, then
inspect various distros’ build logs for this to see whether something
from there needs to be added.

I think the MIPS problem is that -EL is added by the frontend by
default; were it not, or were it to add -mel instead, we wouldn’t
have this problem. So, unclean-ish solution on the GCC side. I do
wonder what changed between GCC 12 and 13 to trigger this, though.

>, but the wrapper is generally low-priority, and there's other stuff
>I'm trying to get to/finish first.

OK, fair. Maybe this can be applied in Debian, and until then, I’m
using a gcc-12 dependency on MIPS for mksh to build with musl-gcc,
I probably could do the same workaround by copying the specs to a
temporary directory, too…

If I can help in any way, e.g. by testing things on Debian project
machines (porterboxen), feel free to ping me (Cc, or on Fedi). I
also have connections to Debian porters for various ports arches
(well, that’s mostly cbmuser these days, though I helped out in
the past as well).

Something else relating to the wrapper: Debian #971937 indicates
that musl’s stdc-predef.h is not automatically included when using
it (but at least neither is glibc’s, so that is a plus compared to
diet (dietlibc wrapper).

────────────────────────────────────────────────────────────────────────

And (I know this has been a contended topic, but from an upstream
PoV, please let me squeeze in this), it would be good if there was
a __MUSL__ predef not because musl needs special handling, but to
not do all the special handling I have to do to get things working
on all the other libcs for Linux (dietlibc, klibc, µClibc, glibc in
modern and ancient versions, and even libc5). If I knew it was musl
I could just skip a whole block of tests and workarounds. It *has*
been the most pleasant Linux libc to work with, and so far the only
one I have not had to contribute bugfixes to yet ;-)

Thanks,
//mirabilos
-- 
„Cool, /usr/share/doc/mksh/examples/uhr.gz ist ja ein Grund,
mksh auf jedem System zu installieren.“
	-- XTaran auf der OpenRheinRuhr, ganz begeistert
(EN: “[…]uhr.gz is a reason to install mksh on every system.”)

Re: [musl] musl-gcc: unusable on mipsel, mips64el: mipsel-linux-gnu-gcc: unrecognised command-line option '-EL'

[Markus Wichmann]

Am Sun, Feb 04, 2024 at 02:14:11AM +0000 schrieb Thorsten Glaser:
> Rich Felker dixit:
>
> >OK, it's not musl that's unusable
>
> That referred to the Debian package musl-tools originally.
>
> >, but the gcc wrapper. This is not
> >the recommended way to use musl except as minimal evaluation setup or
> >for compiling very simple programs
>
> Huh? Where does it say that, and how else is one supposed to do this?
> I have been building and testing mksh with the diet, klcc and musl-gcc
> wrappers on all Debian architectures they are available for years, and
> this is the first time I remember running into problems with it.
>

musl-gcc (and similar wrappers for other libcs) is in principle limited
by the fact that the target libraries it links in are still built
against the system libc. Most of the time there is no issue, since the
target libraries only contain simple code that either does not depend on
libc or (typically) only depends on a protable interface. E.g. on ARM
the 64-bit division code used to call raise(SIGFPE) if it saw the
divisor was zero. This typically works because raise() is implemented as
a function in all libcs and SIGFPE is kernel interface.

But sometimes there are issues anyway, and not just because of ABI
mismatch. Some of GCC's target libraries hook deeply into glibc
implementation internals once they know to be running on a glibc
platform, with libstdc++ being probably the worst offender.

A true cross-compiler avoids that by making the compiler choose a
musl-compatible ABI and letting it know that it is specifically not
building against glibc.

Ciao,
Markus

Re: [musl] musl-gcc: unusable on mipsel, mips64el: mipsel-linux-gnu-gcc: unrecognised command-line option '-EL'

[Thorsten Glaser]

Markus Wichmann dixit:

>> >, but the gcc wrapper. This is not
>> >the recommended way to use musl except as minimal evaluation setup or
>> >for compiling very simple programs
>>
>> Huh? Where does it say that, and how else is one supposed to do this?
[…]
>musl-gcc (and similar wrappers for other libcs) is in principle limited
>by the fact that the target libraries it links in are still built
>against the system libc.

The target library is exactly libc (musl) though.

>mismatch. Some of GCC's target libraries hook deeply into glibc
>implementation internals once they know to be running on a glibc
>platform, with libstdc++ being probably the worst offender.

Ah, ok, good point. I know libgcc.a doesn’t, and for some reason
libgcc_eh.a is also pulled in, though I don’t see anything using
its functions.

bye,
//mirabilos
-- 
FWIW, I'm quite impressed with mksh interactively. I thought it was much
*much* more bare bones. But it turns out it beats the living hell out of
ksh93 in that respect. I'd even consider it for my daily use if I hadn't
wasted half my life on my zsh setup. :-) -- Frank Terbeck in #!/bin/mksh