[musl] `musl-gcc -static` and lld/mold

[musl] `musl-gcc -static` and lld/mold

[Rui Ueyama]

[-- Attachment #1: Type: text/plain, Size: 822 bytes --]

Hi,

I think I found a musl-gcc issue. It looks like musl-gcc always appends
`-dynamic-linker /lib/ld-musl-x86_64.so.1` even if `-static` is given. That
causes a created program to immediately crash on startup as you can see
below:

$ cat hello.c
#include <stdio.h>
int main() { printf("Hello\n"); }

$ musl-gcc -static -fuse-ld=lld hello.c -o hello

$ ./hello
Segmentation fault (core dumped)

$ musl-gcc -static -fuse-ld=lld hello.c -o hello -Wl,-no-dynamic-linker
$ ./hello
Hello

This also happens to my new linker, mold, as well. `-dynamic-linker` option
is passed to the linker, and lld and mold do what it is told to do, so I
don't think it is a linker's bug. Rather, it's a compiler front end's bug
that passes the unnecessary command line option. Can you not to append
`-dynamic-linker` if `-static`?

Rui Ueyama

[-- Attachment #2: Type: text/html, Size: 1108 bytes --]

Re: [musl] `musl-gcc -static` and lld/mold

[Rich Felker]

On Sun, Nov 13, 2022 at 08:11:29AM +0800, Rui Ueyama wrote:
> Hi,
> 
> I think I found a musl-gcc issue. It looks like musl-gcc always appends
> `-dynamic-linker /lib/ld-musl-x86_64.so.1` even if `-static` is given. That
> causes a created program to immediately crash on startup as you can see
> below:
> 
> $ cat hello.c
> #include <stdio.h>
> int main() { printf("Hello\n"); }
> 
> $ musl-gcc -static -fuse-ld=lld hello.c -o hello
> 
> $ ./hello
> Segmentation fault (core dumped)
> 
> $ musl-gcc -static -fuse-ld=lld hello.c -o hello -Wl,-no-dynamic-linker
> $ ./hello
> Hello
> 
> This also happens to my new linker, mold, as well. `-dynamic-linker` option
> is passed to the linker, and lld and mold do what it is told to do, so I
> don't think it is a linker's bug. Rather, it's a compiler front end's bug
> that passes the unnecessary command line option. Can you not to append
> `-dynamic-linker` if `-static`?

Yes, I think this should be fixed. It only works with bfd ld without
static pie (which we're also missing support for) because it just
ignores -dynamic-linker in the ET_EXEC case.

Would you be willing to propose a candidate patch? I believe this has
been raised before in the context of static pie not working with
musl-gcc (it didn't exist when the wrapper was added) so ideally that
will get fixed too.

Rich

Re: [musl] `musl-gcc -static` and lld/mold

[Rui Ueyama]

[-- Attachment #1: Type: text/plain, Size: 1505 bytes --]

Let me try to create a patch.

On Sun, Nov 13, 2022 at 8:38 AM Rich Felker <dalias@libc.org> wrote:

> On Sun, Nov 13, 2022 at 08:11:29AM +0800, Rui Ueyama wrote:
> > Hi,
> >
> > I think I found a musl-gcc issue. It looks like musl-gcc always appends
> > `-dynamic-linker /lib/ld-musl-x86_64.so.1` even if `-static` is given.
> That
> > causes a created program to immediately crash on startup as you can see
> > below:
> >
> > $ cat hello.c
> > #include <stdio.h>
> > int main() { printf("Hello\n"); }
> >
> > $ musl-gcc -static -fuse-ld=lld hello.c -o hello
> >
> > $ ./hello
> > Segmentation fault (core dumped)
> >
> > $ musl-gcc -static -fuse-ld=lld hello.c -o hello -Wl,-no-dynamic-linker
> > $ ./hello
> > Hello
> >
> > This also happens to my new linker, mold, as well. `-dynamic-linker`
> option
> > is passed to the linker, and lld and mold do what it is told to do, so I
> > don't think it is a linker's bug. Rather, it's a compiler front end's bug
> > that passes the unnecessary command line option. Can you not to append
> > `-dynamic-linker` if `-static`?
>
> Yes, I think this should be fixed. It only works with bfd ld without
> static pie (which we're also missing support for) because it just
> ignores -dynamic-linker in the ET_EXEC case.
>
> Would you be willing to propose a candidate patch? I believe this has
> been raised before in the context of static pie not working with
> musl-gcc (it didn't exist when the wrapper was added) so ideally that
> will get fixed too.
>
> Rich
>

[-- Attachment #2: Type: text/html, Size: 2009 bytes --]

Re: [musl] `musl-gcc -static` and lld/mold

[Rui Ueyama]

[-- Attachment #1: Type: text/plain, Size: 2132 bytes --]

The below patch should fix the issue.

diff --git a/tools/musl-gcc.specs.sh b/tools/musl-gcc.specs.sh
index 30492574..ffb46d70 100644
--- a/tools/musl-gcc.specs.sh
+++ b/tools/musl-gcc.specs.sh
@@ -23,7 +23,7 @@ libgcc.a%s %:if-exists(libgcc_eh.a%s)
 crtendS.o%s $libdir/crtn.o

 *link:
--dynamic-linker $ldso -nostdlib %{shared:-shared} %{static:-static}
%{rdynamic:-export-dynamic}
+%{!static:-dynamic-linker $ldso} -nostdlib %{shared:-shared}
%{static:-static} %{rdynamic:-export-dynamic}

 *esp_link:

On Sun, Nov 13, 2022 at 8:46 AM Rui Ueyama <rui314@gmail.com> wrote:

> Let me try to create a patch.
>
> On Sun, Nov 13, 2022 at 8:38 AM Rich Felker <dalias@libc.org> wrote:
>
>> On Sun, Nov 13, 2022 at 08:11:29AM +0800, Rui Ueyama wrote:
>> > Hi,
>> >
>> > I think I found a musl-gcc issue. It looks like musl-gcc always appends
>> > `-dynamic-linker /lib/ld-musl-x86_64.so.1` even if `-static` is given.
>> That
>> > causes a created program to immediately crash on startup as you can see
>> > below:
>> >
>> > $ cat hello.c
>> > #include <stdio.h>
>> > int main() { printf("Hello\n"); }
>> >
>> > $ musl-gcc -static -fuse-ld=lld hello.c -o hello
>> >
>> > $ ./hello
>> > Segmentation fault (core dumped)
>> >
>> > $ musl-gcc -static -fuse-ld=lld hello.c -o hello -Wl,-no-dynamic-linker
>> > $ ./hello
>> > Hello
>> >
>> > This also happens to my new linker, mold, as well. `-dynamic-linker`
>> option
>> > is passed to the linker, and lld and mold do what it is told to do, so I
>> > don't think it is a linker's bug. Rather, it's a compiler front end's
>> bug
>> > that passes the unnecessary command line option. Can you not to append
>> > `-dynamic-linker` if `-static`?
>>
>> Yes, I think this should be fixed. It only works with bfd ld without
>> static pie (which we're also missing support for) because it just
>> ignores -dynamic-linker in the ET_EXEC case.
>>
>> Would you be willing to propose a candidate patch? I believe this has
>> been raised before in the context of static pie not working with
>> musl-gcc (it didn't exist when the wrapper was added) so ideally that
>> will get fixed too.
>>
>> Rich
>>
>

[-- Attachment #2: Type: text/html, Size: 3097 bytes --]

Re: [musl] `musl-gcc -static` and lld/mold

[Fangrui Song]

On 2022-11-13, Rui Ueyama wrote:
>The below patch should fix the issue.
>
>diff --git a/tools/musl-gcc.specs.sh b/tools/musl-gcc.specs.sh
>index 30492574..ffb46d70 100644
>--- a/tools/musl-gcc.specs.sh
>+++ b/tools/musl-gcc.specs.sh
>@@ -23,7 +23,7 @@ libgcc.a%s %:if-exists(libgcc_eh.a%s)
> crtendS.o%s $libdir/crtn.o
>
> *link:
>--dynamic-linker $ldso -nostdlib %{shared:-shared} %{static:-static}
>%{rdynamic:-export-dynamic}
>+%{!static:-dynamic-linker $ldso} -nostdlib %{shared:-shared}
>%{static:-static} %{rdynamic:-export-dynamic}
>
> *esp_link:

I use this patch which handles -static-pie as well: https://github.com/MaskRay/musl/tree/musl-gcc

In addition, I use `*libdir: $libdir` to avoid absolute path references
so that the spec file can be easily fixed after moving the build directory.

>On Sun, Nov 13, 2022 at 8:46 AM Rui Ueyama <rui314@gmail.com> wrote:
>
>> Let me try to create a patch.
>>
>> On Sun, Nov 13, 2022 at 8:38 AM Rich Felker <dalias@libc.org> wrote:
>>
>>> On Sun, Nov 13, 2022 at 08:11:29AM +0800, Rui Ueyama wrote:
>>> > Hi,
>>> >
>>> > I think I found a musl-gcc issue. It looks like musl-gcc always appends
>>> > `-dynamic-linker /lib/ld-musl-x86_64.so.1` even if `-static` is given.
>>> That
>>> > causes a created program to immediately crash on startup as you can see
>>> > below:
>>> >
>>> > $ cat hello.c
>>> > #include <stdio.h>
>>> > int main() { printf("Hello\n"); }
>>> >
>>> > $ musl-gcc -static -fuse-ld=lld hello.c -o hello
>>> >
>>> > $ ./hello
>>> > Segmentation fault (core dumped)
>>> >
>>> > $ musl-gcc -static -fuse-ld=lld hello.c -o hello -Wl,-no-dynamic-linker
>>> > $ ./hello
>>> > Hello
>>> >
>>> > This also happens to my new linker, mold, as well. `-dynamic-linker`
>>> option
>>> > is passed to the linker, and lld and mold do what it is told to do, so I
>>> > don't think it is a linker's bug. Rather, it's a compiler front end's
>>> bug
>>> > that passes the unnecessary command line option. Can you not to append
>>> > `-dynamic-linker` if `-static`?
>>>
>>> Yes, I think this should be fixed. It only works with bfd ld without
>>> static pie (which we're also missing support for) because it just
>>> ignores -dynamic-linker in the ET_EXEC case.
>>>
>>> Would you be willing to propose a candidate patch? I believe this has
>>> been raised before in the context of static pie not working with
>>> musl-gcc (it didn't exist when the wrapper was added) so ideally that
>>> will get fixed too.
>>>
>>> Rich
>>>
>>

Re: [musl] `musl-gcc -static` and lld/mold

[Szabolcs Nagy]

* Fangrui Song <i@maskray.me> [2022-11-12 19:26:53 -0800]:
> On 2022-11-13, Rui Ueyama wrote:
> > diff --git a/tools/musl-gcc.specs.sh b/tools/musl-gcc.specs.sh
> > index 30492574..ffb46d70 100644
> > --- a/tools/musl-gcc.specs.sh
> > +++ b/tools/musl-gcc.specs.sh
> > @@ -23,7 +23,7 @@ libgcc.a%s %:if-exists(libgcc_eh.a%s)
> > crtendS.o%s $libdir/crtn.o
> > 
> > *link:
> > --dynamic-linker $ldso -nostdlib %{shared:-shared} %{static:-static}
> > %{rdynamic:-export-dynamic}
> > +%{!static:-dynamic-linker $ldso} -nostdlib %{shared:-shared}
> > %{static:-static} %{rdynamic:-export-dynamic}
> > 
> > *esp_link:
> 
> I use this patch which handles -static-pie as well: https://github.com/MaskRay/musl/tree/musl-gcc

i guess you meant
https://github.com/MaskRay/musl/commit/e8a9c5489b9be78a4532712045df6f7cd45c4de6
(would be nice if it was submitted to the list)

> 
> In addition, I use `*libdir: $libdir` to avoid absolute path references
> so that the spec file can be easily fixed after moving the build directory.

i think that should only affect the paths used at compile/link time
but not paths at runtime:

%{static|static-pie:; :-dynamic-linker %(libdir)/libc.so}

different runtime path should be a separate option.
(just like --syslibdir is separate from --libdir, e.g. syslibdir
could be a variable too)

there is also interference with the -static-pie handling of gcc's
driver which might cause trouble when -static and -static-pie is mixed:

%{static-pie:-static -pie --no-dynamic-linker -z text}

Re: [musl] `musl-gcc -static` and lld/mold

[Rich Felker]

On Sun, Nov 13, 2022 at 04:51:26PM +0100, Szabolcs Nagy wrote:
> * Fangrui Song <i@maskray.me> [2022-11-12 19:26:53 -0800]:
> > On 2022-11-13, Rui Ueyama wrote:
> > > diff --git a/tools/musl-gcc.specs.sh b/tools/musl-gcc.specs.sh
> > > index 30492574..ffb46d70 100644
> > > --- a/tools/musl-gcc.specs.sh
> > > +++ b/tools/musl-gcc.specs.sh
> > > @@ -23,7 +23,7 @@ libgcc.a%s %:if-exists(libgcc_eh.a%s)
> > > crtendS.o%s $libdir/crtn.o
> > > 
> > > *link:
> > > --dynamic-linker $ldso -nostdlib %{shared:-shared} %{static:-static}
> > > %{rdynamic:-export-dynamic}
> > > +%{!static:-dynamic-linker $ldso} -nostdlib %{shared:-shared}
> > > %{static:-static} %{rdynamic:-export-dynamic}
> > > 
> > > *esp_link:
> > 
> > I use this patch which handles -static-pie as well: https://github.com/MaskRay/musl/tree/musl-gcc
> 
> i guess you meant
> https://github.com/MaskRay/musl/commit/e8a9c5489b9be78a4532712045df6f7cd45c4de6
> (would be nice if it was submitted to the list)
> 
> > 
> > In addition, I use `*libdir: $libdir` to avoid absolute path references
> > so that the spec file can be easily fixed after moving the build directory.
> 
> i think that should only affect the paths used at compile/link time
> but not paths at runtime:
> 
> %{static|static-pie:; :-dynamic-linker %(libdir)/libc.so}
> 
> different runtime path should be a separate option.
> (just like --syslibdir is separate from --libdir, e.g. syslibdir
> could be a variable too)

Yes, this change is wrong. Use of $ldso here was very intentional.

> there is also interference with the -static-pie handling of gcc's
> driver which might cause trouble when -static and -static-pie is mixed:
> 
> %{static-pie:-static -pie --no-dynamic-linker -z text}

While musl-target-patched gcc supports -static -pie (or -static with
default-pie) as static-pie, we can't assume a gcc we're repurposing
does that, so I think we need to make -static just act as classic
static (no pie). This can probably be in the "else case" for
-static-pie though, so that, if both are included, -static-pie takes
precedence.

Note that we can't just implement the desired behavior for -static
afaict because it requires existence of --no-dynamic-linker, which the
host ld might not have. So, something like:

*link:
-nostdlib %{static-pie:-static -pie --no-dynamic-linker -z text;%{static:-static;-dynamic-linker $ldso %{shared:-shared}}} %{rdynamic:-export-dynamic}

Re: [musl] `musl-gcc -static` and lld/mold

[Fangrui Song]

On 2022-11-13, Rich Felker wrote:
>On Sun, Nov 13, 2022 at 04:51:26PM +0100, Szabolcs Nagy wrote:
>> * Fangrui Song <i@maskray.me> [2022-11-12 19:26:53 -0800]:
>> > On 2022-11-13, Rui Ueyama wrote:
>> > > diff --git a/tools/musl-gcc.specs.sh b/tools/musl-gcc.specs.sh
>> > > index 30492574..ffb46d70 100644
>> > > --- a/tools/musl-gcc.specs.sh
>> > > +++ b/tools/musl-gcc.specs.sh
>> > > @@ -23,7 +23,7 @@ libgcc.a%s %:if-exists(libgcc_eh.a%s)
>> > > crtendS.o%s $libdir/crtn.o
>> > >
>> > > *link:
>> > > --dynamic-linker $ldso -nostdlib %{shared:-shared} %{static:-static}
>> > > %{rdynamic:-export-dynamic}
>> > > +%{!static:-dynamic-linker $ldso} -nostdlib %{shared:-shared}
>> > > %{static:-static} %{rdynamic:-export-dynamic}
>> > >
>> > > *esp_link:
>> >
>> > I use this patch which handles -static-pie as well: https://github.com/MaskRay/musl/tree/musl-gcc
>>
>> i guess you meant
>> https://github.com/MaskRay/musl/commit/e8a9c5489b9be78a4532712045df6f7cd45c4de6
>> (would be nice if it was submitted to the list)
>>
>> >
>> > In addition, I use `*libdir: $libdir` to avoid absolute path references
>> > so that the spec file can be easily fixed after moving the build directory.
>>
>> i think that should only affect the paths used at compile/link time
>> but not paths at runtime:
>>
>> %{static|static-pie:; :-dynamic-linker %(libdir)/libc.so}
>>
>> different runtime path should be a separate option.
>> (just like --syslibdir is separate from --libdir, e.g. syslibdir
>> could be a variable too)
>
>Yes, this change is wrong. Use of $ldso here was very intentional.

I agree for installation purposes using $ldso is better.
However, the main purpose of this change is to allow musl-gcc without `make install`.

I actually also use

*cpp_options:
-nostdinc -isystem %(srcdir)/arch/x86_64 -isystem %(srcdir)/arch/generic -isystem %(srcdir)/include -isystem %(objdir)/obj/include -isystem include%s %(old_cpp_options)

*cc1:
%(cc1_cpu) -isystem %(srcdir)/arch/x86_64 -isystem %(srcdir)/arch/generic -isystem %(srcdir)/include -isystem %(objdir)/obj/include -nostdinc -isystem include%s

so that I can use musl-gcc without installing the headers. It's
assuredly hacky but seems to work well for my learning purpose.

I have the musl-gcc.specs template in my utility directory which I'll
use to overwrite musl configure created musl-gcc.specs :) The repo was
created to give a one-off link to someone asked how to use -static-pie.

>> there is also interference with the -static-pie handling of gcc's
>> driver which might cause trouble when -static and -static-pie is mixed:
>>
>> %{static-pie:-static -pie --no-dynamic-linker -z text}
>
>While musl-target-patched gcc supports -static -pie (or -static with
>default-pie) as static-pie, we can't assume a gcc we're repurposing
>does that, so I think we need to make -static just act as classic
>static (no pie). This can probably be in the "else case" for
>-static-pie though, so that, if both are included, -static-pie takes
>precedence.

In glibc gcc, -static-pie -static are not supposed to be used together.
Using both will pass -static -pie --no-dynamic-linker to ld while
crtbeginT.o (only intended for -static) is used, likely causing a linker
error.

(Similarly, while it appears that the default spec file let -shared override -pie/-static/-static-pie
-shared -static cannot be used together in practice because of crtbeginT.o.)

>Note that we can't just implement the desired behavior for -static
>afaict because it requires existence of --no-dynamic-linker, which the
>host ld might not have. So, something like:
>
>*link:
>-nostdlib %{static-pie:-static -pie --no-dynamic-linker -z text;%{static:-static;-dynamic-linker $ldso %{shared:-shared}}} %{rdynamic:-export-dynamic}
>

GNU ld has --no-dynamic-linker since 2.26 (2015)/lld has it since 2017
so I do not worry about the portability that much.  If a user really
uses a very old linker, they can change the musl-gcc.specs file
themselves. (The file is for convenience not fully supported AIUI.)