Re: [BUG] Force to use a dynamic linker in musl-clang wapper cause a crash for static-pie c++ programs.

[Fangrui Song <i@maskray.me>]

>On Tue, Dec 24, 2019 at 11:03 PM Rich Felker <dalias@libc.org> wrote:
>
>    On Tue, Dec 24, 2019 at 10:38:49PM +0800, Youren Shen wrote:
>    > Hey, there,
>    > Recently I'm trying to build a non-gnu toolchain with musl, clang, llvm,
>    > libc++, compiler-rt. While static-pie feature is very useful in our
>    > project, musl-clang force to link a dynamic linker into the binary. This
>    > behavior will cause a crash in c++ programs with compiler-rt and libc++.
>    > For more details and reproduction of this bug, you can read my previous
>    > email to llvm-dev mail lists.[1]
>    > I spend a few days to find the reason -- in function _dlstart_c, the
>    > program will get a "base" of relocation in /lib/ld-musl-x86_64.so.1,
>    which
>    > is not right when it tries to relocate .rel.dyn section in the binary.
>    > Overall, the static-pie program does not need a dynamic linker at all. So
>    > maybe we should remove this argument in wrapper when static-pie is
>    enabled.
>    >
>    > Thank you very much.
>    >
>    > [1]. https://groups.google.com/forum/#!msg/llvm-dev/XPrSPqD2zjM/
>    YtH6Fi2YAgAJ
>
>    I don't think the wrappers (gcc or clang one) have been updated with
>    logic for static pie since it was added, and I'm not sure how easy it
>    is to add. I'd welcome patches for review, though.
>
>    Rich
>
> Thanks for your reply.
>
> Dived deeper, I think this is because the options clang/gcc passed to ld are
> not well handled by ld. Unlike "-static", which is passed directly to the
> linker, the clang pass -no-dynamic-linker when static-pie is enabled. However,
> -dynamic-linker=<file> in ld.musl-clang wrapper conflict with this
> -no-dynamic-linker. As a result, ld accepts the last one it received, which is
> "-dynamic-linker=<file>" in this case. Here is the problem: even we passed
> static-pie to clang, and then the clang pass -no-dynamic-linker to ld, This
> option is still omitted by ld.musl-clang. I suggest a fix on it: move
> -dynamic-linker "$ldso" before the previous user inputs.
> Before:
> ```
> exec $($cc -print-prog-name=ld) -nostdlib "$@" -lc -dynamic-linker "$ldso"
> ```
> After:
> ```
>  exec $($cc -print-prog-name=ld) -nostdlib -dynamic-linker "$ldso" "$@" -lc 
> ```
> As you can see, this is a quite simple patch. Further testes may required. And
> for musl-gcc, as I test, it seems that it has the same problem. But I'm not
> familiar with gcc specs file. So maybe if anyone encounter the same problem,
> they can refer this email and give a solution.

It may be better moving -dynamic-linker from ld.musl-clang to musl-clang.
For -shared, -static, and -static-pie, gcc specs does not pass -dynamic-linker.
musl-clang should not pass -dynamic-linker if one of -shared, -static, -static-pie is used.

(I think the ideal solution is not to have any platform-dependent
default ld.so path (if GNU ld had done this, there would have been no
point to have --no-dynamic-linker in the first place). Let
-dynamic-linker create .interp . See https://reviews.llvm.org/D62765)

[-Wunused-command-line-argument] is another issue:

   % ~/musl/Debug/obj/musl-clang a.c -c
   clang: warning: argument unused during compilation: '-fuse-ld=musl-clang' [-Wunused-command-line-argument]
   clang: warning: argument unused during compilation: '-static-libgcc' [-Wunused-command-line-argument]
   clang: warning: argument unused during compilation: '-L-user-start' [-Wunused-command-line-argument]
   clang: warning: argument unused during compilation: '-L/home/ray/.local/stow/musl/lib' [-Wunused-command-line-argument]
   clang: warning: argument unused during compilation: '-L-user-end' [-Wunused-command-line-argument]

Adding a default -Wno-unused-command-line-argument may be a solution.