From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HTML_MESSAGE,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,URIBL_BLACK,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.4 Received: (qmail 3294 invoked from network); 1 Feb 2021 16:55:36 -0000 Received: from mother.openwall.net (195.42.179.200) by inbox.vuxu.org with ESMTPUTF8; 1 Feb 2021 16:55:36 -0000 Received: (qmail 31902 invoked by uid 550); 1 Feb 2021 16:55:33 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 31871 invoked from network); 1 Feb 2021 16:55:32 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=KEWMLzGtSwlST0Fpp5OHvQHwnNYEpCAu6cetSWmA5hU=; b=rjlezDTUYSBAj/RcWkg47E/xyNT8+2M+P3pW/wLbSKbZv+QcOQmuidA2/nddGd7z41 itlFtqFdZ5HWFaZ/nS3facGaVtzqko9bO39QhSmki5WRbz5IduztP0JaosLrAaz9Cp8r yiUWBRFcQ4kc0Ij6LR1Ba6braSzS3l4Pga1K6ghbqiM/A/eGKeIv+jiN4pkq4Fsw0hhR kJyNs3CLAZ3K593e+IV6gncCpQuDzvriGjb7WpHnDvetpOC4fQqWp0bgFcgZ87rO1qNg LKaMZoWGW6NO9O4DANzLybZRQYhxleLMDYlErLxSkcfrZfmqRD6ICcTeb35ck+RzLSUh xkjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=KEWMLzGtSwlST0Fpp5OHvQHwnNYEpCAu6cetSWmA5hU=; b=RA0WItLbSOVx1BhOSyeYI0W3YgGqGKEebX4Zu86kPOmZmQv8cUjyKqdFAR34U26lDg v2uTtpaPbk6t/eq5BD/fa4bf0xAUkPKtiDbWj4I+Fs7dDE+SC14NcQJpcqDrsHjXz59h SX0flwMGxiL6vMTyt/iCQjs4SalMlpfoLsi+qMgLgsYYEgK9cQRWzALKcOKz01Rh6FHC WsBs3HPGMlcxu+Ytt7vjkADl2oZWxYuGJ1tiXXIpyFHh/O7MNN9Raopfu+4GKyUBy15i kvY+4m4URejSeviyGsY7aZpsTYIg4n1/HYwqYWDza4OCLWnHStUrNG3vS1dF03B0BVZ3 0Rrg== X-Gm-Message-State: AOAM533XvYYIE+yLqkDg10uj6L3yoBV1jFNr/ZKRAP8frCN1SvkpXO9h jNz0Zr1u8eWUAXx5UUzXir+5bzxILxDkIhglzi85+UlX7FKeTw== X-Google-Smtp-Source: ABdhPJwRoLPeLsoLlP26Z6v/RCMgn2MxZ71qzxqLsvR1Dcj9rqEY9ni3W/vkkM7u6zzOrmerYDgg7lta9RJFWqpJ+g8= X-Received: by 2002:a05:6214:186b:: with SMTP id eh11mr16322632qvb.30.1612198520346; Mon, 01 Feb 2021 08:55:20 -0800 (PST) MIME-Version: 1.0 References: <20210130201227.GP23432@brightrain.aerifal.cx> <20210130233012.GR23432@brightrain.aerifal.cx> <20210131050121.GS23432@brightrain.aerifal.cx> <20210131210105.GT23432@brightrain.aerifal.cx> In-Reply-To: From: Nick Bray Date: Mon, 1 Feb 2021 08:55:08 -0800 Message-ID: To: musl@lists.openwall.com Cc: Rich Felker Content-Type: multipart/alternative; boundary="0000000000001d4d2a05ba4938b4" Subject: =?UTF-8?Q?Re=3A_=5Bmusl=5D_Can=E2=80=99t_build_musl_with_lto=3Dthin?= --0000000000001d4d2a05ba4938b4 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Small warning: When I statically linked in Musl and performed LTO on the entire executable with Clang, I ran into a few bugs. Specifically, when Clang could rewrite libc calls (memcmp =3D> bcmp, for example) the rewritten call no longer had any calling convention information in the IR. If LTO then tried to inline the call, Clang got confused and emitted a UDF instead of an actual call. I was working on aarch64. I believe the loss of calling convention is architecture independent, but I don't know what the other backends do with the malformed IR. I was hesitant to mail the list because I _think_ there's a patch in flight and it may have landed, but I have been too busy to follow up. I figured I should at least mention the problems that I have seen, inlining across the libc boundary. On Mon, Feb 1, 2021 at 5:28 AM Jiahao XU wrote: > Thank you very much for helping me, RIch. > > Jiahao XU > ------------------------------ > *From:* Jiahao XU > *Sent:* Monday, February 1, 2021 11:50:04 AM > *To:* Rich Felker > *Cc:* musl@lists.openwall.com > *Subject:* Re: [musl] Can=E2=80=99t build musl with lto=3Dthin > > Correction: The right number should be the executable is cut from 47KB to > 5.0KB. > > The previous numbers are retrieved using du -hs, which calculate disk > usage, not file size. > > Jiahao XU > ------------------------------ > *From:* Jiahao XU > *Sent:* Monday, February 1, 2021 11:44:49 AM > *To:* Rich Felker > *Cc:* musl@lists.openwall.com > *Subject:* Re: [musl] Can=E2=80=99t build musl with lto=3Dthin > > Correction: The executable is cut from 48KB to 8KB. > > I mixed up the numbers from du and ls. > > Jiahao XU > ------------------------------ > *From:* Jiahao XU > *Sent:* Monday, February 1, 2021 11:31:47 AM > *To:* Rich Felker > *Cc:* musl@lists.openwall.com > *Subject:* Re: [musl] Can=E2=80=99t build musl with lto=3Dthin > > Interesting enough, I found =E2=80=94gc-section used along with -flto can= cut the > size of final hello_world executable from 48KB to 5KB. > > After investigating with bloaty, I found that =E2=80=94gc-section along w= ith -flto > is able to cut .text from 25.4 KiB to 3.04 KiB, and cut the .rodata from > 19.5 KiB to 120 bytes. > .data section however, seen an increase from 316 bytes to 372 bytes, but > the VM size is cut from 252 to 244 bytes. > > $ bloaty gc-section-a.out -- no-gc-section.a.out > > FILE SIZE VM SIZE > > -------------- -------------- > > +18% +56 -3.2% -8 .data > > [NEW] +6 [NEW] +6 [LOAD #2 [RX]] > > [DEL] -4 -66.7% -8 [LOAD #4 [RW]] > > -72.7% -8 [ =3D ] 0 [Unmapped] > > -32.0% -64 [ =3D ] 0 .comment > > -99.4% -19.4Ki -99.7% -19.4Ki .rodata > > -88.0% -22.3Ki -88.2% -22.3Ki .text > > -89.4% -41.8Ki -88.5% -41.8Ki TOTAL > > > Jiahao XU > ------------------------------ > *From:* Jiahao XU > *Sent:* Monday, February 1, 2021 11:22:21 AM > *To:* Rich Felker > *Cc:* musl@lists.openwall.com > *Subject:* Re: [musl] Can=E2=80=99t build musl with lto=3Dthin > > I have finally succeded to produce a statically linked hello_world > executable. > > I modified the last line of musl-clang-lld to: > > exec $($cc -print-prog-name=3Dld.lld) -nostdlib =E2=80=9C$@=E2=80=9C = -l:libc.a > =E2=80=94no-dynamic-linker > > and everything works fine now. > > Jiahao XU > ------------------------------ > *From:* Rich Felker > *Sent:* Monday, February 1, 2021 8:01:06 AM > *To:* Jiahao XU > *Cc:* musl@lists.openwall.com > *Subject:* Re: [musl] Can=E2=80=99t build musl with lto=3Dthin > > On Sun, Jan 31, 2021 at 05:32:45AM +0000, Jiahao XU wrote: > > I used `musl-clang -Oz -flto -s -fuse-ld=3Dmusl-clang-lld-static > -Wl,=E2=80=94plugin-opt=3DO3,-O3 hello.c` to produce the executable. > > Where is -static? Normally it does *not* work to add -static just to > the ld command line. The compiler driver has to know that it's > requesting static linking because it will pass a different command > line to the linker based on that. > > > Content of `/usr/local/musl/bin/ld.musl-clang-lld-static` is same as > > the generated `ld.musl-clang`, except for the last line, which I > > modified it to: > > > > exec $($cc -print-prog-name=3Dld.lld) -nostdlib =E2=80=9C$@=E2=80= =9C -static -lc > -dynamic-linker =E2=80=9C$ldso=E2=80=9D > > Try moving -static out from here (i.e. using the script unmodified > except for requesting ld.lld) and see if that works. Note that a > correctly linked executable will not have any INTERP in readelf -a > output, so as long as you see INTERP anywhere there you're doing > something wrong. > > Rich > > > > ________________________________ > > From: Rich Felker > > Sent: Sunday, January 31, 2021 4:01:22 PM > > To: Jiahao XU > > Cc: musl@lists.openwall.com > > Subject: Re: [musl] Can=E2=80=99t build musl with lto=3Dthin > > > > On Sat, Jan 30, 2021 at 11:44:48PM +0000, Jiahao XU wrote: > > > (gdb) bt > > > > > > #0 0x00007ffff7ff5498 in decode_vec () from /lib/ld-musl-x86_64.so.1 > > > > > > #1 0x00007ffff7ff58cb in decode_dyn () from /lib/ld-musl-x86_64.so.1 > > > > > > #2 0x0000000000000000 in ?? () > > > > This is not a static-linked program. decode_dyn is part of the dynamic > > linker. It looks to me like you've created some sort of weird hybrid > > executable that's not valid. Can you show the command lines you used > > to produce it? > > > > Also please keep list on CC when replying. > > > > Rich > > > > > > > (gdb) info r > > > > > > rax 0x1 1 > > > > > > rbx 0x7ffff7ffe2d8 140737354130136 > > > > > > rcx 0x5000 20480 > > > > > > rdx 0x200238 2097720 > > > > > > rsi 0x7fffffffd8d0 140737488345296 > > > > > > rdi 0x0 0 > > > > > > rbp 0x7ffff7ffe2d8 0x7ffff7ffe2d8 <__dls3.app> > > > > > > rsp 0x7fffffffd8c8 0x7fffffffd8c8 > > > > > > r8 0x0 0 > > > > > > r9 0xfffffffffffff000 -4096 > > > > > > r10 0x800000 8388608 > > > > > > r11 0x200000 2097152 > > > > > > r12 0x7fffffffdcb8 140737488346296 > > > > > > r13 0x0 0 > > > > > > r14 0x7fffffffd8d0 140737488345296 > > > > > > r15 0x7fffffffdcb8 140737488346296 > > > > > > rip 0x7ffff7ff5498 0x7ffff7ff5498 > > > > > > eflags 0x10246 [ PF ZF IF RF ] > > > > > > cs 0x33 51 > > > > > > ss 0x2b 43 > > > > > > ds 0x0 0 > > > > > > es 0x0 0 > > > > > > fs 0x0 0 > > > > > > gs 0x0 0 > > > > > > > > > Disassembly of decode_dyn: > > > > > > > > > 0x00007ffff7ff58af <+0>: push %r14 > > > > > > 0x00007ffff7ff58b1 <+2>: push %rbx > > > > > > 0x00007ffff7ff58b2 <+3>: sub $0x108,%rsp > > > > > > 0x00007ffff7ff58b9 <+10>: mov %rdi,%rbx > > > > > > 0x00007ffff7ff58bc <+13>: mov 0x10(%rdi),%rdi > > > > > > 0x00007ffff7ff58c0 <+17>: mov %rsp,%r14 > > > > > > 0x00007ffff7ff58c3 <+20>: mov %r14,%rsi > > > > > > 0x00007ffff7ff58c6 <+23>: call 0x7ffff7ff5483 > > > > > > 0x00007ffff7ff58cb <+28>: mov (%rbx),%rax > > > > > > > > > Disassembly of decode_vec: > > > > > > > > > 0x00007ffff7ff5483 <+0>: xor %eax,%eax > > > > > > 0x00007ffff7ff5485 <+2>: cmp $0x20,%rax > > > > > > 0x00007ffff7ff5489 <+6>: je 0x7ffff7ff5495 > > > > > > 0x00007ffff7ff548b <+8>: andq $0x0,(%rsi,%rax,8) > > > > > > 0x00007ffff7ff5490 <+13>: inc %rax > > > > > > 0x00007ffff7ff5493 <+16>: jmp 0x7ffff7ff5485 > > > > > > 0x00007ffff7ff5495 <+18>: push $0x1 > > > > > > 0x00007ffff7ff5497 <+20>: pop %rax > > > > > > =3D> 0x00007ffff7ff5498 <+21>: mov (%rdi),%rcx > > > > > > 0x00007ffff7ff549b <+24>: test %rcx,%rcx > > > > > > 0x00007ffff7ff549e <+27>: je 0x7ffff7ff54c3 > > > > > > 0x00007ffff7ff54a0 <+29>: lea -0x1(%rcx),%rdx > > > > > > 0x00007ffff7ff54a4 <+33>: cmp $0x1e,%rdx > > > > > > 0x00007ffff7ff54a8 <+37>: ja 0x7ffff7ff54bd > > > > > > 0x00007ffff7ff54aa <+39>: shlx %rcx,%rax,%rcx > > > > > > 0x00007ffff7ff54af <+44>: or %rcx,(%rsi) > > > > > > 0x00007ffff7ff54b2 <+47>: mov (%rdi),%rcx > > > > > > 0x00007ffff7ff54b5 <+50>: mov 0x8(%rdi),%rdx > > > > > > 0x00007ffff7ff54b9 <+54>: mov %rdx,(%rsi,%rcx,8) > > > > > > 0x00007ffff7ff54bd <+58>: add $0x10,%rdi > > > > > > 0x00007ffff7ff54c1 <+62>: jmp 0x7ffff7ff5498 > > > > > > 0x00007ffff7ff54c3 <+64>: ret > > > > > > > > > Jiahao XU > > > > > > Get Outlook for iOS > > > ________________________________ > > > From: Rich Felker > > > Sent: Sunday, January 31, 2021 10:30:12 AM > > > To: Jiahao XU > > > Cc: musl@lists.openwall.com > > > Subject: Re: [musl] Can=E2=80=99t build musl with lto=3Dthin > > > > > > On Sat, Jan 30, 2021 at 11:04:32PM +0000, Jiahao XU wrote: > > > > > So something like (in config.mak): > > > > > > > > > > obj/ldso/dlstart.lo: CFLAGS_ALL +=3D -fno-lto > > > > > > > > Thanks, with this I was able to build libc.so successfully with > clang and created a 3.5 KB hello world program using clang and lld. > > > > > > > > However, I still wasn=E2=80=99t able to statically linked with libc= . > > > > > > > > Once I added =E2=80=98-static=E2=80=99 to the compiler flags, the e= xecutable failed > with =E2=80=98Segmentation fault (core dumped)=E2=80=99. > > > > > > It's libc.a, not libc.so, that will be involved in making a > > > static-linked binary. It's hard to know what's going wrong without > > > more information. Can you run under a debugger and provide a > > > backtrace, disassembly, and register dump for where the crash occurs? > > > > > > Rich > > > > > > > > > > ________________________________ > > > > From: Rich Felker > > > > Sent: Sunday, January 31, 2021 7:12:31 AM > > > > To: Jiahao XU > > > > Cc: musl@lists.openwall.com > > > > Subject: Re: [musl] Can=E2=80=99t build musl with lto=3Dthin > > > > > > > > On Fri, Jan 29, 2021 at 12:19:42PM +0000, Jiahao XU wrote: > > > > > musl-1.2.2 compilation with clang-11 failed to build libc.so at > the final linking stage: > > > > > > > > > > ld.lld: error: undefined hidden symbol: __dls2 > > > > > >>> referenced by ld-temp.o > > > > > >>> lto.tmp:(_dlstart_c) > > > > > >>> did you mean: __dls3 > > > > > >>> defined in: lto.tmp > > > > > > > > > > I am using CFLAGS=3D=E2=80=98-march=3Dnative -mtune=3Dnative -Oz = -flto > > > > > -fmerge-all-constants -fomit-frame-pointer=E2=80=99 and LDFLAGS= =3D=E2=80=98-flto > > > > ^^^^^^^^^^^^^^^^^^^^^ > > > > > > > > The -fmerge-all-constants option gives non-conforming language > > > > semantics and should not be used, but that's a separate issue. > > > > > > > > > -fuse-ld=3Dlld -Wl,=E2=80=94plugin-opt=3DO3,-O3,=E2=80=94icf=3Dsa= fe=E2=80=99. > > > > > > > > > No configure option is supplied. > > > > > > > > Otherwise, it's a known issue that LTO misses references from asm > > > > (both top-level and in functions). I think dlstart.lo and a few oth= er > > > > files should just be built with LTO disabled; any LTO-type > > > > optimization in code that runs at this stage is inherently invalid, > > > > anyway. So something like (in config.mak): > > > > > > > > obj/ldso/dlstart.lo: CFLAGS_ALL +=3D -fno-lto > > > > > > > > Rich > --0000000000001d4d2a05ba4938b4 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Small warning:

When I statically linked= in Musl and performed LTO on the entire executable with Clang, I ran into = a few bugs.=C2=A0 Specifically, when Clang could rewrite libc calls (memcmp= =3D> bcmp, for example) the rewritten call no longer had any calling co= nvention information in the IR.=C2=A0 If LTO then tried to inline the call,= Clang got confused and emitted=C2=A0a UDF instead of an actual call.=C2=A0= I was working on aarch64. I believe=C2=A0 the loss of calling convention i= s architecture=C2=A0independent, but I don't know what the other backen= ds do with the malformed IR.

I was hesitant to mai= l the list because I _think_ there's a patch in flight and it may have = landed, but I have been too busy to follow up.=C2=A0 I figured I should at = least mention the problems that I have seen, inlining across the libc bound= ary.

On Mon, Feb 1, 2021 at 5:28 AM Jiahao XU <Jiahao_XU@outlook.com> wrote:
Thank you very much for helping me, RIch.

Jiahao XU
From= : Jiahao XU <Jiahao_XU@outlook.com>
Sent: Monday, February 1, 2021 11:50:04 AM
To: Rich Felker <dalias@libc.org>
Cc: mus= l@lists.openwall.com <musl@lists.openwall.com>
Subject: Re: [musl] Can=E2=80=99t build musl with lto=3Dthin
=C2=A0
Correction: The right number should be the executable is cut from 47KB= to 5.0KB.

The previous numbers are retrieved using du -hs, which cal= culate disk usage, not file size.

Jiahao XU
Fr= om: Jiahao XU <Jiahao_XU@outlook.com>
Sent: Monday, February 1, 2021 11:44:49 AM
To: Rich Felker <dalias@libc.org>
Cc: mus= l@lists.openwall.com <musl@lists.openwall.com>
Subject: Re: [musl] Can=E2=80=99t build musl with lto=3Dthin
=C2=A0
Correction: The executable is cut from 48KB to 8KB.

I mixed up the numbers from du and ls.

Jiahao XU
= From: Jiahao XU <Jiahao_XU@outlook.com>
Sent: Monday, February 1, 2021 11:31:47 AM
To: Rich Felker <dalias@libc.org>
Cc: mus= l@lists.openwall.com <musl@lists.openwall.com>
Subject: Re: [musl] Can=E2=80=99t build musl with lto=3Dthin
=C2=A0
Interesting enough, I found =E2=80=94gc-section used along with -flto ca= n cut the size of final hello_world executable from 48KB to 5KB.

After investigating with bloaty, I found that =E2=80=94gc-section along = with -flto is able to cut .text from 25.4 KiB to 3.04 KiB, and cut the .rod= ata from 19.5 KiB to 120 bytes.
.data section however, seen an increase from 316 bytes to 372 bytes, but= the VM size is cut from 252 to 244 bytes.

$ bloaty gc-section-a.out --= no-gc-section.a.out =C2=A0

=C2=A0 =C2=A0 FILE SIZE=C2=A0 =C2=A0 =C2=A0 =C2=A0 VM SIZE =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0

=C2=A0---------= -----=C2=A0 --------------=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0

=C2=A0=C2=A0 +18% =C2=A0 =C2=A0 +56=C2=A0 -3.2%= =C2=A0 =C2=A0 =C2=A0 -8=C2=A0 =C2=A0 .data=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0

=C2=A0 [NEW]=C2=A0 =C2=A0 =C2=A0 +6=C2=A0 [NEW]=C2=A0 =C2=A0 =C2=A0 +6=C2=A0 =C2=A0 [LOAD #2 [RX]] =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0

=C2=A0 [DEL]=C2=A0 =C2=A0 =C2=A0 -4 -66.7%=C2=A0 =C2=A0 = =C2=A0 -8=C2=A0 =C2=A0 [LOAD #4 [RW]] =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0

=C2=A0-72.7%=C2=A0 =C2=A0 =C2=A0 -8=C2=A0 [ =3D ] =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 [Unmapped] =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0

=C2=A0-32.0% =C2=A0 =C2=A0 -64=C2=A0 [ =3D ] =C2=A0 =C2= =A0 =C2=A0 0=C2=A0 =C2=A0 .comment =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0

=C2=A0-99.4% -1= 9.4Ki -99.7% -19.4Ki=C2=A0 =C2=A0 .rodata=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0

=C2=A0-88.0% -2= 2.3Ki -88.2% -22.3Ki=C2=A0 =C2=A0 .text=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0

=C2=A0-89.4% -4= 1.8Ki -88.5% -41.8Ki=C2=A0 =C2=A0 TOTAL



Jiahao XU
<= b>From: Jiahao XU <Jiahao_XU@outlook.com>
Sent: Monday, February 1, 2021 11:22:21 AM
To: Rich Felker <dalias@libc.org>
Cc: mus= l@lists.openwall.com <musl@lists.openwall.com>
Subject: Re: [musl] Can=E2=80=99t build musl with lto=3Dthin
=C2=A0
I have finally succeded to produce a statically linked hello_world execu= table.

I modified the last line of musl-clang-lld to:
=C2=A0 =C2=A0=C2=A0
=C2=A0 =C2=A0 exec $($cc -print-prog-name=3Dld.lld) -nostdlib =E2=80=9C$= @=E2=80=9C -l:libc.a =E2=80=94no-dynamic-linker

and everything works fine now.

Jiahao XU
From: Rich Felker <dalias@libc.org>
Sent: Monday, February 1, 2021 8:01:06 AM
To: Jiahao XU <Jiahao_XU@outlook.com>
Cc: mus= l@lists.openwall.com <musl@lists.openwall.com>
Subject: Re: [musl] Can=E2=80=99t build musl with lto=3Dthin
=C2=A0
On Sun, Jan 31, 2021 at 05:32:45AM +0000, Jiahao XU wrote:
> I used `musl-clang -Oz -flto -s -fuse-ld=3Dmusl-clang-lld-static -Wl,= =E2=80=94plugin-opt=3DO3,-O3 hello.c` to produce the executable.

Where is -static? Normally it does *not* work to add -static just to
the ld command line. The compiler driver has to know that it's
requesting static linking because it will pass a different command
line to the linker based on that.

> Content of `/usr/local/musl/bin/ld.musl-clang-lld-static` is same as > the generated `ld.musl-clang`, except for the last line, which I
> modified it to:
>
>=C2=A0=C2=A0=C2=A0=C2=A0 exec $($cc -print-prog-name=3Dld.lld) -nostdli= b =E2=80=9C$@=E2=80=9C -static -lc -dynamic-linker =E2=80=9C$ldso=E2=80=9D<= br>
Try moving -static out from here (i.e. using the script unmodified
except for requesting ld.lld) and see if that works. Note that a
correctly linked executable will not have any INTERP in readelf -a
output, so as long as you see INTERP anywhere there you're doing
something wrong.

Rich


> ________________________________
> From: Rich Felker <dalias@libc.org>
> Sent: Sunday, January 31, 2021 4:01:22 PM
> To: Jiahao XU <Jiahao_XU@outlook.com>
> Cc: musl@= lists.openwall.com <musl@lists.openwall.com>
> Subject: Re: [musl] Can=E2=80=99t build musl with lto=3Dthin
>
> On Sat, Jan 30, 2021 at 11:44:48PM +0000, Jiahao XU wrote:
> > (gdb) bt
> >
> > #0=C2=A0 0x00007ffff7ff5498 in decode_vec () from /lib/ld-musl-x8= 6_64.so.1
> >
> > #1=C2=A0 0x00007ffff7ff58cb in decode_dyn () from /lib/ld-musl-x8= 6_64.so.1
> >
> > #2=C2=A0 0x0000000000000000 in ?? ()
>
> This is not a static-linked program. decode_dyn is part of the dynamic=
> linker. It looks to me like you've created some sort of weird hybr= id
> executable that's not valid. Can you show the command lines you us= ed
> to produce it?
>
> Also please keep list on CC when replying.
>
> Rich
>
>
> > (gdb) info r
> >
> > rax=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 0x1=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 1
> >
> > rbx=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 0x7ffff7ffe2d8=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 140737354130136
> >
> > rcx=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 0x5000=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 20480
> >
> > rdx=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 0x200238=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 2097720
> >
> > rsi=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 0x7fffffffd8d0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 140737488345296
> >
> > rdi=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 0x0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0
> >
> > rbp=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 0x7ffff7ffe2d8=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0x7ffff7ffe2d8 <__dl= s3.app>
> >
> > rsp=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 0x7fffffffd8c8=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0x7fffffffd8c8
> >
> > r8=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 0x0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0
> >
> > r9=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 0xfffffffffffff000=C2=A0 -4096
> >
> > r10=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 0x800000=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 8388608
> >
> > r11=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 0x200000=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 2097152
> >
> > r12=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 0x7fffffffdcb8=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 140737488346296
> >
> > r13=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 0x0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0
> >
> > r14=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 0x7fffffffd8d0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 140737488345296
> >
> > r15=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 0x7fffffffdcb8=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 140737488346296
> >
> > rip=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 0x7ffff7ff5498=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0x7ffff7ff5498 <deco= de_vec+21>
> >
> > eflags=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0x10246=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 [ PF = ZF IF RF ]
> >
> > cs=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 0x33=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 51
> >
> > ss=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 0x2b=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 43
> >
> > ds=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 0x0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0
> >
> > es=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 0x0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0
> >
> > fs=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 0x0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0
> >
> > gs=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 0x0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0
> >
> >
> > Disassembly of decode_dyn:
> >
> >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff58af <+0>:=C2=A0=C2=A0=C2= =A0=C2=A0 push=C2=A0=C2=A0 %r14
> >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff58b1 <+2>:=C2=A0=C2=A0=C2= =A0=C2=A0 push=C2=A0=C2=A0 %rbx
> >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff58b2 <+3>:=C2=A0=C2=A0=C2= =A0=C2=A0 sub=C2=A0=C2=A0=C2=A0 $0x108,%rsp
> >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff58b9 <+10>:=C2=A0=C2=A0=C2= =A0 mov=C2=A0=C2=A0=C2=A0 %rdi,%rbx
> >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff58bc <+13>:=C2=A0=C2=A0=C2= =A0 mov=C2=A0=C2=A0=C2=A0 0x10(%rdi),%rdi
> >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff58c0 <+17>:=C2=A0=C2=A0=C2= =A0 mov=C2=A0=C2=A0=C2=A0 %rsp,%r14
> >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff58c3 <+20>:=C2=A0=C2=A0=C2= =A0 mov=C2=A0=C2=A0=C2=A0 %r14,%rsi
> >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff58c6 <+23>:=C2=A0=C2=A0=C2= =A0 call=C2=A0=C2=A0 0x7ffff7ff5483 <decode_vec>
> >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff58cb <+28>:=C2=A0=C2=A0=C2= =A0 mov=C2=A0=C2=A0=C2=A0 (%rbx),%rax
> >
> >
> > Disassembly of decode_vec:
> >
> >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff5483 <+0>:=C2=A0=C2=A0=C2= =A0=C2=A0 xor=C2=A0=C2=A0=C2=A0 %eax,%eax
> >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff5485 <+2>:=C2=A0=C2=A0=C2= =A0=C2=A0 cmp=C2=A0=C2=A0=C2=A0 $0x20,%rax
> >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff5489 <+6>:=C2=A0=C2=A0=C2= =A0=C2=A0 je=C2=A0=C2=A0=C2=A0=C2=A0 0x7ffff7ff5495 <decode_vec+18> > >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff548b <+8>:=C2=A0=C2=A0=C2= =A0=C2=A0 andq=C2=A0=C2=A0 $0x0,(%rsi,%rax,8)
> >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff5490 <+13>:=C2=A0=C2=A0=C2= =A0 inc=C2=A0=C2=A0=C2=A0 %rax
> >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff5493 <+16>:=C2=A0=C2=A0=C2= =A0 jmp=C2=A0=C2=A0=C2=A0 0x7ffff7ff5485 <decode_vec+2>
> >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff5495 <+18>:=C2=A0=C2=A0=C2= =A0 push=C2=A0=C2=A0 $0x1
> >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff5497 <+20>:=C2=A0=C2=A0=C2= =A0 pop=C2=A0=C2=A0=C2=A0 %rax
> >
> > =3D> 0x00007ffff7ff5498 <+21>:=C2=A0=C2=A0=C2=A0 mov=C2= =A0=C2=A0=C2=A0 (%rdi),%rcx
> >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff549b <+24>:=C2=A0=C2=A0=C2= =A0 test=C2=A0=C2=A0 %rcx,%rcx
> >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff549e <+27>:=C2=A0=C2=A0=C2= =A0 je=C2=A0=C2=A0=C2=A0=C2=A0 0x7ffff7ff54c3 <decode_vec+64>
> >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff54a0 <+29>:=C2=A0=C2=A0=C2= =A0 lea=C2=A0=C2=A0=C2=A0 -0x1(%rcx),%rdx
> >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff54a4 <+33>:=C2=A0=C2=A0=C2= =A0 cmp=C2=A0=C2=A0=C2=A0 $0x1e,%rdx
> >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff54a8 <+37>:=C2=A0=C2=A0=C2= =A0 ja=C2=A0=C2=A0=C2=A0=C2=A0 0x7ffff7ff54bd <decode_vec+58>
> >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff54aa <+39>:=C2=A0=C2=A0=C2= =A0 shlx=C2=A0=C2=A0 %rcx,%rax,%rcx
> >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff54af <+44>:=C2=A0=C2=A0=C2= =A0 or=C2=A0=C2=A0=C2=A0=C2=A0 %rcx,(%rsi)
> >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff54b2 <+47>:=C2=A0=C2=A0=C2= =A0 mov=C2=A0=C2=A0=C2=A0 (%rdi),%rcx
> >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff54b5 <+50>:=C2=A0=C2=A0=C2= =A0 mov=C2=A0=C2=A0=C2=A0 0x8(%rdi),%rdx
> >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff54b9 <+54>:=C2=A0=C2=A0=C2= =A0 mov=C2=A0=C2=A0=C2=A0 %rdx,(%rsi,%rcx,8)
> >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff54bd <+58>:=C2=A0=C2=A0=C2= =A0 add=C2=A0=C2=A0=C2=A0 $0x10,%rdi
> >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff54c1 <+62>:=C2=A0=C2=A0=C2= =A0 jmp=C2=A0=C2=A0=C2=A0 0x7ffff7ff5498 <decode_vec+21>
> >
> >=C2=A0=C2=A0=C2=A0 0x00007ffff7ff54c3 <+64>:=C2=A0=C2=A0=C2= =A0 ret
> >
> >
> > Jiahao XU
> >
> > Get Outlook for iOS<https://aka.ms/o0ukef>
> > ________________________________
> > From: Rich Felker <dalias@libc.org>
> > Sent: Sunday, January 31, 2021 10:30:12 AM
> > To: Jiahao XU <Jiahao_XU@outlook.com>
> > Cc: = musl@lists.openwall.com <musl@lists.openwall.com>
> > Subject: Re: [musl] Can=E2=80=99t build musl with lto=3Dthin
> >
> > On Sat, Jan 30, 2021 at 11:04:32PM +0000, Jiahao XU wrote:
> > > > So something like (in config.mak):
> > > >
> > > > obj/ldso/dlstart.lo: CFLAGS_ALL +=3D -fno-lto
> > >
> > > Thanks, with this I was able to build libc.so successfully w= ith clang and created a 3.5 KB hello world program using clang and lld.
> > >
> > > However, I still wasn=E2=80=99t able to statically linked wi= th libc.
> > >
> > > Once I added =E2=80=98-static=E2=80=99 to the compiler flags= , the executable failed with =E2=80=98Segmentation fault (core dumped)=E2= =80=99.
> >
> > It's libc.a, not libc.so, that will be involved in making a > > static-linked binary. It's hard to know what's going wron= g without
> > more information. Can you run under a debugger and provide a
> > backtrace, disassembly, and register dump for where the crash occ= urs?
> >
> > Rich
> >
> >
> > > ________________________________
> > > From: Rich Felker <dalias@libc.org>
> > > Sent: Sunday, January 31, 2021 7:12:31 AM
> > > To: Jiahao XU <Jiahao_XU@outlook.com>
> > > Cc: musl@lists.openwall.com <musl@lists.openwall.com>
> > > Subject: Re: [musl] Can=E2=80=99t build musl with lto=3Dthin=
> > >
> > > On Fri, Jan 29, 2021 at 12:19:42PM +0000, Jiahao XU wrote: > > > > musl-1.2.2 compilation with clang-11 failed to build li= bc.so at the final linking stage:
> > > >
> > > >=C2=A0=C2=A0=C2=A0=C2=A0 ld.lld: error: undefined hidden= symbol: __dls2
> > > >=C2=A0=C2=A0=C2=A0=C2=A0 >>> referenced by ld-t= emp.o
> > > >=C2=A0=C2=A0=C2=A0=C2=A0 >>>=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 lto.tmp:(_dlstart= _c)
> > > >=C2=A0=C2=A0=C2=A0=C2=A0 >>> did you mean: __dl= s3
> > > >=C2=A0=C2=A0=C2=A0=C2=A0 >>> defined in: lto.tm= p
> > > >
> > > > I am using CFLAGS=3D=E2=80=98-march=3Dnative -mtune=3Dn= ative -Oz -flto
> > > > -fmerge-all-constants -fomit-frame-pointer=E2=80=99 and= LDFLAGS=3D=E2=80=98-flto
> > >=C2=A0=C2=A0 ^^^^^^^^^^^^^^^^^^^^^
> > >
> > > The -fmerge-all-constants option gives non-conforming langua= ge
> > > semantics and should not be used, but that's a separate = issue.
> > >
> > > > -fuse-ld=3Dlld -Wl,=E2=80=94plugin-opt=3DO3,-O3,=E2=80= =94icf=3Dsafe=E2=80=99.
> > >
> > > > No configure option is supplied.
> > >
> > > Otherwise, it's a known issue that LTO misses references= from asm
> > > (both top-level and in functions). I think dlstart.lo and a = few other
> > > files should just be built with LTO disabled; any LTO-type > > > optimization in code that runs at this stage is inherently i= nvalid,
> > > anyway. So something like (in config.mak):
> > >
> > > obj/ldso/dlstart.lo: CFLAGS_ALL +=3D -fno-lto
> > >
> > > Rich
--0000000000001d4d2a05ba4938b4--