From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-1.6 required=5.0 tests=MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,URIBL_BLACK autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 9292 invoked from network); 1 Feb 2021 17:53:31 -0000 Received: from mother.openwall.net (195.42.179.200) by inbox.vuxu.org with ESMTPUTF8; 1 Feb 2021 17:53:31 -0000 Received: (qmail 24480 invoked by uid 550); 1 Feb 2021 17:53:27 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 24462 invoked from network); 1 Feb 2021 17:53:27 -0000 Date: Mon, 1 Feb 2021 12:53:14 -0500 From: Rich Felker To: Jiahao XU Cc: "musl@lists.openwall.com" Message-ID: <20210201175313.GU23432@brightrain.aerifal.cx> References: <20210130201227.GP23432@brightrain.aerifal.cx> <20210130233012.GR23432@brightrain.aerifal.cx> <20210131050121.GS23432@brightrain.aerifal.cx> <20210131210105.GT23432@brightrain.aerifal.cx> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Subject: Re: [musl] =?utf-8?B?Q2Fu4oCZ?= =?utf-8?Q?t?= build musl with lto=thin On Mon, Feb 01, 2021 at 12:31:47AM +0000, Jiahao XU wrote: > Interesting enough, I found —gc-section used along with -flto can cut the size of final hello_world executable from 48KB to 5KB. > > After investigating with bloaty, I found that —gc-section along with -flto is able to cut .text from 25.4 KiB to 3.04 KiB, and cut the .rodata from 19.5 KiB to 120 bytes. > ..data section however, seen an increase from 316 bytes to 372 bytes, but the VM size is cut from 252 to 244 bytes. > > > $ bloaty gc-section-a.out -- no-gc-section.a.out > > FILE SIZE VM SIZE > > -------------- -------------- > > +18% +56 -3.2% -8 .data > > [NEW] +6 [NEW] +6 [LOAD #2 [RX]] > > [DEL] -4 -66.7% -8 [LOAD #4 [RW]] > > -72.7% -8 [ = ] 0 [Unmapped] > > -32.0% -64 [ = ] 0 .comment > > -99.4% -19.4Ki -99.7% -19.4Ki .rodata > > -88.0% -22.3Ki -88.2% -22.3Ki .text > > -89.4% -41.8Ki -88.5% -41.8Ki TOTAL What is included in your hello world? Mine, static linked normally (no LTO) is around 4k of text and virtually no rodata. (This is with GCC; I'm not using clang.) If I compile with -fno-builtin so printf doesn't get transformed to puts, there's about 16k of text and 3k of rodata, but still nowhere near the ~20k you saw. Rich > ________________________________ > From: Jiahao XU > Sent: Monday, February 1, 2021 11:22:21 AM > To: Rich Felker > Cc: musl@lists.openwall.com > Subject: Re: [musl] Can’t build musl with lto=thin > > I have finally succeded to produce a statically linked hello_world executable. > > I modified the last line of musl-clang-lld to: > > exec $($cc -print-prog-name=ld.lld) -nostdlib “$@“ -l:libc.a —no-dynamic-linker > > and everything works fine now. > > Jiahao XU > ________________________________ > From: Rich Felker > Sent: Monday, February 1, 2021 8:01:06 AM > To: Jiahao XU > Cc: musl@lists.openwall.com > Subject: Re: [musl] Can’t build musl with lto=thin > > On Sun, Jan 31, 2021 at 05:32:45AM +0000, Jiahao XU wrote: > > I used `musl-clang -Oz -flto -s -fuse-ld=musl-clang-lld-static -Wl,—plugin-opt=O3,-O3 hello.c` to produce the executable. > > Where is -static? Normally it does *not* work to add -static just to > the ld command line. The compiler driver has to know that it's > requesting static linking because it will pass a different command > line to the linker based on that. > > > Content of `/usr/local/musl/bin/ld.musl-clang-lld-static` is same as > > the generated `ld.musl-clang`, except for the last line, which I > > modified it to: > > > > exec $($cc -print-prog-name=ld.lld) -nostdlib “$@“ -static -lc -dynamic-linker “$ldso” > > Try moving -static out from here (i.e. using the script unmodified > except for requesting ld.lld) and see if that works. Note that a > correctly linked executable will not have any INTERP in readelf -a > output, so as long as you see INTERP anywhere there you're doing > something wrong. > > Rich > > > > ________________________________ > > From: Rich Felker > > Sent: Sunday, January 31, 2021 4:01:22 PM > > To: Jiahao XU > > Cc: musl@lists.openwall.com > > Subject: Re: [musl] Can’t build musl with lto=thin > > > > On Sat, Jan 30, 2021 at 11:44:48PM +0000, Jiahao XU wrote: > > > (gdb) bt > > > > > > #0 0x00007ffff7ff5498 in decode_vec () from /lib/ld-musl-x86_64.so.1 > > > > > > #1 0x00007ffff7ff58cb in decode_dyn () from /lib/ld-musl-x86_64.so.1 > > > > > > #2 0x0000000000000000 in ?? () > > > > This is not a static-linked program. decode_dyn is part of the dynamic > > linker. It looks to me like you've created some sort of weird hybrid > > executable that's not valid. Can you show the command lines you used > > to produce it? > > > > Also please keep list on CC when replying. > > > > Rich > > > > > > > (gdb) info r > > > > > > rax 0x1 1 > > > > > > rbx 0x7ffff7ffe2d8 140737354130136 > > > > > > rcx 0x5000 20480 > > > > > > rdx 0x200238 2097720 > > > > > > rsi 0x7fffffffd8d0 140737488345296 > > > > > > rdi 0x0 0 > > > > > > rbp 0x7ffff7ffe2d8 0x7ffff7ffe2d8 <__dls3.app> > > > > > > rsp 0x7fffffffd8c8 0x7fffffffd8c8 > > > > > > r8 0x0 0 > > > > > > r9 0xfffffffffffff000 -4096 > > > > > > r10 0x800000 8388608 > > > > > > r11 0x200000 2097152 > > > > > > r12 0x7fffffffdcb8 140737488346296 > > > > > > r13 0x0 0 > > > > > > r14 0x7fffffffd8d0 140737488345296 > > > > > > r15 0x7fffffffdcb8 140737488346296 > > > > > > rip 0x7ffff7ff5498 0x7ffff7ff5498 > > > > > > eflags 0x10246 [ PF ZF IF RF ] > > > > > > cs 0x33 51 > > > > > > ss 0x2b 43 > > > > > > ds 0x0 0 > > > > > > es 0x0 0 > > > > > > fs 0x0 0 > > > > > > gs 0x0 0 > > > > > > > > > Disassembly of decode_dyn: > > > > > > > > > 0x00007ffff7ff58af <+0>: push %r14 > > > > > > 0x00007ffff7ff58b1 <+2>: push %rbx > > > > > > 0x00007ffff7ff58b2 <+3>: sub $0x108,%rsp > > > > > > 0x00007ffff7ff58b9 <+10>: mov %rdi,%rbx > > > > > > 0x00007ffff7ff58bc <+13>: mov 0x10(%rdi),%rdi > > > > > > 0x00007ffff7ff58c0 <+17>: mov %rsp,%r14 > > > > > > 0x00007ffff7ff58c3 <+20>: mov %r14,%rsi > > > > > > 0x00007ffff7ff58c6 <+23>: call 0x7ffff7ff5483 > > > > > > 0x00007ffff7ff58cb <+28>: mov (%rbx),%rax > > > > > > > > > Disassembly of decode_vec: > > > > > > > > > 0x00007ffff7ff5483 <+0>: xor %eax,%eax > > > > > > 0x00007ffff7ff5485 <+2>: cmp $0x20,%rax > > > > > > 0x00007ffff7ff5489 <+6>: je 0x7ffff7ff5495 > > > > > > 0x00007ffff7ff548b <+8>: andq $0x0,(%rsi,%rax,8) > > > > > > 0x00007ffff7ff5490 <+13>: inc %rax > > > > > > 0x00007ffff7ff5493 <+16>: jmp 0x7ffff7ff5485 > > > > > > 0x00007ffff7ff5495 <+18>: push $0x1 > > > > > > 0x00007ffff7ff5497 <+20>: pop %rax > > > > > > => 0x00007ffff7ff5498 <+21>: mov (%rdi),%rcx > > > > > > 0x00007ffff7ff549b <+24>: test %rcx,%rcx > > > > > > 0x00007ffff7ff549e <+27>: je 0x7ffff7ff54c3 > > > > > > 0x00007ffff7ff54a0 <+29>: lea -0x1(%rcx),%rdx > > > > > > 0x00007ffff7ff54a4 <+33>: cmp $0x1e,%rdx > > > > > > 0x00007ffff7ff54a8 <+37>: ja 0x7ffff7ff54bd > > > > > > 0x00007ffff7ff54aa <+39>: shlx %rcx,%rax,%rcx > > > > > > 0x00007ffff7ff54af <+44>: or %rcx,(%rsi) > > > > > > 0x00007ffff7ff54b2 <+47>: mov (%rdi),%rcx > > > > > > 0x00007ffff7ff54b5 <+50>: mov 0x8(%rdi),%rdx > > > > > > 0x00007ffff7ff54b9 <+54>: mov %rdx,(%rsi,%rcx,8) > > > > > > 0x00007ffff7ff54bd <+58>: add $0x10,%rdi > > > > > > 0x00007ffff7ff54c1 <+62>: jmp 0x7ffff7ff5498 > > > > > > 0x00007ffff7ff54c3 <+64>: ret > > > > > > > > > Jiahao XU > > > > > > Get Outlook for iOS > > > ________________________________ > > > From: Rich Felker > > > Sent: Sunday, January 31, 2021 10:30:12 AM > > > To: Jiahao XU > > > Cc: musl@lists.openwall.com > > > Subject: Re: [musl] Can’t build musl with lto=thin > > > > > > On Sat, Jan 30, 2021 at 11:04:32PM +0000, Jiahao XU wrote: > > > > > So something like (in config.mak): > > > > > > > > > > obj/ldso/dlstart.lo: CFLAGS_ALL += -fno-lto > > > > > > > > Thanks, with this I was able to build libc.so successfully with clang and created a 3.5 KB hello world program using clang and lld. > > > > > > > > However, I still wasn’t able to statically linked with libc. > > > > > > > > Once I added ‘-static’ to the compiler flags, the executable failed with ‘Segmentation fault (core dumped)’. > > > > > > It's libc.a, not libc.so, that will be involved in making a > > > static-linked binary. It's hard to know what's going wrong without > > > more information. Can you run under a debugger and provide a > > > backtrace, disassembly, and register dump for where the crash occurs? > > > > > > Rich > > > > > > > > > > ________________________________ > > > > From: Rich Felker > > > > Sent: Sunday, January 31, 2021 7:12:31 AM > > > > To: Jiahao XU > > > > Cc: musl@lists.openwall.com > > > > Subject: Re: [musl] Can’t build musl with lto=thin > > > > > > > > On Fri, Jan 29, 2021 at 12:19:42PM +0000, Jiahao XU wrote: > > > > > musl-1.2.2 compilation with clang-11 failed to build libc.so at the final linking stage: > > > > > > > > > > ld.lld: error: undefined hidden symbol: __dls2 > > > > > >>> referenced by ld-temp.o > > > > > >>> lto.tmp:(_dlstart_c) > > > > > >>> did you mean: __dls3 > > > > > >>> defined in: lto.tmp > > > > > > > > > > I am using CFLAGS=‘-march=native -mtune=native -Oz -flto > > > > > -fmerge-all-constants -fomit-frame-pointer’ and LDFLAGS=‘-flto > > > > ^^^^^^^^^^^^^^^^^^^^^ > > > > > > > > The -fmerge-all-constants option gives non-conforming language > > > > semantics and should not be used, but that's a separate issue. > > > > > > > > > -fuse-ld=lld -Wl,—plugin-opt=O3,-O3,—icf=safe’. > > > > > > > > > No configure option is supplied. > > > > > > > > Otherwise, it's a known issue that LTO misses references from asm > > > > (both top-level and in functions). I think dlstart.lo and a few other > > > > files should just be built with LTO disabled; any LTO-type > > > > optimization in code that runs at this stage is inherently invalid, > > > > anyway. So something like (in config.mak): > > > > > > > > obj/ldso/dlstart.lo: CFLAGS_ALL += -fno-lto > > > > > > > > Rich