Re: building musl libc.so with gcc -flto

[Rich Felker <dalias@libc.org>]

On Wed, Apr 22, 2015 at 10:34:40PM -0700, Andre McCurdy wrote:
> On Wed, Apr 22, 2015 at 7:23 PM, Rich Felker <dalias@libc.org> wrote:
> > On Wed, Apr 22, 2015 at 03:48:52PM -0700, Andre McCurdy wrote:
> >> Hi all,
> >>
> >> Below are some observations from building musl libc.so with gcc's -flto
> >> (link time optimization) option.
> >
> > Interesting!
> >
> >> 1) With today's master (afbcac68), adding -flto to CFLAGS causes the
> >> build to fail:
> >>
> >>  | `_dlstart_c' referenced in section `.text' of /tmp/cc8ceNIy.ltrans0.ltrans.o: defined in discarded section `.text' of src/ldso/dlstart.lo (symbol from plugin)
> >>  | collect2: error: ld returned 1 exit status
> >>  | make: *** [lib/libc.so] Error 1
> >>
> >> Reverting f1faa0e1 (make _dlstart_c function use hidden visibility)
> >> seems to be a workaround.
> >
> > I think the problem is that LTO is garbage collecting "unused" symbols
> > before it gets to the step of linking with asm for which there is no
> > IR code, thereby losing anything that's only referenced from asm. A
> > better workaround might be to define _dlstart_c with a different name
> > as a non-hidden function (e.g. call it __dls1) and then make
> > _dlstart_c a hidden alias for it via:
> >
> > __attribute__((__visibility__("hidden")))
> > void _dlstart_c(size_t *, size_t *);
> >
> > weak_alias(__dls1, _dlstart_c);
> >
> > If you get a chance to try that, let me know if it works.
> 
> That change does fix the build, but the resulting binary fails to run:
> 
> $ gdb ./lib/libc.so
> ....
> (gdb) run
> Starting program: /home/andre/.../lib/libc.so
> 
> Program received signal SIGILL, Illegal instruction.
> 0x56572ab8 in _dlstart ()
> (gdb) disassemble
> Dump of assembler code for function _dlstart:
>    0x56572aa0 <+0>:    xor    %ebp,%ebp
>    0x56572aa2 <+2>:    mov    %esp,%eax
>    0x56572aa4 <+4>:    and    $0xfffffff0,%esp
>    0x56572aa7 <+7>:    push   %eax
>    0x56572aa8 <+8>:    push   %eax
>    0x56572aa9 <+9>:    call   0x56572aae <_dlstart+14>
>    0x56572aae <+14>:    addl   $0x7864a,(%esp)
>    0x56572ab5 <+21>:    push   %eax
>    0x56572ab6 <+22>:    call   0x56572ab7 <_dlstart+23>
>    0x56572abb <+27>:    nop
>    0x56572abc <+28>:    lea    0x0(%esi,%eiz,1),%esi
> End of assembler dump.
> (gdb)

OK, it looks like the _dlstart_c symbol got removed before linking the
asm. What about selectively compiling this file with -fno-lto via
something like this in config.mak:

src/ldso/dlstart.lo: CFLAGS += -fno-lto

> > Also seems rather like what I would expect. Any idea if performance is
> > significantly better? It's not very comprehensive but you could try
> > libc-bench.
> 
> I modified libc-bench so that it loops though everything in main() ten
> times and then ran the same libc-bench binary with each version of
> libc.so, sending output to /dev/null.
> 
> The -O3 -flto build seems to be consistently very slightly *slower*
> than the non -flto version...

That makes the whole thing somewhat less interesting. LTO is probably
more interesting for static libc.

Rich