Re: gcc'c crtstuff.c: a musl-related experience

[Rich Felker <dalias@libc.org>]

On Sun, Jun 15, 2014 at 09:41:31PM -0400, Rich Felker wrote:
> On Sun, Jun 15, 2014 at 05:36:37PM -0400, writeonce@midipix.org wrote:
> > The two files at stake are gcc/linux.h, and libgcc/crtstuff.c.  In
> > the former, a built-in macro named __gnu_linux__ is defined when
> > glibc is the toolchain's default libc (OPTION_GLIBC).  In the
> > latter, a macro named USE_PT_GNU_EH_FRAME is defined when
> > __gnu_linux__ is defined, which accordingly prevents
> > USE_EH_FRAME_REGISTRY from being defined shortly thereafter.  The
> > absence of this last macro results in a crtbegin.o that is
> > incompatible with musl, at least in some cases (a simple c++
> > application crashes after an exception has been thrown, which is how
> > I came to notice it).
> 
> I'm not an expert on gcc internals, but I'm doubtful of your
> explanation of the problem. From just the macro names,
> USE_PT_GNU_EH_FRAME is the one you want, and USE_EH_FRAME_REGISTRY
> sounds like the nasty old fallback that's incompatible with static
> linking of libgcc_eh.a.
> 
> However I'm confused what this has to do with crtbegin.o/crtend.o.
> Perhaps someone who's worked with toolchain stuff could comment.
> 
> The reason I'm bringing this up is that there might really be some bug
> in musl or in your toolchain setup, where switching to the old code is
> just covering up the bug.

I followed up on this and it's entirely a toolchain issue. Both
crtstuff and libgcc are involved:

crtstuff can be built in two different ways. One way is for supporting
the old EH_FRAME_REGISTRY stuff, whereby the global ctors/dtors for a
shared library or the main program must register/unregister their
unwind tables with libgcc_s (or libgcc_eh, but this has serious
problems in this configuration except when static linking the whole
program) and thus the registration code is required in crtstart.o and
crtend.o. The other way omits this code, assuming the modern system
based on dl_iterate_phdr will be used lib libgcc.

The libgcc unwind code can also be built in two ways. One has support
for the new PT_GNU_EH_FRAME system, which requires additional code in
libgcc_eh and support from libc. The other only suppors the old way,
where registration of unwind tables by individual modules is required.

In short, crtstuff can be built with or without support for the old
junk (the new way works regardless of how crtstuff was built, as long
as libgcc supports it), and libgcc can be built with or without
support for the new stuff (the old way work regardless of how libgcc
was built).

The current musl-cross patches fix libgcc to support the new way, but
they don't fix crtstuff to drop the old bloated code. If crtstuff is
fixed (or if you copy crtbegin.o and crtend.o from a toolchain built
for glibc), then static-linked programs using exceptions start
crashing. And here's why:

The default GCC specs have:

#define LINK_EH_SPEC "%{!static:--eh-frame-hdr} "

This inhibits the linker from emitting a PT_GNU_EH_FRAME program
header when static linking. I really have no idea how this is supposed
to work, but somehow it does work for glibc. (Perhaps their
dl_iterate_phdr is patching up a fake PT_GNU_EH_FRAME to pass back to
the application?)

So, here's what should be changed in musl-cross, in my opinion:

1. The crtstuff.c preprocessor checks for whether support for the old
   bloated/buggy registration system is needed should be fixed to
   align precisely with the checks used in libgcc unwind code to
   determine whether the latter uses dl_iterate_phdr.

2. Ether the specfile should be changed to provide --eh-frame-hdr to
   the linker even when static-linking, or we should figure out how
   glibc manages to get by without this program header and do the
   same (not sure if this would be on the musl side or the toolchain
   side).

Please note that these changes are not mandatory; at present,
everything works, but crtbegin.o and crtend.o are unnecessarily
bloated and this affects static binary size.

Rich