Re: [0pf] musl/SH-FDPIC progress

[Rich Felker <dalias@libc.org>]

On Sun, Sep 13, 2015 at 06:45:49PM -0500, Rob Landley wrote:
> Jeff Dionne commented on your toolchain:
> 
> > The only issue is he seems to be exclusively targeting fdpic.  The
> > issue is you loose a few registers.   I don't know what gcc will do
> > performance wise in that case, we need to test.  Hopefully we don't
> > need a 'fall back' to bFLT, or something.
> >
> > A few % hit in an embedded system is a lot...
> 
> To which I don't know how to respond. Register starvation mostly seems
> to crop up on x86 (where they have horrible behind the scenes hardware
> hacks with register renaming and multiple register profiles and so on to
> get decent performance out of a lousy assembly design). But sh2 has 16
> general purpose registers, which is much less constained...
> 
> I'd say "get 'em both out and benchmark them" but the binflt toolchain
> I've got (cutting an aboriginal linux release as we speak by the way) is
> gcc 4.2.1+binutils 2.17, and yours is something like 8 years later so
> the whole code generation backend is basically redone. Not remotely
> apples to apples there.

With an aim of assessing what the old bFLT toolchain for sh2 is doing,
I dug through a lot more legacy uClinux docs and other nommu targets
in GCC. The bFLT format is documented as being able to do shared-text
and XIP, but of course for this to be possible, the compiler's codegen
needs to be compatible with data being loaded at an arbitrary
location. In both upstream gcc and the old sh2-uclinux toolchains I
have, the only targets which have the -msep-data option, which is
what's needed for shared-text/XIP, are Blackfin and m68k.

What the old sh2-uclinux bFLT toolchain is producing are "Fully
Relocated Binaries". These are (although they don't necessarily have
to be) non-PIC, and thus have the most efficient possible code in
terms of register availability, code density, and performance. On the
other hand they cannot share text or execute in place, and they have a
lot of relocations, which increases file size and startup time -- but
of course the biggest contributor to startup time is the need to
memcpy the whole file in kernelspace, since it's not shareable.

Aside from a few bytes of header data and slightly different entry
point code sequences, binaries of the above form are completely
equivalent to what you get with my PIE toolchain using the options
-static -fno-pic -pie, an unconventional but valid combination that
gives you an ET_DYN format binary that can be loaded at arbitrary
address but that uses TEXTRELs instead of PIC to achieve that.

If we did have -msep-data bFLT for sh (note: afaik no such gcc patch
exists, but it would not be hard to make one) the results would be
near-identical to my new fdpic toolchain with static linking. Since ld
will optimize any calls to func@PLT to a direct call to func (simply
by resolving the address that way), the GOT register never has to get
reloaded in a static-linked program except in the case of indirect
calls (via a function pointer) in which case there are 1-2 extra
instructions involved in making the call.

TL;DR:

I don't see any cases where bFLT could give a measurable advantage.

- The bFLT toolchain we have now is equivalent to my non-FDPIC PIE
  toolchain with -static -no-pic -pie.

- A hypothetical shared-text/XIP bFLT toolchain is essentially
  equivalent static linking to my FDPIC toolchain.

- Both of my toolchains are much more flexible (and free of hacks)
  than anything bFLT-based, support dynamic linking, etc.

Rich