Re: Eliminating preference for avoiding thread pointer? Cost on MIPS?

[Rich Felker <dalias@libc.org>]

On Mon, May 18, 2015 at 04:16:20PM -0400, Rich Felker wrote:
> On Mon, May 18, 2015 at 12:35:55PM -0700, Andre McCurdy wrote:
> > On Sat, May 16, 2015 at 9:48 AM, Rich Felker <dalias@libc.org> wrote:
> > > On Sat, May 16, 2015 at 09:33:20AM -0700, Isaac Dunham wrote:
> > >> On Fri, May 15, 2015 at 11:55:44PM -0400, Rich Felker wrote:
> > >> > Traditionally, musl has gone to pretty great lengths to avoid
> > >> > depending on the thread pointer. The original reason was that it was
> > >> > not always initialized, and when it was, the init was lazy. This
> > >> > resulted in a lot of cruft, where we would have lots of constructs of
> > >> > the form:
> > >> >
> > >> >     bar = some_predicate ? __pthread_self()->foo : global_foo
> > >> >
> > >> > or similar. Being that these predicates depend(ed) on globals, they
> > >> > were/are rather expensive in position-independent code on most archs.
> > >> > Now that the thread pointer is always initialized at startup (since
> > >> > 1.1.0) and assumed to have succeeded (since 1.1.9; musl now performs
> > >> > HCF if it fails), this seems to be an unnecessary cost. Not only does
> > >> > it cost cycles; it also has a complexity cost in terms of code to
> > >> > maintain the state of the predicates (e.g. the atomics for locale
> > >> > state) and in terms of libc-internal assumptions. So I'd like to just
> > >> > use the thread pointer directly wherever it makes sense, and take
> > >> > advantage of the fact that we have it.
> > >> >
> > >> > Unfortunately, there's one arch where thread-pointer access may be
> > >> > prohibitively costly: old MIPS. On the MIPS o32 ABI, the thread
> > >> > pointer is accessed via the "rdhwr $3,$29" instruction, which was only
> > >> > introduced in MIPS32rev2. MIPS-I, MIPS-II, and possibly the original
> > >> > MIPS32 lack it, and while Linux has a "fast path" trap to emulate it,
> > >> > I'm not clear on how "fast" it is.
> > >> >
> > >> > First, I'd like to find out how slow this trap is. If it's something
> > >> > like 150 cycles, that's ugly but probably acceptable. If it's more
> > >> > like 1000 cycles, that's a big problem. If anyone can run the attached
> > >> > test program on real MIPS-I or MIPS-II hardware and give me the
> > >> > results, please do! Compile it once with -O3 -DDO_RDHWR and once with
> > >> > just -O3 and send the (one-line) output of both to the list. It
> > >> > doesn't matter what libc your MIPS system is using -- any should be
> > >> > fine, but you might need to link with -lrt on glibc or uclibc.
> > >>
> > >> dd-wrt micro on a WRT54Gv8.0:
> > >> \u@\h:\w\$ cat /proc/version
> > >> Linux version 2.4.37 (root@dd-wrt) (gcc version 3.4.6 (OpenWrt-2.0)) #13303 Thu Aug 12 04:47:54 CEST 2010
> > 
> > It looks like rdhwr emulation was first added in linux 2.6.15, so
> > 2.4.37 is likely too old to run this test?
> 
> Ah yes, that would explain it. Linux 2.4 is pre-NPTL and really
> doesn't have any of the stuff needed to support threads. I could look
> and see if LinuxThreads might have had any practical way to do TLS for
> 2.4 though; this may give us a fallback for accessing TLS quickly on
> MIPS-I and MIPS-II.

And nope -- later LinuxThreads used rdhwr; earlier did their hideous
hack of using the high bits of the stack pointer as a thread id and
means of locating the thread structure.

Rich