thread local storage

thread local storage

[John Spencer]

2 out of 14 sabotage followers wanted to use a musl-based system as a 
platform for luajit (and then were never seen again).
so i looked into adding it...

luajit builds without problems on musl, but then crashes due to a lack 
of TLS.

is it planned to add this feature ? iirc it wasn't mentioned on the 
latest roadmap...

Re: thread local storage

[Gregor Richards]

On 07/16/2012 03:02 PM, John Spencer wrote:
> 2 out of 14 sabotage followers wanted to use a musl-based system as a 
> platform for luajit (and then were never seen again).
> so i looked into adding it...
>
> luajit builds without problems on musl, but then crashes due to a lack 
> of TLS.
>
> is it planned to add this feature ? iirc it wasn't mentioned on the 
> latest roadmap...
>
>
>
With a quick perusal of the LuaJIT source, this is the only instance of 
TLS I see:

#if LJ_UNWIND_EXT
#if LJ_TARGET_OSX || defined(__OpenBSD__)
/* Sorry, no thread safety for OSX. Complain to Apple, not me. */
static _Unwind_Exception static_uex;
#else
static __thread _Unwind_Exception static_uex;
#endif

Convince it to use the same exception as OS X and OpenBSD and you should 
be in business.

"Complain to Apple, not me" is an odd way to say "I am too lazy to use 
standard interfaces." TLS has come up on the IRC chat, but I don't 
recall what the conclusions were regarding future implementation or not.

With valediction,
  - Gregor Richards

Re: thread local storage

[Rich Felker]

On Mon, Jul 16, 2012 at 02:57:22PM -0400, Gregor Richards wrote:
> On 07/16/2012 03:02 PM, John Spencer wrote:
> >2 out of 14 sabotage followers wanted to use a musl-based system
> >as a platform for luajit (and then were never seen again).
> >so i looked into adding it...
> >
> >luajit builds without problems on musl, but then crashes due to a
> >lack of TLS.
> >
> >is it planned to add this feature ? iirc it wasn't mentioned on
> >the latest roadmap...
> >
> >
> >
> With a quick perusal of the LuaJIT source, this is the only instance
> of TLS I see:
> 
> #if LJ_UNWIND_EXT
> #if LJ_TARGET_OSX || defined(__OpenBSD__)
> /* Sorry, no thread safety for OSX. Complain to Apple, not me. */
> static _Unwind_Exception static_uex;
> #else
> static __thread _Unwind_Exception static_uex;
> #endif
> 
> Convince it to use the same exception as OS X and OpenBSD and you
> should be in business.

This is broken and non-thread-safe. Not a good idea. Instead try:

#define static_uex (*(_Unwind_Exception *)pthread_getspecific(static_uex_key))

where static_uex_key is a pthread_key_t initialized earlier with:
pthread_key_create(&static_uex_key, 0);

And where the thread-specific value of the key is set in thread
startup as:
_Unwind_Exception static_uex_local;
pthread_setspecific(static_uex_key, &static_uex_local);

The simplicity and generality of this solution is why __thread was
just a stupid idea to begin with...

Rich

Re: thread local storage

[John Spencer]

On 07/16/2012 10:09 PM, Rich Felker wrote:
>
> This is broken and non-thread-safe. Not a good idea. Instead try:

thanks, very interesting.

do you mind if i forward this to the luajit ml ?
looks as if this might fix the problem on OSX/OpenBSD as well.

Re: thread local storage

[Rich Felker]

On Mon, Jul 16, 2012 at 11:30:01PM +0200, John Spencer wrote:
> On 07/16/2012 10:09 PM, Rich Felker wrote:
> >
> >This is broken and non-thread-safe. Not a good idea. Instead try:
> 
> thanks, very interesting.
> 
> do you mind if i forward this to the luajit ml ?
> looks as if this might fix the problem on OSX/OpenBSD as well.

Please do. This is the correct, portable fix. Using __thread or
_Thread_local might be prefered to avoid the init/cleanup need when
possible, but the POSIX fallback should be used rather than falling
back to code that will horribly crash or even yield privilege
elevation under multi-threaded use.

Rich

Re: thread local storage

[Justin Cormack]

[-- Attachment #1: Type: text/plain, Size: 358 bytes --]

On Jul 16, 2012 7:53 PM, "John Spencer" <maillist-musl@barfooze.de> wrote:
>
> 2 out of 14 sabotage followers wanted to use a musl-based system as a
platform for luajit (and then were never seen again).
> so i looked into adding it...
>

I am still around interested in luajit on musl  just had no time since
getting the same stack trace you got....

Justin

[-- Attachment #2: Type: text/html, Size: 486 bytes --]

Re: thread local storage

[Rich Felker]

On Mon, Jul 16, 2012 at 09:02:49PM +0200, John Spencer wrote:
> 2 out of 14 sabotage followers wanted to use a musl-based system as
> a platform for luajit (and then were never seen again).
> so i looked into adding it...
> 
> luajit builds without problems on musl, but then crashes due to a
> lack of TLS.

Then the gcc was built wrong. --disable-tls should be passed when
building gcc so that attempts to use __thread generate a compile error
(to be detected in configure).

> is it planned to add this feature ? iirc it wasn't mentioned on the
> latest roadmap...

Yes, but it's one of the hardest remainind things, for a couple
reasons...

1. It involves introducing ugly arch-specific code into lots of things
that should be arch-agnostic, because the original implementors did
things this way and encoding the knowledge into gcc. This might be
possible to bypass if we drop support for the static/local TLS model
or whatever they call it and force all modules, even the main program
and static binaries, to access TLS through a function call like shared
libs have to do. Actually I'd really like to do it this way if
possible since it would allow us to change things in the future
instead of locking in ABI stuff. At the very least it probably
requires a build option to GCC and/or a special -f flag in the spec
file or GCC default specs to make this possible.

2. Dynamic linker needs to be updtaed to handle all sorts of
TLS-related relocations which I don't yet understand, so I have to
read up on them...

3. Existing implementations of TLS are just _wrong_ and crashingly so.
When a new shared lib is loaded with dlopen, it may require additional
TLS memory, and this memory must be allocated for each thread
currently running. The way glibc/NPTL works is to keep a "generation
counter" and dynamically allocate space for each thread's new TLS on
the first accerss if the ldso generation counter has increased (or
something like that). If allocation fails, there's nothing you can do
but abort the program. Naturally this is unacceptable.

Fixing it requires temporarily placing a lock on new thread creation
and allocating the new TLS space for all currently-running threads as
part of the dlopen operation, then either assigning it to them before
dlopen returns, or placing it in a reserved pool whereby they can get
it on the first access attempt. Solving this problem correctly,
without race conditions or deadlocks, etc. is rather non-trivial.

Rich

Re: thread local storage

[John Spencer]

On 07/16/2012 10:03 PM, Rich Felker wrote:
> On Mon, Jul 16, 2012 at 09:02:49PM +0200, John Spencer wrote:
>> luajit builds without problems on musl, but then crashes due to a
>> lack of TLS.
> Then the gcc was built wrong. --disable-tls should be passed when
> building gcc so that attempts to use __thread generate a compile error
> (to be detected in configure).

aww, i was sure it did, but i have missed that one.
using a fixed gcc4, luajit apparently falls back to the OSX workaround 
and appears to work so far.

both luajit and the fixed gcc build script is now in sabotage. (needs a 
"butch rebuild gcc4" after updating the pkgs.)