* thread local storage @ 2012-07-16 19:02 John Spencer 2012-07-16 18:57 ` Gregor Richards ` (2 more replies) 0 siblings, 3 replies; 8+ messages in thread From: John Spencer @ 2012-07-16 19:02 UTC (permalink / raw) To: musl 2 out of 14 sabotage followers wanted to use a musl-based system as a platform for luajit (and then were never seen again). so i looked into adding it... luajit builds without problems on musl, but then crashes due to a lack of TLS. is it planned to add this feature ? iirc it wasn't mentioned on the latest roadmap... ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: thread local storage 2012-07-16 19:02 thread local storage John Spencer @ 2012-07-16 18:57 ` Gregor Richards 2012-07-16 20:09 ` Rich Felker 2012-07-16 19:58 ` Justin Cormack 2012-07-16 20:03 ` Rich Felker 2 siblings, 1 reply; 8+ messages in thread From: Gregor Richards @ 2012-07-16 18:57 UTC (permalink / raw) To: musl On 07/16/2012 03:02 PM, John Spencer wrote: > 2 out of 14 sabotage followers wanted to use a musl-based system as a > platform for luajit (and then were never seen again). > so i looked into adding it... > > luajit builds without problems on musl, but then crashes due to a lack > of TLS. > > is it planned to add this feature ? iirc it wasn't mentioned on the > latest roadmap... > > > With a quick perusal of the LuaJIT source, this is the only instance of TLS I see: #if LJ_UNWIND_EXT #if LJ_TARGET_OSX || defined(__OpenBSD__) /* Sorry, no thread safety for OSX. Complain to Apple, not me. */ static _Unwind_Exception static_uex; #else static __thread _Unwind_Exception static_uex; #endif Convince it to use the same exception as OS X and OpenBSD and you should be in business. "Complain to Apple, not me" is an odd way to say "I am too lazy to use standard interfaces." TLS has come up on the IRC chat, but I don't recall what the conclusions were regarding future implementation or not. With valediction, - Gregor Richards ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: thread local storage 2012-07-16 18:57 ` Gregor Richards @ 2012-07-16 20:09 ` Rich Felker 2012-07-16 21:30 ` John Spencer 0 siblings, 1 reply; 8+ messages in thread From: Rich Felker @ 2012-07-16 20:09 UTC (permalink / raw) To: musl On Mon, Jul 16, 2012 at 02:57:22PM -0400, Gregor Richards wrote: > On 07/16/2012 03:02 PM, John Spencer wrote: > >2 out of 14 sabotage followers wanted to use a musl-based system > >as a platform for luajit (and then were never seen again). > >so i looked into adding it... > > > >luajit builds without problems on musl, but then crashes due to a > >lack of TLS. > > > >is it planned to add this feature ? iirc it wasn't mentioned on > >the latest roadmap... > > > > > > > With a quick perusal of the LuaJIT source, this is the only instance > of TLS I see: > > #if LJ_UNWIND_EXT > #if LJ_TARGET_OSX || defined(__OpenBSD__) > /* Sorry, no thread safety for OSX. Complain to Apple, not me. */ > static _Unwind_Exception static_uex; > #else > static __thread _Unwind_Exception static_uex; > #endif > > Convince it to use the same exception as OS X and OpenBSD and you > should be in business. This is broken and non-thread-safe. Not a good idea. Instead try: #define static_uex (*(_Unwind_Exception *)pthread_getspecific(static_uex_key)) where static_uex_key is a pthread_key_t initialized earlier with: pthread_key_create(&static_uex_key, 0); And where the thread-specific value of the key is set in thread startup as: _Unwind_Exception static_uex_local; pthread_setspecific(static_uex_key, &static_uex_local); The simplicity and generality of this solution is why __thread was just a stupid idea to begin with... Rich ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: thread local storage 2012-07-16 20:09 ` Rich Felker @ 2012-07-16 21:30 ` John Spencer 2012-07-16 22:18 ` Rich Felker 0 siblings, 1 reply; 8+ messages in thread From: John Spencer @ 2012-07-16 21:30 UTC (permalink / raw) To: musl On 07/16/2012 10:09 PM, Rich Felker wrote: > > This is broken and non-thread-safe. Not a good idea. Instead try: thanks, very interesting. do you mind if i forward this to the luajit ml ? looks as if this might fix the problem on OSX/OpenBSD as well. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: thread local storage 2012-07-16 21:30 ` John Spencer @ 2012-07-16 22:18 ` Rich Felker 0 siblings, 0 replies; 8+ messages in thread From: Rich Felker @ 2012-07-16 22:18 UTC (permalink / raw) To: musl On Mon, Jul 16, 2012 at 11:30:01PM +0200, John Spencer wrote: > On 07/16/2012 10:09 PM, Rich Felker wrote: > > > >This is broken and non-thread-safe. Not a good idea. Instead try: > > thanks, very interesting. > > do you mind if i forward this to the luajit ml ? > looks as if this might fix the problem on OSX/OpenBSD as well. Please do. This is the correct, portable fix. Using __thread or _Thread_local might be prefered to avoid the init/cleanup need when possible, but the POSIX fallback should be used rather than falling back to code that will horribly crash or even yield privilege elevation under multi-threaded use. Rich ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: thread local storage 2012-07-16 19:02 thread local storage John Spencer 2012-07-16 18:57 ` Gregor Richards @ 2012-07-16 19:58 ` Justin Cormack 2012-07-16 20:03 ` Rich Felker 2 siblings, 0 replies; 8+ messages in thread From: Justin Cormack @ 2012-07-16 19:58 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 358 bytes --] On Jul 16, 2012 7:53 PM, "John Spencer" <maillist-musl@barfooze.de> wrote: > > 2 out of 14 sabotage followers wanted to use a musl-based system as a platform for luajit (and then were never seen again). > so i looked into adding it... > I am still around interested in luajit on musl just had no time since getting the same stack trace you got.... Justin [-- Attachment #2: Type: text/html, Size: 486 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: thread local storage 2012-07-16 19:02 thread local storage John Spencer 2012-07-16 18:57 ` Gregor Richards 2012-07-16 19:58 ` Justin Cormack @ 2012-07-16 20:03 ` Rich Felker 2012-07-16 22:06 ` John Spencer 2 siblings, 1 reply; 8+ messages in thread From: Rich Felker @ 2012-07-16 20:03 UTC (permalink / raw) To: musl On Mon, Jul 16, 2012 at 09:02:49PM +0200, John Spencer wrote: > 2 out of 14 sabotage followers wanted to use a musl-based system as > a platform for luajit (and then were never seen again). > so i looked into adding it... > > luajit builds without problems on musl, but then crashes due to a > lack of TLS. Then the gcc was built wrong. --disable-tls should be passed when building gcc so that attempts to use __thread generate a compile error (to be detected in configure). > is it planned to add this feature ? iirc it wasn't mentioned on the > latest roadmap... Yes, but it's one of the hardest remainind things, for a couple reasons... 1. It involves introducing ugly arch-specific code into lots of things that should be arch-agnostic, because the original implementors did things this way and encoding the knowledge into gcc. This might be possible to bypass if we drop support for the static/local TLS model or whatever they call it and force all modules, even the main program and static binaries, to access TLS through a function call like shared libs have to do. Actually I'd really like to do it this way if possible since it would allow us to change things in the future instead of locking in ABI stuff. At the very least it probably requires a build option to GCC and/or a special -f flag in the spec file or GCC default specs to make this possible. 2. Dynamic linker needs to be updtaed to handle all sorts of TLS-related relocations which I don't yet understand, so I have to read up on them... 3. Existing implementations of TLS are just _wrong_ and crashingly so. When a new shared lib is loaded with dlopen, it may require additional TLS memory, and this memory must be allocated for each thread currently running. The way glibc/NPTL works is to keep a "generation counter" and dynamically allocate space for each thread's new TLS on the first accerss if the ldso generation counter has increased (or something like that). If allocation fails, there's nothing you can do but abort the program. Naturally this is unacceptable. Fixing it requires temporarily placing a lock on new thread creation and allocating the new TLS space for all currently-running threads as part of the dlopen operation, then either assigning it to them before dlopen returns, or placing it in a reserved pool whereby they can get it on the first access attempt. Solving this problem correctly, without race conditions or deadlocks, etc. is rather non-trivial. Rich ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: thread local storage 2012-07-16 20:03 ` Rich Felker @ 2012-07-16 22:06 ` John Spencer 0 siblings, 0 replies; 8+ messages in thread From: John Spencer @ 2012-07-16 22:06 UTC (permalink / raw) To: musl On 07/16/2012 10:03 PM, Rich Felker wrote: > On Mon, Jul 16, 2012 at 09:02:49PM +0200, John Spencer wrote: >> luajit builds without problems on musl, but then crashes due to a >> lack of TLS. > Then the gcc was built wrong. --disable-tls should be passed when > building gcc so that attempts to use __thread generate a compile error > (to be detected in configure). aww, i was sure it did, but i have missed that one. using a fixed gcc4, luajit apparently falls back to the OSX workaround and appears to work so far. both luajit and the fixed gcc build script is now in sabotage. (needs a "butch rebuild gcc4" after updating the pkgs.) ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2012-07-16 22:18 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2012-07-16 19:02 thread local storage John Spencer 2012-07-16 18:57 ` Gregor Richards 2012-07-16 20:09 ` Rich Felker 2012-07-16 21:30 ` John Spencer 2012-07-16 22:18 ` Rich Felker 2012-07-16 19:58 ` Justin Cormack 2012-07-16 20:03 ` Rich Felker 2012-07-16 22:06 ` John Spencer
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/musl/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).