From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/1307 Path: news.gmane.org!not-for-mail From: Rich Felker Newsgroups: gmane.linux.lib.musl.general Subject: Re: thread local storage Date: Mon, 16 Jul 2012 16:03:51 -0400 Message-ID: <20120716200350.GN544@brightrain.aerifal.cx> References: <50046559.3090204@barfooze.de> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: dough.gmane.org 1342469083 4834 80.91.229.3 (16 Jul 2012 20:04:43 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Mon, 16 Jul 2012 20:04:43 +0000 (UTC) To: musl@lists.openwall.com Original-X-From: musl-return-1308-gllmg-musl=m.gmane.org@lists.openwall.com Mon Jul 16 22:04:43 2012 Return-path: Envelope-to: gllmg-musl@plane.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by plane.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1SqrX8-00075N-1q for gllmg-musl@plane.gmane.org; Mon, 16 Jul 2012 22:04:42 +0200 Original-Received: (qmail 26426 invoked by uid 550); 16 Jul 2012 20:04:41 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: Original-Received: (qmail 26417 invoked from network); 16 Jul 2012 20:04:41 -0000 Content-Disposition: inline In-Reply-To: <50046559.3090204@barfooze.de> User-Agent: Mutt/1.5.21 (2010-09-15) Xref: news.gmane.org gmane.linux.lib.musl.general:1307 Archived-At: On Mon, Jul 16, 2012 at 09:02:49PM +0200, John Spencer wrote: > 2 out of 14 sabotage followers wanted to use a musl-based system as > a platform for luajit (and then were never seen again). > so i looked into adding it... > > luajit builds without problems on musl, but then crashes due to a > lack of TLS. Then the gcc was built wrong. --disable-tls should be passed when building gcc so that attempts to use __thread generate a compile error (to be detected in configure). > is it planned to add this feature ? iirc it wasn't mentioned on the > latest roadmap... Yes, but it's one of the hardest remainind things, for a couple reasons... 1. It involves introducing ugly arch-specific code into lots of things that should be arch-agnostic, because the original implementors did things this way and encoding the knowledge into gcc. This might be possible to bypass if we drop support for the static/local TLS model or whatever they call it and force all modules, even the main program and static binaries, to access TLS through a function call like shared libs have to do. Actually I'd really like to do it this way if possible since it would allow us to change things in the future instead of locking in ABI stuff. At the very least it probably requires a build option to GCC and/or a special -f flag in the spec file or GCC default specs to make this possible. 2. Dynamic linker needs to be updtaed to handle all sorts of TLS-related relocations which I don't yet understand, so I have to read up on them... 3. Existing implementations of TLS are just _wrong_ and crashingly so. When a new shared lib is loaded with dlopen, it may require additional TLS memory, and this memory must be allocated for each thread currently running. The way glibc/NPTL works is to keep a "generation counter" and dynamically allocate space for each thread's new TLS on the first accerss if the ldso generation counter has increased (or something like that). If allocation fails, there's nothing you can do but abort the program. Naturally this is unacceptable. Fixing it requires temporarily placing a lock on new thread creation and allocating the new TLS space for all currently-running threads as part of the dlopen operation, then either assigning it to them before dlopen returns, or placing it in a reserved pool whereby they can get it on the first access attempt. Solving this problem correctly, without race conditions or deadlocks, etc. is rather non-trivial. Rich