From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-3.1 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.4 Received: from second.openwall.net (second.openwall.net [193.110.157.125]) by inbox.vuxu.org (Postfix) with SMTP id 1A0FE217BA for ; Mon, 22 Jan 2024 03:47:10 +0100 (CET) Received: (qmail 24148 invoked by uid 550); 22 Jan 2024 02:45:06 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 24113 invoked from network); 22 Jan 2024 02:45:06 -0000 Date: Sun, 21 Jan 2024 21:47:12 -0500 From: Rich Felker To: 847567161 <847567161@qq.com> Cc: musl Message-ID: <20240122024712.GE4163@brightrain.aerifal.cx> References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Subject: Re: Re: [musl] =?utf-8?B?44CQTGlua2Vy44CR?= =?utf-8?Q?Does?= MUSL support using TLS for both libc and app dependencies? On Mon, Jan 22, 2024 at 10:28:49AM +0800, 847567161 wrote: > > Indeed, there currently is not support for that. It's kinda an > > omission that should be remedied, as someday we may want to support > > softfloat with fenv where TLS might come into play through linking the > > compiler runtime (libgcc.a) that contains thread-local floating point > > state. But for the time being that's not something anyone seems to be > > interested in doing. > > We discovered the problem when integrating gwp_asan into musl libc. > gwp_asan is a static library that used tls of init-exec mode, It can't work with tsd for some reason. > So we think it is necessary for libc to support tls. I wasn't sure what gwp_asan was so I looked it up, and it appears not to actually be real asan, just a wrapper for detecting malloc usage errors (overflows, uaf/df, etc.). I'm not sure what you expect to gain out of putting it in libc. There is no nontrivial malloc usage in libc itself, and on top of that, mallocng already detects the same types of errors gwp_asan seems to be documented to detect. If you really want to try this, you can probably just put a member in the struct __pthread and have the gwp_asan code access its thread-local data via __pthread_self()->new_member.. But I don't see a good motivation for doing any of this. > > That would also be the answer to your question: If the TLS is too large > > for builtin_tls, it gets allocated (which happens in line 2017). > > builtin_tls is merely an optimization to ensure small TLS sections don't > > need an allocation that can fail. This can also be understood as a > > quality improvement measure since this way, most applications cannot > > randomly on startup for TLS allocation reasons when the free memory runs > > low. Whatever good that does in an environment where the kernel gets to > > arbitrarily kill any process it wants to. > > Why do we set tp in Stage 2b, is builtin_tls used before here? Yes. Basically all libc functions use TLS because they set errno on error. > Can we set it later？ > https://github.com/bminor/musl/blob/master/ldso/dynlink.c#L2034 No, you'll crash very early if it's not setup at all. > > For static linking, main thread TLS is allocated with mmap in > > static_init_tls(). In that case there is at most 1 TLS module. > > > > > > > > 2、About libc and other so use tls at the same time > > > I didn’t see musl modify tls_offset when ldso uses tls, so when another so uses tls later, their tls offsets will conflict. > > > >If we were ever to support TLS in libc/ldso, I think it would always > > > be accessed via the global-dynamic model, so that it doesn't have to > > >move; its DTV slot number would just get changed. > > Why can't we support locl-dynamic and init-exec modes in libc? Using initial-exec from shared libs is a hack, not the intended usage. In this case it can work, but as you've noticed it would require moving the libc TLS after setting up the main program's TLS, whose location relative to the thread pointer is fixed by the local-exec model used in the main program. This is even more of a hack, because it causes objects that have already been seen to change address. And it has very little value. With TLSDESC, GD model is not noticably slow. > Do you mean that 'move' refers to the dlopen scene? Could you give more details? No, just that whatever address the libc TLS got placed at in stage 2b would not necessarily be a valid place to keep it after loading the main program in stage 3. > >As to the question: Libc itself cannot use TLS. It could in the static > >linking case (when the TLS just gets rolled into the same section as the > >application TLS), but not in the dynamic one. The dynlinker currently > >does not set up TLS in libc correctly. Not entirely familiar with the > >list of things that would be needed to allow libc to have TLS, but it is > >likely to be nontrivial. I already foresee an order-of-operations > >problem. See, the dynlinker on startup currently works like this: > > >Stage 1: Process libc relative relocations > >Stage 2a: Process libc symbolic relocations > >Stage 2b: Load initial thread pointer > >Stage 3: Load dependencies, process all of the relocations. > > >With TLS in libc, stage 2a would already encounter relocations > >referencing the TLS which gets allocated in stage 3 at the earliest, > >because that's when the allocator becomes available. A gordian knot. > > Some of our ideas： > 1、Anyway, we need to put the app's tls in the first block. > 2、We can put the libc tls in the first or second part, depending on whether the app uses tls or not. > 3、More details： > In Stage 2a, we can use aux to see whether the app uses tls and decode the tls size of the app. > Then we can know the tls offset of libc, it can be used for libc symbolic relocations. > Skip processing of libc in subsequent processing of TLS functions，like update_tls_size、assign_tls and so on. You can't see the app at stage 2a or 2b. In the general case, the main app has not even been loaded into memory until line 1906 (ldso executed as a command rather than via the kernel). Rich