From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/12694 Path: news.gmane.org!.POSTED!not-for-mail From: Szabolcs Nagy Newsgroups: gmane.linux.lib.musl.general Subject: Re: tcmalloc compatibility Date: Tue, 10 Apr 2018 22:33:55 +0200 Message-ID: <20180410203354.GI4418@port70.net> References: <0ea267bf-ea3a-9810-be1a-50e71b6cfce1@denis.im> <20180410143359.GF3094@brightrain.aerifal.cx> <878t9vlzh1.fsf@mid.deneb.enyo.de> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: blaine.gmane.org 1523392327 11616 195.159.176.226 (10 Apr 2018 20:32:07 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Tue, 10 Apr 2018 20:32:07 +0000 (UTC) User-Agent: Mutt/1.9.1 (2017-09-22) Cc: Rich Felker , Florian Weimer To: musl@lists.openwall.com Original-X-From: musl-return-12710-gllmg-musl=m.gmane.org@lists.openwall.com Tue Apr 10 22:32:03 2018 Return-path: Envelope-to: gllmg-musl@m.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by blaine.gmane.org with smtp (Exim 4.84_2) (envelope-from ) id 1f5zvs-0002pz-OE for gllmg-musl@m.gmane.org; Tue, 10 Apr 2018 22:32:00 +0200 Original-Received: (qmail 15360 invoked by uid 550); 10 Apr 2018 20:34:07 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Original-Received: (qmail 14318 invoked from network); 10 Apr 2018 20:34:07 -0000 Mail-Followup-To: musl@lists.openwall.com, Rich Felker , Florian Weimer Content-Disposition: inline In-Reply-To: <878t9vlzh1.fsf@mid.deneb.enyo.de> Xref: news.gmane.org gmane.linux.lib.musl.general:12694 Archived-At: * Florian Weimer [2018-04-10 19:53:46 +0200]: > We have some documentation nowadays: > > > > The remaining undocumented aspects concern cyclic dependencies, such > as the suitability of certain TLS models for implementing a custom > malloc, or using memory-allocating glibc functions such as fopen or > backtrace from the allocator itself. > it's not documented what the interposed malloc may or may not do. cyclic dependency (because malloc calls a memory-allocating libc function) is one issue, but in general the libc can be in an inconsistent state at the point of an internal malloc call (e.g. some lock is held that should never be held when user code is running) and thus certain other libc functions may not operate as expected in an interposer. documenting the set of libc apis that work as expected in an interposer is difficult. > In practice, malloc interposition works extremely well and causes few > issues due to interposition itself. Obviously, there are bugs, but > most of them would still be bugs if the allocator was non-interposing. > (Examples are lots of initial-exec TLS data, and incorrect alignment > for allocations.) > even with the very small set of libc apis that a malloc interposer may want to use, there known (but undocumented) caveats: - glibc dlsym calls calloc (bites anybody who tries to wrap calloc in the usual way) - general dynamic tls may call malloc and uses global dl lock so using tls can cause recursion or deadlock. - using stdio for logging is problematic because of stdio locks and recursive malloc calls. - malloc may get called before the first ctor and after the last dtor. - malloc may be called on a thread with small stack (e.g. gai helper thread) so it should not have excessive stack usage. - glibc fork tries to reset the malloc state before calling the child atfork handler in an attempt to support non-as-safe fork handlers, code that relies on this can get broken with malloc interposition. interposition only seems to work extremely well because users already fixed their code and things dont change much in this space. > I believe musl uses less malloc internally, so it should be even more > compatible with an interposing malloc implementation than glibc.