From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/12718 Path: news.gmane.org!.POSTED!not-for-mail From: Rich Felker Newsgroups: gmane.linux.lib.musl.general Subject: Re: tcmalloc compatibility Date: Mon, 16 Apr 2018 00:40:06 -0400 Message-ID: <20180416044006.GY3094@brightrain.aerifal.cx> References: <0ea267bf-ea3a-9810-be1a-50e71b6cfce1@denis.im> <20180410143359.GF3094@brightrain.aerifal.cx> <878t9vlzh1.fsf@mid.deneb.enyo.de> <20180410203354.GI4418@port70.net> <20180410204411.GK3094@brightrain.aerifal.cx> <20180410211724.GJ4418@port70.net> <20180416041924.GA23767@voyager> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: blaine.gmane.org 1523853495 25804 195.159.176.226 (16 Apr 2018 04:38:15 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Mon, 16 Apr 2018 04:38:15 +0000 (UTC) User-Agent: Mutt/1.5.21 (2010-09-15) To: musl@lists.openwall.com Original-X-From: musl-return-12734-gllmg-musl=m.gmane.org@lists.openwall.com Mon Apr 16 06:38:11 2018 Return-path: Envelope-to: gllmg-musl@m.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by blaine.gmane.org with smtp (Exim 4.84_2) (envelope-from ) id 1f7vu7-0006du-6K for gllmg-musl@m.gmane.org; Mon, 16 Apr 2018 06:38:11 +0200 Original-Received: (qmail 20368 invoked by uid 550); 16 Apr 2018 04:40:19 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Original-Received: (qmail 20347 invoked from network); 16 Apr 2018 04:40:18 -0000 Content-Disposition: inline In-Reply-To: <20180416041924.GA23767@voyager> Original-Sender: Rich Felker Xref: news.gmane.org gmane.linux.lib.musl.general:12718 Archived-At: On Mon, Apr 16, 2018 at 06:19:24AM +0200, Markus Wichmann wrote: > On Sun, Apr 15, 2018 at 01:52:10PM +0200, ardi wrote: > > On Tue, Apr 10, 2018 at 11:17 PM, Szabolcs Nagy wrote: > > > > > > then the wrappers with dlsym(RTLD_NEXT,sym) would not work. > > > (malloc checkers, valgrind, sanitizers etc all do it) > > > > I've been using ElectricFence, as my only memory debugger since 1996 > > or so; mostly with the libc of commercial Unices, but also with glibc > > in Linux, and with the OSX libc. I never considered I could run into > > the issues commented in this thread, and in fact I never faced these > > issues and it always worked as expected (however, I must admit I only > > use multithreading for accelerating clearly isolated math-intensive > > loops that don't call malloc-related functions from inside the loop). > > > > Said this, when I'm linking with ElectricFence, my brain has the "hack > > mode flag" ON (I mean, I always had the feeling that I'm working with > > a temporary hack that can fail whenever my link line contains -lefence > > , and I'm aware that things can go wrong --I didn't consider thread > > safety, but anyway I know ElectricFence can fail if the OS syscalls > > that allocate protected memory at buffer ends change their behaviour > > in newer versions, or if there's some OS/CPU-dependent subtlety with > > alignment, etc...) > > > > I've not tried to use ElectricFence with musl yet... but reading this, > > can I suppose it won't work? Is there any "hack mode ON" procedure > > (yet easy) that would allow to use ElectricFence (assuming > > non-threaded code, which is always my case). > > > > I agree with your commitment to correctness, and I'm not asking for a > > safe and guaranteed implementation of function interposition, just > > that sometimes I need to break my binaries to make them crash hard as > > soon as pointer accesses a byte it shouldn't access. > > > > Cheers, > > > > ardi > > So long as you refrain from using dynamic linking (because of the memory > donation) This is only a small part of the reason you can't use dynamic linking. The other big part is that references in libc.so are bound at libc.so link time, so functions like getline, open_memstream, strdup, etc. will return pointers that won't be valid for you to free. > and calloc() and memalign() (and posix_memalign()) are unused > or overloaded, you should be fine. Both of these functions use the > internal bookkeeping of musl's malloc. calloc() uses it to figure out if > a chunk was mmapped (in which case no initialization is necessary), and > memalign() uses it to construct a second chunk header to cause the > returned pointer to be aligned. Yes, but this rule always applies for interposing, with any implementation. It's not musl-specific. > Most of the questioning here arose from that first part. Those are the > two big problems, actually, we need an interface to donate memory to the > malloc implementation, This isn't needed. It's fine for donation to donate to the internal (unused) implementation if malloc is interposed, or for donation not to happen at all. I don't think it's a good idea to create a public interposable API for donation. The big thing that does need to happen is getting rid of the call to free() to do the donation, which is unsafe/incorrect if it's interposed. Alexander Monakov's patch (which looks ok to commit with minor changes described in the thread) should fix that. > and the malloc implementation needs to provide > all of the hairier functions like memalign(). And we currently have no > way of enforcing either of these. A way to enforce this was discussed earlier in the thread, so it looks doable. Rich