From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/14960 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Rich Felker Newsgroups: gmane.linux.lib.musl.general Subject: Re: New malloc - first preview Date: Thu, 28 Nov 2019 16:56:42 -0500 Message-ID: <20191128215642.GN16318@brightrain.aerifal.cx> References: <20191022174051.GA24726@brightrain.aerifal.cx> Reply-To: musl@lists.openwall.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="16903"; mail-complaints-to="usenet@blaine.gmane.org" User-Agent: Mutt/1.5.21 (2010-09-15) To: musl@lists.openwall.com Original-X-From: musl-return-14976-gllmg-musl=m.gmane.org@lists.openwall.com Thu Nov 28 22:56:58 2019 Return-path: Envelope-to: gllmg-musl@m.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by blaine.gmane.org with smtp (Exim 4.89) (envelope-from ) id 1iaRmU-0004GM-Iv for gllmg-musl@m.gmane.org; Thu, 28 Nov 2019 22:56:58 +0100 Original-Received: (qmail 27788 invoked by uid 550); 28 Nov 2019 21:56:55 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Original-Received: (qmail 27765 invoked from network); 28 Nov 2019 21:56:54 -0000 Content-Disposition: inline In-Reply-To: <20191022174051.GA24726@brightrain.aerifal.cx> Original-Sender: Rich Felker Xref: news.gmane.org gmane.linux.lib.musl.general:14960 Archived-At: Work on the new malloc is well underway, and I have a draft version now public at: https://github.com/richfelker/mallocng-draft Some highlights: - Smallest size-class now has 12 of 16 bytes usable for storage, compared to 8 of 16 (32-bit archs) or 16 of 32 (64-bit archs) with the old malloc, plus 1.5 bytes (32-bit) or 2.5 bytes (64-bit) of out-of-band metadata. This overhead (5.5 or 6.5 bytes per allocation) is uniform across all sizes. - Small-size allocation granularity/alignment is now 16-byte on both 32-bit and 64-bit archs. No more wasting 64 bytes for a 36-byte structure. - Metadata defining heap state is all out-of-band and protected below by guard pages, where it's not easily reachable through typical attack vectors. - Freed memory is quickly returnable to the kernel. - Common path malloc and free (asymptotically should be ~95% of calls under high use) do not require exclusive locks -- readlock+atomic for malloc, pure-atomic for free. - Size classes are not straight geometric progression but instead 1/7, 1/6, and 1/5 of powers of two in between the whole power-of-two steps, optimizing fit in whole power-of-two numbers of pages. - Code size is very close to the same as the old allocator, and may end up being smaller. This is still early in development, and it won't be merged in musl until the next release cycle, after the new year. There may be plenty of bugs or suboptimal behavior. And I still need to write up the design. For anyone wanting to play around with the draft, it can be linked or used by LD_PRELOAD. A dump_heap function is provided which can show allocator state. In the status masks, a=available and f=freed; these are distinct states to allow some control over interval to reuse of freed memory (which gives increased safety against UAF/DF). The "_" status is in-use. Rich