From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/4771 Path: news.gmane.org!not-for-mail From: Rich Felker Newsgroups: gmane.linux.lib.musl.general Subject: Re: malloc & uncommitting memory Date: Thu, 3 Apr 2014 00:43:23 -0400 Message-ID: <20140403044323.GR26358@brightrain.aerifal.cx> References: <20140402210805.GA852@brightrain.aerifal.cx> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1396525116 14583 80.91.229.3 (3 Apr 2014 11:38:36 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 3 Apr 2014 11:38:36 +0000 (UTC) To: musl@lists.openwall.com Original-X-From: musl-return-4779-gllmg-musl=m.gmane.org@lists.openwall.com Thu Apr 03 13:38:30 2014 Return-path: Envelope-to: gllmg-musl@plane.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by plane.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1WVebA-0002MT-Ib for gllmg-musl@plane.gmane.org; Thu, 03 Apr 2014 12:10:16 +0200 Original-Received: (qmail 23745 invoked by uid 550); 3 Apr 2014 04:43:35 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: Original-Received: (qmail 23736 invoked from network); 3 Apr 2014 04:43:35 -0000 Content-Disposition: inline In-Reply-To: <20140402210805.GA852@brightrain.aerifal.cx> User-Agent: Mutt/1.5.21 (2010-09-15) Xref: news.gmane.org gmane.linux.lib.musl.general:4771 Archived-At: On Wed, Apr 02, 2014 at 05:08:05PM -0400, Rich Felker wrote: > As long as thresholds are chosen right, I don't think this approach > would be costly from a performance standpoint. But it might have > consequences for fragmentation. So I'd like to discuss it further, see > what ideas emerge, and whether it looks reasonable before trying to > implement anything. Basically, the way fragmentation can result from having "frozen" chunks is when small chunks adjacent to them are freed: these free chunks are unable to be merged with the frozen chunk, so they get reused for small allocations despite being adjacent to a large free (albeit frozen) region. I see several possible ways to reduce or avoid this: 1. Always leave large padding (large enough to stay in bin-63) on both sides of the frozen chunk when freezing. This would force adjacent freed chunks to get merged into a bin-63 chunk, preventing them from getting used for small allocations unless nothing smaller were available. However, they could still get used and gradually depleted. 2. When a chunk is freed, check if it's a neighbor to a frozen chunk, and if so, put it at the end of its bin rather than the beginning, so that other free chunks of the appropriate size get used first. However this wouldn't help when it's the only chunk of its size but another slightly-larger chunk would be a better candidate to split. 3. Allow merging directly into frozen chunks, the same way merging into free chunks is done now, with logic to expand the region that's PROT_NONE similar to the current logic for expanding the MADV_DONTNEED region. Of these, option 3 looks the most promising. Rich