From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/4458 Path: news.gmane.org!not-for-mail From: Rich Felker Newsgroups: gmane.linux.lib.musl.general Subject: Re: Re: Removing sbrk and brk Date: Tue, 7 Jan 2014 17:00:53 -0500 Message-ID: <20140107220053.GG24286@brightrain.aerifal.cx> References: <20131223044609.GZ24286@brightrain.aerifal.cx> <20140102220302.GR24286@brightrain.aerifal.cx> <20140103173301.GU24286@brightrain.aerifal.cx> <20140103181906.GV24286@brightrain.aerifal.cx> <20140103190350.GW24286@brightrain.aerifal.cx> <20140106224036.GC24286@brightrain.aerifal.cx> <20140107160627.GD24286@brightrain.aerifal.cx> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Trace: ger.gmane.org 1389132063 20680 80.91.229.3 (7 Jan 2014 22:01:03 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Tue, 7 Jan 2014 22:01:03 +0000 (UTC) Cc: Theo de Raadt To: musl@lists.openwall.com Original-X-From: musl-return-4462-gllmg-musl=m.gmane.org@lists.openwall.com Tue Jan 07 23:01:10 2014 Return-path: Envelope-to: gllmg-musl@plane.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by plane.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1W0ehu-0003Fr-Cu for gllmg-musl@plane.gmane.org; Tue, 07 Jan 2014 23:01:06 +0100 Original-Received: (qmail 11450 invoked by uid 550); 7 Jan 2014 22:01:05 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: Original-Received: (qmail 11439 invoked from network); 7 Jan 2014 22:01:05 -0000 Content-Disposition: inline In-Reply-To: <20140107160627.GD24286@brightrain.aerifal.cx> User-Agent: Mutt/1.5.21 (2010-09-15) Xref: news.gmane.org gmane.linux.lib.musl.general:4458 Archived-At: It seems like I've made some incorrect statements in this thread which the OpenBSD folks weren't too happy about, so now that I'm more clear on the details I'd like to correct those mistakes for the record... On Tue, Jan 07, 2014 at 11:06:27AM -0500, Rich Felker wrote: > On Tue, Jan 07, 2014 at 09:43:26AM +0000, Thorsten Glaser wrote: > > Rich Felker aerifal.cx> writes: > > > > > This seems to be optional behavior; using guard pages with all > > > allocations would blow up memory usage several thousand times and > > > > No, they aren’t accessible, so the kernel (should) never maps them > > to any real RAM. > > The point is that even a 1-byte allocation (in fairness, that would be > rounded up to at least 16 bytes) ends up consuming a whole page and > thus 4k of storage, plus another 4k of virtual address space for the > guard page. On MIPS it may be even worse (16k pages are required on > some hardwasre). This is both incorrect and misleading. I was under the impression that omalloc had a (non-default) option to use guard pages for all allocations, even sub-page-size ones. This is not the case, and my only excuse is that I wrote the above a day or two after last reading the source and seeing that it had configurable options, but without taking the time to confirm their nature. So in reality, guard pages are only used for allocations that take up at least a whole page, and so from what I can tell now, the maximum overhead is something like 100% (not 25600%) committed memory and 200% virtual address space and occurs for allocations just larger than one page (where you end up needing one extra page for the rest of the storage, and one guard page). This is comparable to the overhead for small allocations (less than a page), which are rounded up to a power-of-two size and thus have a worst-case overhead of near-100%. So there's no pathologically huge over-allocation going on, just moderate overhead, which the OpenBSD folks have claimed (and I believe them on this) has caught serious bugs in lots of major software. Hope this clears things up. Rich