From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/2452 Path: news.gmane.org!not-for-mail From: Rich Felker Newsgroups: gmane.linux.lib.musl.general Subject: Re: stdio self-synchronized destruction: does it need fixing? Date: Mon, 10 Dec 2012 18:21:53 -0500 Message-ID: <20121210232153.GN20323@brightrain.aerifal.cx> References: <20121210180508.GA2313@brightrain.aerifal.cx> <20121210225752.GI23126@port70.net> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1355181731 29890 80.91.229.3 (10 Dec 2012 23:22:11 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Mon, 10 Dec 2012 23:22:11 +0000 (UTC) To: musl@lists.openwall.com Original-X-From: musl-return-2453-gllmg-musl=m.gmane.org@lists.openwall.com Tue Dec 11 00:22:24 2012 Return-path: Envelope-to: gllmg-musl@plane.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by plane.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1TiCfy-0008Tk-Mb for gllmg-musl@plane.gmane.org; Tue, 11 Dec 2012 00:22:18 +0100 Original-Received: (qmail 7374 invoked by uid 550); 10 Dec 2012 23:22:06 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: Original-Received: (qmail 7366 invoked from network); 10 Dec 2012 23:22:05 -0000 Content-Disposition: inline In-Reply-To: <20121210225752.GI23126@port70.net> User-Agent: Mutt/1.5.21 (2010-09-15) Xref: news.gmane.org gmane.linux.lib.musl.general:2452 Archived-At: On Mon, Dec 10, 2012 at 11:57:53PM +0100, Szabolcs Nagy wrote: > * Rich Felker [2012-12-10 13:05:09 -0500]: > > memory. The allocation of FILE structues is always performed by libc, > > and always happens via malloc with a small-size allocation, which > > means the memory is managed as part of the heap and never unmapped > > once it's mapped. Thus, as far as I can tell, the worst that can > > happen is a read-only access to memory no longer owned by the FILE, > > at least write a comment there that the invalid read is known This sounds like a perfect solution for now. > (btw at some point someone may rewrite malloc so small allocations > can go to mmapped areas as well which may be reclaimed..) I would like to add the ability to obtain more heap memory from mmap, but I think unmapping unused parts has more cons than pros. The only benefit is that it reclaims the virtual address space so that the kernel can use it for honoring future mmap requests. The disadvantage is that it fragments the heap; once a "hole" has been created in the heap, it's impossible to put it back and merge the adjacent free regions into one large free region in O(1) time. Of course if you're talking about a _whole_ supplemental mmap zone becoming free, then it would be safe to unmap it, and this may be a worthwhile case to handle. In particular, one way of doing supplemental heap via mmap would be to make each map roughly the same size as the threshold for releasing it back to the kernel via mprotect/mmap zeroing. The beginning and end of the region could have specially-flagged chunks, and then, after merging on free(), if both the previous and next neighbor are flagged as the mmap region endpoints, the whole region could be unmapped rather than adding it back to the free list. The alternative would be to do supplemental mmap regions in much larger increments, maybe on the order of 16MB, initially with PROT_NONE, and grow the PROT_READ|PROT_WRITE portion like brk() does for the main heap. Then the same code that releases parts of the main brk heap via zeroing could also work well here. I think this alternative approach would result in less fragmentation for malloc, but more chance of filling up virtual address space and preventing mmap from working. Rich