From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/334 Path: news.gmane.org!not-for-mail From: Szabolcs Nagy Newsgroups: gmane.linux.lib.musl.general Subject: Re: New daily reports - debugging alloc.c et al Date: Sat, 6 Aug 2011 13:15:05 +0200 Message-ID: <20110806111505.GS29562@port70.net> References: <4E39C84F.8060705@gmail.com> <20110803224651.GB11437@openwall.com> <4E3A79B2.8090204@gmail.com> <4E3B331E.7050502@gmail.com> <4E3CC5AC.3070404@gmail.com> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: dough.gmane.org 1312629323 2111 80.91.229.12 (6 Aug 2011 11:15:23 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Sat, 6 Aug 2011 11:15:23 +0000 (UTC) To: musl@lists.openwall.com Original-X-From: musl-return-335-gllmg-musl=m.gmane.org@lists.openwall.com Sat Aug 06 13:15:19 2011 Return-path: Envelope-to: gllmg-musl@lo.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by lo.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1Qpeqc-0007xv-NC for gllmg-musl@lo.gmane.org; Sat, 06 Aug 2011 13:15:18 +0200 Original-Received: (qmail 25930 invoked by uid 550); 6 Aug 2011 11:15:17 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: Original-Received: (qmail 25919 invoked from network); 6 Aug 2011 11:15:17 -0000 Content-Disposition: inline In-Reply-To: <4E3CC5AC.3070404@gmail.com> User-Agent: Mutt/1.5.20 (2009-06-14) Xref: news.gmane.org gmane.linux.lib.musl.general:334 Archived-At: * Luka Mar??eti?? [2011-08-06 06:40:12 +0200]: > here), but the method of allocation that I've been recommended > either fails to fulfill its promise, or I'm blind to my own bug. > Here's a minimal test case: > http://paste.debian.net/125242/ > A quick explanation: Alloc.c needs to squat most of process' virtual > memory. I employ a kind of binary search to find the greatest size > that can be allocated, but then it turns out, mmap and malloc are > able to continue allocating memory still. Not sure why. > it works here as expected the first loop finds the largest area that can be mmaped (about 2.9G here) then the second loop finds extra mmapable smaller regions (this is slow here as there are about 70000 extra mmappable pages, it would be better to do it in bigger chunks like: /*tries to mmap even more now:*/ printf("Number of *additional* bytes mmap'd:\n"); most=0; for(j=1<<20; j; j/=2) { printf("%8zu ", j); for(i=0; mmap(NULL, j*PAGE_SIZE, PROT_NONE, MAP_PRIVATE, fd, 0) != MAP_FAILED; ++i) printf("."); most += i*j*PAGE_SIZE; printf("\n"); } printf("%zu\n", most); ) after this the last loop with malloc immediately fails (there is nothing to mmap anymore, but maybe if the allocator already has some area reserved then a few mallocs can succeed)