From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/11736 Path: news.gmane.org!.POSTED!not-for-mail From: Steven Walter Newsgroups: gmane.linux.lib.musl.general Subject: [PATCH] malloc/expand_heap.c: really try to use all available memory Date: Tue, 18 Jul 2017 08:52:59 -0400 Message-ID: <20170718125259.19932-1-stevenrwalter@gmail.com> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: blaine.gmane.org X-Trace: blaine.gmane.org 1500385094 23763 195.159.176.226 (18 Jul 2017 13:38:14 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Tue, 18 Jul 2017 13:38:14 +0000 (UTC) Cc: Steven Walter To: musl@lists.openwall.com Original-X-From: musl-return-11749-gllmg-musl=m.gmane.org@lists.openwall.com Tue Jul 18 15:38:11 2017 Return-path: Envelope-to: gllmg-musl@m.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by blaine.gmane.org with smtp (Exim 4.84_2) (envelope-from ) id 1dXShT-0005rw-Aq for gllmg-musl@m.gmane.org; Tue, 18 Jul 2017 15:38:07 +0200 Original-Received: (qmail 32093 invoked by uid 550); 18 Jul 2017 13:38:09 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Original-Received: (qmail 7851 invoked from network); 18 Jul 2017 12:53:30 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=Gwdi1SwNyqvrYQV2NPBHZ+mE3TGwCS8/5BGAgS1iXSY=; b=LwGtTZ6sbRf8lA6hWsq6pQerY9gw1FK4qIi7E49OkDs3+Tre83x+zjrnzBNdBXeOqS IYry4+zdTCJ2hMY0vXBr+LSLG2wXM3FNE4SUGVlLFyX/abKqgOvLFloI+ADcgMIJgbd2 8xSBrcF2yNHVILrXKtE0R/FgnI2cW8TVL6jBSTkXz/bfSJ2EDQ6UwwuJmroM+MCAECA0 pxPeV0LBXT2nQ6M21VIIK/Can94hk1HZMgMYkfimsG7A8xMrvdHfK7FCsAXPGkjQUn9w 4MY8PlUGeleIU6fV8fu+8i/M3WJHnpa/1K6odhSjwqStisPPjLjIfKHNIccg7lR0z77F 2q6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=Gwdi1SwNyqvrYQV2NPBHZ+mE3TGwCS8/5BGAgS1iXSY=; b=dfhmcpRVZFJakqjAO4FBQOonW1EYCKRUm/RFEyD9KfwnTFypxR2YhlZpRvQuoKqowl 0CzuVaT/lE8+ewqaxam3zBdDoyVlMQsFs5prhWJxupGFR8fLGpJhWXXygFCKN9bvX3BF HSlvGXYgZsVZPQndmeQ35BxCzxlmWzTzl20T6ZnYem9c/spwp8E2rPnnxFShhWfWPkzf Kqc75ZaXX5bF1qYAvbSI3onRc47m8qlKq25tR4sCYHwzvrH5Hvq6SFmm7Q3IaUhmu2ra oKtk+WLsCWe+Z3vXlikHJ0HGQEBTAGunPOj1dsoGNWKdmU2vE5ycYHozE9bG6v5BYs5n anSA== X-Gm-Message-State: AIVw111DQJIqagN0AXd82MbVp2IQ2LnSFxVSEcYoNkseK4D3JO7ZEdV5 XnjYph2De1w9xJMxyO0= X-Received: by 10.36.139.1 with SMTP id g1mr2473700ite.18.1500382398528; Tue, 18 Jul 2017 05:53:18 -0700 (PDT) X-Mailer: git-send-email 2.13.3 Xref: news.gmane.org gmane.linux.lib.musl.general:11736 Archived-At: Previously expand_heap would ask for increasingly larger mmap areas every time, in order to avoid allocating a bunch of small areas and fragmenting memory. However, after a point, we may ask for an mmap area so large that it can't possibly succeed, even though there is still system memory available. This is particularly likely on a 32-bit system with gigs of RAM, or else on a no-MMU system with pretty much any amount of RAM. Without an MMU to make physically-discontiguous pages appear contigious, the chance of any large mmap succeeding are very low. To fix this, support decreasing mmap_step once we hit an allocation failure. We'll try smaller and smaller amounts until we either ask for a single page or exactly as many pages as we currently need. Only if that fails do we fail the overall request. --- src/malloc/expand_heap.c | 27 +++++++++++++++++++-------- 1 file changed, 19 insertions(+), 8 deletions(-) diff --git a/src/malloc/expand_heap.c b/src/malloc/expand_heap.c index d8c0be7..6a0dae8 100644 --- a/src/malloc/expand_heap.c +++ b/src/malloc/expand_heap.c @@ -61,12 +61,23 @@ void *__expand_heap(size_t *pn) return (void *)(brk-n); } - size_t min = (size_t)PAGE_SIZE << mmap_step/2; - if (n < min) n = min; - void *area = __mmap(0, n, PROT_READ|PROT_WRITE, - MAP_PRIVATE|MAP_ANONYMOUS, -1, 0); - if (area == MAP_FAILED) return 0; - *pn = n; - mmap_step++; - return area; + while (1) { + size_t min = (size_t)PAGE_SIZE << mmap_step/2; + size_t size = n; + if (size < min) size = min; + void *area = __mmap(0, size, PROT_READ|PROT_WRITE, + MAP_PRIVATE|MAP_ANONYMOUS, -1, 0); + if (area != MAP_FAILED) { + *pn = size; + mmap_step++; + return area; + } + + // If we asked for a single page (or the exact allocation + // amount) and still didn't get it, we're toast + if (size == n || mmap_step < 2) + return 0; + + mmap_step--; + } } -- 2.7.4