From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/4742 Path: news.gmane.org!not-for-mail From: Timo Teras Newsgroups: gmane.linux.lib.musl.general Subject: Re: malloc not behaving well when brk space is limited? Date: Sat, 29 Mar 2014 19:15:02 +0200 Message-ID: <20140329191502.072c07f9@vostro> References: <20140329170032.GJ8221@example.net> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Trace: ger.gmane.org 1396113242 32024 80.91.229.3 (29 Mar 2014 17:14:02 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sat, 29 Mar 2014 17:14:02 +0000 (UTC) Cc: u-igbb@aetey.se To: musl@lists.openwall.com Original-X-From: musl-return-4746-gllmg-musl=m.gmane.org@lists.openwall.com Sat Mar 29 18:13:56 2014 Return-path: Envelope-to: gllmg-musl@plane.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by plane.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1WTwpN-0004Dw-Hb for gllmg-musl@plane.gmane.org; Sat, 29 Mar 2014 18:13:53 +0100 Original-Received: (qmail 31883 invoked by uid 550); 29 Mar 2014 17:13:52 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: Original-Received: (qmail 31875 invoked from network); 29 Mar 2014 17:13:52 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; bh=Eh5e06AsPYqn//JVIHS5rgg27YdqcQgMBSZcTJycpY0=; b=WMWIvsqXDYfPT9Q5BXteHPrHWjcryIxZE2FOqFWoLbtP6Y9jT/+HMiSN0EZ0rY3FOo Lh9ujydQpz6jksvr1Cuu6QlvH/kk01AHDzRFG7Vn/X/JmDUSLji8D104Cf4DCMUVpMRx hOvomIN7kLxgtR9KDpcNcJdQ7ZADoZ0Evkx1yE8QVlytOvsmnbwREvAQEJl8KCBckTDM NpdbKAwnI/XuIO0pa/giRvrgdnbcV4LqacRsgp0fawncl6I8RZU/KObmyCP2WntKoLim Y+Q9g26CuQVwzFDHSOCld93HmXq5qF1TZzFptHOWhEkgsEz2D9gPxNiwUQ43KkpHPBCf WOhg== X-Received: by 10.112.52.225 with SMTP id w1mr10422060lbo.22.1396113220928; Sat, 29 Mar 2014 10:13:40 -0700 (PDT) Original-Sender: =?UTF-8?Q?Timo_Ter=C3=A4s?= In-Reply-To: <20140329170032.GJ8221@example.net> X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.20; i486-alpine-linux-uclibc) Xref: news.gmane.org gmane.linux.lib.musl.general:4742 Archived-At: On Sat, 29 Mar 2014 17:00:32 +0000 u-igbb@aetey.se wrote: > Background: > Compiling a native musl-based toolchain for ia32 on Linux 2.6+. > Using the standalone dynamic loader mode. > (The latter seems to lead to a quite limited heap space, by kernel > behaviour/design) > > I encounter out of memory errors. A look at the malloc source does not > find any fallback to mmap when heap is exhausted. What would you > suggest as a suitable approach to make it work? > > Somebody has possibly already encountered and solved this with musl? Yes, been there done that. I patched kernel. The thread that follows on sending the patch upstream is e.g. at: https://groups.google.com/forum/#!msg/linux.kernel/mOf1EWrrhZc/bl96BAE4fyQJ Also using grsec kernel would fix the issue mostly, since grsec creates "better" memory layout for PIE binaries. > I see also reports about a related out of memory problem with > pae-executables which means a solution might help many musl users. > > The other standard libraries I am using (glibc, uclibc) seem to > happily switch to allocation from mmap() when the heap is full. I > understand that this costs some code and performance but a breakup is > no good either. > > Any ideas? Maintaining and using an external libmalloc or substituting > malloc in musl? This feels like quite a burden... > (Would musl internal calls to malloc notice the external library > and resolve to its entry points instead of the internal malloc?) musl does not support external malloc. musl internal calls to malloc() are not overridable. I think you need to fix kernel. Rewrite allocator in musl. Or add the fallback code to mmap - but dalias said it's "hard". Perhaps still should be still reconsidered. - Timo