From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/12985 Path: news.gmane.org!.POSTED!not-for-mail From: Rich Felker Newsgroups: gmane.linux.lib.musl.general Subject: Re: Changing MMAP_THRESHOLD in malloc() implementation. Date: Wed, 4 Jul 2018 10:54:03 -0400 Message-ID: <20180704145403.GP1392@brightrain.aerifal.cx> References: <20180703144331.GL1392@brightrain.aerifal.cx> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: blaine.gmane.org 1530715934 19914 195.159.176.226 (4 Jul 2018 14:52:14 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Wed, 4 Jul 2018 14:52:14 +0000 (UTC) User-Agent: Mutt/1.5.21 (2010-09-15) To: musl@lists.openwall.com Original-X-From: musl-return-13001-gllmg-musl=m.gmane.org@lists.openwall.com Wed Jul 04 16:52:10 2018 Return-path: Envelope-to: gllmg-musl@m.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by blaine.gmane.org with smtp (Exim 4.84_2) (envelope-from ) id 1faj8c-00054r-6w for gllmg-musl@m.gmane.org; Wed, 04 Jul 2018 16:52:10 +0200 Original-Received: (qmail 5827 invoked by uid 550); 4 Jul 2018 14:54:18 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Original-Received: (qmail 5802 invoked from network); 4 Jul 2018 14:54:17 -0000 Content-Disposition: inline In-Reply-To: Original-Sender: Rich Felker Xref: news.gmane.org gmane.linux.lib.musl.general:12985 Archived-At: On Wed, Jul 04, 2018 at 10:54:50AM +0530, ritesh sonawane wrote: > Thank you very much for instant reply.. > > Yes sir it is wasting memory for each shared library. But memory wastage > even worse when program > requesting memory with size more than 224KB(threshold value). > > ->If a program requests 1GB per request, it can use 45GB at the most. > ->If a program requests 512MB per request, it can use 41.5GB at the most. > ->If a program requests 225KB per request, it can use about 167MB at the > most. How does this happen? The behavior you should see is just rounding up of the request to a multiple of the page size, not scaling of the request. Maybe I don't understand what you're saying here. > As we ported musl-1.1.14 for our architecture, we are bounded to make > change in same base code. > we have increased MMAP_THRESHOLD to 1GB and also changes the calculation > for bin index . > after that observed improvement in memory utilization. i.e for size 225KB > memory used is 47.6 GB. > > But now facing problem in multi threaded application. As we haven't changed > the function pretrim() > because there are some hard coded values like '40' and '3' used and unable > to understand how > these values are decided ..? > > static int pretrim(struct chunk *self, size_t n, int i, int j) > { > size_t n1; > struct chunk *next, *split; > > /* We cannot pretrim if it would require re-binning. */ > if (j < 40) return 0; > if (j < i+3) { > if (j != 63) return 0; > n1 = CHUNK_SIZE(self); > if (n1-n <= MMAP_THRESHOLD) return 0; > } else { > n1 = CHUNK_SIZE(self); > } > ..... > ..... > ....... > } > > can we get any clue how these value are decided, it will be very > helpful for us. You can just disable use of pretrim, at a performance cost -- it's an optimization to avoid shuffling in and out of bins when the remainder will go back in the same bin. Unfortunately you're right that the magic numbers in pretrim aren't immediately clear in their purpose so I'd have to take some time to look at what they're doing. Some of the checks might be redundant (just a short circuit to save time). Rich