From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/2309 Path: news.gmane.org!not-for-mail From: Rich Felker Newsgroups: gmane.linux.lib.musl.general Subject: debloating data, bss Date: Sun, 18 Nov 2012 16:03:53 -0500 Message-ID: <20121118210353.GA4844@brightrain.aerifal.cx> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1353272648 5697 80.91.229.3 (18 Nov 2012 21:04:08 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 18 Nov 2012 21:04:08 +0000 (UTC) To: musl@lists.openwall.com Original-X-From: musl-return-2310-gllmg-musl=m.gmane.org@lists.openwall.com Sun Nov 18 22:04:19 2012 Return-path: Envelope-to: gllmg-musl@plane.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by plane.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1TaC2I-000056-Tq for gllmg-musl@plane.gmane.org; Sun, 18 Nov 2012 22:04:15 +0100 Original-Received: (qmail 25912 invoked by uid 550); 18 Nov 2012 21:04:04 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: Original-Received: (qmail 25904 invoked from network); 18 Nov 2012 21:04:04 -0000 Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Xref: news.gmane.org gmane.linux.lib.musl.general:2309 Archived-At: Hi all, I've been doing a bit of checking for unneeded bloat, and here's what I've found in data & bss: 1045 0 288 1333 535 mntent.o (ex lib/libc.a) This is simply a line buffer. It seems to me we could instead use getline or fgetln, but for the latter some consideration is needed to determine whether its semantics suffice. 665 0 32 697 2b9 mq_notify.o (ex lib/libc.a) This is purely gcc being stupid and putting static const char[32] into bss rather than text (wasting 32 bytes of writable memory to save 32 bytes on disk). Since the buffer is junk, we could just use "char[32]" (uninitialized); that would shrink the code but would result in warnings. Or, we could use a pointer to any static object or code of at least 32 bytes in size. 86 0 544 630 276 gethostbyaddr.o (ex lib/libc.a) 78 0 544 622 26e gethostbyname2.o (ex lib/libc.a) Some ugly gigantic buffers for results. Since getaddrinfo requires dynamic allocation anyway, it would be reasonable to dynamically allocate these too; it would not be introducing a failure case that did not already previously exist. 6 0 512 518 206 res_state.o (ex lib/libc.a) This is pure junk; it's just there to satisfy broken programs that try to peek/poke at the resolver state. I wonder if we could make it smaller without breaking anything. 908 160 12 1080 438 random.o (ex lib/libc.a) Unfortunately I think random really does have that much state... 43 0 128 171 ab sigisemptyset.o (ex lib/libc.a) This is another case of gcc stupidly putting uninitialized static const in bss instead of text. I have a better workaround anyway though; anyway this code needs to be fixed because it's comparing the while 1024-bit bit-array even though we treat all but the first 64/128 bits as padding now. 209 0 8192 8401 20d1 pthread_key_create.o (ex lib/libc.a) I'm considering replacing pthread_key_create with a new implementation that makes a fake DSO with TLS instead of having the pthread thread-specific data being part of the main thread block. Aside from this, the main issue that's making libc.so's dirty-page cost so high is that the bss isn't sorted; most of bss is unused in most programs, but because the commonly-used stuff isn't grouped together, several pages end up dirty. With this in mind, I'm considering one of the following 3 approaches to get the commonly-used data all together in one page: 1. Explicitly initialize everything that's always-used, so it ends up in .data rather than .bss, and thus on the first page. 2. Reorder object files in the linking so that the bloated junk is all at the end. 3. Find a way to get the linker to sort it for us, possibly with alignment and alignment-based sorting. With the above changes, I think we should be able to cut 2-3 pages of commit charged off of libc.so and drop the minimum dirty pages for dynamic linking from 20k (5 pages) to 12k (3 pages, only one of which is in libc.so; the others are the main app's data and stack). Major work on debloating will probably not begin until after the next release. Rich