Re: Difficulty emulating F_DUPFD_CLOEXEC

[Rich Felker <dalias@aerifal.cx>]

On Sat, Mar 23, 2013 at 10:57:51PM -0400, Zvi Gilboa wrote:
> >One byte of data should not really matter, but in a sense it does
> >because you have to worry that the linking order might put it in the
> >middle of an otherwise-unused page and thus increase the memory usage
> >of the whole process by a page. This is actually an issue we need to
> >address -- libc.so's memory usage is a lot higher than it should be
> >right now, because the global vars that get touched by EVERY process
> >at startup time are spread out across several pages. Ordering things
> >so that they all end up in the same page would fix it, but I'm still
> >looking for the cleanest way to do that..
> 
> .... could a structure holding all global variables be considered a
> clean way to achieve this?  That would obviously mandate some code
> changes (adding the extern declaration to the relevant modules, and
> accordingly referring to global_vars.var_one instead of just to
> var_one), but at least "guarantee" that they are all kept together
> on the same page (or two or three)...

That's what we do for some data that's known to be needed in _all_
programs (see the "libc" structure in internal/libc.[ch]), but it's
problematic for static linking if the data might or might not be
needed, and you want to let the linker omit it when it's not needed.

So far I've seen two half-decent approaches:

1. Put all the almost-surely-touched data in .data rather than .bss.
The amount of .data is small enough that it all fits in one page
anyway. The trade-off here is that .data takes space on disk (even if
it's zero-filled), so you increase the size of libc.so and
static-linked binaries slightly on disk.

2. Reorder the linking/archiving of .o files so that the ones with
almost-surely-touched data appear first in the link order. The
trade-off here is that the source tree and/or Makefile becomes a bit
uglier.

Of these, I tend to prefer the first. However, it still might not be
enough for fully optimal layout. For example, we'd like the
likely-to-be-used parts of .bss to immediately follow .data (in the
same page): things like the buffers for stdin and stdout, which are
not _necessarily_ used, but fairly likely to be used. Other big .bss
buffers like pthread_key_create tables, which are unlikely to be used,
should be at the end in hopes that the last few pages of .bss won't be
dirtied at all.

Rich