Re: Re: Removing sbrk and brk

[Rich Felker <dalias@aerifal.cx>]

It seems like I've made some incorrect statements in this thread which
the OpenBSD folks weren't too happy about, so now that I'm more clear
on the details I'd like to correct those mistakes for the record...

On Tue, Jan 07, 2014 at 11:06:27AM -0500, Rich Felker wrote:
> On Tue, Jan 07, 2014 at 09:43:26AM +0000, Thorsten Glaser wrote:
> > Rich Felker <dalias <at> aerifal.cx> writes:
> > 
> > > This seems to be optional behavior; using guard pages with all
> > > allocations would blow up memory usage several thousand times and
> > 
> > No, they aren’t accessible, so the kernel (should) never maps them
> > to any real RAM.
> 
> The point is that even a 1-byte allocation (in fairness, that would be
> rounded up to at least 16 bytes) ends up consuming a whole page and
> thus 4k of storage, plus another 4k of virtual address space for the
> guard page. On MIPS it may be even worse (16k pages are required on
> some hardwasre).

This is both incorrect and misleading. I was under the impression that
omalloc had a (non-default) option to use guard pages for all
allocations, even sub-page-size ones. This is not the case, and my
only excuse is that I wrote the above a day or two after last reading
the source and seeing that it had configurable options, but without
taking the time to confirm their nature.

So in reality, guard pages are only used for allocations that take up
at least a whole page, and so from what I can tell now, the maximum
overhead is something like 100% (not 25600%) committed memory and 200%
virtual address space and occurs for allocations just larger than one
page (where you end up needing one extra page for the rest of the
storage, and one guard page).

This is comparable to the overhead for small allocations (less than a
page), which are rounded up to a power-of-two size and thus have a
worst-case overhead of near-100%.

So there's no pathologically huge over-allocation going on, just
moderate overhead, which the OpenBSD folks have claimed (and I believe
them on this) has caught serious bugs in lots of major software.

Hope this clears things up.

Rich