Re: stdio self-synchronized destruction: does it need fixing?

[Rich Felker <dalias@aerifal.cx>]

On Mon, Dec 10, 2012 at 11:57:53PM +0100, Szabolcs Nagy wrote:
> * Rich Felker <dalias@aerifal.cx> [2012-12-10 13:05:09 -0500]:
> > memory. The allocation of FILE structues is always performed by libc,
> > and always happens via malloc with a small-size allocation, which
> > means the memory is managed as part of the heap and never unmapped
> > once it's mapped. Thus, as far as I can tell, the worst that can
> > happen is a read-only access to memory no longer owned by the FILE,
> 
> at least write a comment there that the invalid read is known

This sounds like a perfect solution for now.

> (btw at some point someone may rewrite malloc so small allocations
> can go to mmapped areas as well which may be reclaimed..)

I would like to add the ability to obtain more heap memory from mmap,
but I think unmapping unused parts has more cons than pros. The only
benefit is that it reclaims the virtual address space so that the
kernel can use it for honoring future mmap requests. The disadvantage
is that it fragments the heap; once a "hole" has been created in the
heap, it's impossible to put it back and merge the adjacent free
regions into one large free region in O(1) time.

Of course if you're talking about a _whole_ supplemental mmap zone
becoming free, then it would be safe to unmap it, and this may be a
worthwhile case to handle. In particular, one way of doing
supplemental heap via mmap would be to make each map roughly the same
size as the threshold for releasing it back to the kernel via
mprotect/mmap zeroing. The beginning and end of the region could have
specially-flagged chunks, and then, after merging on free(), if both
the previous and next neighbor are flagged as the mmap region
endpoints, the whole region could be unmapped rather than adding it
back to the free list.

The alternative would be to do supplemental mmap regions in much
larger increments, maybe on the order of 16MB, initially with
PROT_NONE, and grow the PROT_READ|PROT_WRITE portion like brk() does
for the main heap. Then the same code that releases parts of the main
brk heap via zeroing could also work well here. I think this
alternative approach would result in less fragmentation for malloc,
but more chance of filling up virtual address space and preventing
mmap from working.

Rich