Re: New malloc - first preview

[Rich Felker <dalias@libc.org>]

Work on the new malloc is well underway, and I have a draft version
now public at:

https://github.com/richfelker/mallocng-draft

Some highlights:

- Smallest size-class now has 12 of 16 bytes usable for storage,
  compared to 8 of 16 (32-bit archs) or 16 of 32 (64-bit archs) with
  the old malloc, plus 1.5 bytes (32-bit) or 2.5 bytes (64-bit) of
  out-of-band metadata. This overhead (5.5 or 6.5 bytes per
  allocation) is uniform across all sizes.

- Small-size allocation granularity/alignment is now 16-byte on both
  32-bit and 64-bit archs. No more wasting 64 bytes for a 36-byte
  structure.

- Metadata defining heap state is all out-of-band and protected below
  by guard pages, where it's not easily reachable through typical
  attack vectors.

- Freed memory is quickly returnable to the kernel.

- Common path malloc and free (asymptotically should be ~95% of calls
  under high use) do not require exclusive locks -- readlock+atomic
  for malloc, pure-atomic for free.

- Size classes are not straight geometric progression but instead 1/7,
  1/6, and 1/5 of powers of two in between the whole power-of-two
  steps, optimizing fit in whole power-of-two numbers of pages.

- Code size is very close to the same as the old allocator, and may
  end up being smaller.

This is still early in development, and it won't be merged in musl
until the next release cycle, after the new year. There may be plenty
of bugs or suboptimal behavior. And I still need to write up the
design.

For anyone wanting to play around with the draft, it can be linked or
used by LD_PRELOAD. A dump_heap function is provided which can show
allocator state. In the status masks, a=available and f=freed; these
are distinct states to allow some control over interval to reuse of
freed memory (which gives increased safety against UAF/DF). The "_"
status is in-use.

Rich