[musl] upstreaming mallocng

[musl] upstreaming mallocng

[Rich Felker]

Finishing initial development of and upstreaming mallocng is the main
item for this release cycle, and I think we're close to there. The
main question is whether there will be any serious performance or
memory usage regressions relative to oldmalloc, which may be
problematic for some users. With that in mind, I'm considering
supporting a build-time choice of oldmalloc for at least the first
couple releases with mallocng, so that there's an easy path back for
users who are hit by any such regression that can't be solved
immediately.

If offering oldmalloc as an option, I think I'd like to apply the
patch for the runaway heap expansion issue first:

https://www.openwall.com/lists/musl/2019/04/12/4

This does cost some level of performance, but the loss should be
well-understood. Arguably this should have been applied at the time I
wrote it, but I wasn't really excited about the performance hit
without having something better to offer in place of it.

With that said, here are some of the things users should expect to see
from mallocng:

- More predictable behavior. Availability of memory should be
  influenced a lot less by history of allocator activity within the
  process's lifetime, especially on 64-bit archs where virtual address
  space fragmentation is not an issue. Non-essential patterns of
  ballooning fragmentation, like in glibc issue
  https://sourceware.org/bugzilla/show_bug.cgi?id=14581 which musl
  oldmalloc also exhibited, should also not happen anymore.

- Memory getting returned to the system, aggressively unless it
  bounces back and forth enough that we back off to avoid
  pathologically bad performance. Software that builds then frees
  large data structures should expect to mostly return to usage levels
  from before the allocation after it's all freed, rather than getting
  stuck at near-peak usage like what happens with dlmalloc type
  allocator.

- No more runaway heap expansion (see above thread).

- Differences in performance characteristics. Expected to perform
  better at high usage, anywhere between noticably better and
  noticably worse at low usage levels. (Where worse is severe, efforts
  will be made to mitigate it as much as possible, but only if it's
  reported.)

- Lower memory overhead at high usage. Ideally similar, but possibly
  somewhat higher, memory overhead at low usage. Tuning out all of the
  causes of gratuitous excess usage and over-eagar preallocation when
  the nominal usage is low, without clobbering performance, has been
  the big focus of recent development.

- Byte-accurate malloc_usable_size. This is necessary to make accesses
  out to malloc_usable_size offset not produce UB in the eyes of the
  compiler/fortify, which may generate traps (e.g. for memset(p, 42,
  malloc_usable_size(p))).

Now would be a great time for feedback on large-scale use of mallocng,
if anyone's up for it. You can put it in LD_PRELOAD via .profile or
even in init for system-wide use.

Rich