Re: [musl] Failing assertions in get_meta

[Rich Felker <dalias@libc.org>]

On Tue, Jul 20, 2021 at 01:19:58PM -0600, Christopher Hodgkins wrote:
> Hi all,
> I'm working on a modified version of musl for some research.
> We replaced the typical memory provisioning methods (mmap/brk)
> with our own versions which change the backing of the provided memory,
> but maintain the same semantics from the perspective of the caller,
> except that the program break is no longer adjacent to the rest of the
> mapped binary.
> 
> We have been encountering various failed assertions in mallocng during
> debugging
> in the get_meta function. Specifically, we see the assertion at line 141 in
> meta.h (meta == mem->base)
> failing when free()-ing memory previously allocated with malloc(),

If this assertion fails, it means that the group header's pointer to
the out-of-band metadata is either pointing to the wrong place (e.g.
because it was overwritten by some out-of-bounds access to allocated
memory) or points to the correct metadata location but the structure
at the location has been overwritten.

> and the assertion at line 131 ( !((uintptr_t)p & 15) ) failing for calls to
> posix_memalign
> with an alignment of 16 and a length of 704. Could our changes in the
> backing of provisioned memory cause this?

Perhaps, if you're not following the semantics of mmap (that it yield
page-aligned, page-granular memory) or not honoring the obligation to
faithfully store data written to the memory. Without knowing the
specifics of your modification it's hard to say.

> Related question: We are aware of (and have disabled) the "donation"
> functionality in the loader.
> Are there other out-of-band (i.e. not mmap/sbrk) sources of memory used by
> mallocng?

No.

> The errors are caused in some way by our changes, but we haven't changed
> anything inside the allocator,
> except to point the brk() macro to our own implementation rather than the
> syscall.

You could try disabling brk entirely by making it never return a value
equal to its argument, as in:

#define brk(p) ((p)-1)

This would help determine if your problem is caused by your brk or
something else.

Rich