How to get base address of heap arenas

How to get base address of heap arenas

[sva sva]

[-- Attachment #1: Type: text/plain, Size: 269 bytes --]

I am writing a heap walk program in C and would like to know if there is
anything like the concept of arenas in musl. Basically, I need to have a
pointer to the base address of all my allocated heaps. Unfortunately
inspecting the musl code I found none.

Thanks

Vahid

[-- Attachment #2: Type: text/html, Size: 359 bytes --]

Re: How to get base address of heap arenas

[Szabolcs Nagy]

* sva sva <azharivs@gmail.com> [2019-05-30 08:59:46 -0400]:
> I am writing a heap walk program in C and would like to know if there is
> anything like the concept of arenas in musl. Basically, I need to have a
> pointer to the base address of all my allocated heaps. Unfortunately
> inspecting the musl code I found none.

there is no such concept as "heap arena" visible to user
code, so almost surely you don't "need a pointer to the
base address" of it.

try to describe what exactly you want to do (not in terms
of libc internals, but in terms that make sense for user
code)

(do you want to get all memory mappings? -> try /proc/self/maps)
(do you want to track malloc behaviour? -> try malloc interposition)
(etc)

Re: How to get base address of heap arenas

[Rich Felker]

On Thu, May 30, 2019 at 03:27:24PM +0200, Szabolcs Nagy wrote:
> * sva sva <azharivs@gmail.com> [2019-05-30 08:59:46 -0400]:
> > I am writing a heap walk program in C and would like to know if there is
> > anything like the concept of arenas in musl. Basically, I need to have a
> > pointer to the base address of all my allocated heaps. Unfortunately
> > inspecting the musl code I found none.
> 
> there is no such concept as "heap arena" visible to user
> code, so almost surely you don't "need a pointer to the
> base address" of it.
> 
> try to describe what exactly you want to do (not in terms
> of libc internals, but in terms that make sense for user
> code)
> 
> (do you want to get all memory mappings? -> try /proc/self/maps)
> (do you want to track malloc behaviour? -> try malloc interposition)
> (etc)

To further clarify, the current allocator implementation has no global
view of "the heap". It sees only free chunks and the headers or
footers of the immediately adjacent-in-address-space allocated chunks.
It's likely that the future replacement will have global tracking that
further helps ensure integrity against heap corruption attacks, but it
won't be a public API or something necessarily stable between
versions. If you want a debugging malloc, you need to interpose one;
for the past few releases, musl has supported malloc interposition.

Rich

Re: How to get base address of heap arenas

[sva sva]

[-- Attachment #1: Type: text/plain, Size: 2398 bytes --]

Indeed your replies verified my understanding.

Here's what I want to do:
I have a C library that does malloc interposition and adds signatures to
header and footer of chunks by allocating extra bytes before and after it.
It also does a number of other things. Then I have some C code which is
supposed to walk the entire heap to that particular process and spit out
all the heap signatures making sure there are no scribbled ones.

What I am doing right now is that whenever I get a malloc/calloc/etc.,
which is of course overloaded, I follow the returned pointer all the way
back to reach the top chunk (prev size = 0). This would fail when I get
chunks in different memory regions that are not contiguous. So I need to
somehow keep track of the contiguous heap areas and get the top chunk for
all of those to start my heap walk. Last thing I want to do is to patch
musl's libc.

Thanks

Vahid

On Thu, May 30, 2019 at 10:02 AM Rich Felker <dalias@libc.org> wrote:

> On Thu, May 30, 2019 at 03:27:24PM +0200, Szabolcs Nagy wrote:
> > * sva sva <azharivs@gmail.com> [2019-05-30 08:59:46 -0400]:
> > > I am writing a heap walk program in C and would like to know if there
> is
> > > anything like the concept of arenas in musl. Basically, I need to have
> a
> > > pointer to the base address of all my allocated heaps. Unfortunately
> > > inspecting the musl code I found none.
> >
> > there is no such concept as "heap arena" visible to user
> > code, so almost surely you don't "need a pointer to the
> > base address" of it.
> >
> > try to describe what exactly you want to do (not in terms
> > of libc internals, but in terms that make sense for user
> > code)
> >
> > (do you want to get all memory mappings? -> try /proc/self/maps)
> > (do you want to track malloc behaviour? -> try malloc interposition)
> > (etc)
>
> To further clarify, the current allocator implementation has no global
> view of "the heap". It sees only free chunks and the headers or
> footers of the immediately adjacent-in-address-space allocated chunks.
> It's likely that the future replacement will have global tracking that
> further helps ensure integrity against heap corruption attacks, but it
> won't be a public API or something necessarily stable between
> versions. If you want a debugging malloc, you need to interpose one;
> for the past few releases, musl has supported malloc interposition.
>
> Rich
>

[-- Attachment #2: Type: text/html, Size: 3076 bytes --]

Re: How to get base address of heap arenas

[sva sva]

[-- Attachment #1: Type: text/plain, Size: 2772 bytes --]

Correction: I currently only follow the first malloc back to the top chunk
and would NOT like to follow every malloc due to the obvious performance
impact. This, however, fails when there are multiple non contiguous heap
regions.

On Thu, May 30, 2019 at 1:49 PM sva sva <azharivs@gmail.com> wrote:

> Indeed your replies verified my understanding.
>
> Here's what I want to do:
> I have a C library that does malloc interposition and adds signatures to
> header and footer of chunks by allocating extra bytes before and after it.
> It also does a number of other things. Then I have some C code which is
> supposed to walk the entire heap to that particular process and spit out
> all the heap signatures making sure there are no scribbled ones.
>
> What I am doing right now is that whenever I get a malloc/calloc/etc.,
> which is of course overloaded, I follow the returned pointer all the way
> back to reach the top chunk (prev size = 0). This would fail when I get
> chunks in different memory regions that are not contiguous. So I need to
> somehow keep track of the contiguous heap areas and get the top chunk for
> all of those to start my heap walk. Last thing I want to do is to patch
> musl's libc.
>
> Thanks
>
> Vahid
>
> On Thu, May 30, 2019 at 10:02 AM Rich Felker <dalias@libc.org> wrote:
>
>> On Thu, May 30, 2019 at 03:27:24PM +0200, Szabolcs Nagy wrote:
>> > * sva sva <azharivs@gmail.com> [2019-05-30 08:59:46 -0400]:
>> > > I am writing a heap walk program in C and would like to know if there
>> is
>> > > anything like the concept of arenas in musl. Basically, I need to
>> have a
>> > > pointer to the base address of all my allocated heaps. Unfortunately
>> > > inspecting the musl code I found none.
>> >
>> > there is no such concept as "heap arena" visible to user
>> > code, so almost surely you don't "need a pointer to the
>> > base address" of it.
>> >
>> > try to describe what exactly you want to do (not in terms
>> > of libc internals, but in terms that make sense for user
>> > code)
>> >
>> > (do you want to get all memory mappings? -> try /proc/self/maps)
>> > (do you want to track malloc behaviour? -> try malloc interposition)
>> > (etc)
>>
>> To further clarify, the current allocator implementation has no global
>> view of "the heap". It sees only free chunks and the headers or
>> footers of the immediately adjacent-in-address-space allocated chunks.
>> It's likely that the future replacement will have global tracking that
>> further helps ensure integrity against heap corruption attacks, but it
>> won't be a public API or something necessarily stable between
>> versions. If you want a debugging malloc, you need to interpose one;
>> for the past few releases, musl has supported malloc interposition.
>>
>> Rich
>>
>

[-- Attachment #2: Type: text/html, Size: 3674 bytes --]

Re: How to get base address of heap arenas

[Markus Wichmann]

On Thu, May 30, 2019 at 01:49:10PM -0400, sva sva wrote:
> Indeed your replies verified my understanding.
>
> Here's what I want to do:
> I have a C library that does malloc interposition and adds signatures to
> header and footer of chunks by allocating extra bytes before and after it.
> It also does a number of other things. Then I have some C code which is
> supposed to walk the entire heap to that particular process and spit out
> all the heap signatures making sure there are no scribbled ones.
>

You know, if you just want to debug your dynamic memory, you could just
use valgrind, or electric fence.

As for your problem, if you are already allocating more memory, why not
enqueue all allocated chunks into global lists? This way, the actual
heap layout will not matter to you, your analyzer can always find all
allocated chunks.

Musl's malloc tries to use the brk() heap if it can, but will resort to
mmap() if brk() fails for any reason. Therefore the actual chunks may be
discontiguous. Searching memory for random numbers does not seem like a
good idea.

Ciao,
Markus

PS: Please don't top-post.