Best Regards BaiYang baiyang@gmail.com http://i.baiy.cn |
From: Rich FelkerDate: 2022-09-20 11:28To: baiyangCC: muslSubject: Re: Re: [musl] The heap memory performance (malloc/free/realloc) is significantly degraded in musl 1.2 (compared to 1.1)On Tue, Sep 20, 2022 at 10:35:02AM +0800, baiyang wrote:> > You seem to think that if the group stride was 8100, calling realloc might memcpy up to 8100 bytes. This is not the case.>> Yes, I already understood that mallocng would only memcpy 6600 bytes> when I was told that malloc_usable_size will return the size> requested by the user.>> But AFAIK, many other malloc implementations basically don't keep> 6600 bytes of data. So they're actually going to memcpy the 8100> bytes.You have made up a problem that does not exist. Specifically, at leastas far as I can tell, no such implementation exists.The ones that return some value larger than the requested size arereturning "the requested size, rounded up to a multiple of 16" orsimilar. Not "the requested size plus 1500 bytes". For thedlmalloc-style allocators, this is because they do not have uniformslots for size classes; they have arbitrary splits, and only careabout aligning start/end on a properly aligned boundary. And for moreslab-like allocators, they all keep track of at least a coarse "actualsize" specifically so that realloc does not gratuitously copy largeamounts of unnecessary data (among other reasons).> > You also seem to be under the impression that the work to determine> > that the size was 6600 and not 8100 is where most (or at least a> > significant portion of) the time is spent. This is also not the case.> > The majority of the metadata processing time is chasing pointers back> > to the out-of-band metadata, validating it, validating that it> > round-trips back, and validating various other things. Some of these> > could in principle be omitted at the cost of loss-of-hardening.>> Yes, according to my previous understanding (which seems wrong now),> since other malloc_usable_size implementations that directly return> 8100 (the actual allocated size class length)They don't return 8100. They return something like 6608 or 6624.> such as tcmalloc are> all very fast, so I can only understand that mallocng is so much> slower than them because it has to return 6600, not 8100.This does not follow at all. tcmalloc is fast because it does not haveglobal consistency, does not have any notable hardening, and (see thename) keeps large numbers of freed slots *cached* to reuse, therebyusing lots of extra memory. Its malloc_usable_size is not fast becauseof returning the wrong value, if it even does return the wrong value(I have no idea). It's fast because they store the size in-band rightnext to the allocated memory and trust that it's valid, rather thancomputing it from out-of-band metadata that is not subject tofalsification unless the attacker already has nearly full control ofexecution.> Apart from> this difference, there is no reason it is slower than other> implementations of malloc_usable_size as I understand it.Comparing the speed of malloc_usable_size is utterly pointless. Thatis not where the time is spent in any real-world load that's notgratuitously doing stupid things. I imagine tcmalloc's is faster, forthe reasons explained above (which have nothing to do with whether thevalue returned is the exact size you requested). But this has nothingto do with why the overall performance is higher.> If this is not the main reason, can we speed up this algorithm with> the help of a fast lookup table mechanism like tcmalloc? As I said> before, this not only greatly increases the performance of> malloc_usable_size , but also the performance of realloc and free .No, because the fundamental difference is where you're storing thedata, and the *whole point* of mallocng is that we're storing it in aplace where it's not subject to attack via overflows from otherobjects, UAF/DF, etc. This makes it somewhat costlier (but still veryreasonable) to obtain.If you have a particular application where you actually want to tradesafety and low memory usage for performance, you're perfectly free tolink that application to whatever malloc implementation you like. muslhas supported doing that since 1.1.20. It makes no sense to do thissystem-wide. You would just increase memory usage and attack surfacedrastically with nothing to show for it, since the vast majority ofprograms *don't care* how long malloc takes.Rich