From: Rich Felker <dalias@libc.org>
To: baiyang <baiyang@gmail.com>
Cc: musl@lists.openwall.com
Subject: Re: Re: [musl] The heap memory performance (malloc/free/realloc) is significantly degraded in musl 1.2 (compared to 1.1)
Date: Mon, 19 Sep 2022 14:15:56 -0400 [thread overview]
Message-ID: <20220919181556.GT9709@brightrain.aerifal.cx> (raw)
In-Reply-To: <202209200132289145679@gmail.com>
On Tue, Sep 20, 2022 at 01:32:31AM +0800, baiyang wrote:
> Hi Rich,
>
> Thanks for your reply.
>
> > Unless you have an application that's explicitly using
> > malloc_usable_size all over the place, it's highly unlikely that this
> > is the cause of your real-world performance problems.
>
> 1. Yes, we have a real scenario where `malloc_usable_size` is called
> frequently: we need to optimize the realloc experience. We add an
> extra parameter to realloc - minimalCopyBytes: it represents the
> actual size of data that needs to be copied after fallback to
> malloc-copy-free mode. We will judge whether to call realloc or
> complete malloc-memcpy-free by ourself based on factors such as the
> size of the data that realloc needs to copy (obtained through
> `malloc_usable_size`), the size that we actually need to copy when
> we doing malloc-memcpy-free ourself (minimalCopyBytes) and the
> chance of merging chunks (small blocks) or mremap (large blocks) in
> the underlayer realloc. So, this is a real scenario, we need to call
> `malloc_usable_size` frequently.
Is there a reason you're relying on an unreliable and nonstandard
function (malloc_usable_size) to do this rather than your program
keeping track of its own knowledge of the allocated size? This is what
the C language expects you to do. For example if you have a structure
that contains a pointer to a dynamically sized buffer, normally you
store the size in a size_t member right next to that pointer, allowing
you to make these kind of decisions without having to probe anything.
> 2. As I mentioned before, this isn't just a problem with
> `malloc_usable_size`, since we actually include a full
> `malloc_usable_size` procedure in both `realloc` and `free`, it
> actually slows down The speed of other calls such as `free` and
> `realloc`. So this problem actually slows down not only the
> `malloc_usable_size` call itself, but also the realloc and free
> calls.
If this is affecting you too, that's a separate issue. But I can't
tell from what you've reported so far whether you're just claiming
this on a theoretical basis or whether you're actually experiencing
unacceptable performance.
next prev parent reply other threads:[~2022-09-19 18:16 UTC|newest]
Thread overview: 68+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-09-19 7:53 baiyang
2022-09-19 11:08 ` Szabolcs Nagy
2022-09-19 12:36 ` Florian Weimer
2022-09-19 13:46 ` Rich Felker
2022-09-19 13:53 ` James Y Knight
2022-09-19 17:40 ` baiyang
2022-09-19 18:14 ` Szabolcs Nagy
2022-09-19 18:40 ` baiyang
2022-09-19 19:07 ` Gabriel Ravier
2022-09-19 19:21 ` Rich Felker
2022-09-19 21:02 ` Gabriel Ravier
2022-09-19 21:47 ` Rich Felker
2022-09-19 22:31 ` Gabriel Ravier
2022-09-19 22:46 ` baiyang
2022-09-19 20:46 ` Nat!
2022-09-20 8:51 ` Szabolcs Nagy
2022-09-20 0:13 ` James Y Knight
2022-09-20 0:25 ` baiyang
2022-09-20 0:38 ` Rich Felker
2022-09-20 0:47 ` baiyang
2022-09-20 1:00 ` Rich Felker
2022-09-20 1:18 ` baiyang
2022-09-20 2:15 ` Rich Felker
2022-09-20 2:35 ` baiyang
2022-09-20 3:28 ` Rich Felker
2022-09-20 3:53 ` baiyang
2022-09-20 5:41 ` Rich Felker
2022-09-20 5:56 ` baiyang
2022-09-20 12:16 ` Rich Felker
2022-09-20 17:21 ` baiyang
2022-09-20 8:33 ` Florian Weimer
2022-09-20 13:54 ` Siddhesh Poyarekar
2022-09-20 16:59 ` James Y Knight
2022-09-20 17:34 ` Szabolcs Nagy
2022-09-20 19:53 ` James Y Knight
2022-09-24 8:55 ` Fangrui Song
2022-09-20 17:39 ` baiyang
2022-09-20 18:12 ` Quentin Rameau
2022-09-20 18:19 ` Rich Felker
2022-09-20 18:26 ` Alexander Monakov
2022-09-20 18:35 ` baiyang
2022-09-20 20:33 ` Gabriel Ravier
2022-09-20 20:45 ` baiyang
2022-09-21 8:42 ` NRK
2022-09-20 18:37 ` Quentin Rameau
2022-09-21 10:15 ` [musl] " 王志强
2022-09-21 16:11 ` [musl] " 王志强
2022-09-21 17:15 ` [musl] " Rich Felker
2022-09-21 17:58 ` Rich Felker
2022-09-22 3:34 ` [musl] " 王志强
2022-09-22 9:10 ` [musl] " 王志强
2022-09-22 9:39 ` [musl] " 王志强
2022-09-20 17:28 ` baiyang
2022-09-20 17:44 ` Siddhesh Poyarekar
2022-10-10 14:13 ` Florian Weimer
2022-09-19 13:43 ` Rich Felker
2022-09-19 17:32 ` baiyang
2022-09-19 18:15 ` Rich Felker [this message]
2022-09-19 18:44 ` baiyang
2022-09-19 19:18 ` Rich Felker
2022-09-19 19:45 ` baiyang
2022-09-19 20:07 ` Rich Felker
2022-09-19 20:17 ` baiyang
2022-09-19 20:28 ` Rich Felker
2022-09-19 20:38 ` baiyang
2022-09-19 22:02 ` Quentin Rameau
2022-09-19 20:17 ` Joakim Sindholt
2022-09-19 20:33 ` baiyang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220919181556.GT9709@brightrain.aerifal.cx \
--to=dalias@libc.org \
--cc=baiyang@gmail.com \
--cc=musl@lists.openwall.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.vuxu.org/mirror/musl/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).