mailing list of musl libc
 help / color / mirror / code / Atom feed
From: Rich Felker <dalias@libc.org>
To: musl@lists.openwall.com
Subject: Re: [musl] mallocng progress and growth chart
Date: Fri, 15 May 2020 23:29:01 -0400	[thread overview]
Message-ID: <20200516032901.GO21576@brightrain.aerifal.cx> (raw)
In-Reply-To: <20200516002912.GN21576@brightrain.aerifal.cx>

On Fri, May 15, 2020 at 08:29:13PM -0400, Rich Felker wrote:
> The same thing happens at the next doubling for malloc(8192), and the
> same mitigation applies. However with that:
> 
>  9340:   3x10912  3x10912  3x10912  3x10912  3x9344   7x9344   ...
> 
> the coarse size classing is dubious because the size is sufficiently
> large that a 7->3 count reduction can be used, with the same count the
> coarsse class would have, but with a 28k rather than 32k mapping.
> 
> Unfortunately the decision here depends on knowing page size, which
> isn't constant at the point where it needs to be made. For integration
> with musl, page size is initially known even if it's variable, so we
> could possibly make a decision not to use coarse sizing based on that
> here, but standalone mallocng lacks that knowledge (page size isn't
> known until after first alloc_meta). This might could be reworked.
> There's a fairly small range of sizes that would benefit (larger ones
> are better off with individual mmap because page size quickly becomes
> "finer than" size classes), but the benefit seems fairly significant
> (not wasting an extra 1.3k each for the first 12 malloc(8192)'s) at
> the sizes where it helps.

It might just make sense to always disable coarse size classing
starting at this size. The absolute amount of over-allocation is
sufficiently high that it's probably not justified. On archs with
larger page size, more pages may be mapped (e.g. 7x9344 can't be
reduced to 3x if page size is over 4k, and can't be reduced to 5x if
page size is over 16k) but having too much memory mapped is generally
an expected consequence of ridiculous page sizes.

Another possibly horrible idea for dealing with exact page sizes at
low usage: pair groups of short length. Rather than needing to choose
between coarse classing 3x5440 (16k) or a 5x4672 (5 whole slots, 24k)
for a malloc(4095), at low useage we could create a pseudo-2x4672
where the second slot is truncated and contains a nested 1x3264 that's
only freeable together with the outer group. This relation always
works: if size class k is just under a power of two (k == 3 mod 4),
classes k+1 and k-1 add up to just under 2 times class k. (This
follows from n/5 + 2n/7 == (7n+10n)/35 == 17n/35 <= n/2 == 2*n/4,
where n/5, 2n/7, and n/4 are the sizes of class k-1, k+1, and k,
respectively.)

This gives a strategy that always works for allocating very-low-count
of arbitrary size classes, as long as we're willing to allocate a slot
for the complementary size at the same time. And in some ways it's
nicer than coarse classing -- rather than overallocating the requested
slot in hopes that the grouped slots will be useful for larger
allocations too, it allocates a smaller complementary pair in hopes
that the complementary size will be useful.


  reply	other threads:[~2020-05-16  3:29 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-05-10 18:09 Rich Felker
2020-05-11 17:51 ` Rich Felker
2020-05-16  0:29 ` Rich Felker
2020-05-16  3:29   ` Rich Felker [this message]
2020-05-17  3:30     ` Rich Felker
2020-05-18 18:53       ` Rich Felker
2020-05-25 15:45         ` Pirmin Walthert
2020-05-25 17:54           ` Rich Felker
2020-05-25 18:13             ` Pirmin Walthert
2020-06-04  7:04               ` Pirmin Walthert
2020-06-04 15:11                 ` Rich Felker
2020-05-18 18:35   ` Rich Felker

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200516032901.GO21576@brightrain.aerifal.cx \
    --to=dalias@libc.org \
    --cc=musl@lists.openwall.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/musl/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).