From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-3.1 required=5.0 tests=DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 27802 invoked from network); 6 Jan 2022 20:37:32 -0000 Received: from mother.openwall.net (195.42.179.200) by inbox.vuxu.org with ESMTPUTF8; 6 Jan 2022 20:37:32 -0000 Received: (qmail 14172 invoked by uid 550); 6 Jan 2022 20:37:28 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 14134 invoked from network); 6 Jan 2022 20:37:27 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:message-id:mime-version:subject:from:to:cc; bh=lHGUlBxT7Bb3IguyWSAYISq1CPBhNY2G51++XG5ZKoQ=; b=gCanDLacjMtIdQacPBvYY/KS1DKCUycxxq15Qx7Uj4v/heDwMp9nqoCJLHLh/+lVts Y4AqGz0WBKjuTpFAFtCtsPv+A8ExOql5oABgmG/Auz4Ei7EV1S/9z8KvDKOKlXiQcjuu 3zHH4M+tU0+b+ns0Hn0K8Fxu8YfjaZoSPnCXQ+JubEsyjGVaGG2KdWyjU5IwRVUFyZ+c gjR5xAdSYXXxG6QphpIHG8otx7Nmp01OitlPRSYZp71u/lOh6HhhEQ+Ra/w3tDQEgoJO XwPk92wShk+jlRa/ayeI7TzSgxHNa1Z248KzXEZ5nnPmO7KgGrnFFjJceAtZBJSf+fqg gAVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=lHGUlBxT7Bb3IguyWSAYISq1CPBhNY2G51++XG5ZKoQ=; b=r6BRsoGSDEQM6UGRuvvBjGFTMA3CZk7gDL+Iy38D6h7tt94jWHUNzYA8uU8qrNFDNh Sn7q3ubhGNzrzijaEFK3O264zpITlS8SVAThqFJ6ZVfNLQ8Na09tifT7XyenZ1vNUuQL LDKksTwyfJ+iVRUTLnwlVAtY+B1PzG0wJtLDClZKw8afEWOdEGxVbHz7Rm7yZGmrJQOP 4Ys3RYLVs5QuXfet+8/3XCwQQfR235C1kP/vEnumW5wRdIuQHP0D+7siNrvhVULIdH0y m2J23BWLTrPEaUue35tQ0cNXbOp8wusL+4ErvVtgiIbp0ujur6KxDdJFeKNaPSGP8w43 EX3g== X-Gm-Message-State: AOAM533YsMrnZU8mZvOjC5U3/bgigQtKxOF6yVs2K4yfMVzu++1QxqxA El/AbT2dUvLJrI/SkixQqYFE6omEqzlFwy9QUUNtAaWyWeg6SBhZ1OlYv8tyB5V3/KXQbDFenRp 8qS7phaBnI3M0j9kymv4aCEoG0yXBg8nFbNkQNuEFDC86CK6hrHJN6CIV4d3qdnk= X-Google-Smtp-Source: ABdhPJygaCA/ywNlwkfwH+BPz2nebK7+5EaLPFzlz2GdZa6md6Ee+NCm6GJMuZgvAozzublyztiNDvYBSx8= X-Received: from walnut.mtv.corp.google.com ([2620:15c:211:202:a0f4:ac27:f00c:61fc]) (user=ccross job=sendgmr) by 2002:a25:9849:: with SMTP id k9mr20269854ybo.349.1641501434227; Thu, 06 Jan 2022 12:37:14 -0800 (PST) Date: Thu, 6 Jan 2022 12:37:09 -0800 Message-Id: <20220106203709.1525763-1-ccross@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.34.1.448.ga2b2bfdf31-goog From: Colin Cross To: musl@lists.openwall.com Cc: Colin Cross Content-Type: text/plain; charset="UTF-8" Subject: [musl] [PATCH] Add mallinfo2 and mallinfo glibc introduced mallinfo2 [1], which solves some of the arguments [2] against including mallinfo in musl by expanding the width of the returned counters from int to size_t. This patch implements mallinfo2 without requiring any additional metadata. It iterates through the meta_areas and metas in order to count mmap, large and small allocations, and produces ordblks, hblks, hblkhd, uordblks and fordblks values. Once mallinfo2 exists, it is trivial to implement mallinfo that caps the mallinfo2 outputs such that they fit in the int fields returned by mallinfo. [1] https://sourceware.org/git/?p=glibc.git;a=commit;h=e3960d1c57e57f33e0e846d615788f4ede73b945 [2] https://www.openwall.com/lists/musl/2018/01/17/2 --- The motivation for this patch is an attempt to use musl instead of glibc to build host tools used when building the Android platform and the tools that are distributed to app developers as part of the Android SDK. mallinfo is used in a variety of third-party code built as part of building Android, and tests and benchmarks in the Android tree. The implementation has been lightly tested with bionic's malloc.mallinfo and malloc.mallinfo2 tests, which verify that a variety of different allocation sizes result in an increase of the uordblks value by at least the usable size of the returned allocation. I can keep this as a local patch in Android if it is still not acceptable for musl. dynamic.list | 1 + include/malloc.h | 30 ++++++++++++++ src/malloc/mallocng/mallinfo.c | 73 ++++++++++++++++++++++++++++++++++ 3 files changed, 104 insertions(+) create mode 100644 src/malloc/mallocng/mallinfo.c diff --git a/dynamic.list b/dynamic.list index ee0d363b..84d13c53 100644 --- a/dynamic.list +++ b/dynamic.list @@ -14,6 +14,7 @@ memalign; posix_memalign; aligned_alloc; malloc_usable_size; +mallinfo; timezone; daylight; diff --git a/include/malloc.h b/include/malloc.h index 35f8b19c..98ba0100 100644 --- a/include/malloc.h +++ b/include/malloc.h @@ -18,6 +18,36 @@ void *memalign(size_t, size_t); size_t malloc_usable_size(void *); +struct mallinfo { + int arena; + int ordblks; + int smblks; + int hblks; + int hblkhd; + int usmblks; + int fsmblks; + int uordblks; + int fordblks; + int keepcost; +}; + +struct mallinfo mallinfo(void); + +struct mallinfo2 { + size_t arena; + size_t ordblks; + size_t smblks; + size_t hblks; + size_t hblkhd; + size_t usmblks; + size_t fsmblks; + size_t uordblks; + size_t fordblks; + size_t keepcost; +}; + +struct mallinfo2 mallinfo2(void); + #ifdef __cplusplus } #endif diff --git a/src/malloc/mallocng/mallinfo.c b/src/malloc/mallocng/mallinfo.c new file mode 100644 index 00000000..c60840b1 --- /dev/null +++ b/src/malloc/mallocng/mallinfo.c @@ -0,0 +1,73 @@ +#include +#include +#include + +#include "glue.h" +#include "meta.h" + +static void accumulate_meta(struct mallinfo2 *mi, struct meta *g) { + int sc = g->sizeclass; + if (sc >= 48) { + // Large mmap allocation + mi->hblks++; + mi->uordblks += g->maplen*4096; + mi->hblkhd += g->maplen*4096; + } else { + if (g->freeable && !g->maplen) { + // Small size slots are embedded in a larger slot, avoid double counting + // by subtracing the size of the larger slot from the total used memory. + struct meta* outer_g = get_meta((void*)g->mem); + int outer_sc = outer_g->sizeclass; + int outer_sz = size_classes[outer_sc]*UNIT; + mi->uordblks -= outer_sz; + } + int sz = size_classes[sc]*UNIT; + int mask = g->avail_mask | g->freed_mask; + int nr_unused = __builtin_popcount(mask); + mi->ordblks += nr_unused; + mi->uordblks += sz*(g->last_idx+1-nr_unused); + mi->fordblks += sz*nr_unused; + } +} + +static void accumulate_meta_area(struct mallinfo2 *mi, struct meta_area *ma) { + for (int i=0; inslots; i++) { + if (ma->slots[i].mem) { + accumulate_meta(mi, &ma->slots[i]); + } + } +} + +struct mallinfo2 mallinfo2() { + struct mallinfo2 mi = {0}; + + rdlock(); + struct meta_area *ma = ctx.meta_area_head; + while (ma) { + accumulate_meta_area(&mi, ma); + ma = ma->next; + } + unlock(); + + return mi; +} + +#define cap(x) ((x > INT_MAX) ? INT_MAX : x) + +struct mallinfo mallinfo() { + struct mallinfo mi = {0}; + struct mallinfo2 mi2 = mallinfo2(); + + mi.arena = cap(mi2.arena); + mi.ordblks = cap(mi2.ordblks); + mi.smblks = cap(mi2.smblks); + mi.hblks = cap(mi2.hblks); + mi.hblkhd = cap(mi2.hblkhd); + mi.usmblks = cap(mi2.usmblks); + mi.fsmblks = cap(mi2.fsmblks); + mi.uordblks = cap(mi2.uordblks); + mi.fordblks = cap(mi2.fordblks); + mi.keepcost = cap(mi2.keepcost); + + return mi; +} -- 2.34.1.448.ga2b2bfdf31-goog