From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-3.3 required=5.0 tests=MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 30347 invoked from network); 11 Jun 2020 04:33:20 -0000 Received: from mother.openwall.net (195.42.179.200) by inbox.vuxu.org with ESMTPUTF8; 11 Jun 2020 04:33:20 -0000 Received: (qmail 3892 invoked by uid 550); 11 Jun 2020 04:33:15 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 3872 invoked from network); 11 Jun 2020 04:33:14 -0000 Date: Thu, 11 Jun 2020 00:33:02 -0400 From: Rich Felker To: musl@lists.openwall.com Message-ID: <20200611043301.GM1079@brightrain.aerifal.cx> References: <20200609035010.GE1079@brightrain.aerifal.cx> <20200611034934.GL1079@brightrain.aerifal.cx> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="9JSHP372f+2dzJ8X" Content-Disposition: inline In-Reply-To: <20200611034934.GL1079@brightrain.aerifal.cx> User-Agent: Mutt/1.5.21 (2010-09-15) Subject: Re: [musl] mallocng switchover - opportunity to test --9JSHP372f+2dzJ8X Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Wed, Jun 10, 2020 at 11:49:34PM -0400, Rich Felker wrote: > On Mon, Jun 08, 2020 at 11:50:10PM -0400, Rich Felker wrote: > > I just pushed a series of changes in preparation for upstreaming > > mallocng. Before it's actually imported, it can be tested by > > performing the following simple 4 steps: > > > > 1. mkdir src/malloc/mallocng > > 2. echo "MALLOC_DIR = mallocng" >> config.mak > > 3. Dropping the attached files into src/malloc/mallocng > > 4. Symlinking or copying meta.h, malloc.c, realloc.c, free.c, > > malloc_usable_size.c, and aligned_alloc.c from mallocng source dir > > to src/malloc/mallocng. (You can also include dump.c if desired.) > > > > This produces a near-fully-integrated malloc, including support for > > reclaim_gaps donation from ldso. The only functionality missing, which > > I expect to flesh out before actual import, is handling of the case of > > incomplete malloc replacement by interposition (__malloc_replaced!=0). > > > > Please report any problems encountered. > > For reference -- I should have mentioned in the original post -- the > above is with musl commit 384c0131ccda2656dec23a0416ad3f14101151a7 > and mallocng-draft commit c0d6d87596f565e652e126f54aa1a2afaecc0e52. > > I'll have an update to these posted soon. Updated upsteam. Reduction in what's needed to integrate, only 2 files now, smaller than before (attached). musl: ca36573ecf mallocng-draft: 55c1e76bb0 Files needed from mallocng: meta.h, malloc.c, realloc.c, free.c, malloc_usable_size.c, and aligned_alloc.c Rich --9JSHP372f+2dzJ8X Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="glue.h" #ifndef MALLOC_GLUE_H #define MALLOC_GLUE_H #include #include #include #include #include #include #include "atomic.h" #include "syscall.h" #include "libc.h" #include "lock.h" #include "dynlink.h" // use macros to appropriately namespace these. #define size_classes __malloc_size_classes #define ctx __malloc_context #define alloc_meta __malloc_alloc_meta #define is_allzero __malloc_allzerop #define dump_heap __dump_heap #if USE_REAL_ASSERT #include #else #undef assert #define assert(x) do { if (!(x)) a_crash(); } while(0) #endif #define brk(p) ((uintptr_t)__syscall(SYS_brk, p)) #define mmap __mmap #define madvise __madvise #define mremap __mremap #define DISABLE_ALIGNED_ALLOC (__malloc_replaced && !__aligned_alloc_replaced) static inline uint64_t get_random_secret() { uint64_t secret = (uintptr_t)&secret * 1103515245; for (size_t i=0; libc.auxv[i]; i+=2) if (libc.auxv[i]==AT_RANDOM) memcpy(&secret, (char *)libc.auxv[i+1]+8, sizeof secret); return secret; } #ifndef PAGESIZE #define PAGESIZE PAGE_SIZE #endif #define MT (libc.need_locks) #define RDLOCK_IS_EXCLUSIVE 1 __attribute__((__visibility__("hidden"))) extern int __malloc_lock[1]; #define LOCK_OBJ_DEF \ int __malloc_lock[1]; static inline void rdlock() { if (MT) LOCK(__malloc_lock); } static inline void wrlock() { if (MT) LOCK(__malloc_lock); } static inline void unlock() { UNLOCK(__malloc_lock); } static inline void upgradelock() { } #endif --9JSHP372f+2dzJ8X Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="donate.c" #include #include #include #include #include #include #include "meta.h" static void donate(unsigned char *base, size_t len) { uintptr_t a = (uintptr_t)base; uintptr_t b = a + len; a += -a & (UNIT-1); b -= b & (UNIT-1); memset(base, 0, len); for (int sc=47; sc>0 && b>a; sc-=4) { if (b-a < (size_classes[sc]+1)*UNIT) continue; struct meta *m = alloc_meta(); m->avail_mask = 0; m->freed_mask = 1; m->mem = (void *)a; m->mem->meta = m; m->last_idx = 0; m->freeable = 0; m->sizeclass = sc; m->maplen = 0; *((unsigned char *)m->mem+UNIT-4) = 0; *((unsigned char *)m->mem+UNIT-3) = 255; m->mem->storage[size_classes[sc]*UNIT-4] = 0; queue(&ctx.active[sc], m); a += (size_classes[sc]+1)*UNIT; } } void __malloc_donate(char *start, char *end) { donate((void *)start, end-start); } --9JSHP372f+2dzJ8X--