From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/12166 Path: news.gmane.org!.POSTED!not-for-mail From: Rich Felker Newsgroups: gmane.linux.lib.musl.general Subject: Re: [PATCH] Wasm support patch 1 (support systems without mmap) Date: Tue, 28 Nov 2017 10:46:34 -0500 Message-ID: <20171128154634.GZ1627@brightrain.aerifal.cx> References: Reply-To: musl@lists.openwall.com NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: blaine.gmane.org 1511884008 17454 195.159.176.226 (28 Nov 2017 15:46:48 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Tue, 28 Nov 2017 15:46:48 +0000 (UTC) User-Agent: Mutt/1.5.21 (2010-09-15) To: musl@lists.openwall.com Original-X-From: musl-return-12182-gllmg-musl=m.gmane.org@lists.openwall.com Tue Nov 28 16:46:44 2017 Return-path: Envelope-to: gllmg-musl@m.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by blaine.gmane.org with smtp (Exim 4.84_2) (envelope-from ) id 1eJi5r-0004AS-U9 for gllmg-musl@m.gmane.org; Tue, 28 Nov 2017 16:46:44 +0100 Original-Received: (qmail 13750 invoked by uid 550); 28 Nov 2017 15:46:48 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Original-Received: (qmail 13725 invoked from network); 28 Nov 2017 15:46:47 -0000 Content-Disposition: inline In-Reply-To: Original-Sender: Rich Felker Xref: news.gmane.org gmane.linux.lib.musl.general:12166 Archived-At: On Tue, Nov 28, 2017 at 11:50:08AM +0000, Nicholas Wilson wrote: > I'm hoping the first patch is uncontroversial. > > WebAssembly has a linear/flat memory model, whereby it's simply > impossible for the addressable memory to contain "holes". Therefore, > mmap can't really be emulated, and all memory has to be allocated > via brk. This is not supported or supportable in musl's malloc. The heap that can be serviced by brk is only usable for allocation sizes less than ~128k. > I've done this by allowing malloc to fall back to brk (even for > allocations above MMAP_THRESHOLD) if mmap returns ENOSYS. > > It's one line of code that will do harm in "normal" systems, and > allows platforms that are emulating syscalls to choose not to > support mmap, and still get a working malloc if brk is working. The patch as written almost surely corrupts the heap structures or at least produces new heap expansion (no possibility of reuse) each time an allocation larger than ~128k is requested. This is because bin_index[_up] is not defined for n > MMAP_THRESHOLD. There are other places in musl that also depend on mmap working, like the TLS init code. brk on the other hand is only used as an optimization. I think what you should do, if it's hard to support both brk and mmap, is omit brk instead (have it return -ENOSYS), then have mmap always allocate lowest-available-page and keep some data structure (even just a bit array) representing which pages are free. This way you still keep everything in a contiguous range (possibly with some reusable gaps due to munmap) like you want. Rich > diff --git a/src/malloc/malloc.c b/src/malloc/malloc.c > index 9e05e1d6..572232e1 100644 > --- a/src/malloc/malloc.c > +++ b/src/malloc/malloc.c > @@ -328,13 +328,17 @@ void *malloc(size_t n) > size_t len = n + OVERHEAD + PAGE_SIZE - 1 & -PAGE_SIZE; > char *base = __mmap(0, len, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_ANONYMOUS, -1, 0); > - if (base == (void *)-1) return 0; > + if (base == MAP_FAILED) { > + if (errno == ENOSYS) goto nommap; > + return 0; > + } > c = (void *)(base + SIZE_ALIGN - OVERHEAD); > c->csize = len - (SIZE_ALIGN - OVERHEAD); > c->psize = SIZE_ALIGN - OVERHEAD; > return CHUNK_TO_MEM(c); > } > > +nommap: > i = bin_index_up(n); > for (;;) { > uint64_t mask = mal.binmap & -(1ULL< @@ -405,7 +409,7 @@ void *realloc(void *p, size_t n) > newlen = (newlen + PAGE_SIZE-1) & -PAGE_SIZE; > if (oldlen == newlen) return p; > base = __mremap(base, oldlen, newlen, MREMAP_MAYMOVE); > - if (base == (void *)-1) > + if (base == MAP_FAILED) > goto copy_realloc; > self = (void *)(base + extra); > self->csize = newlen - extra;