From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 3480 invoked by alias); 11 Oct 2015 08:39:42 -0000 Mailing-List: contact zsh-workers-help@zsh.org; run by ezmlm Precedence: bulk X-No-Archive: yes List-Id: Zsh Workers List List-Post: List-Help: X-Seq: 36829 Received: (qmail 567 invoked from network); 11 Oct 2015 08:39:40 -0000 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on f.primenet.com.au X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM, T_DKIM_INVALID autolearn=ham autolearn_force=no version=3.4.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=qeOFDSKJC6sq3H9tC/YrEk6baNuzg+BppvE8VIy2ov8=; b=FAvFOtlHqe/MtVYMAHlG1CehT1fCDvJ+G3g5aBIYRH0vQFZM9irxkYkLErw7IphWMV Ai7Jz4iODwGm+zpfLxfVuMdcaMwmmLjP6GqM7ecDLQQCMywiFYfvDbp9XPKSNJtQD50n SRUFnmaMspGQ6HbTQ6p9FJZ7SNfHWFXyutqCgLlz03bBZvF+zSVzVHHzz4ew3LCiIrpx HwjLOg9XJc8ePeyT8gtFzY7LaNVqM7owU1t0LcKWafVb69K0KVxm/VSRavDENqOZcGZi 0Jn4yB+OZu7AnfTVPvqilOWZRYupsfOKPb+vPzhIUIghHJIMZ8JCyQHpEU7j4T1OzPwY 9yXA== X-Received: by 10.112.157.99 with SMTP id wl3mr9961561lbb.98.1444552776233; Sun, 11 Oct 2015 01:39:36 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <151010232045.ZM12931@torch.brasslantern.com> References: <151010105849.ZM10144@torch.brasslantern.com> <151010170623.ZM16166@torch.brasslantern.com> <151010232045.ZM12931@torch.brasslantern.com> From: Sebastian Gniazdowski Date: Sun, 11 Oct 2015 10:39:16 +0200 Message-ID: Subject: Re: Slowdown around 5.0.5-dev-0 To: Bart Schaefer Cc: zsh-workers@zsh.org Content-Type: multipart/mixed; boundary=001a11c342ee1660dd0521d02763 --001a11c342ee1660dd0521d02763 Content-Type: text/plain; charset=UTF-8 Zprof says there is no change: https://youtu.be/6A-AKesvZKM I also attach zprof result in file. BTW, the patch didn't apply to head or to 23f98c3, I had to make changes manually. Best regards, Sebastian Gniazdowski On 11 October 2015 at 08:20, Bart Schaefer wrote: > On Oct 10, 5:06pm, Bart Schaefer wrote: > } > } This suggests a couple of possible fixes, but I've run out of time to > } experiment right now. > > OK, this is the least intrusive change I can think of that might make > a difference. What this attempts to do is maintain a second pointer > into the list of heaps that tells freeheap() where to start working. > It's set on pushheap() and reset on popheap() so it freeheap() should > avoid searching for free space in some arenas that were completely > filled before pushheap() was called. However, it won't improve the > performance if there are a whole lot of heap arenas each with a small > amount of space available at the time of pushheap(). > > To improve THAT situation, I think we'd have to be willing to "leak" > those small pieces of heap until a popheap() happens, and instead > use at least one new arena following most calls to pushheap(). This > might mean that zsh's memory footprint grows, but on modern hardware > that may not be an issue. > > All tests pass with this change in place, let's see what it does with > Sebastian's 89k array elements. > > > diff --git a/Src/mem.c b/Src/mem.c > index b9569ea..01072a3 100644 > --- a/Src/mem.c > +++ b/Src/mem.c > @@ -127,6 +127,12 @@ static Heap heaps; > > static Heap fheap; > > +/* same as fheap, except in the preceding stack context (it will be the > + * first heap from which space in the current context may be recovered > + * when heap space is freed) */ > + > +static Heap fpop; > + > /**/ > #ifdef ZSH_HEAP_DEBUG > /* > @@ -293,8 +299,11 @@ pushheap(void) > h_push++; > #endif > > + fpop = NULL; > for (h = heaps; h; h = h->next) { > DPUTS(!h->used, "BUG: empty heap"); > + if (!fpop && h->used < ARENA_SIZEOF(h)) > + fpop = h; > hs = (Heapstack) zalloc(sizeof(*hs)); > hs->next = h->sp; > h->sp = hs; > @@ -341,9 +350,10 @@ freeheap(void) > * > * However, if the arena to which fheap points is unused, we want to > * free it, so we have no choice but to do the sweep for a new fheap. > + * fpop is the first heap with free space following pushheap(). > */ > if (fheap && !fheap->sp) > - fheap = NULL; /* We used to do this unconditionally */ > + fheap = fpop; > /* > * In other cases, either fheap is already correct, or it has never > * been set and this loop will do it, or it'll be reset from scratch > @@ -417,7 +427,7 @@ popheap(void) > h_pop++; > #endif > > - fheap = NULL; > + fheap = fpop = NULL; > for (h = heaps; h; h = hn) { > hn = h->next; > if ((hs = h->sp)) { > @@ -443,6 +453,8 @@ popheap(void) > #endif > if (!fheap && h->used < ARENA_SIZEOF(h)) > fheap = h; > + if (!fpop && h->sp && h->sp->used < ARENA_SIZEOF(h)) > + fpop = h; > zfree(hs, sizeof(*hs)); > > hl = h; --001a11c342ee1660dd0521d02763 Content-Type: text/plain; charset=US-ASCII; name="result.txt" Content-Disposition: attachment; filename="result.txt" Content-Transfer-Encoding: base64 X-Attachment-Id: f_ifm8awgs0 cGF0Y2hlZDoKIyB0aW1lICggbnBhbmVsaXplIDwgfi9sc29mbHNvZiApCm51bSAgY2FsbHMgICAg ICAgICAgICAgICAgdGltZSAgICAgICAgICAgICAgICAgICAgICAgc2VsZiAgICAgICAgICAgIG5h bWUKLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KIDEpICAxNjEgICAgICAgIDU0NTMsMDQgICAg MzMsODcgICA0Myw1MyUgICA1NDUzLDA0ICAgIDMzLDg3ICAgNDMsNTMlICBfbmxpc3RfcHJpbnRf d2l0aF9hbnNpCiAyKSAgICAxICAgICAgIDEyNTAwLDI4IDEyNTAwLDI4ICAgOTksNzklICAgNDA1 Miw2NSAgNDA1Miw2NSAgIDMyLDM1JSAgbi1saXN0CiAzKSAgICA3ICAgICAgICA4MDQxLDkxICAx MTQ4LDg0ICAgNjQsMjAlICAgMjU4OCw4NyAgIDM2OSw4NCAgIDIwLDY3JSAgbi1saXN0LWRyYXcK IDQpICAgIDcgICAgICAgICAzMjcsOTEgICAgNDYsODQgICAgMiw2MiUgICAgMzIyLDMwICAgIDQ2 LDA0ICAgIDIsNTclICBuLWxpc3QtaW5wdXQKIDUpICAgIDcgICAgICAgICAgNDAsNzcgICAgIDUs ODIgICAgMCwzMyUgICAgIDMxLDk3ICAgICA0LDU3ICAgIDAsMjYlICBfbmxpc3Rfc2V0dXBfdXNl cl92YXJzCiA2KSAgICA3ICAgICAgICAgIDI4LDkxICAgICA0LDEzICAgIDAsMjMlICAgICAyOCw5 MSAgICAgNCwxMyAgICAwLDIzJSAgX25saXN0X3N0YXR1c19tc2cKIDcpICAgIDIgICAgICAgICAg MjQsOTQgICAgMTIsNDcgICAgMCwyMCUgICAgIDI0LDk0ICAgIDEyLDQ3ICAgIDAsMjAlICBfbmxp c3RfY3Vyc29yX3Zpc2liaWxpdHkKIDgpICAgIDEgICAgICAgICAgMjYsNzkgICAgMjYsNzkgICAg MCwyMSUgICAgICA5LDk3ICAgICA5LDk3ICAgIDAsMDglICBfbmxpc3RfZXhpdAooIG4tcGFuZWxp emUgPCB+L2xzb2Zsc29mOyApICAxNCwzM3MgdXNlciAwLDI3cyBzeXN0ZW0gOTQlIGNwdSAxNSw0 ODggdG90YWwKCmNsZWFuOgojIHRpbWUgKCBucGFuZWxpemUgPCB+L2xzb2Zsc29mICkKbnVtICBj YWxscyAgICAgICAgICAgICAgICB0aW1lICAgICAgICAgICAgICAgICAgICAgICBzZWxmICAgICAg ICAgICAgbmFtZQotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQogMSkgIDE2MSAgICAgICAgNTQx MCw0OSAgICAzMyw2MSAgIDQzLDU3JSAgIDU0MTAsNDkgICAgMzMsNjEgICA0Myw1NyUgIF9ubGlz dF9wcmludF93aXRoX2Fuc2kKIDIpICAgIDEgICAgICAgMTIzOTEsNDkgMTIzOTEsNDkgICA5OSw3 OCUgICA0MDMwLDYyICA0MDMwLDYyICAgMzIsNDYlICBuLWxpc3QKIDMpICAgIDcgICAgICAgIDc5 NjEsNzYgIDExMzcsMzkgICA2NCwxMSUgICAyNTUxLDI3ICAgMzY0LDQ3ICAgMjAsNTQlICBuLWxp c3QtZHJhdwogNCkgICAgNyAgICAgICAgIDMyNCwzNiAgICA0NiwzNCAgICAyLDYxJSAgICAzMTgs MzYgICAgNDUsNDggICAgMiw1NiUgIG4tbGlzdC1pbnB1dAogNSkgICAgNyAgICAgICAgICAzOSwy MiAgICAgNSw2MCAgICAwLDMyJSAgICAgMzEsNDQgICAgIDQsNDkgICAgMCwyNSUgIF9ubGlzdF9z ZXR1cF91c2VyX3ZhcnMKIDYpICAgIDcgICAgICAgICAgMjcsODYgICAgIDMsOTggICAgMCwyMiUg ICAgIDI3LDg2ICAgICAzLDk4ICAgIDAsMjIlICBfbmxpc3Rfc3RhdHVzX21zZwogNykgICAgMiAg ICAgICAgICAyNCwzNSAgICAxMiwxOCAgICAwLDIwJSAgICAgMjQsMzUgICAgMTIsMTggICAgMCwy MCUgIF9ubGlzdF9jdXJzb3JfdmlzaWJpbGl0eQogOCkgICAgMSAgICAgICAgICAyNywwNSAgICAy NywwNSAgICAwLDIyJSAgICAgMTAsMzcgICAgMTAsMzcgICAgMCwwOCUgIF9ubGlzdF9leGl0Cigg bi1wYW5lbGl6ZSA8IH4vbHNvZmxzb2Y7ICkgIDE0LDE5cyB1c2VyIDAsMjlzIHN5c3RlbSA5NCUg Y3B1IDE1LDMxOSB0b3RhbAoK --001a11c342ee1660dd0521d02763--