From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 2971 invoked by alias); 11 Oct 2015 18:05:40 -0000 Mailing-List: contact zsh-workers-help@zsh.org; run by ezmlm Precedence: bulk X-No-Archive: yes List-Id: Zsh Workers List List-Post: List-Help: X-Seq: 36837 Received: (qmail 21033 invoked from network); 11 Oct 2015 18:05:39 -0000 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on f.primenet.com.au X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM, T_DKIM_INVALID autolearn=ham autolearn_force=no version=3.4.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=a6b82V8HfvTe/31Ebgog/YC2qR6sHBOhy0C8jJqo2LM=; b=v/abDDTl4+RW+VkjrVyofkbypOlQzQCgknXfvKxsDOErhf6iRUQyEeByevg5vf9E8V F9YvZE/h3VMBl9oMe0b3Zu2basYGxu5YpNXEDMMacj1fA/hHmXNubiPTTbkTah1+l1kH VR1297AMH0y4AmcCIxDssSJVlPdxbBclZpn7o9KnYBRUMcx6/N1VH0x/gKRHseCaJaQf WwbHTP0J14QbB/Q9UZxI8s2gbMrB8RzVy4C4+n75WC5A/b9sgs1Qd4OORjZQ0CIctkgY VMxZrti5S9KqxnABjFupJsK+J960EGgJpd7Ao2mRCad8McegCSGW9C6cqKdqiDZXW96V CHBA== X-Received: by 10.25.88.67 with SMTP id m64mr7131870lfb.23.1444586735093; Sun, 11 Oct 2015 11:05:35 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <151011103121.ZM8814@torch.brasslantern.com> References: <151010105849.ZM10144@torch.brasslantern.com> <151010170623.ZM16166@torch.brasslantern.com> <151010232045.ZM12931@torch.brasslantern.com> <151011091757.ZM27755@torch.brasslantern.com> <151011103121.ZM8814@torch.brasslantern.com> From: Sebastian Gniazdowski Date: Sun, 11 Oct 2015 20:05:15 +0200 Message-ID: Subject: Re: Slowdown around 5.0.5-dev-0 To: Bart Schaefer Cc: zsh-workers@zsh.org Content-Type: multipart/mixed; boundary=001a1141a07c31c9190521d80f54 --001a1141a07c31c9190521d80f54 Content-Type: text/plain; charset=UTF-8 The patch (alone) much helped, zprof time is now 3027 and there is no after-while slow down. Htop shows RES of 152 MB. For the first patch RES is 166 MB. For the clean version it's 152 MB. I didn't combine the patches (BTW. the problems with applying patches were caused by expand tabs that occured on copy-pasting). On 11 October 2015 at 19:31, Bart Schaefer wrote: > On Oct 11, 6:48pm, Sebastian Gniazdowski wrote: > } > } It's now much faster, zprof time 11851 vs 3433. That's still strongly > } lagging behavior. One other trait is that it slows down after a while. > > The whole algorithm is designed to maximally fill the allocated space > and thereby minimize the memory footprint, not to maximize speed. So > the more often you grow/free the heap, the more time it spends looking > for unused space in the existing arenas. > > Besides zprof, it would be interesting to see the maximum memory usage > of the patched and unpatched shells. > > Does this change in zhalloc() make any difference? This goes farther > in the direction of abandoning small bits of earlier arenas until the > popheap() comes around. > > diff --git a/Src/mem.c b/Src/mem.c > index b9569ea..63a5afe 100644 > --- a/Src/mem.c > +++ b/Src/mem.c > @@ -543,9 +543,14 @@ zhalloc(size_t size) > > /* find a heap with enough free space */ > > - for (h = ((fheap && ARENA_SIZEOF(fheap) >= (size + fheap->used)) > - ? fheap : heaps); > - h; h = h->next) { > + /* > + * This previously assigned: > + * h = ((fheap && ARENA_SIZEOF(fheap) >= (size + fheap->used)) > + * ? fheap : heaps); > + * but we think that nothing upstream of fheap has more free space, > + * so why start over at heaps just because fheap has too little? > + */ > + for (h = (fheap ? fheap : heaps); h; h = h->next) { > if (ARENA_SIZEOF(h) >= (n = size + h->used)) { > void *ret; > --001a1141a07c31c9190521d80f54 Content-Type: text/plain; charset=US-ASCII; name="result3.txt" Content-Disposition: attachment; filename="result3.txt" Content-Transfer-Encoding: base64 X-Attachment-Id: f_ifmtk0pt0 cGF0Y2hlZCAoemhhbGxvYyk6Cm51bSAgY2FsbHMgICAgICAgICAgICAgICAgdGltZSAgICAgICAg ICAgICAgICAgICAgICAgc2VsZiAgICAgICAgICAgIG5hbWUKLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0KIDEpICAgIDEgICAgICAgIDMwMjcsNDEgIDMwMjcsNDEgICA5OSw1MyUgICAyMDI1LDUw ICAyMDI1LDUwICAgNjYsNTklICBuLWxpc3QKIDIpICAxNjEgICAgICAgICA2NzQsMDIgICAgIDQs MTkgICAyMiwxNiUgICAgNjc0LDAyICAgICA0LDE5ICAgMjIsMTYlICBfbmxpc3RfcHJpbnRfd2l0 aF9hbnNpCiAzKSAgICA3ICAgICAgICAgOTYyLDUyICAgMTM3LDUwICAgMzEsNjQlICAgIDI4OCw1 MCAgICA0MSwyMSAgICA5LDQ4JSAgbi1saXN0LWRyYXcKIDQpICAgIDcgICAgICAgICAgMjQsNTAg ICAgIDMsNTAgICAgMCw4MSUgICAgIDI0LDMxICAgICAzLDQ3ICAgIDAsODAlICBuLWxpc3QtaW5w dXQKIDUpICAgIDIgICAgICAgICAgMTMsMDkgICAgIDYsNTQgICAgMCw0MyUgICAgIDEzLDA5ICAg ICA2LDU0ICAgIDAsNDMlICBfbmxpc3RfY3Vyc29yX3Zpc2liaWxpdHkKIDYpICAgIDcgICAgICAg ICAgIDksMTggICAgIDEsMzEgICAgMCwzMCUgICAgICA4LDk5ICAgICAxLDI4ICAgIDAsMzAlICBf bmxpc3Rfc2V0dXBfdXNlcl92YXJzCiA3KSAgICAxICAgICAgICAgIDE0LDMzICAgIDE0LDMzICAg IDAsNDclICAgICAgNiwyMyAgICAgNiwyMyAgICAwLDIwJSAgX25saXN0X2V4aXQKIDgpICAgIDcg ICAgICAgICAgIDAsNzEgICAgIDAsMTAgICAgMCwwMiUgICAgICAwLDcxICAgICAwLDEwICAgIDAs MDIlICBfbmxpc3Rfc3RhdHVzX21zZwooIG4tcGFuZWxpemUgPCB+L2xzb2Zsc29mOyApICA0LDYy cyB1c2VyIDAsMThzIHN5c3RlbSA4OCUgY3B1IDUsNDMyIHRvdGFsCgpwYXRjaGVkICh0aGUgZmly c3QgcGF0Y2gpOgpudW0gIGNhbGxzICAgICAgICAgICAgICAgIHRpbWUgICAgICAgICAgICAgICAg ICAgICAgIHNlbGYgICAgICAgICAgICBuYW1lCi0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCiAx KSAgICAxICAgICAgICAzNTQ3LDQzICAzNTQ3LDQzICAgOTksNjMlICAgMjU2MCw1OCAgMjU2MCw1 OCAgIDcxLDkyJSAgbi1saXN0CiAyKSAgMTYxICAgICAgICAgNjM2LDU2ICAgICAzLDk1ICAgMTcs ODglICAgIDYzNiw1NiAgICAgMyw5NSAgIDE3LDg4JSAgX25saXN0X3ByaW50X3dpdGhfYW5zaQog MykgICAgNyAgICAgICAgIDkyMSwyOCAgIDEzMSw2MSAgIDI1LDg4JSAgICAyODQsNzEgICAgNDAs NjcgICAgOCwwMCUgIG4tbGlzdC1kcmF3CiA0KSAgICA3ICAgICAgICAgIDQ5LDQyICAgICA3LDA2 ICAgIDEsMzklICAgICA0OSwzMyAgICAgNywwNSAgICAxLDM5JSAgbi1saXN0LWlucHV0CiA1KSAg ICAyICAgICAgICAgIDExLDk2ICAgICA1LDk4ICAgIDAsMzQlICAgICAxMSw5NiAgICAgNSw5OCAg ICAwLDM0JSAgX25saXN0X2N1cnNvcl92aXNpYmlsaXR5CiA2KSAgICA3ICAgICAgICAgIDEwLDgw ICAgICAxLDU0ICAgIDAsMzAlICAgICAxMCw2NCAgICAgMSw1MiAgICAwLDMwJSAgX25saXN0X3Nl dHVwX3VzZXJfdmFycwogNykgICAgMSAgICAgICAgICAxMywwNSAgICAxMywwNSAgICAwLDM3JSAg ICAgIDYsMTcgICAgIDYsMTcgICAgMCwxNyUgIF9ubGlzdF9leGl0CiA4KSAgICA3ICAgICAgICAg ICAwLDI4ICAgICAwLDA0ICAgIDAsMDElICAgICAgMCwyOCAgICAgMCwwNCAgICAwLDAxJSAgX25s aXN0X3N0YXR1c19tc2cKKCBuLXBhbmVsaXplIDwgfi9sc29mbHNvZjsgKSAgNSwxN3MgdXNlciAw LDE4cyBzeXN0ZW0gODglIGNwdSA2LDAzMiB0b3RhbAoKY2xlYW46Cm51bSAgY2FsbHMgICAgICAg ICAgICAgICAgdGltZSAgICAgICAgICAgICAgICAgICAgICAgc2VsZiAgICAgICAgICAgIG5hbWUK LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KIDEpICAxNjEgICAgICAgIDUxNzYsNzcgICAgMzIs MTUgICA0Myw4NCUgICA1MTc2LDc3ICAgIDMyLDE1ICAgNDMsODQlICBfbmxpc3RfcHJpbnRfd2l0 aF9hbnNpCiAyKSAgICAxICAgICAgIDExNzg1LDE2IDExNzg1LDE2ICAgOTksNzklICAgMzc1NSwy MyAgMzc1NSwyMyAgIDMxLDgwJSAgbi1saXN0CiAzKSAgICA3ICAgICAgICA3NjQwLDcwICAxMDkx LDUzICAgNjQsNzAlICAgMjQ2Myw5MyAgIDM1MSw5OSAgIDIwLDg2JSAgbi1saXN0LWRyYXcKIDQp ICAgIDcgICAgICAgICAzMTYsNDUgICAgNDUsMjEgICAgMiw2OCUgICAgMzEwLDk3ICAgIDQ0LDQy ICAgIDIsNjMlICBuLWxpc3QtaW5wdXQKIDUpICAgIDcgICAgICAgICAgMzcsNzIgICAgIDUsMzkg ICAgMCwzMiUgICAgIDMwLDAwICAgICA0LDI5ICAgIDAsMjUlICBfbmxpc3Rfc2V0dXBfdXNlcl92 YXJzCiA2KSAgICA3ICAgICAgICAgIDI2LDUyICAgICAzLDc5ICAgIDAsMjIlICAgICAyNiw1MiAg ICAgMyw3OSAgICAwLDIyJSAgX25saXN0X3N0YXR1c19tc2cKIDcpICAgIDIgICAgICAgICAgMjMs NjggICAgMTEsODQgICAgMCwyMCUgICAgIDIzLDY4ICAgIDExLDg0ICAgIDAsMjAlICBfbmxpc3Rf Y3Vyc29yX3Zpc2liaWxpdHkKIDgpICAgIDEgICAgICAgICAgMjQsNDUgICAgMjQsNDUgICAgMCwy MSUgICAgICA5LDMyICAgICA5LDMyICAgIDAsMDglICBfbmxpc3RfZXhpdAooIG4tcGFuZWxpemUg PCB+L2xzb2Zsc29mOyApICAxMyw2NHMgdXNlciAwLDIwcyBzeXN0ZW0gOTUlIGNwdSAxNCw1NDgg dG90YWwK --001a1141a07c31c9190521d80f54--