From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 51 invoked by alias); 22 Nov 2012 22:33:39 -0000 Mailing-List: contact zsh-workers-help@zsh.org; run by ezmlm Precedence: bulk X-No-Archive: yes List-Id: Zsh Workers List List-Post: List-Help: X-Seq: 30818 Received: (qmail 10188 invoked from network); 22 Nov 2012 22:33:37 -0000 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on f.primenet.com.au X-Spam-Level: X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.2 Received-SPF: none (ns1.primenet.com.au: domain at linux.vnet.ibm.com does not designate permitted sender hosts) Date: Fri, 23 Nov 2012 06:33:08 +0800 From: Han Pingtian To: zsh-workers@zsh.org Subject: Re: argv subscript range uses too many memory Message-ID: <20121122223308.GB2576@localhost.localdomain> References: <20121108084001.GA7594@localhost.localdomain> <20121108100226.575b0788@pwslap01u.europe.root.pri> <20121110105811.GA7136@localhost.localdomain> <121110065709.ZM4781@torch.brasslantern.com> <20121120130457.GD2500@localhost.localdomain> <121120090300.ZM5552@torch.brasslantern.com> <121120094443.ZM5584@torch.brasslantern.com> <20121120210552.19eb87a9@pws-pc.ntlworld.com> <121120152257.ZM5937@torch.brasslantern.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <121120152257.ZM5937@torch.brasslantern.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-Content-Scanned: Fidelis XPS MAILER x-cbid: 12112222-5806-0000-0000-00001C138B25 Hi Bart, I have tried your suggestion. Looks like this patch on 5.0 resolve both problem. The for loop of 29175 run about 5 seconds on my loptop and the while loop only used ~180M memory on my loptop, it does't trigger oom-killer any more. diff --git a/Src/mem.c b/Src/mem.c index 5275c6c..0e4d258 100644 --- a/Src/mem.c +++ b/Src/mem.c @@ -106,7 +106,7 @@ union mem_align { }; #define H_ISIZE sizeof(union mem_align) -#define HEAPSIZE (16384 - H_ISIZE) +#define HEAPSIZE (163840 - H_ISIZE) /* Memory available for user data in default arena size */ #define HEAP_ARENA_SIZE (HEAPSIZE - sizeof(struct heap)) #define HEAPFREE (16384 - H_ISIZE) @@ -319,28 +319,8 @@ freeheap(void) h_free++; #endif - /* At this point we used to do: fheap = NULL; - * - * When pushheap() is called, it sweeps over the entire heaps list of - * arenas and marks every one of them with the amount of free space in - * that arena at that moment. zhalloc() is then allowed to grab bits - * out of any of those arenas that have free space. - * - * With the above reset of fheap, the loop below sweeps back over the - * entire heap list again, resetting the free space in every arena to - * the amount stashed by pushheap() and finding the first arena with - * free space to optimize zhalloc()'s next search. When there's a lot - * of stuff already on the heap, this is an enormous amount of work, - * and performance goes to hell. - * - * However, there doesn't seem to be any reason to reset fheap before - * beginning this loop. Either it's already correct, or it has never - * been set and this loop will do it, or it'll be reset from scratch - * on the next popheap(). So all that's needed here is to pick up - * the scan wherever the last pass [or the last popheap()] left off. - */ - for (h = (fheap ? fheap : heaps); h; h = hn) { + for (h = heaps; h; h = hn) { hn = h->next; if (h->sp) { #ifdef ZSH_MEM_DEBUG @@ -377,7 +357,7 @@ freeheap(void) if (hl) hl->next = NULL; else - heaps = fheap = NULL; + heaps = NULL; unqueue_signals(); }