From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 27328 invoked by alias); 10 Nov 2012 21:23:45 -0000 Mailing-List: contact zsh-workers-help@zsh.org; run by ezmlm Precedence: bulk X-No-Archive: yes List-Id: Zsh Workers List List-Post: List-Help: X-Seq: 30791 Received: (qmail 19653 invoked from network); 10 Nov 2012 21:23:32 -0000 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on f.primenet.com.au X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.2 Received-SPF: neutral (ns1.primenet.com.au: 209.85.212.177 is neither permitted nor denied by SPF record at ntlworld.com) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-proxyuser-ip:date:from:to:subject:message-id:in-reply-to :references:x-mailer:mime-version:content-type :content-transfer-encoding:x-gm-message-state; bh=xUQPOhelifxnvIisuZEKYy/EmFKi+UXZarnf1acymB4=; b=Bof+Y31vWivLp1EYJRmzPkrdKB5wGjqBgw37LLvovaw36hmoC9bogbfGyQ7yT/rns0 Eo1Hkc1TxMngZYTF9dBZHgc8f1i06rV05WJ/lCT8yfdgKTQGU7k4eG7ToDbAvlqk0q4T CtBH4812UF9z7wYqUBuBcSIWXuy9ajlBzkNE2ERjUTgnZWJljbePOq3vKqp0hxkpnL9W Io2sQNgmf8LyQuLwk/sDI6ZH4a+X8VrMZRcO5CYl+Cvek0/SGmnn4TFkA8omxX2nvj/s bTE99eP5ckN0tPN9FzrmHWir0UcRLwMZJt8+N6oyr4Zy1ugrssNgVm5dRgAKSVaH9cBS ro7g== X-ProxyUser-IP: 82.8.55.192 Date: Sat, 10 Nov 2012 21:16:12 +0000 From: Peter Stephenson To: Zsh Hackers' List Subject: Re: argv subscript range uses too many memory Message-ID: <20121110211612.436c5ce3@pws-pc.ntlworld.com> In-Reply-To: <121110065709.ZM4781@torch.brasslantern.com> References: <20121108084001.GA7594@localhost.localdomain> <20121108100226.575b0788@pwslap01u.europe.root.pri> <20121110105811.GA7136@localhost.localdomain> <121110065709.ZM4781@torch.brasslantern.com> X-Mailer: Claws Mail 3.8.0 (GTK+ 2.24.7; x86_64-redhat-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Gm-Message-State: ALoCoQk5HBR1b2sBM20J4M+8DrmjVE8TP2vlTDPFwIVZy/lX+vbJnG7eNirAInCyFBCJa5bxpTl/ On Sat, 10 Nov 2012 06:57:09 -0800 Bart Schaefer wrote: > Further discussion probably should be re-routed to zsh-workers. I've done that. > In a loop, the heap allocations are not popped until the loop is done, > IIRC, so you'll end up with a large number of copies of the original > array in the heap with slice results pointing into different parts of > each copy. Maybe there's a narrower scope in which a pushheap/popheap > could be inserted. What's puzzling me is that loops, including the "while" involved here, execute freeheap() at the end of each iteration. That should restore the pristine state of the loop when it was pushed each time around, shouldn't it? Maybe I'm missing some subtlety of freeheap(). I'm a little bit worried there might be some pathology with permanent, i.e. non-heap, allocation, in particular with zsh's own implementation of malloc() (though I don't think anyone's compared between system and zsh versions). That would be something along the lines of: each time around the loop we create an array that is just a little bit bigger than the one we created last time, so doesn't fit in the memory we freed, so we allocate yet another chunk of memory. That's probably nonsense because (i) if we allocated another chunk, the combination of the that with the previous one would be order twice as large as the original, so we shouldn't see pathological behaviour unless something was doubling each time, and there's nothing obviously like that in this case (ii) indeed we're looking at arrays getting smaller each time. Still, I can't quite get this out of my head. That's probably because my head's a funny shape inside. -- Peter Stephenson Web page now at http://homepage.ntlworld.com/p.w.stephenson/