From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 19729 invoked by alias); 1 Oct 2015 08:51:50 -0000 Mailing-List: contact zsh-workers-help@zsh.org; run by ezmlm Precedence: bulk X-No-Archive: yes List-Id: Zsh Workers List List-Post: List-Help: X-Seq: 36731 Received: (qmail 27402 invoked from network); 1 Oct 2015 08:51:49 -0000 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on f.primenet.com.au X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham autolearn_force=no version=3.4.0 X-AuditID: cbfec7f5-f794b6d000001495-c3-560cf1c5979d Date: Thu, 01 Oct 2015 09:41:40 +0100 From: Peter Stephenson To: zsh-workers@zsh.org Subject: Re: Substitution ${...///} slows down when certain UTF character occurs Message-id: <20151001094140.413b5401@pwslap01u.europe.root.pri> In-reply-to: <150930141937.ZM22962@torch.brasslantern.com> References: <150926134410.ZM17546@torch.brasslantern.com> <150927091121.ZM25721@torch.brasslantern.com> <20150928095142.385a33eb@pwslap01u.europe.root.pri> <20150928202312.6679b38e@ntlworld.com> <20150929094436.32b62692@pwslap01u.europe.root.pri> <20150929193726.38235c76@ntlworld.com> <150929122356.ZM30421@torch.brasslantern.com> <20150930095950.6c9c583b@pwslap01u.europe.root.pri> <20150930150433.21f70e13@pwslap01u.europe.root.pri> <150930141937.ZM22962@torch.brasslantern.com> Organization: Samsung Cambridge Solution Centre X-Mailer: Claws Mail 3.7.9 (GTK+ 2.22.0; i386-redhat-linux-gnu) MIME-version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7bit X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrELMWRmVeSWpSXmKPExsVy+t/xa7rHPvKEGZzlsjjY/JDJgdFj1cEP TAGMUVw2Kak5mWWpRfp2CVwZN/8cYi7oEq7Y83QdUwPjX74uRk4OCQETiV1/JrBA2GISF+6t Z+ti5OIQEljKKHFz+ScoZwaTxIsn75kgnK2MEiv/TWHsYuTgYBFQlVjxpQCkm03AUGLqptmM ILaIgLjE2bXnWUBKhAUCJHZ/DgYJ8wrYSzya84EdxOYUsJK4sfoeK8TIZlaJ+cvbwa7gF9CX uPr3ExPERfYSM6+cYYRoFpT4MfkeWA2zgJbE5m1NrBC2vMTmNW+ZQWwhAXWJG3d3s09gFJqF pGUWkpZZSFoWMDKvYhRNLU0uKE5KzzXSK07MLS7NS9dLzs/dxAgJ2a87GJceszrEKMDBqMTD K5HOEybEmlhWXJl7iFGCg1lJhFfmLVCINyWxsiq1KD++qDQntfgQozQHi5I478xd70OEBNIT S1KzU1MLUotgskwcnFINjBmCljPaw/i2s5rG3jLh542VuTrzQHR141G1Hf7f3jBbXTq4Pe9F zaRr23484/dezsmrZZD/83uGo97/sy4/ZBd9KNGzzZ0dvUwwZvPvrg9BD1JCqw5WTF4a42Nd cvPs4bjjDV/5tZUjfe4XCu2707g5umDaXaZol/nPcqwX2DiJWci8OqCorcRSnJFoqMVcVJwI AM/WocZVAgAA On Wed, 30 Sep 2015 14:19:37 -0700 Bart Schaefer wrote: > On Sep 30, 3:04pm, Peter Stephenson wrote: > } In some complex git completion --- reproducible only in one set up --- > } the heap allocation is apparently going out of bounds. Yet I've checked > } and nothing seems able to happen that can affect this. If I add > } debugging, the same pointer is reported as returned from zhalloc() that > } is later claimed to be out of bounds. Yet with more debugging there's > } no popheap() or switch_heap() happening, as by eye there can't be. I'm > } a bit stuck; how can a piece of memory returned by zhalloc() suddenly go > } invalid? > } > } valgrind agrees with gdb but doesn't give any more help. > > Did you --enable-zsh-heap-debug and --enable-zsh-valgrind ? I always > forget at least the latter of those ... I don't have the full valgrind library framework, just the executable, on the system where this is happening. (This is 64-bit and I haven't seen it with a 32-bit system I have more control over in the same repo.) But I tried heap debug and the last bit of output doesn't look obviously suspicious --- I don't see a sign it's allocating masses of memory, for example. Indeed, the last thing it did was a push, not a pop. > } Maybe we're just running out of memory after all? > > That seems really unlikely, unless the explicit zalloc version shows a > similar issue. Yes, zhalloc() gets memory from zalloc() which explicitly tests, and there's no obvious way you could get a pointer of the sort I've got. > But I suppose if the heap is never popped it might be > possible to run out of memory on a deep recursive glob. Perhaps a > call to freeheap() is needed somewhere prior to the popheap()? Or > maybe there *is* a freeheap() happening where it isn't any longer safe > to do so -- did you check for that? Yes, I looked for popping, freeing and swapping of heaps and there doesn't appear to be any in the immediate vicint. > Even if out of memory, I wouldn't expect an existing pointer to go bad. > You'd just fail to get a new one. And heaps are mmap()d if possible so > in that case you'd have to be out of address space, not just out of > memory? I tried not using mmap and the symptom is the same. I get the feeling there's something here that simply switching back to zalloc() is likely to obscure rather than fix. pws