From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 6708 invoked from network); 3 Mar 2009 22:09:56 -0000 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on f.primenet.com.au X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 Received: from news.dotsrc.org (HELO a.mx.sunsite.dk) (130.225.247.88) by ns1.primenet.com.au with SMTP; 3 Mar 2009 22:09:56 -0000 Received-SPF: none (ns1.primenet.com.au: domain at sunsite.dk does not designate permitted sender hosts) Received: (qmail 52742 invoked from network); 3 Mar 2009 22:09:52 -0000 Received: from sunsite.dk (130.225.247.90) by a.mx.sunsite.dk with SMTP; 3 Mar 2009 22:09:52 -0000 Received: (qmail 24963 invoked by alias); 3 Mar 2009 22:09:46 -0000 Mailing-List: contact zsh-workers-help@sunsite.dk; run by ezmlm Precedence: bulk X-No-Archive: yes X-Seq: 26686 Received: (qmail 24945 invoked from network); 3 Mar 2009 22:09:46 -0000 Received: from bifrost.dotsrc.org (130.225.254.106) by sunsite.dk with SMTP; 3 Mar 2009 22:09:46 -0000 Received: from mtaout02-winn.ispmail.ntl.com (mtaout02-winn.ispmail.ntl.com [81.103.221.48]) by bifrost.dotsrc.org (Postfix) with ESMTP id E43DB8058F82 for ; Tue, 3 Mar 2009 23:09:41 +0100 (CET) Received: from aamtaout03-winn.ispmail.ntl.com ([81.103.221.35]) by mtaout02-winn.ispmail.ntl.com (InterMail vM.7.08.04.00 201-2186-134-20080326) with ESMTP id <20090303220941.XOXT4080.mtaout02-winn.ispmail.ntl.com@aamtaout03-winn.ispmail.ntl.com> for ; Tue, 3 Mar 2009 22:09:41 +0000 Received: from pws-pc ([81.107.42.185]) by aamtaout03-winn.ispmail.ntl.com (InterMail vG.2.02.00.01 201-2161-120-102-20060912) with ESMTP id <20090303220941.QFJB2093.aamtaout03-winn.ispmail.ntl.com@pws-pc> for ; Tue, 3 Mar 2009 22:09:41 +0000 Date: Tue, 3 Mar 2009 22:09:29 +0000 From: Peter Stephenson To: zsh workers Subject: Re: zsh segfaults with lots of data in one variable Message-ID: <20090303220929.21d1727b@pws-pc> In-Reply-To: <20090303210729.GU4167@fsst.voodoo.lan> References: <20090303210729.GU4167@fsst.voodoo.lan> X-Mailer: Claws Mail 3.7.0 (GTK+ 2.14.7; x86_64-redhat-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Cloudmark-Analysis: v=1.0 c=1 a=SrumVsuinmQA:10 a=8aonB6xhPzwA:10 a=02rWKl3eAAAA:8 a=NLZqzBF-AAAA:8 a=WwvQ7B7coh9x-awlOt4A:9 a=CEzUQcxTqveJnKVHhGgA:7 a=GmmAaXf3E0n0vqwqzLJRl78FQhkA:4 a=VkNpryeOj00A:10 a=iVFYgQKhOj0A:10 a=_dQi-Dcv4p4A:10 X-Virus-Scanned: ClamAV 0.92.1/9065/Tue Mar 3 11:43:41 2009 on bifrost X-Virus-Status: Clean [This seemed to disappear the first time I sent it but it may be queued somewhere and sendmail is lying to me.] On Tue, 3 Mar 2009 22:07:29 +0100 Frank Terbeck wrote: > I just wanted to see if lines="$(<&0)" plays nice with binary input > data, when zsh crashed on me. > > What I did is this: > > [snip] > zsh% slurp() { lines="$(<&0)" } > zsh% slurp < ./a-big-71MiB-mp3-file.mp3 > zsh% printf '%s' $lines > foo.mp3 This is down to a combination of implementation issues including (I suspect, depending on your system) how gcc works. For every command the shell saves the previous last argument in $_. This happens after expansion, so in that last line it was the complete value of "$lines". (This indicates how inefficient it can be to get the shell to do things that aren't really it's job...) When executing a shell function, $_ is saved and restored. If gcc or some other compiler that supports variable length arrays is around, we'll use that. This memory then goes on the stack. In the case of the multimegabyte file this caused bad karma. valgrind gave me an odd message about the stack being switched around when this happened, so I can't exclude the possibility that the compiler or the system is getting unnecessarily muddled. Someone more au fait with the internals might want to try this with an unpatched version of the shell. However, in our case it's easy to fix just using ordindary malloc memory for saving $_. It would be much more efficient to mark underscore as "push old value to stack on write", or something similar, but if I try that at ten o'clock in the evening I'll get it wrong. Index: Src/exec.c =================================================================== RCS file: /cvsroot/zsh/zsh/Src/exec.c,v retrieving revision 1.163 diff -u -r1.163 exec.c --- Src/exec.c 11 Feb 2009 20:42:16 -0000 1.163 +++ Src/exec.c 3 Mar 2009 21:54:34 -0000 @@ -4431,10 +4431,12 @@ mod_export void runshfunc(Eprog prog, FuncWrap wrap, char *name) { - int cont; - VARARR(char, ou, underscoreused); + int cont, ouu; + char *ou; - memcpy(ou, underscore, underscoreused); + ou = zalloc(ouu = underscoreused); + if (ou) + memcpy(ou, underscore, underscoreused); while (wrap) { wrap->module->wrapper++; @@ -4445,13 +4447,19 @@ (wrap->module->node.flags & MOD_UNLOAD)) unload_module(wrap->module); - if (!cont) + if (!cont) { + if (ou) + zfree(ou, ouu); return; + } wrap = wrap->next; } startparamscope(); execode(prog, 1, 0); - setunderscore(ou); + if (ou) { + setunderscore(ou); + zfree(ou, ouu); + } endparamscope(); } Index: Src/utils.c =================================================================== RCS file: /cvsroot/zsh/zsh/Src/utils.c,v retrieving revision 1.212 diff -u -r1.212 utils.c --- Src/utils.c 3 Mar 2009 05:22:57 -0000 1.212 +++ Src/utils.c 3 Mar 2009 21:54:35 -0000 @@ -1340,9 +1340,13 @@ fprintf(shout, "You have new mail.\n"); fflush(shout); } else { - VARARR(char, usav, underscoreused); + char *usav; + int uusav = underscoreused; - memcpy(usav, underscore, underscoreused); + usav = zalloc(underscoreused); + + if (usav) + memcpy(usav, underscore, underscoreused); setunderscore(*s); @@ -1353,7 +1357,10 @@ fputc('\n', shout); fflush(shout); } - setunderscore(usav); + if (usav) { + setunderscore(usav); + zfree(usav, uusav); + } } } if (isset(MAILWARNING) && st.st_atime > st.st_mtime && -- Peter Stephenson Web page now at http://homepage.ntlworld.com/p.w.stephenson/