From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 21243 invoked from network); 3 Mar 2009 22:04:17 -0000 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on f.primenet.com.au X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 Received: from news.dotsrc.org (HELO a.mx.sunsite.dk) (130.225.247.88) by ns1.primenet.com.au with SMTP; 3 Mar 2009 22:04:17 -0000 Received-SPF: none (ns1.primenet.com.au: domain at sunsite.dk does not designate permitted sender hosts) Received: (qmail 50583 invoked from network); 3 Mar 2009 22:04:11 -0000 Received: from sunsite.dk (130.225.247.90) by a.mx.sunsite.dk with SMTP; 3 Mar 2009 22:04:11 -0000 Received: (qmail 22195 invoked by alias); 3 Mar 2009 22:04:05 -0000 Mailing-List: contact zsh-workers-help@sunsite.dk; run by ezmlm Precedence: bulk X-No-Archive: yes X-Seq: 26685 Received: (qmail 22184 invoked from network); 3 Mar 2009 22:04:04 -0000 Received: from bifrost.dotsrc.org (130.225.254.106) by sunsite.dk with SMTP; 3 Mar 2009 22:04:04 -0000 Received: from mtaout03-winn.ispmail.ntl.com (mtaout03-winn.ispmail.ntl.com [81.103.221.49]) by bifrost.dotsrc.org (Postfix) with ESMTP id 7E5888058F82 for ; Tue, 3 Mar 2009 23:04:00 +0100 (CET) Received: from aamtaout01-winn.ispmail.ntl.com ([81.103.221.35]) by mtaout03-winn.ispmail.ntl.com (InterMail vM.7.08.04.00 201-2186-134-20080326) with ESMTP id <20090303220400.HTCR7670.mtaout03-winn.ispmail.ntl.com@aamtaout01-winn.ispmail.ntl.com> for ; Tue, 3 Mar 2009 22:04:00 +0000 Received: from pws-pc ([81.107.42.185]) by aamtaout01-winn.ispmail.ntl.com (InterMail vG.2.02.00.01 201-2161-120-102-20060912) with ESMTP id <20090303220359.ZAUN13254.aamtaout01-winn.ispmail.ntl.com@pws-pc> for ; Tue, 3 Mar 2009 22:03:59 +0000 Date: Tue, 3 Mar 2009 22:03:47 +0000 From: Peter Stephenson To: zsh workers Subject: Re: zsh segfaults with lots of data in one variable Message-ID: <20090303220347.36279ac1@pws-pc> In-Reply-To: <20090303210729.GU4167@fsst.voodoo.lan> References: <20090303210729.GU4167@fsst.voodoo.lan> X-Mailer: Claws Mail 3.7.0 (GTK+ 2.14.7; x86_64-redhat-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Cloudmark-Analysis: v=1.0 c=1 a=SrumVsuinmQA:10 a=8aonB6xhPzwA:10 a=02rWKl3eAAAA:8 a=NLZqzBF-AAAA:8 a=EdSKNgDi4mOZPcenhuoA:9 a=nwSxU1Vv_9pPnoZWqX0A:7 a=rd4iXbNYAQW2jdTOqPxn_Yws7qsA:4 a=LY0hPdMaydYA:10 a=iVFYgQKhOj0A:10 a=_dQi-Dcv4p4A:10 X-Virus-Scanned: ClamAV 0.92.1/9065/Tue Mar 3 11:43:41 2009 on bifrost X-Virus-Status: Clean On Tue, 3 Mar 2009 22:07:29 +0100 Frank Terbeck wrote: > I just wanted to see if lines="$(<&0)" plays nice with binary input > data, when zsh crashed on me. > > What I did is this: > > [snip] > zsh% slurp() { lines="$(<&0)" } > zsh% slurp < ./a-big-71MiB-mp3-file.mp3 > zsh% printf '%s' $lines > foo.mp3 This is down to a combination of implementation issues including (I suspect, depending on your system) how gcc works. For every command the shell saves the previous last argument in $_. This happens after expansion, so in that last line it was the complete value of "$lines". (This indicates how inefficient it can be to get the shell to do things that aren't really it's job...) When executing a shell function, $_ is saved and restored. If gcc or some other compiler that supports variable length arrays is around, we'll use that. This memory then goes on the stack. In the case of the multimegabyte file this caused bad karma. valgrind gave me an odd message about the stack being switched around when this happened, so I can't exclude the possibility that the compiler or the system is getting unnecessarily muddled. Someone more au fait with the internals might want to try this with an unpatched version of the shell. However, in our case it's easy to fix just using ordindary malloc memory for saving $_. It would be much more efficient to mark underscore as "push old value to stack on write", or something similar, but if I try that at ten o'clock in the evening I'll get it wrong. Index: Src/exec.c =================================================================== RCS file: /cvsroot/zsh/zsh/Src/exec.c,v retrieving revision 1.163 diff -u -r1.163 exec.c --- Src/exec.c 11 Feb 2009 20:42:16 -0000 1.163 +++ Src/exec.c 3 Mar 2009 21:54:34 -0000 @@ -4431,10 +4431,12 @@ mod_export void runshfunc(Eprog prog, FuncWrap wrap, char *name) { - int cont; - VARARR(char, ou, underscoreused); + int cont, ouu; + char *ou; - memcpy(ou, underscore, underscoreused); + ou = zalloc(ouu = underscoreused); + if (ou) + memcpy(ou, underscore, underscoreused); while (wrap) { wrap->module->wrapper++; @@ -4445,13 +4447,19 @@ (wrap->module->node.flags & MOD_UNLOAD)) unload_module(wrap->module); - if (!cont) + if (!cont) { + if (ou) + zfree(ou, ouu); return; + } wrap = wrap->next; } startparamscope(); execode(prog, 1, 0); - setunderscore(ou); + if (ou) { + setunderscore(ou); + zfree(ou, ouu); + } endparamscope(); } Index: Src/utils.c =================================================================== RCS file: /cvsroot/zsh/zsh/Src/utils.c,v retrieving revision 1.212 diff -u -r1.212 utils.c --- Src/utils.c 3 Mar 2009 05:22:57 -0000 1.212 +++ Src/utils.c 3 Mar 2009 21:54:35 -0000 @@ -1340,9 +1340,13 @@ fprintf(shout, "You have new mail.\n"); fflush(shout); } else { - VARARR(char, usav, underscoreused); + char *usav; + int uusav = underscoreused; - memcpy(usav, underscore, underscoreused); + usav = zalloc(underscoreused); + + if (usav) + memcpy(usav, underscore, underscoreused); setunderscore(*s); @@ -1353,7 +1357,10 @@ fputc('\n', shout); fflush(shout); } - setunderscore(usav); + if (usav) { + setunderscore(usav); + zfree(usav, uusav); + } } } if (isset(MAILWARNING) && st.st_atime > st.st_mtime && -- Peter Stephenson Web page now at http://homepage.ntlworld.com/p.w.stephenson/