From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 4001 invoked by alias); 13 Apr 2017 14:30:30 -0000 Mailing-List: contact zsh-workers-help@zsh.org; run by ezmlm Precedence: bulk X-No-Archive: yes List-Id: Zsh Workers List List-Post: List-Help: X-Seq: 40962 Received: (qmail 5447 invoked from network); 13 Apr 2017 14:30:30 -0000 X-Qmail-Scanner-Diagnostics: from mx1.redhat.com by f.primenet.com.au (envelope-from , uid 7791) with qmail-scanner-2.11 (clamdscan: 0.99.2/21882. spamassassin: 3.4.1. Clear:RC:0(209.132.183.28):SA:0(-5.0/5.0):. Processed in 0.900123 secs); 13 Apr 2017 14:30:30 -0000 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on f.primenet.com.au X-Spam-Level: X-Spam-Status: No, score=-5.0 required=5.0 tests=RCVD_IN_DNSWL_HI, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,RP_MATCHES_RCVD,SPF_HELO_PASS,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.1 X-Envelope-From: kdudka@redhat.com X-Qmail-Scanner-Mime-Attachments: | X-Qmail-Scanner-Zip-Files: | Received-SPF: pass (ns1.primenet.com.au: SPF record at _spf1.redhat.com designates 209.132.183.28 as permitted sender) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 12923C04BD50 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=kdudka@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 12923C04BD50 From: Kamil Dudka To: Bart Schaefer Cc: "zsh-workers@zsh.org" Subject: Re: unbounded recursive call in a shell script crashes zsh Date: Thu, 13 Apr 2017 16:30:31 +0200 Message-ID: <2350280.olGvC23INb@kdudka-nb> User-Agent: KMail/4.14.10 (Linux/4.9.16-gentoo; KDE/4.14.29; x86_64; ; ) In-Reply-To: References: <2960832.nVDpiBkaWZ@kdudka-nb> <2420758.31stuSQeAV@kdudka-nb> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="nextPart9127894.cXZEuNdQal" Content-Transfer-Encoding: 7Bit X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Thu, 13 Apr 2017 14:30:23 +0000 (UTC) --nextPart9127894.cXZEuNdQal Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" On Wednesday, April 12, 2017 15:11:05 Bart Schaefer wrote: > On Wed, Apr 12, 2017 at 12:30 AM, Kamil Dudka wrote: > > On Tuesday, April 11, 2017 19:12:41 Bart Schaefer wrote: > >> In fact 4.3.11 running out of job table space means that it consumed > >> all malloc() memory before consuming all of the stack; > > > > I cannot confirm your hypothesis. > > Sorry, I forgot about (and missed because I was scanning for diffs in > job handling and that part hasn't changed) MAX_MAXJOBS. Either way it > means 4.3.11 ran out of job table space and never had a chance to > exercise the function call recursion limit. Still the question remains: Is it expected to run out of stack after <1000 nested calls of a trivial shell function when 1000 is the default limit for function call depth? I was trying to reduce the stack usage of zsh but was not really successful, mainly because I do not know how to efficiently find the automatic variables that consumed the biggest portion of the stack. The attached patch reduces the peak stack allocation on my example from 5.558 MB to 5.345 MB, so it is probably not worth applying at all. Do you have any estimation about where else the stack allocation could be reduced? Kamil --nextPart9127894.cXZEuNdQal Content-Disposition: attachment; filename="0001-doshfunc-reduce-stack-allocation-in-favour-of-heap.patch" Content-Transfer-Encoding: 7Bit Content-Type: text/x-patch; charset="UTF-8"; name="0001-doshfunc-reduce-stack-allocation-in-favour-of-heap.patch" >>From 758a8ee2b36f73766b1387974e0b3bf4bc87d1d8 Mon Sep 17 00:00:00 2001 From: Kamil Dudka Date: Thu, 13 Apr 2017 14:08:49 +0200 Subject: [PATCH] doshfunc: reduce stack allocation in favour of heap --- Src/exec.c | 31 ++++++++++++++++++------------- 1 file changed, 18 insertions(+), 13 deletions(-) diff --git a/Src/exec.c b/Src/exec.c index f021a08..cde549e 100644 --- a/Src/exec.c +++ b/Src/exec.c @@ -5358,13 +5358,13 @@ doshfunc(Shfunc shfunc, LinkList doshargs, int noreturnval) char **pptab, **x, *oargv0; int oldzoptind, oldlastval, oldoptcind, oldnumpipestats, ret; int *oldpipestats = NULL; - char saveopts[OPT_SIZE], *oldscriptname = scriptname; + char *saveopts, *oldscriptname = scriptname; char *name = shfunc->node.nam; int flags = shfunc->node.flags, ooflags; char *fname = dupstring(name); int obreaks, ocontflag, oloops, saveemulation, restore_sticky; Eprog prog; - struct funcstack fstack; + struct funcstack *fstack; static int oflags; Emulation_options save_sticky = NULL; #ifdef MAX_FUNCTION_DEPTH @@ -5409,6 +5409,7 @@ doshfunc(Shfunc shfunc, LinkList doshargs, int noreturnval) /* We need to save the current options even if LOCALOPTIONS is * * not currently set. That's because if it gets set in the * * function we need to restore the original options on exit. */ + saveopts = (char *)zalloc(OPT_SIZE * sizeof(char)); memcpy(saveopts, opts, sizeof(opts)); saveemulation = emulation; save_sticky = sticky; @@ -5501,21 +5502,22 @@ doshfunc(Shfunc shfunc, LinkList doshargs, int noreturnval) goto undoshfunc; } #endif - fstack.name = dupstring(name); + fstack = zalloc(sizeof(struct funcstack)); + fstack->name = dupstring(name); /* * The caller is whatever is immediately before on the stack, * unless we're at the top, in which case it's the script * or interactive shell name. */ - fstack.caller = funcstack ? funcstack->name : + fstack->caller = funcstack ? funcstack->name : dupstring(oargv0 ? oargv0 : argzero); - fstack.lineno = lineno; - fstack.prev = funcstack; - fstack.tp = FS_FUNC; - funcstack = &fstack; + fstack->lineno = lineno; + fstack->prev = funcstack; + fstack->tp = FS_FUNC; + funcstack = fstack; - fstack.flineno = shfunc->lineno; - fstack.filename = getshfuncfile(shfunc); + fstack->flineno = shfunc->lineno; + fstack->filename = getshfuncfile(shfunc); prog = shfunc->funcdef; if (prog->flags & EF_RUN) { @@ -5523,7 +5525,7 @@ doshfunc(Shfunc shfunc, LinkList doshargs, int noreturnval) prog->flags &= ~EF_RUN; - runshfunc(prog, NULL, fstack.name); + runshfunc(prog, NULL, fstack->name); if (!(shf = (Shfunc) shfunctab->getnode(shfunctab, (name = fname)))) { @@ -5536,9 +5538,10 @@ doshfunc(Shfunc shfunc, LinkList doshargs, int noreturnval) } prog = shf->funcdef; } - runshfunc(prog, wrappers, fstack.name); + runshfunc(prog, wrappers, fstack->name); doneshfunc: - funcstack = fstack.prev; + funcstack = fstack->prev; + zfree(fstack, sizeof(struct funcstack)); #ifdef MAX_FUNCTION_DEPTH undoshfunc: --funcdepth; @@ -5586,6 +5589,8 @@ doshfunc(Shfunc shfunc, LinkList doshargs, int noreturnval) opts[WARNNESTEDVAR] = saveopts[WARNNESTEDVAR]; } + zfree(saveopts, OPT_SIZE * sizeof(char)); + if (opts[LOCALLOOPS]) { if (contflag) zwarn("`continue' active at end of function scope"); -- 2.10.2 --nextPart9127894.cXZEuNdQal--