From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <zsh-workers-return-41134-mason-zsh=primenet.com.au@zsh.org>
Received: (qmail 23502 invoked by alias); 21 May 2017 20:47:49 -0000
Mailing-List: contact zsh-workers-help@zsh.org; run by ezmlm
Precedence: bulk
X-No-Archive: yes
List-Id: Zsh Workers List <zsh-workers.zsh.org>
List-Post: <mailto:zsh-workers@zsh.org>
List-Help: <mailto:zsh-workers-help@zsh.org>
X-Seq: 41134
Received: (qmail 21442 invoked from network); 21 May 2017 20:47:49 -0000
X-Qmail-Scanner-Diagnostics: from mxout1.bln1.prohost.de by f.primenet.com.au (envelope-from <mail@mpy.de>, uid 7791) with qmail-scanner-2.11 
 (clamdscan: 0.99.2/21882. spamassassin: 3.4.1.  
 Clear:RC:0(91.233.87.26):SA:0(-0.7/5.0):. 
 Processed in 2.240303 secs); 21 May 2017 20:47:49 -0000
X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on f.primenet.com.au
X-Spam-Level: 
X-Spam-Status: No, score=-0.7 required=5.0 tests=RCVD_IN_DNSWL_LOW,
	RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL autolearn=unavailable autolearn_force=no
	version=3.4.1
X-Envelope-From: mail@mpy.de
X-Qmail-Scanner-Mime-Attachments: |
X-Qmail-Scanner-Zip-Files: |
Received-SPF: none (ns1.primenet.com.au: domain at mpy.de does not designate permitted sender hosts)
Message-ID: <201705212247400859.0480A18C@gateway.core.mpy.ch>
In-Reply-To: <170521130349.ZM4506@torch.brasslantern.com>
References: <etPan.59214193.41a7c4c9.6b4c@MacMini.local>
 <170521130349.ZM4506@torch.brasslantern.com>
X-Mailer: Courier 3.50.00.09.1098 (http://www.rosecitysoftware.com) (3)
Date: Sun, 21 May 2017 22:47:40 +0200
Reply-To: mail@mpy.de
From: "Manuel Presnitz" <mail@mpy.de>
To: zsh-workers@zsh.org
Subject: Re[2]: High memory usage on // substitution in one situation,
  normal usage in other
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
X-Null-Tag: e0a6828328780a00283a00ef1555d178

Good evening, Ladies and Gentlemen,

I can reproduce the extraordinary high memory usage (until the process is=
 killed) of Sebastian's first code example with zsh-5.3.1-88-gd8c66e6=
 running on Linux; but only if EXTENDEDGLOB is set.

Best regards,
Manuel.




Bart Schaefer hat am 21.05.2017 um 13:03 folgendes geschrieben:

> On May 21,  9:28am, Sebastian Gniazdowski wrote:
> }
> } following will cause like 55 GB of VIRT memory usage (process will
> } be killed), having RES memory at 1.8 MB, which roughly equals to
> } ${#__text}.
> 
> I can't reproduce this.  The for loop takes a lot longer to create the
> array than does the simple assignment, but neither example uses a huge
> amount of memory.
> 
> Since you had a "local -A arr2" in your first example, I ran each of
> the examples in an anon function in a fresh shell.  I also tried the
> second example both with "local -a arr" added, and as you posted it,
> but that didn't really make any difference.
> 
> On my system, the associative arr2 case looks like this:
> 
> swap    free
> 412780  81244
> (skip gradual shrinking during "for" loop)
> 412780  33260
> 412780  33260
> 412780  58668
> 
> The second like this:
> 
> swap    free
> 412780  81420
> 412780  38092
> 412780  36748
> 412780  36772
> 412780  31524
> 412780  22692
> 412780  66532
> 
> The $__text value is global in both cases so the delta between the first
> and last "free" numbers is the memory consumed by that plus any unused
> space in the last unpopped zsh heap block.
> 
> In both cases the // expression is almost instantaneous, much faster than
> creating the $__text string in the first place.  Neither uses any swap.
> My config.h defines HAVE_MMAP 1, so the zsh heap is allocated as an
> anonymous mapfile, but I don't have 55GB free on the filesystem so I
> don't think that can be a factor.