From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 20349 invoked by alias); 21 May 2017 20:03:29 -0000 Mailing-List: contact zsh-workers-help@zsh.org; run by ezmlm Precedence: bulk X-No-Archive: yes List-Id: Zsh Workers List List-Post: List-Help: X-Seq: 41133 Received: (qmail 12448 invoked from network); 21 May 2017 20:03:29 -0000 X-Qmail-Scanner-Diagnostics: from mail-vk0-f53.google.com by f.primenet.com.au (envelope-from , uid 7791) with qmail-scanner-2.11 (clamdscan: 0.99.2/21882. spamassassin: 3.4.1. Clear:RC:0(209.85.213.53):SA:0(-0.0/5.0):. Processed in 1.421263 secs); 21 May 2017 20:03:29 -0000 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on f.primenet.com.au X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_PASS,T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.1 X-Envelope-From: schaefer@brasslantern.com X-Qmail-Scanner-Mime-Attachments: | X-Qmail-Scanner-Zip-Files: | Received-SPF: pass (ns1.primenet.com.au: SPF record at _netblocks.google.com designates 209.85.213.53 as permitted sender) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=brasslantern-com.20150623.gappssmtp.com; s=20150623; h=from:message-id:date:in-reply-to:comments:references:to:subject :mime-version; bh=Il53vzfVYG57AXZ11ll7HffYDyNaYe8Kf525/j5VkCw=; b=RCJYGMdGfe0qkp3tFgtj6LKAT0z26ztXQgObdK0OapR+mVYsYjhhHLgapM1Rn8K+z4 AGyd+vDD+bUSqm9ZG96GmeoVYSUlkHwr+X4PHfGAu/2Z/M+YfePJlDrroOzTyTaGWXfU /F3jiYR85jKGfM0HJZyi5ZR6jc2MKX2GKloelHf9zLlIrwdpw0qCQ/x1ud16xj22Sd87 i8PAJMZva3Uqj2llXrQHbSySWiLyuP9aCb3koXEK/Mxq0+Fxege/P/ybXAFYQs/6JAPA 2Hq5OYGfzvVSn2sQHQBG7ynTYrb8XbMPY+K5Ye64Q6TlTts/wdvHoJDtMH2b1/9sLEdv X4hQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:date:in-reply-to:comments :references:to:subject:mime-version; bh=Il53vzfVYG57AXZ11ll7HffYDyNaYe8Kf525/j5VkCw=; b=n8mIgE9S694pXfnGwGldhpeqtra/Ff/lIy9gyBJsJghku/Vbb7hoCQLnj2apw+tWWc zM9uCcy1vay75YJZbXD/xd/S4MLcnKA1LdL02WjPIBxshUiVlSUHPqdahBMOo5AbmUUD YubVWCkr05qd8j8JBixdp8ezIdipY9FpC94NtkKSvPebnkz8tt6EIOSecXwBZ3WeTSPf /KskSSQHzVmvh9zx3VPuIzUS47z2rfLrdv6O/xh9K3W9nPi0MIKKIzX1R6szfEMN9yIM y/psqHJwzM3WL/AqINFCHarhnecaeZUhK1V7tCwGYBQH1Uer0N5kTiMwtlIgH0IMt2P3 uMxw== X-Gm-Message-State: AODbwcDNlnTLgCG1zy+zW4pDtvSZRzi1IJez7ImiIciJ0RUUIgV7rS8B rtd/50u2ok9xyXvvH4w= X-Received: by 10.31.229.7 with SMTP id c7mr6721738vkh.105.1495396998501; Sun, 21 May 2017 13:03:18 -0700 (PDT) From: Bart Schaefer Message-Id: <170521130349.ZM4506@torch.brasslantern.com> Date: Sun, 21 May 2017 13:03:49 -0700 In-Reply-To: Comments: In reply to Sebastian Gniazdowski "High memory usage on // substitution in one situation, normal usage in other" (May 21, 9:28am) References: X-Mailer: OpenZMail Classic (0.9.2 24April2005) To: zsh-workers@zsh.org Subject: Re: High memory usage on // substitution in one situation, normal usage in other MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii On May 21, 9:28am, Sebastian Gniazdowski wrote: } } following will cause like 55 GB of VIRT memory usage (process will } be killed), having RES memory at 1.8 MB, which roughly equals to } ${#__text}. I can't reproduce this. The for loop takes a lot longer to create the array than does the simple assignment, but neither example uses a huge amount of memory. Since you had a "local -A arr2" in your first example, I ran each of the examples in an anon function in a fresh shell. I also tried the second example both with "local -a arr" added, and as you posted it, but that didn't really make any difference. On my system, the associative arr2 case looks like this: swap free 412780 81244 (skip gradual shrinking during "for" loop) 412780 33260 412780 33260 412780 58668 The second like this: swap free 412780 81420 412780 38092 412780 36748 412780 36772 412780 31524 412780 22692 412780 66532 The $__text value is global in both cases so the delta between the first and last "free" numbers is the memory consumed by that plus any unused space in the last unpopped zsh heap block. In both cases the // expression is almost instantaneous, much faster than creating the $__text string in the first place. Neither uses any swap. My config.h defines HAVE_MMAP 1, so the zsh heap is allocated as an anonymous mapfile, but I don't have 55GB free on the filesystem so I don't think that can be a factor.