From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <zsh-users-return-17388-mason-zsh=primenet.com.au@zsh.org>
Received: (qmail 23660 invoked by alias); 10 Nov 2012 14:57:42 -0000
Mailing-List: contact zsh-users-help@zsh.org; run by ezmlm
Precedence: bulk
X-No-Archive: yes
List-Id: Zsh Users List <zsh-users.zsh.org>
List-Post: <mailto:zsh-users@zsh.org>
List-Help: <mailto:zsh-users-help@zsh.org>
X-Seq: 17388
Received: (qmail 4165 invoked from network); 10 Nov 2012 14:57:31 -0000
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on f.primenet.com.au
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_NONE
	autolearn=ham version=3.3.2
Received-SPF: none (ns1.primenet.com.au: domain at closedmail.com does not designate permitted sender hosts)
From: Bart Schaefer <schaefer@brasslantern.com>
Message-id: <121110065709.ZM4781@torch.brasslantern.com>
Date: Sat, 10 Nov 2012 06:57:09 -0800
In-reply-to: <20121110105811.GA7136@localhost.localdomain>
Comments: In reply to Han Pingtian <hanpt@linux.vnet.ibm.com>
 "Re: argv subscript range uses too many memory" (Nov 10,  6:58pm)
References: <20121108084001.GA7594@localhost.localdomain>
	<20121108100226.575b0788@pwslap01u.europe.root.pri>
	<20121110105811.GA7136@localhost.localdomain>
X-Mailer: OpenZMail Classic (0.9.2 24April2005)
To: zsh-users@zsh.org
Subject: Re: argv subscript range uses too many memory
MIME-version: 1.0
Content-type: text/plain; charset=us-ascii

Further discussion probably should be re-routed to zsh-workers.

On Nov 10,  6:58pm, Han Pingtian wrote:
}
} Looks like when running with 'print -- "$argv[1,3]", the call trace is
} something like this:
} 
} (gdb) bt
} #0  mmap_heap_alloc (n=0x7fff0852e880) at mem.c:449
} #1  0x000000000045f5f9 in zhalloc (size=1594456) at mem.c:542
} #2  0x00000000004a4dfa in arrdup (s=0x313b008) at utils.c:3648
} #3  0x0000000000471fa9 in getarrvalue (v=0x7fff0852ea20) at params.c:2174

Ah, yes.  Array slices are implemented by copying the entire array and
then extracting the desired subset from the copy.  Individual array
elements are string references and therefore copy only the one element.

Unfortunately this is pretty deeply ingrained in zsh's parameter
expansion implementation and likely requires some serious rewriting to
fix.  It might be easier to come up with a way to garbage-collect more
frequently.

In a loop, the heap allocations are not popped until the loop is done,
IIRC, so you'll end up with a large number of copies of the original
array in the heap with slice results pointing into different parts of
each copy.  Maybe there's a narrower scope in which a pushheap/popheap
could be inserted.