From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 28066 invoked by alias); 16 Sep 2017 21:04:42 -0000 Mailing-List: contact zsh-workers-help@zsh.org; run by ezmlm Precedence: bulk X-No-Archive: yes List-Id: Zsh Workers List List-Post: List-Help: X-Seq: 41719 Received: (qmail 10566 invoked by uid 1010); 16 Sep 2017 21:04:42 -0000 X-Qmail-Scanner-Diagnostics: from vulcan.omnia.fi by f.primenet.com.au (envelope-from , uid 7791) with qmail-scanner-2.11 (clamdscan: 0.99.2/21882. spamassassin: 3.4.1. Clear:RC:0(212.68.17.166):SA:0(-2.6/5.0):. Processed in 6.451655 secs); 16 Sep 2017 21:04:42 -0000 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on f.primenet.com.au X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,HTML_MESSAGE, RCVD_IN_DNSWL_LOW,RP_MATCHES_RCVD,T_DKIM_INVALID,T_SPF_HELO_TEMPERROR, T_SPF_PERMERROR autolearn=ham autolearn_force=no version=3.4.1 X-Envelope-From: Anssi.Palin@edu.omnia.fi X-Qmail-Scanner-Mime-Attachments: | X-Qmail-Scanner-Zip-Files: | X-Virus-Scanned: amavisd-new at omnia.fi DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudomnia.onmicrosoft.com; s=selector1-edu-omnia-fi; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=Nj9YqojTFg6ntCt4WcjefkULbT/xV8WYncKgmQXqFow=; b=IFT8XbgOhXGQqiFzCCDcfKZvEGKEShCEoUGfeEKIFnRaqcd1pSdMzHxTNHgCx/7gEibO3vm8ermS5qzJtXWuy7Bc+qUTkPA//BvjfjjJxY74uQKB8cyi2kj6pa0jlmHZgCQU5U+r7+GXwxMC/yA2/i60wGlPTFUw3Z9xvYWZDuE= From: Anssi Palin To: "zsh-workers@zsh.org" Subject: Memory leak when working with undefined associative array keys & problems with unset Thread-Topic: Memory leak when working with undefined associative array keys & problems with unset Thread-Index: AQHTLy3wuTzbYeraakiQ/7v849gZ8Q== Date: Sat, 16 Sep 2017 20:57:06 +0000 Message-ID: Accept-Language: fi-FI, en-US Content-Language: fi-FI X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Anssi.Palin@edu.omnia.fi; x-originating-ip: [93.106.225.143] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1;VI1PR0501MB2589;6:hYVKR465TatjcyhcFcC1uxP16xZJu7UCtZqgV0BZYdxL6xfAV81rfpkxprWFcFtqVmx2bKIyR5c2EXrrnj57ivlld6XSmP+FV4LBqSNRj87y+JlsaMYtg1Ip6O+3vg+JBAJEGmizoctCp0YuYAkeO406eWDhuZnOhOSKPLv76g+OFNz7t4vqR5NDvekl5ztUjQK65cgMTOmETDybb7y5JUJ8vEwCM6PFfEoJav0ep6FYPiOhDTkJWix3qHDzzhV/YDJRo12/UiCYwPskjIn85eQmOxYypOXBfNAzuk1+puFIWdB7MrNYy5FH1nits0QhJ0nRmssoUJabNNZmAdtnQA==;5:H5NEUnQ/V6Kap6Y7XYIwA+nMMpm1lK9nwZti8Y4CXuzh8camG/C+N6O0YQpOBDvKiYJ/kH6/BhQTMYY8tgzU2+ERakrDvnrnMmotiE3Mc/n/yZ2PK/P1tjVGyiYJRX4kBaQtvJUCb+FWOmuUWlRXzLlb6P9RhAfG46D3QXtOI18=;24:E+poA4PvvEOoBuRL9Q5NWjhfqB+EnPYWbQBsMgoWVQ69LpOLuMNNYYqBvNYK8kb0v7dfHQxP0KPX0iXlHtIlNyfmBDORaZELVu0dlA2IGvE=;7:uyjCE9IHG6ySWBN3seLvdpUnYT1V1aIsFiyvkP1DEvlNzHsPB/h5zb+EgeJaWXi4BRDjeP6u2yU9/QI4KGcI4v1oGQLFksIGn1HMCrr6nqNHPAmPh+voDFh6jzyZZzS+yidNQI2BKqsJ2tNeGikmF9qwgZdBAF4Id8t26xOS3+HdrhGhKRlkG9h0h8W/nq0tWwcsLT+yGwflZTc0gT5CGtfUYZbdIqD00h82Sig3EUQ= x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-correlation-id: 8a825e07-59ca-4b0c-394e-08d4fd457dc7 x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(2017082002075)(300000503095)(300135400095)(2017052603199)(201703131423075)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);SRVR:VI1PR0501MB2589; x-ms-traffictypediagnostic: VI1PR0501MB2589: x-exchange-antispam-report-test: UriScan:; x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(10201501046)(3002001)(100000703101)(100105400095)(93006095)(93001095)(6041248)(201703131423075)(201702281529075)(201702281528075)(201703061421075)(201703061406153)(20161123564025)(20161123558100)(20161123562025)(20161123560025)(2016111802025)(20161123555025)(6072148)(6043046)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);SRVR:VI1PR0501MB2589;BCL:0;PCL:0;RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);SRVR:VI1PR0501MB2589; x-forefront-prvs: 0432A04947 x-forefront-antispam-report: SFV:NSPM;SFS:(10009020)(6009001)(376002)(39830400002)(346002)(189002)(199003)(3846002)(5660300001)(786003)(6116002)(6506006)(5640700003)(6606003)(25786009)(101416001)(2501003)(316002)(478600001)(6436002)(53936002)(72206003)(86362001)(5250100002)(106356001)(2900100001)(19627405001)(14454004)(102836003)(105586002)(50986999)(54356999)(2351001)(74316002)(54896002)(9686003)(55016002)(99286003)(53336002)(8936002)(110136004)(6916009)(3280700002)(81156014)(81166006)(3660700001)(7736002)(42882006)(7696004)(97736004)(2906002)(74482002)(68736007)(189998001)(8676002)(66066001)(33656002);DIR:OUT;SFP:1101;SCL:1;SRVR:VI1PR0501MB2589;H:VI1PR0501MB2351.eurprd05.prod.outlook.com;FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:0;LANG:en; received-spf: None (protection.outlook.com: edu.omnia.fi does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: multipart/alternative; boundary="_000_VI1PR0501MB23514D1D7839FDF243840C91D76D0VI1PR0501MB2351_" MIME-Version: 1.0 X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Sep 2017 20:57:06.9611 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 5e5f299e-d6ef-4688-ae09-f6e124ff1786 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0501MB2589 X-OriginatorOrg: edu.omnia.fi --_000_VI1PR0501MB23514D1D7839FDF243840C91D76D0VI1PR0501MB2351_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hello, I've run into two problems with the way Zsh handles associative array keys,= both tested on Linux with Zsh 5.4.1 compiled from source and 5.2 from the = Ubuntu repositories. Zsh appears to permanently allocate some memory when just checking if a key= is defined in an associative array. This behavior resulted in system memor= y getting exhausted on my machine with a script that checks random generate= d strings against a word lookup table. The following snippet replicates the= bug and results in some 100 megabytes of memory being allocated: typeset -A a for (( i =3D 0; i < 1000000; i++ )); do (( ${+a[$i]} )) done Iterating over the same set of undefined keys a second time does not seem t= o cause more memory to be allocated. Moreover, unsetting or emptying the ar= ray doesn't appear to free all of the memory even though the array is destr= oyed or, in the latter case, emptied as expected. On 5.4.1 unsetting or emp= tying a second time looks to finally be freeing all of the taken up memory.= Unsetting individual keys afterwards in a loop similar to the example abov= e has the same problem, but unsetting each key immediately after checking i= t seems to mitigate this. The second issue I have pertains to special characters in associative array= keys when unsetting them individually: $ key=3D'hello * [ world' $ typeset -A a=3D("$key" val) $ unset "a[$key]" unset: a[hello * [ world]: invalid parameter name Since characters such as '\', '[', ']', '(' and ')' must be escaped but oth= ers like space or '*' shouldn't be, using the q or b parameter expansion fl= ags is out of the question: $ unset "a[${(q)key}]" unset: a[hello\ \*\ \[\ world]: invalid parameter name $ typeset -p a typeset -A a=3D( 'hello * [ world' val ) The latter unset error message only shows up on 5.2 but in both cases the k= ey remains set. Similar problems with special characters seem to affect [['= s -v flag when checking for existence of associative array keys. Thank you. --_000_VI1PR0501MB23514D1D7839FDF243840C91D76D0VI1PR0501MB2351_--