From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 15149 invoked by alias); 18 Nov 2016 15:51:33 -0000 Mailing-List: contact zsh-workers-help@zsh.org; run by ezmlm Precedence: bulk X-No-Archive: yes List-Id: Zsh Workers List List-Post: List-Help: X-Seq: 39977 Received: (qmail 16286 invoked from network); 18 Nov 2016 15:51:33 -0000 X-Qmail-Scanner-Diagnostics: from new2-smtp.messagingengine.com by f.primenet.com.au (envelope-from , uid 7791) with qmail-scanner-2.11 (clamdscan: 0.99.2/21882. spamassassin: 3.4.1. Clear:RC:0(66.111.4.224):SA:0(-0.7/5.0):. Processed in 2.287218 secs); 18 Nov 2016 15:51:33 -0000 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on f.primenet.com.au X-Spam-Level: X-Spam-Status: No, score=-0.7 required=5.0 tests=FREEMAIL_FROM, RCVD_IN_DNSWL_LOW,SPF_PASS,T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.1 X-Envelope-From: psprint@fastmail.com X-Qmail-Scanner-Mime-Attachments: |testopt4bb.zsh|testopt8.zsh| X-Qmail-Scanner-Zip-Files: | Received-SPF: pass (ns1.primenet.com.au: SPF record at spf.messagingengine.com designates 66.111.4.224 as permitted sender) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.com; h= content-transfer-encoding:content-type:date:from:message-id :mime-version:subject:to:x-me-sender:x-me-sender:x-sasl-enc; s= mesmtp; bh=r/VoLBdn/OYVpk1vm78U3ELp5YA=; b=HIgzHIxz/Xj2p1g90aiPT BxvOlAh1bWrUBfcF5QOxHHMu/oLJUPqPAN/Dl3YJNgEjRVx+YgmYXAYIVQuemHjO BOVXBUPahfUu/QKQGlHuvnas5L85tHwIF8Xo+/vl/BrHTascAfV92effUF1mPso2 ho8JifbigJ/vDSzA6BzvnQ= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:message-id:mime-version:subject:to:x-me-sender :x-me-sender:x-sasl-enc; s=smtpout; bh=r/VoLBdn/OYVpk1vm78U3ELp5 YA=; b=H6eDWpHOb++BsSbg3PPpAFUnOv+yHiU2fHxfeUk36h2BLxE+3tXEL6ztk y6YCTtrZWdqYW5gzIOgMrbYk4+A4tQzj9Eyga3ChqdZf2X26ehP/5K3aQ7AgQwrH 4IPQz7ZRx6XjgjOZkTZH74WOXtFMGW5W39W+7he5NswH2NPmpY= X-ME-Sender: Message-Id: <1479484285.1417315.792234625.763D6EEB@webmail.messagingengine.com> From: Sebastian Gniazdowski To: zsh-workers@zsh.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: multipart/mixed; boundary="_----------=_147948428514173151" X-Mailer: MessagingEngine.com Webmail Interface - ajax-461bce03 Subject: Significant assignstrvalue optimization Date: Fri, 18 Nov 2016 07:51:25 -0800 --_----------=_147948428514173151 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Hello, first thing =E2=80=93 the dupstring_glen() that was in the code isn't neede= d =E2=80=93 the value allocated was only read, not written to in any place in the code: - z =3D dupstring_glen(v->pm->gsu.s->getfn(v->pm), (unsigned*) &zlen); + z =3D v->pm->gsu.s->getfn(v->pm); + zlen =3D strlen(z); This gives significant speed up =E2=80=93 although only for very large buff= ers =E2=80=93 attached testopt4bb.zsh: i=3D$(( 20000 )) while (( i -- )); do a+=3D"aaaaaaaaaaaaaaaaaaaaaaaaa" done runs for 953 ms with optimizations, 1576 ms without them.=20 Second: the same that I did with setarrvalue =E2=80=93 if we assign a[10]= =3D"a", and 10-th element already exists, i.e. size of string doesn't change, then instead of: x =3D (char *) zalloc(newsize + 1); strncpy(x, z, v->start); strcpy(x + v->start, val); strcat(x + v->start, z + v->end); v->pm->gsu.s->setfn(v->pm, x); It is done: /* Size doesn't change, can limit actions to only * overwriting bytes in already allocated string */ strncpy(z + v->start, val, vlen); Condition excludes special parameters: /* Does new size differ? */ if ( newsize !=3D zlen || v->pm->node.flags & PM_SPECIAL ) { Test code: a=3D"exampletext" repeat 12 a+=3D"$a" strtest() { repeat 60000; do a[10]=3D"a" done } runs 366 ms vs 876 ms. The string has length 45056 elements, and there are as much as 60k repetitions, so previous code was actually already fast despite the extra zalloc, strncpy, strcat =E2=80=93 which is little we= ird. That said, the optimization can save one's day in certain situations. Also added extensive tests of string assignments. PS. The code again removes "+ 1" from new size computation. The comment explains why: /* Characters preceding start index + characters of what is assigned + characters following end index */ newsize =3D v->start + vlen + (zlen - v->end); --=20 Sebastian Gniazdowski psprint@fastmail.com --_----------=_147948428514173151 Content-Disposition: attachment; filename="string_big_opt.diff" Content-Id: <1479484150.1414973.40873083ddf9e097e8ffaee28d399365b8c1c16f.523E44C0@content.messagingengine.com> Content-Transfer-Encoding: base64 Content-Type: text/plain; name="string_big_opt.diff" ZGlmZiAtLWdpdCBhL1NyYy9wYXJhbXMuYyBiL1NyYy9wYXJhbXMuYwppbmRl eCBlZjcyY2JhLi45MGY2ZmRmIDEwMDY0NAotLS0gYS9TcmMvcGFyYW1zLmMK KysrIGIvU3JjL3BhcmFtcy5jCkBAIC0yNDM2LDkgKzI0MzYsMTAgQEAgYXNz aWduc3RydmFsdWUoVmFsdWUgdiwgY2hhciAqdmFsLCBpbnQgZmxhZ3MpCiAJ CXYtPnBtLT53aWR0aCA9IHN0cmxlbih2YWwpOwogCX0gZWxzZSB7CiAJICAg IGNoYXIgKnosICp4OwotCSAgICBpbnQgemxlbjsKKyAgICAgICAgICAgIGlu dCB6bGVuLCB2bGVuLCBuZXdzaXplOwogCi0JICAgIHogPSBkdXBzdHJpbmdf Z2xlbih2LT5wbS0+Z3N1LnMtPmdldGZuKHYtPnBtKSwgKHVuc2lnbmVkKikg JnpsZW4pOworICAgICAgICAgICAgeiA9IHYtPnBtLT5nc3Uucy0+Z2V0Zm4o di0+cG0pOworICAgICAgICAgICAgemxlbiA9IHN0cmxlbih6KTsKIAogCSAg ICBpZiAoKHYtPmZsYWdzICYgVkFMRkxBR19JTlYpICYmIHVuc2V0KEtTSEFS UkFZUykpCiAJCXYtPnN0YXJ0LS0sIHYtPmVuZC0tOwpAQCAtMjQ2OSwxMiAr MjQ3MCwyNiBAQCBhc3NpZ25zdHJ2YWx1ZShWYWx1ZSB2LCBjaGFyICp2YWws IGludCBmbGFncykKIAkgICAgfQogCSAgICBlbHNlIGlmICh2LT5lbmQgPiB6 bGVuKQogCQl2LT5lbmQgPSB6bGVuOwotCSAgICB4ID0gKGNoYXIgKikgemFs bG9jKHYtPnN0YXJ0ICsgc3RybGVuKHZhbCkgKyB6bGVuIC0gdi0+ZW5kICsg MSk7Ci0JICAgIHN0cm5jcHkoeCwgeiwgdi0+c3RhcnQpOwotCSAgICBzdHJj cHkoeCArIHYtPnN0YXJ0LCB2YWwpOwotCSAgICBzdHJjYXQoeCArIHYtPnN0 YXJ0LCB6ICsgdi0+ZW5kKTsKLQkgICAgdi0+cG0tPmdzdS5zLT5zZXRmbih2 LT5wbSwgeCk7Ci0JICAgIHpzZnJlZSh2YWwpOworCisgICAgICAgICAgICB2 bGVuID0gc3RybGVuKHZhbCk7CisgICAgICAgICAgICAvKiBDaGFyYWN0ZXJz IHByZWNlZGluZyBzdGFydCBpbmRleCArCisgICAgICAgICAgICAgICBjaGFy YWN0ZXJzIG9mIHdoYXQgaXMgYXNzaWduZWQgKworICAgICAgICAgICAgICAg Y2hhcmFjdGVycyBmb2xsb3dpbmcgZW5kIGluZGV4ICovCisgICAgICAgICAg ICBuZXdzaXplID0gdi0+c3RhcnQgKyB2bGVuICsgKHpsZW4gLSB2LT5lbmQp OworCisgICAgICAgICAgICAvKiBEb2VzIG5ldyBzaXplIGRpZmZlcj8gKi8K KyAgICAgICAgICAgIGlmICggbmV3c2l6ZSAhPSB6bGVuIHx8IHYtPnBtLT5u b2RlLmZsYWdzICYgUE1fU1BFQ0lBTCApIHsKKyAgICAgICAgICAgICAgICB4 ID0gKGNoYXIgKikgemFsbG9jKG5ld3NpemUgKyAxKTsKKyAgICAgICAgICAg ICAgICBzdHJuY3B5KHgsIHosIHYtPnN0YXJ0KTsKKyAgICAgICAgICAgICAg ICBzdHJjcHkoeCArIHYtPnN0YXJ0LCB2YWwpOworICAgICAgICAgICAgICAg IHN0cmNhdCh4ICsgdi0+c3RhcnQsIHogKyB2LT5lbmQpOworICAgICAgICAg ICAgICAgIHYtPnBtLT5nc3Uucy0+c2V0Zm4odi0+cG0sIHgpOworICAgICAg ICAgICAgfSBlbHNlIHsKKyAgICAgICAgICAgICAgICAvKiBTaXplIGRvZXNu J3QgY2hhbmdlLCBjYW4gbGltaXQgYWN0aW9ucyB0byBvbmx5CisgICAgICAg ICAgICAgICAgICogb3ZlcndyaXRpbmcgYnl0ZXMgaW4gYWxyZWFkeSBhbGxv Y2F0ZWQgc3RyaW5nICovCisgICAgICAgICAgICAgICAgc3RybmNweSh6ICsg di0+c3RhcnQsIHZhbCwgdmxlbik7CisgICAgICAgICAgICB9CisgICAgICAg ICAgICB6c2ZyZWUodmFsKTsKIAl9CiAJYnJlYWs7CiAgICAgY2FzZSBQTV9J TlRFR0VSOgpkaWZmIC0tZ2l0IGEvVGVzdC9BMDZhc3NpZ24uenRzdCBiL1Rl c3QvQTA2YXNzaWduLnp0c3QKaW5kZXggZGE0ZTNiMC4uYmYzOWFlZSAxMDA2 NDQKLS0tIGEvVGVzdC9BMDZhc3NpZ24uenRzdAorKysgYi9UZXN0L0EwNmFz c2lnbi56dHN0CkBAIC00ODksMyArNDg5LDE0MyBAQAogIHByaW50ICRhcnJh eQogMDpzbGljZSBiZXlvbmQgbGVuZ3RoIG9mIGFycmF5CiA+RklSU1QKKwor IyB0ZXN0cyBvZiBzdHJpbmcgYXNzaWdubWVudHMKKworIGE9ImFiYyIKKyBh WzFdPXgKKyBwcmludCAkYQorMDpvdmVyd3JpdGUgZmlyc3QgY2hhcmFjdGVy IGluIHN0cmluZworPnhiYworCisgYT0iYWJjIgorIGFbMl09IngiCisgcHJp bnQgJGEKKzA6b3ZlcndyaXRlIG1pZGRsZSBjaGFyYWN0ZXIgaW4gc3RyaW5n Cis+YXhjCisKKyBhPSJhYmMiCisgYVszXT0ieCIKKyBwcmludCAkYQorMDpv dmVyd3JpdGUgbGFzdCBjaGFyYWN0ZXIgaW4gc3RyaW5nCis+YWJ4CisKKyBh PSJhYmMiCisgYVstMV09IngiCisgcHJpbnQgJGEKKzA6b3ZlcndyaXRlIC0x IGNoYXJhY3RlciBpbiBzdHJpbmcKKz5hYngKKworIGE9ImFiYyIKKyBhWy0y XT0ieCIKKyBwcmludCAkYQorMDpvdmVyd3JpdGUgLTIgY2hhcmFjdGVyICht aWRkbGUpIGluIHN0cmluZworPmF4YworCisgYT0iYWIiCisgYVstMl09Ingi CisgcHJpbnQgJGEKKzA6b3ZlcndyaXRlIC0yIGNoYXJhY3RlciAoZmlyc3Qp IGluIHN0cmluZworPnhiCisKKyBhPSJhYmMiCisgYVstM109IngiCisgcHJp bnQgJGEKKzA6b3ZlcndyaXRlIC0zIGNoYXJhY3RlciAoZmlyc3QpIGluIHN0 cmluZworPnhiYworCisgYT0iYWJjIgorIGFbLTRdPSJ4IgorIHByaW50ICRh CiswOm92ZXJ3cml0ZSAtNCBjaGFyYWN0ZXIgKGJlZm9yZSBmaXJzdCkgaW4g c3RyaW5nCis+eGFiYworCisgYT0iYWJjIgorIGFbLTVdPSJ4IgorIHByaW50 ICRhCiswOm92ZXJ3cml0ZSAtNSBjaGFyYWN0ZXIgKGJlZm9yZS1iZWZvcmUg Zmlyc3QpIGluIHN0cmluZworPnhhYmMKKworIGE9ImFiYyIKKyBhWy00LDBd PSJ4IgorIHByaW50ICRhCiswOm92ZXJ3cml0ZSBbLTQsMF0gY2hhcmFjdGVy cyAoYmVmb3JlIGZpcnN0KSBpbiBzdHJpbmcKKz54YWJjCisKKyBhPSJhYmMi CisgYVstNCwtNF09IngiCisgcHJpbnQgJGEKKzA6b3ZlcndyaXRlIFstNCwt NF0gY2hhcmFjdGVyIChiZWZvcmUgZmlyc3QpIGluIHN0cmluZworPnhhYmMK KworIGE9ImFiYyIKKyBhWy00MCwtMzBdPSJ4IgorIHByaW50ICRhCiswOm92 ZXJ3cml0ZSBbLTQwLC0zMF0gY2hhcmFjdGVycyAoZmFyIGJlZm9yZSBmaXJz dCkgaW4gc3RyaW5nCis+eGFiYworCisgYT0iYWJjIgorIGFbLTQwLDFdPSJ4 IgorIHByaW50ICRhCiswOm92ZXJ3cml0ZSBbLTQwLDFdIGNoYXJhY3RlcnMg aW4gc2hvcnQgc3RyaW5nCis+eGJjCisKKyBhPSJhYmMiCisgYVstNDAsNDBd PSJ4IgorIHByaW50ICRhCiswOm92ZXJ3cml0ZSBbLTQwLDQwXSBjaGFyYWN0 ZXJzIGluIHNob3J0IHN0cmluZworPngKKworIGE9ImFiYyIKKyBhWzIsNDBd PSJ4IgorIHByaW50ICRhCiswOm92ZXJ3cml0ZSBbMiw0MF0gY2hhcmFjdGVy cyBpbiBzaG9ydCBzdHJpbmcKKz5heAorCisgYT0iYWJjIgorIGFbMiwtMV09 IngiCisgcHJpbnQgJGEKKzA6b3ZlcndyaXRlIFsyLC0xXSBjaGFyYWN0ZXJz IGluIHNob3J0IHN0cmluZworPmF4CisKKyBhPSJhYmMiCisgYVstMiwtMV09 IngiCisgcHJpbnQgJGEKKzA6b3ZlcndyaXRlIFstMiwtMV0gY2hhcmFjdGVy cyBpbiBzaG9ydCBzdHJpbmcKKz5heAorCisgYT0iYSIKKyBhWy0xXT0ieHgi CisgcHJpbnQgJGEKKzA6b3ZlcndyaXRlIFstMV0gY2hhcmFjdGVyIHdpdGgg Inh4IgorPnh4CisKKyBhPSJhIgorIGFbLTJdPSJ4eCIKKyBwcmludCAkYQor MDpvdmVyd3JpdGUgWy0yXSBjaGFyYWN0ZXIgKGJlZm9yZSBmaXJzdCkgd2l0 aCAieHgiCis+eHhhCisKKyBhPSJhIgorIGFbMl09Inh4IgorIHByaW50ICRh CiswOm92ZXJ3cml0ZSBbMl0gY2hhcmFjdGVyIChhZnRlciBsYXN0KSB3aXRo ICJ4eCIKKz5heHgKKworIGE9IiIKKyBhWzFdPSJ4eCIKKyBwcmludCAkYQor MDpvdmVyd3JpdGUgWzFdIGNoYXJhY3RlciAoc3RyaW5nOiAiIikgd2l0aCAi eHgiCis+eHgKKworIGE9IiIKKyBhWy0xXT0ieHgiCisgcHJpbnQgJGEKKzA6 b3ZlcndyaXRlIFstMV0gY2hhcmFjdGVyIChzdHJpbmc6ICIiKSB3aXRoICJ4 eCIKKz54eAorCisgYT0iIgorIGFbMl09Inh4IgorIHByaW50ICRhCiswOm92 ZXJ3cml0ZSBbMl0gY2hhcmFjdGVyIChzdHJpbmc6ICIiKSB3aXRoICJ4eCIK Kz54eAo= --_----------=_147948428514173151 Content-Disposition: attachment; filename="testopt4bb.zsh" Content-Id: <1479484155.1414999.1ce05c6bd32d22bc32dac6674d18d5e2bfda6d16.57892975@content.messagingengine.com> Content-Transfer-Encoding: base64 Content-Type: application/octet-stream; name="testopt4bb.zsh" IyFTcmMvenNoLW5vZHVwCiMhU3JjL3pzaC1ub2R1cC1iZWZvcmUKCnptb2Rs b2FkIHpzaC96cHJvZgoKc3RydGVzdCgpIHsKICAgIGE9IiIKCiAgICBpPSQo KCAyMDAwMCApKQogICAgd2hpbGUgKCggaSAtLSApKTsgZG8KICAgICAgICBh Kz0iYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYSIKICAgIGRvbmUKfQoKc3Ry dGVzdAoKenByb2YgfCBoZWFkIC1uIDI1Cg== --_----------=_147948428514173151 Content-Disposition: attachment; filename="testopt8.zsh" Content-Id: <1479484159.1414972.66f4f858683edb09c74d93d59b05882424ca53c7.2E8C6137@content.messagingengine.com> Content-Transfer-Encoding: base64 Content-Type: application/octet-stream; name="testopt8.zsh" IyFTcmMvenNoCiMhU3JjL3pzaC1ub2R1cC1iZWZvcmUKIyFTcmMvenNoLW5v ZHVwCgp6bW9kbG9hZCB6c2gvenByb2YKCnN0cnRlc3QoKSB7CiAgICByZXBl YXQgNjAwMDA7IGRvCiAgICAgICAgYVsxMF09ImEiCiAgICBkb25lCn0KCmE9 ImV4YW1wbGV0ZXh0IgpyZXBlYXQgMTIgYSs9IiRhIgoKc3RydGVzdAoKenBy b2YgfCBoZWFkIC1uIDI1CgplY2hvICJSZXN1bHRpbmcgbGVuZ3RoOiAkeyNh fSIK --_----------=_147948428514173151--