From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham autolearn_force=no version=3.4.2 Received: from primenet.com.au (ns1.primenet.com.au [203.24.36.2]) by inbox.vuxu.org (OpenSMTPD) with ESMTP id 92dfb0c7 for ; Fri, 7 Feb 2020 11:34:22 +0000 (UTC) Received: (qmail 23068 invoked by alias); 7 Feb 2020 11:34:17 -0000 Mailing-List: contact zsh-workers-help@zsh.org; run by ezmlm Precedence: bulk X-No-Archive: yes List-Id: Zsh Workers List List-Post: List-Help: List-Unsubscribe: X-Seq: 45396 Received: (qmail 6082 invoked by uid 1010); 7 Feb 2020 11:34:17 -0000 X-Qmail-Scanner-Diagnostics: from mail-oi1-f195.google.com by f.primenet.com.au (envelope-from , uid 7791) with qmail-scanner-2.11 (clamdscan: 0.102.1/25713. spamassassin: 3.4.2. Clear:RC:0(209.85.167.195):SA:0(-1.6/5.0):. Processed in 1.403932 secs); 07 Feb 2020 11:34:17 -0000 X-Envelope-From: michael@i3wm.org X-Qmail-Scanner-Mime-Attachments: | X-Qmail-Scanner-Zip-Files: | Received-SPF: none (ns1.primenet.com.au: domain at i3wm.org does not designate permitted sender hosts) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ljQ+FJD2VZYSHfcjv6a1QF06gBtnj9vd6q/27lBrrPA=; b=b2qkZWhEEkNKwwxMn01yEgPz3UgdYIzZUDmuN36+xoY1Bi8jQ9Z41KIVilV8+JFnm/ JKw4fOYpKRxhL6HQ3jFaP/CCvVfVj5LwVGAnfIliwVXxoIPOQkiwukU73yPQqFqMNokc VuqVNsZXG3WT4VDTQ/V56uzoTzfsnK/NVLq7n6Y8sL+uch0ROz8FfF06EQXBqCm4uaxw aKXykL8Ll+VXWZWhKSNmYPGXnT1uAvO9saMJ4/fW26TxIS5m4SnB1r7pVnsPcXx2yhcf odYvVkijJlyoUBibfGDUk5vOxZfgkBXRp0WK9vdCKZbwwf3Dx7TvEUQl8YYov4X26bDJ JW4A== X-Gm-Message-State: APjAAAWcPk+e5jgQ4fAUTOvkLNaLIuFTwaHu8cGExPB1+1aQbOToaWqw gr1nZoM82rgeXAZckDpCfWXNKfy8ezd2c3003Fo+7A== X-Google-Smtp-Source: APXvYqxCNrOVJC6EgPHUYJIWvpl+vTGuXKNpg22iypjTJIlQuYnF+SyTXQNUHTye2PBwriEo5SpiD64tXcXKPZNL2VQ= X-Received: by 2002:a05:6808:8ca:: with SMTP id k10mr1734116oij.164.1581075223593; Fri, 07 Feb 2020 03:33:43 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Michael Stapelberg Date: Fri, 7 Feb 2020 12:33:32 +0100 Message-ID: Subject: Re: [PATCH] readhistfile: avoid thousands of lseek(2) syscalls via ftell() To: Mikael Magnusson Cc: zsh-workers@zsh.org Content-Type: multipart/mixed; boundary="00000000000010ff45059dfac345" --00000000000010ff45059dfac345 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Done, thanks. On Fri, Feb 7, 2020 at 12:30 PM Mikael Magnusson wrote: > > On 2/7/20, Michael Stapelberg wrote: > > [please cc me in replies, as I=E2=80=99m not subscribed to this list] > > > > Before this change, zsh startup time was dominated by lseek(2) system c= alls > > on > > the history file, as shown by strace -c: > > > > time seconds usecs/call calls errors syscall > > ------ ----------- ----------- --------- --------- ---------------- > > 97,35 1,112890 1 697153 1 lseek > > 0,99 0,011314 2 5277 read > > [=E2=80=A6] > > > > This change keeps track of read bytes and the position within the file, > > removing > > all of these system calls. > > > > I verified correctness of the change by comparing fpos with ftell(in) i= n > > every > > loop iteration. > > > + *readbytes +=3D strlen(buf + start); > > int len =3D start + strlen(buf + start); > > int len =3D strlen(buf + start); > *readbytes +=3D len; > len +=3D start; > > Even if you're convinced the compiler will optimize the double strlen > calls, the declaration has to come before code (I don't think we > require new enough C to not require this yet?). > > -- > Mikael Magnusson --00000000000010ff45059dfac345 Content-Type: text/x-patch; charset="UTF-8"; name="0001-readhistfile-avoid-thousands-of-lseek-2-syscalls-via.patch" Content-Disposition: attachment; filename="0001-readhistfile-avoid-thousands-of-lseek-2-syscalls-via.patch" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_k6c3cmze0 RnJvbSA0YTExMDgwNzU4MWViYWZlZWQ4MTc4ZmQxNzdlOTk4N2YzMzRjZTljIE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBNaWNoYWVsIFN0YXBlbGJlcmcgPHN0YXBlbGJlcmdAZ29vZ2xl LmNvbT4KRGF0ZTogRnJpLCA3IEZlYiAyMDIwIDA4OjQxOjI2ICswMTAwClN1YmplY3Q6IFtQQVRD SF0gcmVhZGhpc3RmaWxlOiBhdm9pZCB0aG91c2FuZHMgb2YgbHNlZWsoMikgc3lzY2FsbHMgdmlh CiBmdGVsbCgpCk1JTUUtVmVyc2lvbjogMS4wCkNvbnRlbnQtVHlwZTogdGV4dC9wbGFpbjsgY2hh cnNldD1VVEYtOApDb250ZW50LVRyYW5zZmVyLUVuY29kaW5nOiA4Yml0CgpCZWZvcmUgdGhpcyBj aGFuZ2UsIHpzaCBzdGFydHVwIHRpbWUgd2FzIGRvbWluYXRlZCBieSBsc2VlaygyKSBzeXN0ZW0g Y2FsbHMgb24KdGhlIGhpc3RvcnkgZmlsZSwgYXMgc2hvd24gYnkgc3RyYWNlIC1jOgoKdGltZSAg ICAgc2Vjb25kcyAgdXNlY3MvY2FsbCAgICAgY2FsbHMgICAgZXJyb3JzIHN5c2NhbGwKLS0tLS0t IC0tLS0tLS0tLS0tIC0tLS0tLS0tLS0tIC0tLS0tLS0tLSAtLS0tLS0tLS0gLS0tLS0tLS0tLS0t LS0tLQogOTcsMzUgICAgMSwxMTI4OTAgICAgICAgICAgIDEgICAgNjk3MTUzICAgICAgICAgMSBs c2VlawogIDAsOTkgICAgMCwwMTEzMTQgICAgICAgICAgIDIgICAgICA1Mjc3ICAgICAgICAgICBy ZWFkClvigKZdCgpUaGlzIGNoYW5nZSBrZWVwcyB0cmFjayBvZiByZWFkIGJ5dGVzIGFuZCB0aGUg cG9zaXRpb24gd2l0aGluIHRoZSBmaWxlLCByZW1vdmluZwphbGwgb2YgdGhlc2Ugc3lzdGVtIGNh bGxzLgoKSSB2ZXJpZmllZCBjb3JyZWN0bmVzcyBvZiB0aGUgY2hhbmdlIGJ5IGNvbXBhcmluZyBm cG9zIHdpdGggZnRlbGwoaW4pIGluIGV2ZXJ5Cmxvb3AgaXRlcmF0aW9uLgotLS0KIFNyYy9oaXN0 LmMgfCAxNiArKysrKysrKysrLS0tLS0tCiAxIGZpbGUgY2hhbmdlZCwgMTAgaW5zZXJ0aW9ucygr KSwgNiBkZWxldGlvbnMoLSkKCmRpZmYgLS1naXQgYS9TcmMvaGlzdC5jIGIvU3JjL2hpc3QuYwpp bmRleCA1MjgxZTg3MTguLmI0NDQ2M2UxYiAxMDA2NDQKLS0tIGEvU3JjL2hpc3QuYworKysgYi9T cmMvaGlzdC5jCkBAIC0yNTc1LDExICsyNTc1LDEzIEBAIHJlc2l6ZWhpc3RlbnRzKHZvaWQpCiB9 CiAKIHN0YXRpYyBpbnQKLXJlYWRoaXN0bGluZShpbnQgc3RhcnQsIGNoYXIgKipidWZwLCBpbnQg KmJ1ZnNpeiwgRklMRSAqaW4pCityZWFkaGlzdGxpbmUoaW50IHN0YXJ0LCBjaGFyICoqYnVmcCwg aW50ICpidWZzaXosIEZJTEUgKmluLCBpbnQgKnJlYWRieXRlcykKIHsKICAgICBjaGFyICpidWYg PSAqYnVmcDsKICAgICBpZiAoZmdldHMoYnVmICsgc3RhcnQsICpidWZzaXogLSBzdGFydCwgaW4p KSB7Ci0JaW50IGxlbiA9IHN0YXJ0ICsgc3RybGVuKGJ1ZiArIHN0YXJ0KTsKKwlpbnQgbGVuID0g c3RybGVuKGJ1ZiArIHN0YXJ0KTsKKwkqcmVhZGJ5dGVzICs9IGxlbjsKKwlsZW4gKz0gc3RhcnQ7 CiAJaWYgKGxlbiA9PSBzdGFydCkKIAkgICAgcmV0dXJuIC0xOwogCWlmIChidWZbbGVuIC0gMV0g IT0gJ1xuJykgewpAQCAtMjU4OCw3ICsyNTkwLDcgQEAgcmVhZGhpc3RsaW5lKGludCBzdGFydCwg Y2hhciAqKmJ1ZnAsIGludCAqYnVmc2l6LCBGSUxFICppbikKIAkJICAgIHJldHVybiAtMTsKIAkJ KmJ1ZnAgPSB6cmVhbGxvYyhidWYsIDIgKiAoKmJ1ZnNpeikpOwogCQkqYnVmc2l6ID0gMiAqICgq YnVmc2l6KTsKLQkJcmV0dXJuIHJlYWRoaXN0bGluZShsZW4sIGJ1ZnAsIGJ1ZnNpeiwgaW4pOwor CQlyZXR1cm4gcmVhZGhpc3RsaW5lKGxlbiwgYnVmcCwgYnVmc2l6LCBpbiwgcmVhZGJ5dGVzKTsK IAkgICAgfQogCX0KIAllbHNlIHsKQEAgLTI1OTYsNyArMjU5OCw3IEBAIHJlYWRoaXN0bGluZShp bnQgc3RhcnQsIGNoYXIgKipidWZwLCBpbnQgKmJ1ZnNpeiwgRklMRSAqaW4pCiAJICAgIGlmIChs ZW4gPiAxICYmIGJ1ZltsZW4gLSAyXSA9PSAnXFwnKSB7CiAJCWJ1ZlstLWxlbiAtIDFdID0gJ1xu JzsKIAkJaWYgKCFmZW9mKGluKSkKLQkJICAgIHJldHVybiByZWFkaGlzdGxpbmUobGVuLCBidWZw LCBidWZzaXosIGluKTsKKwkJICAgIHJldHVybiByZWFkaGlzdGxpbmUobGVuLCBidWZwLCBidWZz aXosIGluLCByZWFkYnl0ZXMpOwogCSAgICB9CiAJfQogCXJldHVybiBsZW47CkBAIC0yNjE2LDcg KzI2MTgsNyBAQCByZWFkaGlzdGZpbGUoY2hhciAqZm4sIGludCBlcnIsIGludCByZWFkZmxhZ3Mp CiAgICAgc2hvcnQgKndvcmRzOwogICAgIHN0cnVjdCBzdGF0IHNiOwogICAgIGludCBud29yZHBv cywgbndvcmRzLCBidWZzaXo7Ci0gICAgaW50IHNlYXJjaGluZywgbmV3ZmxhZ3MsIGwsIHJldCwg dXNlbGV4OworICAgIGludCBzZWFyY2hpbmcsIG5ld2ZsYWdzLCBsLCByZXQsIHVzZWxleCwgcmVh ZGJ5dGVzOwogCiAgICAgaWYgKCFmbiAmJiAhKGZuID0gZ2V0c3BhcmFtKCJISVNURklMRSIpKSkK IAlyZXR1cm47CkBAIC0yNjU4LDEzICsyNjYwLDE1IEBAIHJlYWRoaXN0ZmlsZShjaGFyICpmbiwg aW50IGVyciwgaW50IHJlYWRmbGFncykKIAl9IGVsc2UKIAkgICAgc2VhcmNoaW5nID0gMDsKIAor CWZwb3MgPSBmdGVsbChpbik7CisJcmVhZGJ5dGVzID0gMDsKIAluZXdmbGFncyA9IEhJU1RfT0xE IHwgSElTVF9SRUFEOwogCWlmIChyZWFkZmxhZ3MgJiBIRklMRV9GQVNUKQogCSAgICBuZXdmbGFn cyB8PSBISVNUX0ZPUkVJR047CiAJaWYgKHJlYWRmbGFncyAmIEhGSUxFX1NLSVBPTEQKIAkgfHwg KGhpc3RfaWdub3JlX2FsbF9kdXBzICYmIG5ld2ZsYWdzICYgaGlzdF9za2lwX2ZsYWdzKSkKIAkg ICAgbmV3ZmxhZ3MgfD0gSElTVF9NQUtFVU5JUVVFOwotCXdoaWxlIChmcG9zID0gZnRlbGwoaW4p LCAobCA9IHJlYWRoaXN0bGluZSgwLCAmYnVmLCAmYnVmc2l6LCBpbikpKSB7CisJd2hpbGUgKGZw b3MgKz0gcmVhZGJ5dGVzLCByZWFkYnl0ZXMgPSAwLCAobCA9IHJlYWRoaXN0bGluZSgwLCAmYnVm LCAmYnVmc2l6LCBpbiwgJnJlYWRieXRlcykpKSB7CiAJICAgIGNoYXIgKnB0OwogCSAgICBpbnQg cmVtZXRhID0gMDsKIAotLSAKMi4yNS4wCgo= --00000000000010ff45059dfac345--