From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <zsh-workers-return-40876-mason-zsh=primenet.com.au@zsh.org>
Received: (qmail 15225 invoked by alias); 21 Mar 2017 06:04:51 -0000
Mailing-List: contact zsh-workers-help@zsh.org; run by ezmlm
Precedence: bulk
X-No-Archive: yes
List-Id: Zsh Workers List <zsh-workers.zsh.org>
List-Post: <mailto:zsh-workers@zsh.org>
List-Help: <mailto:zsh-workers-help@zsh.org>
X-Seq: 40876
Received: (qmail 25674 invoked from network); 21 Mar 2017 06:04:51 -0000
X-Qmail-Scanner-Diagnostics: from new1-smtp.messagingengine.com by f.primenet.com.au (envelope-from <psprint3@fastmail.com>, uid 7791) with qmail-scanner-2.11 
 (clamdscan: 0.99.2/21882. spamassassin: 3.4.1.  
 Clear:RC:0(66.111.4.221):SA:0(-0.4/5.0):. 
 Processed in 3.025678 secs); 21 Mar 2017 06:04:51 -0000
X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on f.primenet.com.au
X-Spam-Level: 
X-Spam-Status: No, score=-0.4 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,
	FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SPF_PASS,T_DKIM_INVALID autolearn=unavailable
	autolearn_force=no version=3.4.1
X-Envelope-From: psprint3@fastmail.com
X-Qmail-Scanner-Mime-Attachments: |
X-Qmail-Scanner-Zip-Files: |
Received-SPF: pass (ns1.primenet.com.au: SPF record at spf.messagingengine.com designates 66.111.4.221 as permitted sender)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastmail.com; h=
	content-transfer-encoding:content-type:date:from:message-id
	:mime-version:subject:to:x-me-sender:x-me-sender:x-sasl-enc; s=
	fm1; bh=KxTe8jYY8gy6wXb24C2+PHyNFs24JhXyT4BVe9iydqs=; b=t0e5oG1x
	hAeDzGUFde3d9QM8m3eOmZ7SR0bN/wB63HU8KyXxJp+xgpvJycXplNmoixMzkaFq
	pU4aRSzALjqUrXDoJIWbrL+TOaL6tuDO47YzGTIJgR8x6tTCYYVzD5CzkInPIBHm
	O20wGBPVAWCVVuvX5y5oswM1FaawPAGIDZxubgMVAmOmUjkB2Cs3yN+89V8jzGCU
	wXFJdGaHy6J2SnnKr/JRPOkvdfO/ULdRtM0SNkya3/9pE9v0FRIQ7J8HKq1i3ae3
	i7QvkhCCkBERxMZeJjyrDPe0fnmnbLQ1/Flr9KB027HtWjJ7N9Ufqx3IOGNs6y6n
	uy4LJ1U4USDxWQ==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
	messagingengine.com; h=content-transfer-encoding:content-type
	:date:from:message-id:mime-version:subject:to:x-me-sender
	:x-me-sender:x-sasl-enc; s=fm1; bh=KxTe8jYY8gy6wXb24C2+PHyNFs24J
	hXyT4BVe9iydqs=; b=Zu741SclH/OiH4Ev2MMN2BUd1AUiGrN3M8g+/KXl15tMs
	ooIRZeeUJnR0uQLXCv8wptEy63/YqDPiJ6TyVvGymQELpX8mnsCROGkb6bgRpjS6
	qmN0bZVceALdFi3zBqh7Medr+gEOyzor7CUWyO7wgjnjG/MCEeGeUzbaCIlPYJCA
	5pPEQYmKJMi73O0WrqvlbG4rgw6Hy0HztQjoyYleFmemW/vRl247psRNbMRYTkx4
	5W1cVz84ga/ujtUeStZPSa88Rtw9CVgBhTjNwY4/CF14F5htpOsgH4mU1zB8V796
	CEq5dmvVXMHz1clr4siWRta/NaQz6tcj7Jx1qJsZw==
X-ME-Sender: <xms:esLQWPC2HuUouIOwIA-JhQDaiLnjqkQh5lWZBVuhnoUsmuUbC1QmQA>
Message-Id: <1490076282.321781.918060736.22D2C432@webmail.messagingengine.com>
From: Sebastian Gniazdowski <psprint3@fastmail.com>
To: zsh-workers@zsh.org
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset="utf-8"
X-Mailer: MessagingEngine.com Webmail Interface - ajax-aac97429
Subject: An idea for fast "last-N-lines" read
Date: Mon, 20 Mar 2017 23:04:42 -0700

Hello
I read somewhere that to read "last-N-lines" it is good to memory-map
the file. Cannot check with Zsh:

    for (( i=3Dsize; i>=3D1; --i)); do
        if [[ ${${mapfile[input.db]}[i]} =3D $'\n' ]]; then
            echo Got newline / $SECONDS
    ...

This gives:

Got newline / 0.1383100000
Got newline / 16.0876810000
Got newline / 26.8089250000

for 2 MB file =E2=80=93 apparently because it memory-maps the file on each
newline check.

So the idea is to add such feature. It would allow to run Zsh on
machines where e.g. periodic random check of last 1000 lines of
gigabyte-logs would be needed. It's possible that even Perl doesn't have
this. I'm thinking about: $(<10<filepath) syntax, that would return
buffer with 10 last lines, to be splited with (@f). Or maybe
$(<10L<filepath) for lines, and $(<10<filepath) for bytes. Maybe it's
easy to add? Otherwise, extension to zsh/mapfile could be added, or a
new module written.

BTW, (@f) skips trailing \n\n... That's quite problematic and there's
probably no workaround?
--=20
  Sebastian Gniazdowski
  psprint3@fastmail.com