From: Sebastian Gniazdowski <psprint3@fastmail.com>
To: zsh-workers@zsh.org
Subject: An idea for fast "last-N-lines" read
Date: Mon, 20 Mar 2017 23:04:42 -0700 [thread overview]
Message-ID: <1490076282.321781.918060736.22D2C432@webmail.messagingengine.com> (raw)
Hello
I read somewhere that to read "last-N-lines" it is good to memory-map
the file. Cannot check with Zsh:
for (( i=size; i>=1; --i)); do
if [[ ${${mapfile[input.db]}[i]} = $'\n' ]]; then
echo Got newline / $SECONDS
...
This gives:
Got newline / 0.1383100000
Got newline / 16.0876810000
Got newline / 26.8089250000
for 2 MB file – apparently because it memory-maps the file on each
newline check.
So the idea is to add such feature. It would allow to run Zsh on
machines where e.g. periodic random check of last 1000 lines of
gigabyte-logs would be needed. It's possible that even Perl doesn't have
this. I'm thinking about: $(<10<filepath) syntax, that would return
buffer with 10 last lines, to be splited with (@f). Or maybe
$(<10L<filepath) for lines, and $(<10<filepath) for bytes. Maybe it's
easy to add? Otherwise, extension to zsh/mapfile could be added, or a
new module written.
BTW, (@f) skips trailing \n\n... That's quite problematic and there's
probably no workaround?
--
Sebastian Gniazdowski
psprint3@fastmail.com
next reply other threads:[~2017-03-21 6:04 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-03-21 6:04 Sebastian Gniazdowski [this message]
2017-03-23 3:53 ` Bart Schaefer
[not found] ` <etPan.58d39baa.74b0dc51.10ab3@MacMini.local>
2017-03-23 10:17 ` Sebastian Gniazdowski
2017-03-23 16:46 ` Bart Schaefer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1490076282.321781.918060736.22D2C432@webmail.messagingengine.com \
--to=psprint3@fastmail.com \
--cc=zsh-workers@zsh.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.vuxu.org/mirror/zsh/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).