zsh-workers
 help / color / mirror / code / Atom feed
From: Sebastian Gniazdowski <psprint3@fastmail.com>
To: zsh-workers@zsh.org
Subject: An idea for fast "last-N-lines" read
Date: Mon, 20 Mar 2017 23:04:42 -0700	[thread overview]
Message-ID: <1490076282.321781.918060736.22D2C432@webmail.messagingengine.com> (raw)

Hello
I read somewhere that to read "last-N-lines" it is good to memory-map
the file. Cannot check with Zsh:

    for (( i=size; i>=1; --i)); do
        if [[ ${${mapfile[input.db]}[i]} = $'\n' ]]; then
            echo Got newline / $SECONDS
    ...

This gives:

Got newline / 0.1383100000
Got newline / 16.0876810000
Got newline / 26.8089250000

for 2 MB file – apparently because it memory-maps the file on each
newline check.

So the idea is to add such feature. It would allow to run Zsh on
machines where e.g. periodic random check of last 1000 lines of
gigabyte-logs would be needed. It's possible that even Perl doesn't have
this. I'm thinking about: $(<10<filepath) syntax, that would return
buffer with 10 last lines, to be splited with (@f). Or maybe
$(<10L<filepath) for lines, and $(<10<filepath) for bytes. Maybe it's
easy to add? Otherwise, extension to zsh/mapfile could be added, or a
new module written.

BTW, (@f) skips trailing \n\n... That's quite problematic and there's
probably no workaround?
-- 
  Sebastian Gniazdowski
  psprint3@fastmail.com


             reply	other threads:[~2017-03-21  6:04 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-03-21  6:04 Sebastian Gniazdowski [this message]
2017-03-23  3:53 ` Bart Schaefer
     [not found]   ` <etPan.58d39baa.74b0dc51.10ab3@MacMini.local>
2017-03-23 10:17     ` Sebastian Gniazdowski
2017-03-23 16:46       ` Bart Schaefer

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1490076282.321781.918060736.22D2C432@webmail.messagingengine.com \
    --to=psprint3@fastmail.com \
    --cc=zsh-workers@zsh.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/zsh/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).