* Re: Why sourcing a file is not faster than doing a loop with eval, zle -N
[not found] ` <20170619161601.GB9294@chaz.gmail.com>
@ 2017-06-19 19:28 ` Peter Stephenson
2017-06-19 19:57 ` Bart Schaefer
` (2 more replies)
0 siblings, 3 replies; 10+ messages in thread
From: Peter Stephenson @ 2017-06-19 19:28 UTC (permalink / raw)
To: Zsh hackers list
On Mon, 19 Jun 2017 17:16:01 +0100
Stephane Chazelas <stephane.chazelas@gmail.com> wrote:
> That defeats a benefit of stdio saving read() systems calls by
> reading in chunk if we end up doing one system call per byte
> anyway.
Yes, if it's line buffered we should ideally only be unblocking
interrupts once around reading the line, which will dominate the
timing.
How about something like this? As far as I can tell, fgets is designed
from the ground up as Gets Done Properly, so if you have it on your
system it will work correctly. I can't think of a case where this
wouldn't do the right thing --- fgets will read at most one line and if
it does we were going to get the big STDIO overhead at that point
anyway.
pws
diff --git a/Src/input.c b/Src/input.c
index 92abaec..03d6476 100644
--- a/Src/input.c
+++ b/Src/input.c
@@ -147,6 +147,53 @@ shingetline(void)
for (;;) {
winch_unblock();
dont_queue_signals();
+#ifdef HAVE_FGETS
+ do {
+ errno = 0;
+ p = fgets(buf, BUFSIZ, bshin);
+ } while (!p && errno == EINTR);
+ winch_block();
+ if (!p) {
+ restore_queue_signals(q);
+ return NULL;
+ }
+ c = 0;
+ while (*p) {
+ if (imeta(STOUC(*p++)))
+ c++;
+ }
+ if (p > buf) {
+ /* p is pointing to '\0' */
+ ptrdiff_t nread = p - buf;
+ queue_signals();
+ line = zrealloc(line, ll + c + nread + 1);
+ if (c) {
+ char *dest = line + ll;
+ char *src = buf;
+ while (src < p) {
+ if (imeta(STOUC(*src))) {
+ *dest++ = Meta;
+ *dest++ = STOUC(*src++) ^ 32;
+ } else
+ *dest++ = *src++;
+ }
+ *dest++ = '\0';
+ } else {
+ /* copy null */
+ memcpy(line + ll, buf, nread + 1);
+ }
+ unqueue_signals();
+ /* fgets stops at first newline but stores '\0' after */
+ if (p[-1] == '\n') {
+ restore_queue_signals(q);
+ return line;
+ }
+ ll += nread;
+ } else {
+ restore_queue_signals(q);
+ return NULL;
+ }
+#else
do {
errno = 0;
c = fgetc(bshin);
@@ -178,6 +225,7 @@ shingetline(void)
p = buf;
unqueue_signals();
}
+#endif
}
}
diff --git a/configure.ac b/configure.ac
index 88da89e..2f19a73 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1325,7 +1325,8 @@ AC_CHECK_FUNCS(strftime strptime mktime timelocal \
cygwin_conv_path \
nanosleep \
srand_deterministic \
- setutxent getutxent endutxent getutent)
+ setutxent getutxent endutxent getutent \
+ fgets)
AC_FUNC_STRCOLL
AH_TEMPLATE([REALPATH_ACCEPTS_NULL],
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Why sourcing a file is not faster than doing a loop with eval, zle -N
2017-06-19 19:28 ` Why sourcing a file is not faster than doing a loop with eval, zle -N Peter Stephenson
@ 2017-06-19 19:57 ` Bart Schaefer
2017-06-19 20:38 ` Bart Schaefer
2017-06-19 20:52 ` Why sourcing a file is not faster than doing a loop with eval, zle -N Peter Stephenson
2 siblings, 0 replies; 10+ messages in thread
From: Bart Schaefer @ 2017-06-19 19:57 UTC (permalink / raw)
To: Zsh hackers list
On Jun 19, 8:28pm, Peter Stephenson wrote:
}
} How about something like this?
If that's really OK, then I think there's a simpler patch:
diff --git a/Src/input.c b/Src/input.c
index 92abaec..73e8b26 100644
--- a/Src/input.c
+++ b/Src/input.c
@@ -144,9 +144,9 @@ shingetline(void)
int q = queue_signal_level();
p = buf;
+ winch_unblock();
+ dont_queue_signals();
for (;;) {
- winch_unblock();
- dont_queue_signals();
do {
errno = 0;
c = fgetc(bshin);
@@ -176,7 +176,8 @@ shingetline(void)
ll += p - buf;
line[ll] = '\0';
p = buf;
- unqueue_signals();
+ winch_unblock();
+ dont_queue_signals();
}
}
}
--
Barton E. Schaefer
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Why sourcing a file is not faster than doing a loop with eval, zle -N
2017-06-19 19:28 ` Why sourcing a file is not faster than doing a loop with eval, zle -N Peter Stephenson
2017-06-19 19:57 ` Bart Schaefer
@ 2017-06-19 20:38 ` Bart Schaefer
2017-06-19 20:44 ` Peter Stephenson
2017-06-19 20:52 ` Why sourcing a file is not faster than doing a loop with eval, zle -N Peter Stephenson
2 siblings, 1 reply; 10+ messages in thread
From: Bart Schaefer @ 2017-06-19 20:38 UTC (permalink / raw)
To: Zsh hackers list
Parsing 10,000 lines, each a colon-command followed by misc. text, and
repeated 10 times.
HEAD from git:
Src/zsh -fs 0.31s user 1.24s system 94% cpu 1.634 total
Src/zsh -fs 0.45s user 1.31s system 98% cpu 1.795 total
Src/zsh -fs 0.39s user 1.41s system 95% cpu 1.877 total
Src/zsh -fs 0.31s user 1.16s system 88% cpu 1.652 total
Src/zsh -fs 0.43s user 1.33s system 94% cpu 1.854 total
Src/zsh -fs 0.43s user 1.22s system 92% cpu 1.784 total
Src/zsh -fs 0.37s user 1.32s system 91% cpu 1.848 total
Src/zsh -fs 0.31s user 1.48s system 91% cpu 1.948 total
Src/zsh -fs 0.42s user 1.37s system 92% cpu 1.926 total
Src/zsh -fs 0.30s user 1.52s system 92% cpu 1.964 total
Peter's patch:
Src/zsh -fs 0.11s user 0.59s system 73% cpu 0.953 total
Src/zsh -fs 0.10s user 0.54s system 68% cpu 0.940 total
Src/zsh -fs 0.17s user 0.56s system 71% cpu 1.017 total
Src/zsh -fs 0.19s user 0.58s system 79% cpu 0.969 total
Src/zsh -fs 0.13s user 0.65s system 74% cpu 1.040 total
Src/zsh -fs 0.05s user 0.65s system 74% cpu 0.945 total
Src/zsh -fs 0.19s user 0.64s system 82% cpu 1.009 total
Src/zsh -fs 0.18s user 0.59s system 81% cpu 0.944 total
Src/zsh -fs 0.34s user 0.47s system 80% cpu 1.002 total
Src/zsh -fs 0.35s user 0.52s system 89% cpu 0.971 total
My "simpler" patch:
Src/zsh -fs 0.17s user 0.79s system 85% cpu 1.119 total
Src/zsh -fs 0.28s user 0.77s system 90% cpu 1.160 total
Src/zsh -fs 0.19s user 0.82s system 88% cpu 1.146 total
Src/zsh -fs 0.20s user 0.70s system 82% cpu 1.087 total
Src/zsh -fs 0.20s user 0.65s system 84% cpu 1.011 total
Src/zsh -fs 0.31s user 0.57s system 86% cpu 1.021 total
Src/zsh -fs 0.22s user 0.72s system 81% cpu 1.148 total
Src/zsh -fs 0.21s user 0.60s system 81% cpu 0.999 total
Src/zsh -fs 0.19s user 0.63s system 78% cpu 1.046 total
Src/zsh -fs 0.23s user 0.81s system 83% cpu 1.242 total
The additional speedup in PWS's patch is probably due to fewer stdio
function calls, and skip of the check for buffer overflow on every
character read.
I don't think the HAVE_FGETS + configure change is needed, there are
already at least four places where we assume fgets() is available.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Why sourcing a file is not faster than doing a loop with eval, zle -N
2017-06-19 20:38 ` Bart Schaefer
@ 2017-06-19 20:44 ` Peter Stephenson
2017-06-20 11:28 ` fgets() portability (Was: Why sourcing a file is not faster than doing a loop with eval, zle -N) Stephane Chazelas
0 siblings, 1 reply; 10+ messages in thread
From: Peter Stephenson @ 2017-06-19 20:44 UTC (permalink / raw)
To: Zsh hackers list
On Mon, 19 Jun 2017 13:38:12 -0700
Bart Schaefer <schaefer@brasslantern.com> wrote:
> I don't think the HAVE_FGETS + configure change is needed, there are
> already at least four places where we assume fgets() is available.
It's been there since 2001 and I added one of them myself in 2011.
pws
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Why sourcing a file is not faster than doing a loop with eval, zle -N
2017-06-19 19:28 ` Why sourcing a file is not faster than doing a loop with eval, zle -N Peter Stephenson
2017-06-19 19:57 ` Bart Schaefer
2017-06-19 20:38 ` Bart Schaefer
@ 2017-06-19 20:52 ` Peter Stephenson
2017-06-19 23:01 ` Bart Schaefer
2017-12-23 15:01 ` Sebastian Gniazdowski
2 siblings, 2 replies; 10+ messages in thread
From: Peter Stephenson @ 2017-06-19 20:52 UTC (permalink / raw)
To: Zsh hackers list
On Mon, 19 Jun 2017 20:28:35 +0100
Peter Stephenson <p.w.stephenson@ntlworld.com> wrote:
> How about something like this? As far as I can tell, fgets is designed
> from the ground up as Gets Done Properly, so if you have it on your
> system it will work correctly. I can't think of a case where this
> wouldn't do the right thing --- fgets will read at most one line and if
> it does we were going to get the big STDIO overhead at that point
> anyway.
I know what the problem with it is --- we can't tell the difference
betwen a terminating '\0' and a '\0' that's part of the input stream.
pws
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Why sourcing a file is not faster than doing a loop with eval, zle -N
2017-06-19 20:52 ` Why sourcing a file is not faster than doing a loop with eval, zle -N Peter Stephenson
@ 2017-06-19 23:01 ` Bart Schaefer
2017-12-15 11:01 ` Sebastian Gniazdowski
2017-12-23 15:01 ` Sebastian Gniazdowski
1 sibling, 1 reply; 10+ messages in thread
From: Bart Schaefer @ 2017-06-19 23:01 UTC (permalink / raw)
To: Zsh hackers list
On Jun 19, 9:52pm, Peter Stephenson wrote:
}
} I know what the problem with it is --- we can't tell the difference
} betwen a terminating '\0' and a '\0' that's part of the input stream.
I'll add a comment to that effect and commit my patch from w/41322.
^ permalink raw reply [flat|nested] 10+ messages in thread
* fgets() portability (Was: Why sourcing a file is not faster than doing a loop with eval, zle -N)
2017-06-19 20:44 ` Peter Stephenson
@ 2017-06-20 11:28 ` Stephane Chazelas
0 siblings, 0 replies; 10+ messages in thread
From: Stephane Chazelas @ 2017-06-20 11:28 UTC (permalink / raw)
To: Peter Stephenson; +Cc: Zsh hackers list
2017-06-19 21:44:14 +0100, Peter Stephenson:
> On Mon, 19 Jun 2017 13:38:12 -0700
> Bart Schaefer <schaefer@brasslantern.com> wrote:
> > I don't think the HAVE_FGETS + configure change is needed, there are
> > already at least four places where we assume fgets() is available.
>
> It's been there since 2001 and I added one of them myself in 2011.
[...]
Out of curiosity, are there really systems that have fgetc() and not
fgets()? AFAICT, both were already there in the "original"
stdio implementation in Unix v7 in the 70s.
--
Stephane
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Why sourcing a file is not faster than doing a loop with eval, zle -N
2017-06-19 23:01 ` Bart Schaefer
@ 2017-12-15 11:01 ` Sebastian Gniazdowski
0 siblings, 0 replies; 10+ messages in thread
From: Sebastian Gniazdowski @ 2017-12-15 11:01 UTC (permalink / raw)
To: Bart Schaefer, Zsh hackers list
On 20 czerwca 2017 at 01:01:26, Bart Schaefer (schaefer@brasslantern.com) wrote:
> On Jun 19, 9:52pm, Peter Stephenson wrote:
> }
> } I know what the problem with it is --- we can't tell the difference
> } betwen a terminating '\0' and a '\0' that's part of the input stream.
>
> I'll add a comment to that effect and commit my patch from w/41322.
I have just noted a huge speed up of Zsh startup and ran bisect. I do:
repeat 10 { time /usr/local/bin/zsh -i -c exit }
and before 41322, speed is 400 ms, 350 ms when everything is compiled. With this patch, it is 220/200 ms. I thought this effect will be only for sourcing not-compiled files.
Basically this means that all the frameworks like Oh-My-Zsh can grow in size a "little" more ;)
With my hasher optimization that's 205/185 ms, but 41322 is just paradigm-shift.
--
Sebastian Gniazdowski
psprint /at/ zdharma.org
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Why sourcing a file is not faster than doing a loop with eval, zle -N
2017-06-19 20:52 ` Why sourcing a file is not faster than doing a loop with eval, zle -N Peter Stephenson
2017-06-19 23:01 ` Bart Schaefer
@ 2017-12-23 15:01 ` Sebastian Gniazdowski
2017-12-23 15:47 ` Sebastian Gniazdowski
1 sibling, 1 reply; 10+ messages in thread
From: Sebastian Gniazdowski @ 2017-12-23 15:01 UTC (permalink / raw)
To: Zsh hackers list, Peter Stephenson
On 19 czerwca 2017 at 22:52:37, Peter Stephenson (p.w.stephenson@ntlworld.com) wrote:
> On Mon, 19 Jun 2017 20:28:35 +0100
> Peter Stephenson wrote:
> > How about something like this? As far as I can tell, fgets is designed
> > from the ground up as Gets Done Properly, so if you have it on your
> > system it will work correctly. I can't think of a case where this
> > wouldn't do the right thing --- fgets will read at most one line and if
> > it does we were going to get the big STDIO overhead at that point
> > anyway.
>
> I know what the problem with it is --- we can't tell the difference
> betwen a terminating '\0' and a '\0' that's part of the input stream.
We could do a pair of calls, first fgets(), then fgetc(), and this way solve this problem?
--
Sebastian Gniazdowski
psprint /at/ zdharma.org
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Why sourcing a file is not faster than doing a loop with eval, zle -N
2017-12-23 15:01 ` Sebastian Gniazdowski
@ 2017-12-23 15:47 ` Sebastian Gniazdowski
0 siblings, 0 replies; 10+ messages in thread
From: Sebastian Gniazdowski @ 2017-12-23 15:47 UTC (permalink / raw)
To: Zsh hackers list, Peter Stephenson
On 23 grudnia 2017 at 16:01:38, Sebastian Gniazdowski (psprint@zdharma.org) wrote:
> We could do a pair of calls, first fgets(), then fgetc(), and this way solve this problem?
Sorry rather no, seems more complex than that.
--
Sebastian Gniazdowski
psprint /at/ zdharma.org
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2017-12-23 15:47 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <etPan.594513a8.516100cd.10b2e__10513.1716504276$1497699329$gmane$org@zdharma.org>
[not found] ` <20170619122413.GA9294@chaz.gmail.com>
[not found] ` <170619083116.ZM17323__41722.0601499595$1497886320$gmane$org@torch.brasslantern.com>
[not found] ` <20170619161601.GB9294@chaz.gmail.com>
2017-06-19 19:28 ` Why sourcing a file is not faster than doing a loop with eval, zle -N Peter Stephenson
2017-06-19 19:57 ` Bart Schaefer
2017-06-19 20:38 ` Bart Schaefer
2017-06-19 20:44 ` Peter Stephenson
2017-06-20 11:28 ` fgets() portability (Was: Why sourcing a file is not faster than doing a loop with eval, zle -N) Stephane Chazelas
2017-06-19 20:52 ` Why sourcing a file is not faster than doing a loop with eval, zle -N Peter Stephenson
2017-06-19 23:01 ` Bart Schaefer
2017-12-15 11:01 ` Sebastian Gniazdowski
2017-12-23 15:01 ` Sebastian Gniazdowski
2017-12-23 15:47 ` Sebastian Gniazdowski
Code repositories for project(s) associated with this public inbox
https://git.vuxu.org/mirror/zsh/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).