* [BUG?] Very strange behavior, execution of a function is interrupted @ 2018-08-31 22:01 Sebastian Gniazdowski 2018-09-01 19:12 ` Sebastian Gniazdowski 0 siblings, 1 reply; 4+ messages in thread From: Sebastian Gniazdowski @ 2018-08-31 22:01 UTC (permalink / raw) To: Zsh hackers list Hello, it's quite a lost game to explain this, but I'll try: - there is a scheduler called from sched +1 and once from zle -F - it detects timed-out tasks, moves them to ZPLG_RUN, then executes them reading them from this array Problem: sometimes execution never reaches this line and beyond: https://github.com/zdharma/zplugin/blob/7a24478e5862a1359b6cfb2887a792213be07e3a/zplugin.zsh#L1485 Some cycles of the preceding loop are then also skipped. I feel very helpless in explaining this, maybe there's standard questions sheet for situation where bug in exec.c is suspected? I know `break 2' inside a function will break out of loop that contains the function call. But I don't know what could exit a function. I have debug prints and observe this clearly. At the mentioned line, no execution occurs, ZPLG_RUN isn't cleared, I get to run some tasks twice. Twice when I checked, it was happening when zle -F was called so near next second, that timeout'd tasks detection included two time slots: "0" (0 seconds after first precmd, handled by zle -F) and "1" (1 second after first precmd, handled by sched +1). -- Sebastian Gniazdowski News: https://twitter.com/ZdharmaI IRC: https://kiwiirc.com/client/chat.freenode.net:+6697/#zplugin Blog: http://zdharma.org ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [BUG?] Very strange behavior, execution of a function is interrupted 2018-08-31 22:01 [BUG?] Very strange behavior, execution of a function is interrupted Sebastian Gniazdowski @ 2018-09-01 19:12 ` Sebastian Gniazdowski 2018-09-02 11:53 ` Sebastian Gniazdowski 0 siblings, 1 reply; 4+ messages in thread From: Sebastian Gniazdowski @ 2018-09-01 19:12 UTC (permalink / raw) To: Zsh hackers list I've found the cause. `compinit' does `read -q' when occurring insecure directories. If this happens from `sched' with possibly Zle being active, then following call inside bin_read() uses Zle's raw_getbyte() to read a character: zleentry(ZLE_CMD_GET_KEY, izle_timeout, NULL, &val); This means that `read' can make all the rich side-effects of the regular command-line to activate. I think this has significant importance, is severe. I wouldn't expect the `read' builtin to bring up whole `zle -F' handling, waiting for inputs, invoking callbacks. Below test code confirms the above thesis. Its execution is recorded here: https://asciinema.org/a/199265 Basically, my problem is that Zplugin's scheduler gets invoked in nested manner because of compinit's `read -q' (I was doing stress tests with compaudit activating) and data structures go to improper state. FD=1337; setup() { exec {FD}< <( sleep 1; echo run ) zle -F $FD -my-scheduler echo "Sched call reporting being called" read -q "?[y/n]?" echo "Sched call reporting exiting" } -my-scheduler() { local THEFD="$1"; zle -F "$THEFD"; exec {THEFD}<&- # remove zle -F handler echo "Zle -F handler reporting being called" } sched +1 setup On Sat, 1 Sep 2018 at 00:01, Sebastian Gniazdowski <sgniazdowski@gmail.com> wrote: > > Hello, > it's quite a lost game to explain this, but I'll try: > > - there is a scheduler called from sched +1 and once from zle -F > - it detects timed-out tasks, moves them to ZPLG_RUN, then executes > them reading them from this array > > Problem: sometimes execution never reaches this line and beyond: > https://github.com/zdharma/zplugin/blob/7a24478e5862a1359b6cfb2887a792213be07e3a/zplugin.zsh#L1485 > > Some cycles of the preceding loop are then also skipped. > > I feel very helpless in explaining this, maybe there's standard > questions sheet for situation where bug in exec.c is suspected? > > I know `break 2' inside a function will break out of loop that > contains the function call. But I don't know what could exit a > function. I have debug prints and observe this clearly. At the > mentioned line, no execution occurs, ZPLG_RUN isn't cleared, I get to > run some tasks twice. > > Twice when I checked, it was happening when zle -F was called so near > next second, that timeout'd tasks detection included two time slots: > "0" (0 seconds after first precmd, handled by zle -F) and "1" (1 > second after first precmd, handled by sched +1). > -- > Sebastian Gniazdowski > News: https://twitter.com/ZdharmaI > IRC: https://kiwiirc.com/client/chat.freenode.net:+6697/#zplugin > Blog: http://zdharma.org -- Sebastian Gniazdowski News: https://twitter.com/ZdharmaI IRC: https://kiwiirc.com/client/chat.freenode.net:+6697/#zplugin Blog: http://zdharma.org ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [BUG?] Very strange behavior, execution of a function is interrupted 2018-09-01 19:12 ` Sebastian Gniazdowski @ 2018-09-02 11:53 ` Sebastian Gniazdowski 2018-09-03 8:16 ` Peter Stephenson 0 siblings, 1 reply; 4+ messages in thread From: Sebastian Gniazdowski @ 2018-09-02 11:53 UTC (permalink / raw) To: Zsh hackers list Guys to state this more simply but also FUD-like: the little `read -q` brings up front whole Zle-command-line machinery, with sched, zle -F, waiting for inputs (select) on multiple file descriptors, running callbacks. Author of a script that asks y/n with `read -q` would never expect that he will run `sched' by it, embedding foreign scripts in middle of his script. This should have some attention, i.e. should be fixed. On Sat, 1 Sep 2018 at 21:12, Sebastian Gniazdowski <sgniazdowski@gmail.com> wrote: > > I've found the cause. `compinit' does `read -q' when occurring > insecure directories. If this happens from `sched' with possibly Zle > being active, then following call inside bin_read() uses Zle's > raw_getbyte() to read a character: > > zleentry(ZLE_CMD_GET_KEY, izle_timeout, NULL, &val); > > This means that `read' can make all the rich side-effects of the regular > command-line to activate. I think this has significant importance, is > severe. I wouldn't expect the `read' builtin to bring up whole `zle -F' > handling, waiting for inputs, invoking callbacks. > > Below test code confirms the above thesis. Its execution is recorded > here: https://asciinema.org/a/199265 Basically, my problem is that > Zplugin's scheduler gets invoked in nested manner because of > compinit's `read -q' (I was doing stress tests with compaudit > activating) and data structures go to improper state. > > > FD=1337; setup() { > exec {FD}< <( sleep 1; echo run ) > zle -F $FD -my-scheduler > echo "Sched call reporting being called" > read -q "?[y/n]?" > echo "Sched call reporting exiting" > } > -my-scheduler() { > local THEFD="$1"; zle -F "$THEFD"; exec {THEFD}<&- # remove zle -F handler > echo "Zle -F handler reporting being called" > } > sched +1 setup > > > On Sat, 1 Sep 2018 at 00:01, Sebastian Gniazdowski > <sgniazdowski@gmail.com> wrote: > > > > Hello, > > it's quite a lost game to explain this, but I'll try: > > > > - there is a scheduler called from sched +1 and once from zle -F > > - it detects timed-out tasks, moves them to ZPLG_RUN, then executes > > them reading them from this array > > > > Problem: sometimes execution never reaches this line and beyond: > > https://github.com/zdharma/zplugin/blob/7a24478e5862a1359b6cfb2887a792213be07e3a/zplugin.zsh#L1485 > > > > Some cycles of the preceding loop are then also skipped. > > > > I feel very helpless in explaining this, maybe there's standard > > questions sheet for situation where bug in exec.c is suspected? > > > > I know `break 2' inside a function will break out of loop that > > contains the function call. But I don't know what could exit a > > function. I have debug prints and observe this clearly. At the > > mentioned line, no execution occurs, ZPLG_RUN isn't cleared, I get to > > run some tasks twice. > > > > Twice when I checked, it was happening when zle -F was called so near > > next second, that timeout'd tasks detection included two time slots: > > "0" (0 seconds after first precmd, handled by zle -F) and "1" (1 > > second after first precmd, handled by sched +1). > > -- > > Sebastian Gniazdowski > > News: https://twitter.com/ZdharmaI > > IRC: https://kiwiirc.com/client/chat.freenode.net:+6697/#zplugin > > Blog: http://zdharma.org > > > > -- > Sebastian Gniazdowski > News: https://twitter.com/ZdharmaI > IRC: https://kiwiirc.com/client/chat.freenode.net:+6697/#zplugin > Blog: http://zdharma.org -- Sebastian Gniazdowski News: https://twitter.com/ZdharmaI IRC: https://kiwiirc.com/client/chat.freenode.net:+6697/#zplugin Blog: http://zdharma.org ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [BUG?] Very strange behavior, execution of a function is interrupted 2018-09-02 11:53 ` Sebastian Gniazdowski @ 2018-09-03 8:16 ` Peter Stephenson 0 siblings, 0 replies; 4+ messages in thread From: Peter Stephenson @ 2018-09-03 8:16 UTC (permalink / raw) To: Zsh hackers list On Sun, 2 Sep 2018 13:53:40 +0200 Sebastian Gniazdowski <sgniazdowski@gmail.com> wrote: > Guys to state this more simply but also FUD-like: the little `read -q` > brings up front whole Zle-command-line machinery, with sched, zle -F, > waiting for inputs (select) on multiple file descriptors, running > callbacks. This should stop the function execution feature if we're just reading a key, but I haven't checked the details work. diff --git a/Src/Zle/zle_keymap.c b/Src/Zle/zle_keymap.c index 2e96ac7..a5cf101 100644 --- a/Src/Zle/zle_keymap.c +++ b/Src/Zle/zle_keymap.c @@ -1518,7 +1518,7 @@ getrestchar_keybuf(void) * arrive together. If we don't do this the input can * get stuck if an invalid byte sequence arrives. */ - inchar = getbyte(1L, &timeout); + inchar = getbyte(1L, &timeout, 1); /* getbyte deliberately resets lastchar_wide_valid */ lastchar_wide_valid = 1; if (inchar == EOF) { @@ -1673,7 +1673,7 @@ addkeybuf(int c) static int getkeybuf(int w) { - int c = getbyte((long)w, NULL); + int c = getbyte((long)w, NULL, 1); if(c < 0) return EOF; diff --git a/Src/Zle/zle_main.c b/Src/Zle/zle_main.c index db70e7d..3487b5d 100644 --- a/Src/Zle/zle_main.c +++ b/Src/Zle/zle_main.c @@ -452,7 +452,7 @@ struct ztmout { */ static void -calc_timeout(struct ztmout *tmoutp, long do_keytmout) +calc_timeout(struct ztmout *tmoutp, long do_keytmout, int full) { if (do_keytmout && (keytimeout > 0 || do_keytmout < 0)) { if (do_keytmout < 0) @@ -465,7 +465,7 @@ calc_timeout(struct ztmout *tmoutp, long do_keytmout) } else tmoutp->tp = ZTM_NONE; - if (timedfns) { + if (full && timedfns) { for (;;) { LinkNode tfnode = firstnode(timedfns); Timedfn tfdat; @@ -504,7 +504,7 @@ calc_timeout(struct ztmout *tmoutp, long do_keytmout) /* see calc_timeout for use of do_keytmout */ static int -raw_getbyte(long do_keytmout, char *cptr) +raw_getbyte(long do_keytmout, char *cptr, int full) { int ret; struct ztmout tmout; @@ -519,7 +519,7 @@ raw_getbyte(long do_keytmout, char *cptr) # endif #endif - calc_timeout(&tmout, do_keytmout); + calc_timeout(&tmout, do_keytmout, full); /* * Handle timeouts and watched fd's. If a watched fd or a function @@ -684,7 +684,7 @@ raw_getbyte(long do_keytmout, char *cptr) * reconsider the key timeout from scratch. * The effect of this is microscopic. */ - calc_timeout(&tmout, do_keytmout); + calc_timeout(&tmout, do_keytmout, full); break; } /* @@ -810,7 +810,7 @@ raw_getbyte(long do_keytmout, char *cptr) # endif } /* If looping, need to recalculate timeout */ - calc_timeout(&tmout, do_keytmout); + calc_timeout(&tmout, do_keytmout, full); } # ifdef HAVE_POLL zfree(fds, sizeof(struct pollfd) * nfds); @@ -852,7 +852,7 @@ raw_getbyte(long do_keytmout, char *cptr) /**/ mod_export int -getbyte(long do_keytmout, int *timeout) +getbyte(long do_keytmout, int *timeout, int full) { char cc; unsigned int ret; @@ -877,7 +877,7 @@ getbyte(long do_keytmout, int *timeout) for (;;) { int q = queue_signal_level(); dont_queue_signals(); - r = raw_getbyte(do_keytmout, &cc); + r = raw_getbyte(do_keytmout, &cc, full); restore_queue_signals(q); if (r == -2) { /* timeout */ @@ -956,7 +956,7 @@ getbyte(long do_keytmout, int *timeout) mod_export ZLE_INT_T getfullchar(int do_keytmout) { - int inchar = getbyte((long)do_keytmout, NULL); + int inchar = getbyte((long)do_keytmout, NULL, 1); #ifdef MULTIBYTE_SUPPORT return getrestchar(inchar, NULL, NULL); @@ -1021,7 +1021,7 @@ getrestchar(int inchar, char *outstr, int *outcount) * arrive together. If we don't do this the input can * get stuck if an invalid byte sequence arrives. */ - inchar = getbyte(1L, &timeout); + inchar = getbyte(1L, &timeout, 1); /* getbyte deliberately resets lastchar_wide_valid */ lastchar_wide_valid = 1; if (inchar == EOF) { @@ -2139,7 +2139,7 @@ zle_main_entry(int cmd, va_list ap) do_keytmout = va_arg(ap, long); timeout = va_arg(ap, int *); chrp = va_arg(ap, int *); - *chrp = getbyte(do_keytmout, timeout); + *chrp = getbyte(do_keytmout, timeout, 0); break; } diff --git a/Src/Zle/zle_misc.c b/Src/Zle/zle_misc.c index 898b552..612ac21 100644 --- a/Src/Zle/zle_misc.c +++ b/Src/Zle/zle_misc.c @@ -756,7 +756,7 @@ bracketedstring(void) while (endesc[endpos]) { if (current + 1 >= psize) pbuf = zrealloc(pbuf, psize *= 2); - if ((next = getbyte(1L, &timeout)) == EOF) + if ((next = getbyte(1L, &timeout, 1)) == EOF) break; if (!endpos || next != endesc[endpos++]) endpos = (next == *endesc); @@ -970,7 +970,7 @@ universalargument(char **args) * * Hence for now this remains byte-by-byte. */ - while ((gotk = getbyte(0L, NULL)) != EOF) { + while ((gotk = getbyte(0L, NULL, 1)) != EOF) { if (gotk == '-' && !digcnt) { minus = -1; digcnt++; diff --git a/Src/Zle/zle_vi.c b/Src/Zle/zle_vi.c index e0923db..a5ff920 100644 --- a/Src/Zle/zle_vi.c +++ b/Src/Zle/zle_vi.c @@ -131,7 +131,7 @@ vigetkey(void) char m[3], *str; Thingy cmd; - if (getbyte(0L, NULL) == EOF) + if (getbyte(0L, NULL, 1) == EOF) return ZLEEOF; m[0] = lastchar; ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2018-09-03 8:24 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2018-08-31 22:01 [BUG?] Very strange behavior, execution of a function is interrupted Sebastian Gniazdowski 2018-09-01 19:12 ` Sebastian Gniazdowski 2018-09-02 11:53 ` Sebastian Gniazdowski 2018-09-03 8:16 ` Peter Stephenson
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/zsh/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).