Where to start debugging zle recursive-edit? / Ctrl-C

Where to start debugging zle recursive-edit? / Ctrl-C

[Sebastian Gniazdowski]

Hello,
I use a POSTDISPLAY interface for Ctrl-R, that works inside
.zle-recursive edit. Occurring problems with Ctrl-C every day. Some
limited response, not instant cancel of search, once even full
blockade of interruption. Funny that it became much more intensive
after adding few custom keymap's bindings for left/right cursors, page
up/page down keys. Also one user reports that Ctrl-C crashes Zsh.
Would add a debug print somewhere and observe, then extend the
debugging. But where?

Best regards,
Sebastian Gniazdowski

Re: Where to start debugging zle recursive-edit? / Ctrl-C

[Bart Schaefer]

On Sep 23,  6:26pm, Sebastian Gniazdowski wrote:
}
} I use a POSTDISPLAY interface for Ctrl-R, that works inside
} .zle-recursive edit. Occurring problems with Ctrl-C every day.
} [...] Funny that it became much more intensive after adding
} few custom keymap's bindings for left/right cursors

This probably means that the ^C is arriving while zle is waiting
to see if a binding has multiple "keystrokes" before determining
what widget to call.

Does emacs mode vs. viins or vicmd matter?  Does (un)setopt multibyte?

} Also one user reports that Ctrl-C crashes Zsh.

If this is with the latest build from git, it might be related to
the recently-introduced signal-handling problem with functions that
Daniel and I have been discussing elsewhere.

} Would add a debug print somewhere and observe, then extend the
} debugging. But where?

You're probably interested in getbyte() in zle_main.c, but you might
also attach with a debugger and put a breakpoint in zhandler so you
can get a stack trace of where the handler is being called [if it is].
(Starting the debugger first and running zsh inside it won't work as
well because the debugger itself will trap the signals).

Re: Where to start debugging zle recursive-edit? / Ctrl-C

[Sebastian Gniazdowski]

On 24 September 2016 at 22:31, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Sep 23,  6:26pm, Sebastian Gniazdowski wrote:
> }
> } I use a POSTDISPLAY interface for Ctrl-R, that works inside
> } .zle-recursive edit. Occurring problems with Ctrl-C every day.
> } [...] Funny that it became much more intensive after adding
> } few custom keymap's bindings for left/right cursors
>
> This probably means that the ^C is arriving while zle is waiting
> to see if a binding has multiple "keystrokes" before determining
> what widget to call.

That would explain what at first glance looks surprising.

> Does emacs mode vs. viins or vicmd matter?  Does (un)setopt multibyte?

I create custom keymap from emac's:

https://github.com/psprint/history-search-multi-word/blob/master/history-search-multi-word#L155

Rather hard to test viins and vicmd? Might try with multibyte when I
spot this happening and repeating. And one session now had this. Doing
unsetopt multibyte stopped this, restoring the option didn't cause the
issue to come back.

> } Also one user reports that Ctrl-C crashes Zsh.
>
> If this is with the latest build from git, it might be related to
> the recently-introduced signal-handling problem with functions that
> Daniel and I have been discussing elsewhere.

It's 5.1.1, and I've tested rather identical setup in coincidence:

https://github.com/psprint/history-search-multi-word/issues/2

Proposed a possible fix but user didn't respond if it helped:

- bindkey -N zhcw main
+ bindkey -N zhcw emacs

> } Would add a debug print somewhere and observe, then extend the
> } debugging. But where?
>
> You're probably interested in getbyte() in zle_main.c, but you might
> also attach with a debugger and put a breakpoint in zhandler so you
> can get a stack trace of where the handler is being called [if it is].
> (Starting the debugger first and running zsh inside it won't work as
> well because the debugger itself will trap the signals).

Thanks, will sure get on that tomorrow. I also sometimes get crashes.
Terminal window closes. Any tips for generating coredumps on OS X? I'm
able to google that when will have time, but also wanted to disable
-O, and ./configure --help | grep -i opt didn't show any option.

Best regards,
Sebastian Gniazdowski

Re: Where to start debugging zle recursive-edit? / Ctrl-C

[Daniel Shahaf]

Sebastian Gniazdowski wrote on Sat, Sep 24, 2016 at 22:58:19 +0200:
> but also wanted to disable -O, and ./configure --help | grep -i opt
> didn't show any option.

--enable-zsh-debug does that and more.

And for future reference CFLAGS=-O0 should have done it too.

Re: Where to start debugging zle recursive-edit? / Ctrl-C

[Sebastian Gniazdowski]

Hello,
have unoptimized HEAD Zsh, coredumps are set up, added some stressing
things to zshrc: TMOUT=1 function, PERIOD=1 hook, repeat 50; do sleep
1000000 &; done, coproc cat – and I'm waiting. The issue wasn't very
active in past two days and there was no good reason to start lldb.
PS. Any ideas of how to stress Zsh more?

Best regards,
Sebastian Gniazdowski

Re: Where to start debugging zle recursive-edit? / Ctrl-C

[Bart Schaefer]

On Sep 27, 12:16am, Sebastian Gniazdowski wrote:
}
} have unoptimized HEAD Zsh, coredumps are set up, added some stressing

If you're not seeing the problem under these circumstances, then it's
probably a race condition that's eliminated either by taking out the
code optimization or by adding the other processing.  Try first to
reproduce with the minimum differences from when you were seeing it
before, i.e., just debugging symbols and core dumps enabled, before
you start throwing in theoretical stressors.

Re: Where to start debugging zle recursive-edit? / Ctrl-C

[Sebastian Gniazdowski]

On 27 September 2016 at 18:09, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Sep 27, 12:16am, Sebastian Gniazdowski wrote:
> }
> } have unoptimized HEAD Zsh, coredumps are set up, added some stressing
>
> If you're not seeing the problem under these circumstances, then it's
> probably a race condition that's eliminated either by taking out the
> code optimization or by adding the other processing.

It seems so, hours after I switched to CFLAGS=-O2 compiled binary,
I've got a suspicious effect, after smoothly running no-opt binary for
few days – couldn't recall if I spotted anything suspicious. The
effect is message at shell startup:

TRAPALRM: scalar parameter abcdef created globally in function TRAPALRM"

It appeared just once, not long after switching to -O2. I don't use
warn_create_global, and did a test that Mikael suggested to ensure:

function setopt() { print -r - "$funcfiletrace[1]: $0 ${(q-)@}" >>!
/tmp/setopt_data; builtin $0 "$@" }

and only "setopt localoptions warncreateglobal" from z-sy-h was
reported. Zshrc has at the end:

TRAPALRM() { abcdef=$(( abcdef + 1 )); }
TMOUT=1
PERIODIC_HOOK() { efghij=$(( efghij + 1 )); }
PERIOD=1
add-zsh-hook periodic PERIODIC_HOOK
repeat 50; do sleep 1000000 &; done
coproc cat

My aim is .recursive-edit and Ctrl-C, but just want to hook at
something first, have any backtrace, I suspect it will be easy to
connect with problems with .recursive-edit even when not directly
related to it. Crash related to Ctrl-C appeared in uncanny situations
AFAIR like Ctrl-R, Ctrl-C multiple times, then menu search completion
with also Ctrl-C, something like this, hard to reproduce.

Best regards,
Sebastian Gniazdowski

Re: Where to start debugging zle recursive-edit? / Ctrl-C

[Bart Schaefer]

On Sep 29, 11:30am, Sebastian Gniazdowski wrote:
} Subject: Re: Where to start debugging zle recursive-edit? / Ctrl-C
} 
} I've got a suspicious effect, after smoothly running no-opt binary for
} few days - couldn't recall if I spotted anything suspicious. The
} effect is message at shell startup:
} 
} TRAPALRM: scalar parameter abcdef created globally in function TRAPALRM"
} 
} [...]
} 
} and only "setopt localoptions warncreateglobal" from z-sy-h was
} reported. Zshrc has at the end:
} 
} TRAPALRM() { abcdef=$(( abcdef + 1 )); }
} TMOUT=1

So what's happening is that z-sy-h begins executing, and then *while* it
is executing, the first TMOUT occurs and TRAPALRM executes.  It therefore
picks up the warncreateglobal from the z-sy-h context and so you get the
warning about the assignment to abcdef.

This could happen any time that z-sy-h is active simultaneously with the
TMOUT alarm.

} Crash related to Ctrl-C appeared in uncanny situations
} AFAIR like Ctrl-R, Ctrl-C multiple times, then menu search completion
} with also Ctrl-C, something like this, hard to reproduce.

Memory management for menu selection is known to be somewhat fragile;
other crashes have occurred there.  Usually this seems to happen when
tracking of where things should be drawn on the screen gets out of
sync with the data structures describing what to draw, see for example
workers/37961 (commit f07a1bd0).  It would not surprise me to find
that an ill-timed interrupt could result in the same sort of problem.

There are some queue_signals() regions in complist.c but they're very
narrowly defined.

Re: Where to start debugging zle recursive-edit? / Ctrl-C

[Sebastian Gniazdowski]

[-- Attachment #1: Type: text/plain, Size: 1358 bytes --]

On 24 September 2016 at 22:31, Bart Schaefer <schaefer@brasslantern.com> wrote:
> You're probably interested in getbyte() in zle_main.c, but you might
> also attach with a debugger and put a breakpoint in zhandler so you
> can get a stack trace of where the handler is being called [if it is].
> (Starting the debugger first and running zsh inside it won't work as
> well because the debugger itself will trap the signals).

I've set breakpoint to zhandler, it was called on each Ctrl-C. And
.recursiveedit never exited. Following execution paths I've spotted
crucial places and added debug prints. Luckily it's easy to trigger
".recursiveedit not canceled" and ".rec. cancelled" paths when not in
debugger. Attached is 10-context lines diff. Exit from .recursiveedit
is:

-- Doing break / zle_main.c: errflag: 2, retflag: 0, breaks: 0, exit_pending: 0
-- Got EINTR 1
-- Setting error in zlecore.c

I've also observed that following possible, though rare:

-- Got EINTR 1
-- Setting error in zlecore.c

No reaction on Ctrl-C, i.e. no-exit from .recursiveedit, is:

-- Trying again !errtry / zle_main.c
(now another Ctrl-C)
-- Doing break / zle_main.c: errflag: 2, retflag: 0, breaks: 0, exit_pending: 0
-- Got EINTR 2
-- Setting error in zlecore.c

I will go further into this but apparently zhandler should set errflag?

Best regards,
Sebastian Gniazdowski

[-- Attachment #2: rec_edit.diff.txt --]
[-- Type: text/plain, Size: 3571 bytes --]

diff --git a/Src/Zle/zle_main.c b/Src/Zle/zle_main.c
index 9a83d41..36884c4 100644
--- a/Src/Zle/zle_main.c
+++ b/Src/Zle/zle_main.c
@@ -612,28 +612,40 @@ raw_getbyte(long do_keytmout, char *cptr)
 		tvptr = NULL;
 
 	    winch_unblock();
 	    selret = select(fdmax+1, (SELECT_ARG_2_T) & foofd,
 			    NULL, NULL, tvptr);
 	    winch_block();
 # endif
 	    /*
 	     * Make sure a user interrupt gets passed on straight away.
 	     */
-	    if (selret < 0 && (errflag || retflag || breaks || exit_pending))
+	    if (selret < 0 && (errflag || retflag || breaks || exit_pending)) {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "-- Doing break / zle_main.c: errflag: %d, retflag: %d, breaks: %d, exit_pending: %d\n",
+                            errflag, retflag, breaks, exit_pending );
+                fclose(_F);
 		break;
+            }
+
 	    /*
 	     * Try to avoid errors on our special fd's from
 	     * messing up reads from the terminal.  Try first
 	     * with all fds, then try unsetting the special ones.
 	     */
 	    if (selret < 0 && !errtry) {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "-- Trying again !errtry / zle_main.c\n" );
+                fclose(_F);
+
 		errtry = 1;
 		continue;
 	    }
 	    if (selret == 0) {
 		/*
 		 * Nothing ready and no error, so we timed out.
 		 */
 		switch (tmout.tp) {
 		case ZTM_NONE:
 		    /* keeps compiler happy if not debugging */
@@ -888,22 +900,36 @@ getbyte(long do_keytmout, int *timeout)
 		   the counter (icnt) so that this happens 20 times and than
 		   the shell gives up (yes, this is a bit dirty...). */
 		if ((zlereadflags & ZLRF_IGNOREEOF) && icnt++ < 20)
 		    continue;
 		stopmsg = 1;
 		zexit(1, 0);
 	    }
 	    icnt = 0;
 	    if (errno == EINTR) {
 		die = 0;
+                static int counter = 0;
+
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "-- Got EINTR %d\n", ++counter );
+                fclose(_F);
+
 		if (!errflag && !retflag && !breaks && !exit_pending)
+                {
+                    // MY DEBUG
+                    FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                    fprintf( _F, "-- Continuing despite EINTR / zle_main.c: errflag: %d, retflag: %d, breaks: %d, exit_pending: %d\n",
+                                errflag, retflag, breaks, exit_pending );
+                    fclose(_F);
 		    continue;
+                }
 		errflag &= ~ERRFLAG_ERROR;
 		breaks = obreaks;
 		errno = old_errno;
 		return lastchar = EOF;
 	    } else if (errno == EWOULDBLOCK) {
 		fcntl(0, F_SETFL, 0);
 	    } else if (errno == EIO && !die) {
 		ret = opts[MONITOR];
 		opts[MONITOR] = 1;
 		attachtty(mypgrp);
@@ -1115,20 +1141,24 @@ zlecore(void)
 		if (eofsent)
 		    break;
 	    }
 	    handleprefixes();
 	    /* for vi mode, make sure the cursor isn't somewhere illegal */
 	    if (invicmdmode() && zlecs > findbol() &&
 		(zlecs == zlell || zleline[zlecs] == ZWC('\n')))
 		DECCS();
 	    handleundo();
 	} else {
+            // MY DEBUG
+            FILE *_F = fopen("/tmp/recursive.txt", "a+");
+            fprintf( _F, "-- Setting error in zlecore.c\n" );
+            fclose(_F);
 	    errflag |= ERRFLAG_ERROR;
 	    break;
 	}
 
 	redrawhook();
 #ifdef HAVE_POLL
 	if (baud && !(lastcmd & ZLE_MENUCMP)) {
 	    struct pollfd pfd;
 	    int to = cost * costmult / 1000; /* milliseconds */
 

Re: Where to start debugging zle recursive-edit? / Ctrl-C

[Sebastian Gniazdowski]

[-- Attachment #1: Type: text/plain, Size: 2040 bytes --]

On 24 September 2016 at 22:31, Bart Schaefer <schaefer@brasslantern.com> wrote:
> You're probably interested in getbyte() in zle_main.c, but you might
> also attach with a debugger and put a breakpoint in zhandler so you
> can get a stack trace of where the handler is being called [if it is].
> (Starting the debugger first and running zsh inside it won't work as
> well because the debugger itself will trap the signals).

It turned out that lldb stops at Ctrl-C even without b zhandler and
this is what I've been receiving. The stack trace is at the end.

But I've catched another thing. After Ctrl-C in .recursiveedit the
timeouts at select() in raw_getbyte() stop occuring. Here is simple
video that very well covers this. It shows debug prints stop appearing
(attached 10-context lines diff). And Zconvey stops working, i.e.
sched doesn't call its functions:

https://asciinema.org/a/7s5dofinxlnn1i80ojunzybp7



----------------- BT 10 / LLDB ----------------

(lldb) bt 10
* thread #1: tid = 0x1c441b, 0x00007fff897b107a
libsystem_kernel.dylib`__select + 10, queue = 'com.apple.main-thread',
stop reason = signal SIGINT
  * frame #0: 0x00007fff897b107a libsystem_kernel.dylib`__select + 10
    frame #1: 0x00000001082b70e0 zle.so`raw_getbyte(do_keytmout=0,
cptr="") + 640 at zle_main.c:615
    frame #2: 0x00000001082b6abf zle.so`getbyte(do_keytmout=0,
timeout=0x0000000000000000) + 319 at zle_main.c:893
    frame #3: 0x00000001082b5798 zle.so`getkeybuf(w=0) + 24 at zle_keymap.c:1663
    frame #4: 0x00000001082b54b6
zle.so`getkeymapcmd(km=0x00007fbe3201ce00, funcp=0x00007fff57c81c10,
strp=0x00007fff57c81bf8) + 102 at zle_keymap.c:1581
    frame #5: 0x00000001082b5903 zle.so`getkeycmd + 35 at zle_keymap.c:1692
    frame #6: 0x00000001082b8430 zle.so`zlecore + 320 at zle_main.c:1136
    frame #7: 0x00000001082b951b
zle.so`recursiveedit(args=0x00000001083d3a00) + 27 at zle_main.c:1903
    frame #8: 0x00000001082b7ea2
zle.so`execzlefunc(func=0x00000001082ea408, args=0x00000001083d3a00,
set_bindk=1) + 770 at zle_main.c:1455

[-- Attachment #2: rec2_edit.2.diff.txt --]
[-- Type: text/plain, Size: 4228 bytes --]

diff --git a/Src/Zle/zle_main.c b/Src/Zle/zle_main.c
index 9a83d41..7f3da8d 100644
--- a/Src/Zle/zle_main.c
+++ b/Src/Zle/zle_main.c
@@ -612,31 +612,55 @@ raw_getbyte(long do_keytmout, char *cptr)
 		tvptr = NULL;
 
 	    winch_unblock();
 	    selret = select(fdmax+1, (SELECT_ARG_2_T) & foofd,
 			    NULL, NULL, tvptr);
 	    winch_block();
 # endif
 	    /*
 	     * Make sure a user interrupt gets passed on straight away.
 	     */
-	    if (selret < 0 && (errflag || retflag || breaks || exit_pending))
+	    if (selret < 0 && (errflag || retflag || breaks || exit_pending)) {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "-- Doing break / zle_main.c: errflag: %d, retflag: %d, breaks: %d, exit_pending: %d\n",
+                            errflag, retflag, breaks, exit_pending );
+                fclose(_F);
 		break;
+            } else {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "-- NOT doing break / zle_main.c: errflag: %d, retflag: %d, breaks: %d, exit_pending: %d\n",
+                            errflag, retflag, breaks, exit_pending );
+                fclose(_F);
+            }
+
 	    /*
 	     * Try to avoid errors on our special fd's from
 	     * messing up reads from the terminal.  Try first
 	     * with all fds, then try unsetting the special ones.
 	     */
 	    if (selret < 0 && !errtry) {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "-- Trying again !errtry / zle_main.c\n" );
+                fclose(_F);
+
 		errtry = 1;
 		continue;
-	    }
+	    } else {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "--  Passed !errtry(errtry:%d) / zle_main.c\n", errtry );
+                fclose(_F);
+            }
+
 	    if (selret == 0) {
 		/*
 		 * Nothing ready and no error, so we timed out.
 		 */
 		switch (tmout.tp) {
 		case ZTM_NONE:
 		    /* keeps compiler happy if not debugging */
 #ifdef DEBUG
 		    dputs("BUG: timeout fired with no timeout set.");
 #endif
@@ -888,22 +912,36 @@ getbyte(long do_keytmout, int *timeout)
 		   the counter (icnt) so that this happens 20 times and than
 		   the shell gives up (yes, this is a bit dirty...). */
 		if ((zlereadflags & ZLRF_IGNOREEOF) && icnt++ < 20)
 		    continue;
 		stopmsg = 1;
 		zexit(1, 0);
 	    }
 	    icnt = 0;
 	    if (errno == EINTR) {
 		die = 0;
+                static int counter = 0;
+
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "-- Got EINTR %d\n", ++counter );
+                fclose(_F);
+
 		if (!errflag && !retflag && !breaks && !exit_pending)
+                {
+                    // MY DEBUG
+                    FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                    fprintf( _F, "-- Continuing despite EINTR / zle_main.c: errflag: %d, retflag: %d, breaks: %d, exit_pending: %d\n",
+                                errflag, retflag, breaks, exit_pending );
+                    fclose(_F);
 		    continue;
+                }
 		errflag &= ~ERRFLAG_ERROR;
 		breaks = obreaks;
 		errno = old_errno;
 		return lastchar = EOF;
 	    } else if (errno == EWOULDBLOCK) {
 		fcntl(0, F_SETFL, 0);
 	    } else if (errno == EIO && !die) {
 		ret = opts[MONITOR];
 		opts[MONITOR] = 1;
 		attachtty(mypgrp);
@@ -1115,20 +1153,24 @@ zlecore(void)
 		if (eofsent)
 		    break;
 	    }
 	    handleprefixes();
 	    /* for vi mode, make sure the cursor isn't somewhere illegal */
 	    if (invicmdmode() && zlecs > findbol() &&
 		(zlecs == zlell || zleline[zlecs] == ZWC('\n')))
 		DECCS();
 	    handleundo();
 	} else {
+            // MY DEBUG
+            FILE *_F = fopen("/tmp/recursive.txt", "a+");
+            fprintf( _F, "-- Setting error in zlecore.c\n" );
+            fclose(_F);
 	    errflag |= ERRFLAG_ERROR;
 	    break;
 	}
 
 	redrawhook();
 #ifdef HAVE_POLL
 	if (baud && !(lastcmd & ZLE_MENUCMP)) {
 	    struct pollfd pfd;
 	    int to = cost * costmult / 1000; /* milliseconds */
 

Re: Where to start debugging zle recursive-edit? / Ctrl-C

[Sebastian Gniazdowski]

[-- Attachment #1: Type: text/plain, Size: 1843 bytes --]

On 30 September 2016 at 13:28, Sebastian Gniazdowski
<sgniazdowski@gmail.com> wrote:
> But I've catched another thing. After Ctrl-C in .recursiveedit the
> timeouts at select() in raw_getbyte() stop occuring. Here is simple

I've investigated this further. There is calc_timeout() called in the
select-loop:

  case ZTM_MAX:
      /*...*/
      calc_timeout(&tmout, do_keytmout);
      // MY DEBUG
      FILE *_F = fopen("/tmp/recursive.txt", "a+");
      fprintf( _F, "^^^ LOOP-CALLED calc_timeout: tmout.tp == %d\n", tmout.tp );
      fclose(_F);
      break;


The calc_timeout is capable of setting tmout.tp to ZTM_NONE. I have 3
debug prints:

-- tmoutp->tp <- ZTM_NONE / calc_timeout( do_keytmout: 0 ), keytimeout: 4
== CALC_TIMEOUT() one more chance (timedfns)
-- CALC_TIMEOUT() !tfnode break

Code of the last debug message is below:

            LinkNode tfnode = firstnode(timedfns);
            Timedfn tfdat;
            time_t diff, exp100ths;

            if (!tfnode) {
                // MY DEBUG
                FILE *_F = fopen("/tmp/recursive.txt", "a+");
                fprintf( _F, "-- CALC_TIMEOUT() !tfnode break\n",
tmoutp->exp100ths );
                fclose(_F);
                break;
            }

So, the linked list doesn't have any elements, and calc_timeout()
exits with ZTM_NONE instead of ZTM_FUNC like it does all the time (so
normally it executes further the for(;;) there and overwrites initial
ZTM_NONE with ZTM_FUNC). This is after the lucky Ctrl-C. Wonder why no
ZTM_KEY? Also, believe me, I occured 2 spontaneous stopping of
timeouting right after shell started. Tried to catch this on asciinema
but no luck. It looket like – few timeouts occured and then they
stopped. Didn't have precise debug prints then..

Best regards,
Sebastian Gniazdowski

[-- Attachment #2: rec3_edit.3.diff.txt --]
[-- Type: text/plain, Size: 9621 bytes --]

diff --git a/Src/Zle/zle_main.c b/Src/Zle/zle_main.c
index 9a83d41..1f993fe 100644
--- a/Src/Zle/zle_main.c
+++ b/Src/Zle/zle_main.c
@@ -450,47 +450,82 @@ static void
 calc_timeout(struct ztmout *tmoutp, long do_keytmout)
 {
     if (do_keytmout && (keytimeout > 0 || do_keytmout < 0)) {
 	if (do_keytmout < 0)
 	    tmoutp->exp100ths = (time_t)-do_keytmout;
 	else if (keytimeout > ZMAXTIMEOUT * 100 /* 24 days for a keypress???? */)
 	    tmoutp->exp100ths = ZMAXTIMEOUT * 100;
 	else
 	    tmoutp->exp100ths = keytimeout;
 	tmoutp->tp = ZTM_KEY;
-    } else
+        // MY DEBUG
+        FILE *_F = fopen("/tmp/recursive.txt", "a+");
+        fprintf( _F, "-- tmoutp->tp <- ZTM_KEY (%d) / calc_timeout() zle_main.c\n", tmoutp->exp100ths );
+        fclose(_F);
+    } else {
+        // MY DEBUG
+        FILE *_F = fopen("/tmp/recursive.txt", "a+");
+        fprintf( _F, "-- tmoutp->tp <- ZTM_NONE / calc_timeout( do_keytmout: %d ), keytimeout: %d / zle_main.c\n", do_keytmout );
+        fclose(_F);
+
 	tmoutp->tp = ZTM_NONE;
+    }
 
     if (timedfns) {
+        // MY DEBUG
+        FILE *_F = fopen("/tmp/recursive.txt", "a+");
+        fprintf( _F, "== CALC_TIMEOUT() one more chance (timedfns)\n", tmoutp->exp100ths );
+        fclose(_F);
 	for (;;) {
 	    LinkNode tfnode = firstnode(timedfns);
 	    Timedfn tfdat;
 	    time_t diff, exp100ths;
 
-	    if (!tfnode)
+	    if (!tfnode) {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "-- CALC_TIMEOUT() !tfnode break\n", tmoutp->exp100ths );
+                fclose(_F);
 		break;
+            }
 
 	    tfdat = (Timedfn)getdata(tfnode);
 	    diff = tfdat->when - time(NULL);
+
+            // MY DEBUG
+            FILE *_F = fopen("/tmp/recursive.txt", "a+");
+            fprintf( _F, "-- CALC_TIMEOUT() tfnode TRUE no break >> DIFF=%d <<\n", diff );
+            fclose(_F);
+
 	    if (diff < 0) {
 		/* Already due; call it and rescan. */
 		tfdat->func();
 		continue;
 	    }
 
 	    if (diff > ZMAXTIMEOUT) {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "-- CALC_TIMEOUT() %d > %d ZTM_MAX(%d)\n", diff, ZMAXTIMEOUT, ZTM_MAX );
+                fclose(_F);
+
 		tmoutp->exp100ths = ZMAXTIMEOUT * 100;
 		tmoutp->tp = ZTM_MAX;
 	    } else if (diff > 0) {
 		exp100ths = diff * 100;
 		if (tmoutp->tp != ZTM_KEY ||
 		    exp100ths < tmoutp->exp100ths) {
+                    // MY DEBUG
+                    FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                    fprintf( _F, "-- CALC_TIMEOUT() %d != %d || %d < %d ZTM_FUNC(%d)\n", tmoutp->tp, ZTM_KEY, exp100ths, tmoutp->exp100ths, ZTM_FUNC );
+                    fclose(_F);
+
 		    tmoutp->exp100ths = exp100ths;
 		    tmoutp->tp = ZTM_FUNC;
 		}
 	    }
 	    break;
 	}
 	/* In case we called a function which messed up the display... */
 	if (resetneeded)
 	    zrefresh();
     }
@@ -508,20 +543,24 @@ raw_getbyte(long do_keytmout, char *cptr)
     struct ttyinfo ti;
 #endif
 #ifndef HAVE_POLL
 # ifdef HAVE_SELECT
     fd_set foofd, errfd;
     FD_ZERO(&errfd);
 # endif
 #endif
 
     calc_timeout(&tmout, do_keytmout);
+    // MY DEBUG
+    FILE *_F = fopen("/tmp/recursive.txt", "a+");
+    fprintf( _F, "-- INIT tmout.tp(%d) ZTM_NONE(%d) ZTM_KEY(%d) / BEGIN RAW_GETBYTE() zle_main.c\n", tmout.tp, ZTM_NONE, ZTM_KEY );
+    fclose(_F);
 
     /*
      * Handle timeouts and watched fd's.  If a watched fd or a function
      * timeout triggers we restart any key timeout.  This is likely to
      * be harmless: the combination is extremely rare and a function
      * is likely to occupy the user for a little while anyway.  We used
      * to make timeouts take precedence, but we can't now that the
      * timeouts may be external, so we may have both a permanent watched
      * fd and a long-term timeout.
      */
@@ -600,43 +639,78 @@ raw_getbyte(long do_keytmout, char *cptr)
 		    if (fd > fdmax)
 			fdmax = fd;
 		}
 	    }
 	    FD_ZERO(&errfd);
 
 	    if (tmout.tp != ZTM_NONE) {
 		expire_tv.tv_sec = tmout.exp100ths / 100;
 		expire_tv.tv_usec = (tmout.exp100ths % 100) * 10000L;
 		tvptr = &expire_tv;
+
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "\n-- tmoutp != ZTM_NONE / raw_getbyte() zle_main.c\n" );
+                fclose(_F);
 	    }
-	    else
+	    else {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "\n-- FINAL (STOP) tmoutp == ZTM_NONE / raw_getbyte() zle_main.c\n" );
+                fclose(_F);
+
 		tvptr = NULL;
+            }
 
 	    winch_unblock();
 	    selret = select(fdmax+1, (SELECT_ARG_2_T) & foofd,
 			    NULL, NULL, tvptr);
 	    winch_block();
 # endif
 	    /*
 	     * Make sure a user interrupt gets passed on straight away.
 	     */
-	    if (selret < 0 && (errflag || retflag || breaks || exit_pending))
+	    if (selret < 0 && (errflag || retflag || breaks || exit_pending)) {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "-- Doing break / zle_main.c: errflag: %d, retflag: %d, breaks: %d, exit_pending: %d\n",
+                            errflag, retflag, breaks, exit_pending );
+                fclose(_F);
 		break;
+            } else {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "-- NOT doing break / zle_main.c: errflag: %d, retflag: %d, breaks: %d, exit_pending: %d\n",
+                            errflag, retflag, breaks, exit_pending );
+                fclose(_F);
+            }
+
 	    /*
 	     * Try to avoid errors on our special fd's from
 	     * messing up reads from the terminal.  Try first
 	     * with all fds, then try unsetting the special ones.
 	     */
 	    if (selret < 0 && !errtry) {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "-- Trying again !errtry / zle_main.c\n" );
+                fclose(_F);
+
 		errtry = 1;
 		continue;
-	    }
+	    } else {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "--  Passed !errtry(errtry:%d) / zle_main.c\n", errtry );
+                fclose(_F);
+            }
+
 	    if (selret == 0) {
 		/*
 		 * Nothing ready and no error, so we timed out.
 		 */
 		switch (tmout.tp) {
 		case ZTM_NONE:
 		    /* keeps compiler happy if not debugging */
 #ifdef DEBUG
 		    dputs("BUG: timeout fired with no timeout set.");
 #endif
@@ -669,20 +743,24 @@ raw_getbyte(long do_keytmout, char *cptr)
 		case ZTM_MAX:
 		    /*
 		     * Reached the limit of our range, but not the
 		     * actual timeout; recalculate the timeout.
 		     * We're cheating with the key timeout here:
 		     * if one clashed with a function timeout we
 		     * reconsider the key timeout from scratch.
 		     * The effect of this is microscopic.
 		     */
 		    calc_timeout(&tmout, do_keytmout);
+                    // MY DEBUG
+                    FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                    fprintf( _F, "^^^ LOOP-CALLED calc_timeout: tmout.tp == %d / zle_main.c\n", tmout.tp );
+                    fclose(_F);
 		    break;
 		}
 		/*
 		 * If we handled the timeout successfully,
 		 * carry on.
 		 */
 		if (selret == 0)
 		    continue;
 	    }
 	    /* If error or unhandled timeout, give up. */
@@ -888,22 +966,36 @@ getbyte(long do_keytmout, int *timeout)
 		   the counter (icnt) so that this happens 20 times and than
 		   the shell gives up (yes, this is a bit dirty...). */
 		if ((zlereadflags & ZLRF_IGNOREEOF) && icnt++ < 20)
 		    continue;
 		stopmsg = 1;
 		zexit(1, 0);
 	    }
 	    icnt = 0;
 	    if (errno == EINTR) {
 		die = 0;
+                static int counter = 0;
+
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "-- Got EINTR %d\n", ++counter );
+                fclose(_F);
+
 		if (!errflag && !retflag && !breaks && !exit_pending)
+                {
+                    // MY DEBUG
+                    FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                    fprintf( _F, "-- Continuing despite EINTR / zle_main.c: errflag: %d, retflag: %d, breaks: %d, exit_pending: %d\n",
+                                errflag, retflag, breaks, exit_pending );
+                    fclose(_F);
 		    continue;
+                }
 		errflag &= ~ERRFLAG_ERROR;
 		breaks = obreaks;
 		errno = old_errno;
 		return lastchar = EOF;
 	    } else if (errno == EWOULDBLOCK) {
 		fcntl(0, F_SETFL, 0);
 	    } else if (errno == EIO && !die) {
 		ret = opts[MONITOR];
 		opts[MONITOR] = 1;
 		attachtty(mypgrp);
@@ -1115,20 +1207,24 @@ zlecore(void)
 		if (eofsent)
 		    break;
 	    }
 	    handleprefixes();
 	    /* for vi mode, make sure the cursor isn't somewhere illegal */
 	    if (invicmdmode() && zlecs > findbol() &&
 		(zlecs == zlell || zleline[zlecs] == ZWC('\n')))
 		DECCS();
 	    handleundo();
 	} else {
+            // MY DEBUG
+            FILE *_F = fopen("/tmp/recursive.txt", "a+");
+            fprintf( _F, "-- Setting error in zlecore.c\n" );
+            fclose(_F);
 	    errflag |= ERRFLAG_ERROR;
 	    break;
 	}
 
 	redrawhook();
 #ifdef HAVE_POLL
 	if (baud && !(lastcmd & ZLE_MENUCMP)) {
 	    struct pollfd pfd;
 	    int to = cost * costmult / 1000; /* milliseconds */
 

Re: Where to start debugging zle recursive-edit? / Ctrl-C

[Bart Schaefer]

On Sep 30,  3:30pm, Sebastian Gniazdowski wrote:
}
} So, the linked list doesn't have any elements, and calc_timeout()
} exits with ZTM_NONE instead of ZTM_FUNC

If I recall correctly how this is architected ...

This must mean that you are interrupting the scheduled function itself,
before it has a chance to execute "sched" and reset its timer.

Because you are in recursive-edit, interrupting does not go all they
way back to the top-level prompt where the precmd hook would restore
the timer, so it remains unscheduled until the current zle session
is done and precmd runs again.

You may need to find a more aggressive place to reset the timer, such
as in the zle-line-pre-redraw hook, or else create a wrapper widget
for recursive-edit.

Either way you'd want to examine $zle_scheduled_events to see if the
timer is still pending, before scheduling it again.

Or possibly you could switch from doing this with a timer scheduled
function to using a "zle -F" handler function.

Re: Where to start debugging zle recursive-edit? / Ctrl-C

[Sebastian Gniazdowski]

[-- Attachment #1: Type: text/plain, Size: 4029 bytes --]

On 30 September 2016 at 22:44, Bart Schaefer <schaefer@brasslantern.com> wrote:
> Because you are in recursive-edit, interrupting does not go all they
> way back to the top-level prompt where the precmd hook would restore
> the timer, so it remains unscheduled until the current zle session
> is done and precmd runs again.

I'm looking at the outputs from some time today and what you say seems
to ring bells thinking what I observe but not yet clear. However, what
I do observe clearly is:

-- tmoutp != ZTM_NONE / raw_getbyte() zle_main.c
-- (**) NOT doing break selret[0] / zle_main.c: errflag: 0, retflag:
0, breaks: 0, exit_pending: 0
--  Passed !errtry(errtry:1) selret[0] / zle_main.c
## set errflag to 2 (ERRFLAG_INT:2) / signals.c

This means: SIGINT is received – select breaks. Then errflag is checked (**):

            if (selret < 0 && (errflag || retflag || breaks || exit_pending)) {
                fprintf( _F, "-- Doing break [selret:%d] / zle_main.c:
errflag: %d, retflag: %d, breaks: %d, exit_pending: %d\>
                            selret, errflag, retflag, breaks, exit_pending );
                break;
            } else {
                fprintf( _F, "-- NOT doing break selret[%d] /
zle_main.c: errflag: %d, retflag: %d, breaks: %d, exit_pending: >
                            selret, errflag, retflag, breaks, exit_pending );
            }

(fopen/fclose removed). Result: no break is done, no errflag to
trigger it. Then the check runs:

            if (selret < 0 && !errtry) {
                fprintf( _F, "-- Trying again selret[%d], !errtry[%d]
/ zle_main.c\n", selret, errtry );
                errtry = 1;
                continue;
            } else

And select is run again – .recedit ignores SIGNT, does not return.
This is the thing that I observe, multiple Ctrl-C to break out from
widget. After this zhandler() sets the errflag:

        if (!handletrap(SIGINT)) {
            if ((isset(PRIVILEGED) || isset(RESTRICTED)) &&
                isset(INTERACTIVE) && noerrexit < 0)
                zexit(SIGINT, 1);
            if (list_pipe || chline || simple_pline) {
                breaks = loops;
                errflag |= ERRFLAG_INT;
                inerrflush();
                check_cursh_sig(SIGINT);

                fprintf( _F, "## set errflag to %d (ERRFLAG_INT:%d) /
signals.c\n", errflag, ERRFLAG_INT );
            }

I wonder if this can result also in problems with rescheduling. Code
runs with errflag set and as you said this stops scheduled function
before any execution is done.

Can something be done to make zhandler() set errflag faster, or to
make the raw_getbyte's after-select() checks wait for zhandler's
resolution?

> You may need to find a more aggressive place to reset the timer, such
> as in the zle-line-pre-redraw hook, or else create a wrapper widget
> for recursive-edit.
>
> Either way you'd want to examine $zle_scheduled_events to see if the
> timer is still pending, before scheduling it again.

I use this trick:

ZCONVEY_SCHEDULE_ORIGIN="$SECONDS"
sched +"${ZCONVEY_CONFIG[check_interval]}" __zconvey_on_period_passed
"$ZCONVEY_SCHEDULE_ORIGIN"

Then inside the handler a check is done to make only the current
"*SCHEDULE_ORIGIN" do rescheduling. So I can quite freely
overschedule, in the end only last schedule goes int rescheduling
loop.

>
> Or possibly you could switch from doing this with a timer scheduled
> function to using a "zle -F" handler function.

Would be great to also have in 5.3 .recedit answering predictably to
Ctrl-C. What's also interesting is that I could not get setting
errflag late when not in .recedit. The logs always look like this:

-- tmoutp != ZTM_NONE / raw_getbyte() zle_main.c
## set errflag to 2 (ERRFLAG_INT:2) / signals.c
-- Doing break [selret:-1] / zle_main.c: errflag: 2, retflag: 0,
breaks: 0, exit_pending: 0
-- Got EINTR 3
-- Setting error in zlecore.c

Best regards,
Sebastian Gniazdowski

[-- Attachment #2: rec4_edit.4.diff.txt --]
[-- Type: text/plain, Size: 11023 bytes --]

diff --git a/Src/Zle/zle_main.c b/Src/Zle/zle_main.c
index 9a83d41..7bcf450 100644
--- a/Src/Zle/zle_main.c
+++ b/Src/Zle/zle_main.c
@@ -450,47 +450,87 @@ static void
 calc_timeout(struct ztmout *tmoutp, long do_keytmout)
 {
     if (do_keytmout && (keytimeout > 0 || do_keytmout < 0)) {
 	if (do_keytmout < 0)
 	    tmoutp->exp100ths = (time_t)-do_keytmout;
 	else if (keytimeout > ZMAXTIMEOUT * 100 /* 24 days for a keypress???? */)
 	    tmoutp->exp100ths = ZMAXTIMEOUT * 100;
 	else
 	    tmoutp->exp100ths = keytimeout;
 	tmoutp->tp = ZTM_KEY;
-    } else
+        // MY DEBUG
+        FILE *_F = fopen("/tmp/recursive.txt", "a+");
+        fprintf( _F, "-- tmoutp->tp <- ZTM_KEY (%d) / calc_timeout() zle_main.c\n", tmoutp->exp100ths );
+        fclose(_F);
+    } else {
+        // MY DEBUG
+        FILE *_F = fopen("/tmp/recursive.txt", "a+");
+        fprintf( _F, "-- tmoutp->tp <- ZTM_NONE / calc_timeout( do_keytmout: %d ), keytimeout: %d / zle_main.c\n", do_keytmout, keytimeout );
+        fclose(_F);
+
 	tmoutp->tp = ZTM_NONE;
+    }
 
     if (timedfns) {
+        // MY DEBUG
+        FILE *_F = fopen("/tmp/recursive.txt", "a+");
+        fprintf( _F, "== CALC_TIMEOUT() one more chance (timedfns exp100ths: %d)\n", tmoutp->exp100ths );
+        fclose(_F);
 	for (;;) {
 	    LinkNode tfnode = firstnode(timedfns);
 	    Timedfn tfdat;
 	    time_t diff, exp100ths;
 
-	    if (!tfnode)
+	    if (!tfnode) {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "-- CALC_TIMEOUT() !tfnode break\n" );
+                fclose(_F);
 		break;
+            }
 
 	    tfdat = (Timedfn)getdata(tfnode);
 	    diff = tfdat->when - time(NULL);
+
 	    if (diff < 0) {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "-- CALC_TIMEOUT() tfnode TRUE no break CALLING >> DIFF=%d <<\n", diff );
+                fclose(_F);
+
 		/* Already due; call it and rescan. */
 		tfdat->func();
 		continue;
-	    }
+	    } else {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "-- CALC_TIMEOUT() tfnode TRUE no break NOT calling >> DIFF=%d <<\n", diff );
+                fclose(_F);
+            }
 
 	    if (diff > ZMAXTIMEOUT) {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "-- CALC_TIMEOUT() %d > %d ZTM_MAX(%d)\n", diff, ZMAXTIMEOUT, ZTM_MAX );
+                fclose(_F);
+
 		tmoutp->exp100ths = ZMAXTIMEOUT * 100;
 		tmoutp->tp = ZTM_MAX;
-	    } else if (diff > 0) {
+	    } else if (diff >= 0) {
 		exp100ths = diff * 100;
 		if (tmoutp->tp != ZTM_KEY ||
-		    exp100ths < tmoutp->exp100ths) {
+		    exp100ths <= tmoutp->exp100ths) {
+                    // MY DEBUG
+                    FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                    fprintf( _F, "-- CALC_TIMEOUT() %d != %d || %d <= %d ZTM_FUNC(%d)\n", tmoutp->tp, ZTM_KEY, exp100ths, tmoutp->exp100ths, ZTM_FUNC );
+                    fclose(_F);
+
 		    tmoutp->exp100ths = exp100ths;
 		    tmoutp->tp = ZTM_FUNC;
 		}
 	    }
 	    break;
 	}
 	/* In case we called a function which messed up the display... */
 	if (resetneeded)
 	    zrefresh();
     }
@@ -508,20 +548,24 @@ raw_getbyte(long do_keytmout, char *cptr)
     struct ttyinfo ti;
 #endif
 #ifndef HAVE_POLL
 # ifdef HAVE_SELECT
     fd_set foofd, errfd;
     FD_ZERO(&errfd);
 # endif
 #endif
 
     calc_timeout(&tmout, do_keytmout);
+    // MY DEBUG
+    FILE *_F = fopen("/tmp/recursive.txt", "a+");
+    fprintf( _F, "-- INIT tmout.tp(%d) ZTM_NONE(%d) ZTM_KEY(%d) / BEGIN RAW_GETBYTE() zle_main.c\n", tmout.tp, ZTM_NONE, ZTM_KEY );
+    fclose(_F);
 
     /*
      * Handle timeouts and watched fd's.  If a watched fd or a function
      * timeout triggers we restart any key timeout.  This is likely to
      * be harmless: the combination is extremely rare and a function
      * is likely to occupy the user for a little while anyway.  We used
      * to make timeouts take precedence, but we can't now that the
      * timeouts may be external, so we may have both a permanent watched
      * fd and a long-term timeout.
      */
@@ -600,43 +644,78 @@ raw_getbyte(long do_keytmout, char *cptr)
 		    if (fd > fdmax)
 			fdmax = fd;
 		}
 	    }
 	    FD_ZERO(&errfd);
 
 	    if (tmout.tp != ZTM_NONE) {
 		expire_tv.tv_sec = tmout.exp100ths / 100;
 		expire_tv.tv_usec = (tmout.exp100ths % 100) * 10000L;
 		tvptr = &expire_tv;
+
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "\n-- tmoutp != ZTM_NONE / raw_getbyte() zle_main.c\n" );
+                fclose(_F);
 	    }
-	    else
+	    else {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "\n-- FINAL (STOP) tmoutp == ZTM_NONE / raw_getbyte() zle_main.c\n" );
+                fclose(_F);
+
 		tvptr = NULL;
+            }
 
 	    winch_unblock();
 	    selret = select(fdmax+1, (SELECT_ARG_2_T) & foofd,
 			    NULL, NULL, tvptr);
 	    winch_block();
 # endif
 	    /*
 	     * Make sure a user interrupt gets passed on straight away.
 	     */
-	    if (selret < 0 && (errflag || retflag || breaks || exit_pending))
+	    if (selret < 0 && (errflag || retflag || breaks || exit_pending)) {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "-- Doing break [selret:%d] / zle_main.c: errflag: %d, retflag: %d, breaks: %d, exit_pending: %d\n",
+                            selret, errflag, retflag, breaks, exit_pending );
+                fclose(_F);
 		break;
+            } else {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "-- NOT doing break selret[%d] / zle_main.c: errflag: %d, retflag: %d, breaks: %d, exit_pending: %d\n",
+                            selret, errflag, retflag, breaks, exit_pending );
+                fclose(_F);
+            }
+
 	    /*
 	     * Try to avoid errors on our special fd's from
 	     * messing up reads from the terminal.  Try first
 	     * with all fds, then try unsetting the special ones.
 	     */
 	    if (selret < 0 && !errtry) {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "-- Trying again selret[%d], !errtry[%d] / zle_main.c\n", selret, errtry );
+                fclose(_F);
+
 		errtry = 1;
 		continue;
-	    }
+	    } else {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "--  Passed !errtry(errtry:%d) selret[%d] / zle_main.c\n", errtry, selret );
+                fclose(_F);
+            }
+
 	    if (selret == 0) {
 		/*
 		 * Nothing ready and no error, so we timed out.
 		 */
 		switch (tmout.tp) {
 		case ZTM_NONE:
 		    /* keeps compiler happy if not debugging */
 #ifdef DEBUG
 		    dputs("BUG: timeout fired with no timeout set.");
 #endif
@@ -669,20 +748,24 @@ raw_getbyte(long do_keytmout, char *cptr)
 		case ZTM_MAX:
 		    /*
 		     * Reached the limit of our range, but not the
 		     * actual timeout; recalculate the timeout.
 		     * We're cheating with the key timeout here:
 		     * if one clashed with a function timeout we
 		     * reconsider the key timeout from scratch.
 		     * The effect of this is microscopic.
 		     */
 		    calc_timeout(&tmout, do_keytmout);
+                    // MY DEBUG
+                    FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                    fprintf( _F, "^^^ LOOP-CALLED calc_timeout: tmout.tp == %d / zle_main.c\n", tmout.tp );
+                    fclose(_F);
 		    break;
 		}
 		/*
 		 * If we handled the timeout successfully,
 		 * carry on.
 		 */
 		if (selret == 0)
 		    continue;
 	    }
 	    /* If error or unhandled timeout, give up. */
@@ -888,22 +971,36 @@ getbyte(long do_keytmout, int *timeout)
 		   the counter (icnt) so that this happens 20 times and than
 		   the shell gives up (yes, this is a bit dirty...). */
 		if ((zlereadflags & ZLRF_IGNOREEOF) && icnt++ < 20)
 		    continue;
 		stopmsg = 1;
 		zexit(1, 0);
 	    }
 	    icnt = 0;
 	    if (errno == EINTR) {
 		die = 0;
+                static int counter = 0;
+
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "-- Got EINTR %d\n", ++counter );
+                fclose(_F);
+
 		if (!errflag && !retflag && !breaks && !exit_pending)
+                {
+                    // MY DEBUG
+                    FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                    fprintf( _F, "-- Continuing despite EINTR / zle_main.c: errflag: %d, retflag: %d, breaks: %d, exit_pending: %d\n",
+                                errflag, retflag, breaks, exit_pending );
+                    fclose(_F);
 		    continue;
+                }
 		errflag &= ~ERRFLAG_ERROR;
 		breaks = obreaks;
 		errno = old_errno;
 		return lastchar = EOF;
 	    } else if (errno == EWOULDBLOCK) {
 		fcntl(0, F_SETFL, 0);
 	    } else if (errno == EIO && !die) {
 		ret = opts[MONITOR];
 		opts[MONITOR] = 1;
 		attachtty(mypgrp);
@@ -1115,20 +1212,24 @@ zlecore(void)
 		if (eofsent)
 		    break;
 	    }
 	    handleprefixes();
 	    /* for vi mode, make sure the cursor isn't somewhere illegal */
 	    if (invicmdmode() && zlecs > findbol() &&
 		(zlecs == zlell || zleline[zlecs] == ZWC('\n')))
 		DECCS();
 	    handleundo();
 	} else {
+            // MY DEBUG
+            FILE *_F = fopen("/tmp/recursive.txt", "a+");
+            fprintf( _F, "-- Setting error in zlecore.c\n" );
+            fclose(_F);
 	    errflag |= ERRFLAG_ERROR;
 	    break;
 	}
 
 	redrawhook();
 #ifdef HAVE_POLL
 	if (baud && !(lastcmd & ZLE_MENUCMP)) {
 	    struct pollfd pfd;
 	    int to = cost * costmult / 1000; /* milliseconds */
 
diff --git a/Src/signals.c b/Src/signals.c
index e2587dc..0bbda97 100644
--- a/Src/signals.c
+++ b/Src/signals.c
@@ -638,20 +638,25 @@ zhandler(int sig)
     case SIGINT:
         if (!handletrap(SIGINT)) {
 	    if ((isset(PRIVILEGED) || isset(RESTRICTED)) &&
 		isset(INTERACTIVE) && noerrexit < 0)
 		zexit(SIGINT, 1);
             if (list_pipe || chline || simple_pline) {
                 breaks = loops;
                 errflag |= ERRFLAG_INT;
 		inerrflush();
 		check_cursh_sig(SIGINT);
+
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "## set errflag to %d (ERRFLAG_INT:%d) / signals.c\n", errflag, ERRFLAG_INT );
+                fclose(_F);
             }
 	    lastval = 128 + SIGINT;
         }
         break;
 
 #ifdef SIGWINCH
     case SIGWINCH:
         adjustwinsize(1);  /* check window size and adjust */
 	(void) handletrap(SIGWINCH);
         break;

Re: Where to start debugging zle recursive-edit? / Ctrl-C

[Sebastian Gniazdowski]

[-- Attachment #1: Type: text/plain, Size: 1459 bytes --]

Investigated this further. The handler isn't exactly late. It is
called before errflag checks, but exits here without setting errflag:

    /* Are we queueing signals now?      */
    if (queueing_enabled) {

The log looks now like this:

-- tmoutp != ZTM_NONE / raw_getbyte() zle_main.c
## zhandler(2) signals.c (debug message in first line of zhandler)
## zhandler(2) EXIT 1 signals.c (debug message in if(queueing_enabled))
-- NOT doing break selret[-1/4/4] / zle_main.c: errflag: 0, retflag:
0, breaks: 0, exit_pending: 0
-- Trying again selret[-1], !errtry[0] / zle_main.c

-- tmoutp != ZTM_NONE / raw_getbyte() zle_main.c
-- (**) NOT doing break selret[0/4/4] / zle_main.c: errflag: 0,
retflag: 0, breaks: 0, exit_pending: 0
--  Passed !errtry(errtry:1) selret[0] / zle_main.c
## zhandler(2) signals.c
## set errflag to 2 (ERRFLAG_INT:2) / signals.c
-- tmoutp->tp <- ZTM_NONE / calc_timeout( do_keytmout: 0 ),
keytimeout: 40 / zle_main.c
== CALC_TIMEOUT() one more chance (timedfns exp100ths: 100)
-- CALC_TIMEOUT() tfnode TRUE no break NOT calling >> DIFF=1 <<
-- CALC_TIMEOUT() 0 != 1 || 100 <= 100 ZTM_FUNC(2)
^^^ LOOP-CALLED calc_timeout: tmout.tp == 2 / zle_main.c

Interesting is (**) line, it still has errno == 4 (last number in
[%d/%d/%d]), apparently errno=0 is needed before select, doing so
clears the value. What's with the queueing and early exit from
zhandler()? It brings disorder to .recursiveedit.

Best regards,
Sebastian Gniazdowski

[-- Attachment #2: rec5_edit.5.diff.txt --]
[-- Type: text/plain, Size: 12304 bytes --]

diff --git a/Src/Zle/zle_main.c b/Src/Zle/zle_main.c
index 9a83d41..79d6e20 100644
--- a/Src/Zle/zle_main.c
+++ b/Src/Zle/zle_main.c
@@ -450,47 +450,87 @@ static void
 calc_timeout(struct ztmout *tmoutp, long do_keytmout)
 {
     if (do_keytmout && (keytimeout > 0 || do_keytmout < 0)) {
 	if (do_keytmout < 0)
 	    tmoutp->exp100ths = (time_t)-do_keytmout;
 	else if (keytimeout > ZMAXTIMEOUT * 100 /* 24 days for a keypress???? */)
 	    tmoutp->exp100ths = ZMAXTIMEOUT * 100;
 	else
 	    tmoutp->exp100ths = keytimeout;
 	tmoutp->tp = ZTM_KEY;
-    } else
+        // MY DEBUG
+        FILE *_F = fopen("/tmp/recursive.txt", "a+");
+        fprintf( _F, "-- tmoutp->tp <- ZTM_KEY (%d) / calc_timeout() zle_main.c\n", tmoutp->exp100ths );
+        fclose(_F);
+    } else {
+        // MY DEBUG
+        FILE *_F = fopen("/tmp/recursive.txt", "a+");
+        fprintf( _F, "-- tmoutp->tp <- ZTM_NONE / calc_timeout( do_keytmout: %d ), keytimeout: %d / zle_main.c\n", do_keytmout, keytimeout );
+        fclose(_F);
+
 	tmoutp->tp = ZTM_NONE;
+    }
 
     if (timedfns) {
+        // MY DEBUG
+        FILE *_F = fopen("/tmp/recursive.txt", "a+");
+        fprintf( _F, "== CALC_TIMEOUT() one more chance (timedfns exp100ths: %d)\n", tmoutp->exp100ths );
+        fclose(_F);
 	for (;;) {
 	    LinkNode tfnode = firstnode(timedfns);
 	    Timedfn tfdat;
 	    time_t diff, exp100ths;
 
-	    if (!tfnode)
+	    if (!tfnode) {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "-- CALC_TIMEOUT() !tfnode break\n" );
+                fclose(_F);
 		break;
+            }
 
 	    tfdat = (Timedfn)getdata(tfnode);
 	    diff = tfdat->when - time(NULL);
+
 	    if (diff < 0) {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "-- CALC_TIMEOUT() tfnode TRUE no break CALLING >> DIFF=%d <<\n", diff );
+                fclose(_F);
+
 		/* Already due; call it and rescan. */
 		tfdat->func();
 		continue;
-	    }
+	    } else {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "-- CALC_TIMEOUT() tfnode TRUE no break NOT calling >> DIFF=%d <<\n", diff );
+                fclose(_F);
+            }
 
 	    if (diff > ZMAXTIMEOUT) {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "-- CALC_TIMEOUT() %d > %d ZTM_MAX(%d)\n", diff, ZMAXTIMEOUT, ZTM_MAX );
+                fclose(_F);
+
 		tmoutp->exp100ths = ZMAXTIMEOUT * 100;
 		tmoutp->tp = ZTM_MAX;
-	    } else if (diff > 0) {
+	    } else if (diff >= 0) {
 		exp100ths = diff * 100;
 		if (tmoutp->tp != ZTM_KEY ||
-		    exp100ths < tmoutp->exp100ths) {
+		    exp100ths <= tmoutp->exp100ths) {
+                    // MY DEBUG
+                    FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                    fprintf( _F, "-- CALC_TIMEOUT() %d != %d || %d <= %d ZTM_FUNC(%d)\n", tmoutp->tp, ZTM_KEY, exp100ths, tmoutp->exp100ths, ZTM_FUNC );
+                    fclose(_F);
+
 		    tmoutp->exp100ths = exp100ths;
 		    tmoutp->tp = ZTM_FUNC;
 		}
 	    }
 	    break;
 	}
 	/* In case we called a function which messed up the display... */
 	if (resetneeded)
 	    zrefresh();
     }
@@ -508,20 +548,24 @@ raw_getbyte(long do_keytmout, char *cptr)
     struct ttyinfo ti;
 #endif
 #ifndef HAVE_POLL
 # ifdef HAVE_SELECT
     fd_set foofd, errfd;
     FD_ZERO(&errfd);
 # endif
 #endif
 
     calc_timeout(&tmout, do_keytmout);
+    // MY DEBUG
+    FILE *_F = fopen("/tmp/recursive.txt", "a+");
+    fprintf( _F, "-- INIT tmout.tp(%d) ZTM_NONE(%d) ZTM_KEY(%d) / BEGIN RAW_GETBYTE() zle_main.c\n", tmout.tp, ZTM_NONE, ZTM_KEY );
+    fclose(_F);
 
     /*
      * Handle timeouts and watched fd's.  If a watched fd or a function
      * timeout triggers we restart any key timeout.  This is likely to
      * be harmless: the combination is extremely rare and a function
      * is likely to occupy the user for a little while anyway.  We used
      * to make timeouts take precedence, but we can't now that the
      * timeouts may be external, so we may have both a permanent watched
      * fd and a long-term timeout.
      */
@@ -600,43 +644,78 @@ raw_getbyte(long do_keytmout, char *cptr)
 		    if (fd > fdmax)
 			fdmax = fd;
 		}
 	    }
 	    FD_ZERO(&errfd);
 
 	    if (tmout.tp != ZTM_NONE) {
 		expire_tv.tv_sec = tmout.exp100ths / 100;
 		expire_tv.tv_usec = (tmout.exp100ths % 100) * 10000L;
 		tvptr = &expire_tv;
+
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "\n-- tmoutp != ZTM_NONE / raw_getbyte() zle_main.c\n" );
+                fclose(_F);
 	    }
-	    else
+	    else {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "\n-- FINAL (STOP) tmoutp == ZTM_NONE / raw_getbyte() zle_main.c\n" );
+                fclose(_F);
+
 		tvptr = NULL;
+            }
 
 	    winch_unblock();
 	    selret = select(fdmax+1, (SELECT_ARG_2_T) & foofd,
 			    NULL, NULL, tvptr);
 	    winch_block();
 # endif
 	    /*
 	     * Make sure a user interrupt gets passed on straight away.
 	     */
-	    if (selret < 0 && (errflag || retflag || breaks || exit_pending))
+	    if (selret < 0 && (errflag || retflag || breaks || exit_pending)) {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "-- Doing break [selret:%d/%d/%d] / zle_main.c: errflag: %d, retflag: %d, breaks: %d, exit_pending: %d\n",
+                            selret, EINTR, errno, errflag, retflag, breaks, exit_pending );
+                fclose(_F);
 		break;
+            } else {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "-- NOT doing break selret[%d/%d/%d] / zle_main.c: errflag: %d, retflag: %d, breaks: %d, exit_pending: %d\n",
+                            selret, EINTR, errno, errflag, retflag, breaks, exit_pending );
+                fclose(_F);
+            }
+
 	    /*
 	     * Try to avoid errors on our special fd's from
 	     * messing up reads from the terminal.  Try first
 	     * with all fds, then try unsetting the special ones.
 	     */
 	    if (selret < 0 && !errtry) {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "-- Trying again selret[%d], !errtry[%d] / zle_main.c\n", selret, errtry );
+                fclose(_F);
+
 		errtry = 1;
 		continue;
-	    }
+	    } else {
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "--  Passed !errtry(errtry:%d) selret[%d] / zle_main.c\n", errtry, selret );
+                fclose(_F);
+            }
+
 	    if (selret == 0) {
 		/*
 		 * Nothing ready and no error, so we timed out.
 		 */
 		switch (tmout.tp) {
 		case ZTM_NONE:
 		    /* keeps compiler happy if not debugging */
 #ifdef DEBUG
 		    dputs("BUG: timeout fired with no timeout set.");
 #endif
@@ -669,20 +748,24 @@ raw_getbyte(long do_keytmout, char *cptr)
 		case ZTM_MAX:
 		    /*
 		     * Reached the limit of our range, but not the
 		     * actual timeout; recalculate the timeout.
 		     * We're cheating with the key timeout here:
 		     * if one clashed with a function timeout we
 		     * reconsider the key timeout from scratch.
 		     * The effect of this is microscopic.
 		     */
 		    calc_timeout(&tmout, do_keytmout);
+                    // MY DEBUG
+                    FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                    fprintf( _F, "^^^ LOOP-CALLED calc_timeout: tmout.tp == %d / zle_main.c\n", tmout.tp );
+                    fclose(_F);
 		    break;
 		}
 		/*
 		 * If we handled the timeout successfully,
 		 * carry on.
 		 */
 		if (selret == 0)
 		    continue;
 	    }
 	    /* If error or unhandled timeout, give up. */
@@ -888,22 +971,36 @@ getbyte(long do_keytmout, int *timeout)
 		   the counter (icnt) so that this happens 20 times and than
 		   the shell gives up (yes, this is a bit dirty...). */
 		if ((zlereadflags & ZLRF_IGNOREEOF) && icnt++ < 20)
 		    continue;
 		stopmsg = 1;
 		zexit(1, 0);
 	    }
 	    icnt = 0;
 	    if (errno == EINTR) {
 		die = 0;
+                static int counter = 0;
+
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "-- Got EINTR %d\n", ++counter );
+                fclose(_F);
+
 		if (!errflag && !retflag && !breaks && !exit_pending)
+                {
+                    // MY DEBUG
+                    FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                    fprintf( _F, "-- Continuing despite EINTR / zle_main.c: errflag: %d, retflag: %d, breaks: %d, exit_pending: %d\n",
+                                errflag, retflag, breaks, exit_pending );
+                    fclose(_F);
 		    continue;
+                }
 		errflag &= ~ERRFLAG_ERROR;
 		breaks = obreaks;
 		errno = old_errno;
 		return lastchar = EOF;
 	    } else if (errno == EWOULDBLOCK) {
 		fcntl(0, F_SETFL, 0);
 	    } else if (errno == EIO && !die) {
 		ret = opts[MONITOR];
 		opts[MONITOR] = 1;
 		attachtty(mypgrp);
@@ -1115,20 +1212,24 @@ zlecore(void)
 		if (eofsent)
 		    break;
 	    }
 	    handleprefixes();
 	    /* for vi mode, make sure the cursor isn't somewhere illegal */
 	    if (invicmdmode() && zlecs > findbol() &&
 		(zlecs == zlell || zleline[zlecs] == ZWC('\n')))
 		DECCS();
 	    handleundo();
 	} else {
+            // MY DEBUG
+            FILE *_F = fopen("/tmp/recursive.txt", "a+");
+            fprintf( _F, "-- Setting error in zlecore.c\n" );
+            fclose(_F);
 	    errflag |= ERRFLAG_ERROR;
 	    break;
 	}
 
 	redrawhook();
 #ifdef HAVE_POLL
 	if (baud && !(lastcmd & ZLE_MENUCMP)) {
 	    struct pollfd pfd;
 	    int to = cost * costmult / 1000; /* milliseconds */
 
diff --git a/Src/signals.c b/Src/signals.c
index e2587dc..ea2f472 100644
--- a/Src/signals.c
+++ b/Src/signals.c
@@ -555,20 +555,23 @@ wait_for_processes(void)
 	unqueue_signals();
     }
 }
 
 /* the signal handler */
  
 /**/
 mod_export RETSIGTYPE
 zhandler(int sig)
 {
+    FILE *_F = fopen("/tmp/recursive.txt", "a+");
+    fprintf( _F, "## zhandler(%d) signals.c\n", sig );
+    fclose( _F );
     sigset_t newmask, oldmask;
 
 #if defined(NO_SIGNAL_BLOCKING)
     int do_jump;
     signal_jmp_buf jump_to;
 #endif
  
     last_signal = sig;
     signal_process(sig);
  
@@ -599,20 +602,24 @@ zhandler(int sig)
 	/* Make sure it's not full (extremely unlikely) */
         if (temp_rear != queue_front) {
 	    /* ok, not full, so add to queue   */
             queue_rear = temp_rear;
 	    /* save signal caught              */
             signal_queue[queue_rear] = sig;
 	    /* save current signal mask        */
             signal_mask_queue[queue_rear] = oldmask;
         }
         signal_reset(sig);
+
+        FILE *_F = fopen("/tmp/recursive.txt", "a+");
+        fprintf( _F, "## zhandler(%d) EXIT 1 signals.c\n", sig );
+        fclose( _F );
         return;
     }
  
     /* Reset signal mask, signal traps ok now */
     signal_setmask(oldmask);
  
     switch (sig) {
     case SIGCHLD:
 	wait_for_processes();
         break;
@@ -638,20 +645,25 @@ zhandler(int sig)
     case SIGINT:
         if (!handletrap(SIGINT)) {
 	    if ((isset(PRIVILEGED) || isset(RESTRICTED)) &&
 		isset(INTERACTIVE) && noerrexit < 0)
 		zexit(SIGINT, 1);
             if (list_pipe || chline || simple_pline) {
                 breaks = loops;
                 errflag |= ERRFLAG_INT;
 		inerrflush();
 		check_cursh_sig(SIGINT);
+
+                // MY DEBUG
+                FILE *_F = fopen("/tmp/recursive.txt", "a+");
+                fprintf( _F, "## set errflag to %d (ERRFLAG_INT:%d) / signals.c\n", errflag, ERRFLAG_INT );
+                fclose(_F);
             }
 	    lastval = 128 + SIGINT;
         }
         break;
 
 #ifdef SIGWINCH
     case SIGWINCH:
         adjustwinsize(1);  /* check window size and adjust */
 	(void) handletrap(SIGWINCH);
         break;

Re: Where to start debugging zle recursive-edit? / Ctrl-C

[Sebastian Gniazdowski]

On 30 September 2016 at 22:44, Bart Schaefer <schaefer@brasslantern.com> wrote:
> You may need to find a more aggressive place to reset the timer, such
> as in the zle-line-pre-redraw hook, or else create a wrapper widget
> for recursive-edit.

Was going to do this with periodic hook. Now as 5.3 is fixed I could
say: "Share PERIOD with this plugin, set it to value 6...10, best is
10. And if you really want to own PERIOD, then use latest Zsh".
However periodic hook rather suffers too from no-timeout select()?

Wrapper around recursive-edit would solve Ctrl-C problems, or you
meant something about rescheduling?

> Either way you'd want to examine $zle_scheduled_events to see if the
> timer is still pending, before scheduling it again.

Tried accessing the variable from zle widget and no luck. I must be
doing something wrong as even ack '.*zle.*sched.*eve.*' doesn't return
results.

> Or possibly you could switch from doing this with a timer scheduled
> function to using a "zle -F" handler function.

Cool thing the zle -F, thanks for the tip. It would obviously install
where main symptom of problem occurs, at the select(). But how could I
periodically change state of the FD, without forking background
process for each instance of Zsh?

A half-ready solution is to schedule no-op function 24 hours ahead
(saw something about that maximum schedule time), this should make
calc_timeout() always return ZTM_FUNC. But this requires something
more, as the main scheduled function might still stop running because
of errflag. Maybe this should be done together with periodic hook.
select() would not block (ZTM_FUNC), and periodic hook is maybe immune
to errflag rescheduling-break?

Best regards,
Sebastian Gniazdowski

Re: Where to start debugging zle recursive-edit? / Ctrl-C

[Bart Schaefer]

On Oct 5,  7:31am, Sebastian Gniazdowski wrote:
} Subject: Re: Where to start debugging zle recursive-edit? / Ctrl-C
}
} On 30 September 2016 at 22:44, Bart Schaefer <schaefer@brasslantern.com> wrote:
} > You may need to find a more aggressive place to reset the timer, such
} > as in the zle-line-pre-redraw hook, or else create a wrapper widget
} > for recursive-edit.
} 
} Was going to do this with periodic hook.

For your purposes, the trouble with periodic is that it only executes
right before a prompt.  Unless there's more to what you're saying than
I'm understanding.

} However periodic hook rather suffers too from no-timeout select()?

I wouldn't expect so, no.  It's not related to the select at all.

} Wrapper around recursive-edit would solve Ctrl-C problems, or you
} meant something about rescheduling?

I meant something about scheduling.

} > Either way you'd want to examine $zle_scheduled_events
} 
} Tried accessing the variable from zle widget and no luck

My typo, it's $zsh_scheduled_events.  It's in the documentation for the
sched module.

} Cool thing the zle -F, thanks for the tip. It would obviously install
} where main symptom of problem occurs, at the select(). But how could I
} periodically change state of the FD, without forking background
} process for each instance of Zsh?

Either open a named pipe file (mknod p filename), or use the zsh/tcp
module as in the example that appears in the documentation.

Another thing I've done quite successfully for a very long time is to
assign to the TMOUT variable and use TRAPALRM() to take some action
(such as update the time in my prompt).  This is close to the same
behavior as sched but uses the alarm() system function so doesn't
rely on zle read timeouts.

Re: Where to start debugging zle recursive-edit? / Ctrl-C

[Sebastian Gniazdowski]

On 5 October 2016 at 08:14, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Oct 5,  7:31am, Sebastian Gniazdowski wrote:
> } Subject: Re: Where to start debugging zle recursive-edit? / Ctrl-C
> }
> } On 30 September 2016 at 22:44, Bart Schaefer <schaefer@brasslantern.com> wrote:
> } > You may need to find a more aggressive place to reset the timer, such
> } > as in the zle-line-pre-redraw hook, or else create a wrapper widget
> } > for recursive-edit.
> }
> } Was going to do this with periodic hook.
>
> For your purposes, the trouble with periodic is that it only executes
> right before a prompt.  Unless there's more to what you're saying than
> I'm understanding.

Had some obstacles to use the precmd but really not recalling them now.

> } Wrapper around recursive-edit would solve Ctrl-C problems, or you
> } meant something about rescheduling?
>
> I meant something about scheduling.

Hmm, could you reveal? I thought about trapping INT when detected that
in .recursive-edit, but directly that's "Ctrl-C problems" stuff

> } > Either way you'd want to examine $zle_scheduled_events
> }
> } Tried accessing the variable from zle widget and no luck
>
> My typo, it's $zsh_scheduled_events.  It's in the documentation for the
> sched module.

So no need to parse sched output at all, as I once wrote

> Either open a named pipe file (mknod p filename), or use the zsh/tcp
> module as in the example that appears in the documentation.

That are rather heavy-implementation things

> Another thing I've done quite successfully for a very long time is to
> assign to the TMOUT variable and use TRAPALRM() to take some action
> (such as update the time in my prompt).  This is close to the same
> behavior as sched but uses the alarm() system function so doesn't
> rely on zle read timeouts.

Thanks for info that this trap doesn't cause problems, I would be
definitely anxious about using it (expecting things like e.g. the
pick-up of z-sy-h state, but that's maybe plain simple lack of opt
localoptions), don't want to serve users heavy-implementation stuff,
but if the trap is proven to be robust, then..

Best regards,
Sebastian Gniazdowski

Re: Where to start debugging zle recursive-edit? / Ctrl-C

[Bart Schaefer]

On Oct 5, 12:27pm, Sebastian Gniazdowski wrote:
}
} > } Wrapper around recursive-edit would solve Ctrl-C problems, or you
} > } meant something about rescheduling?
} >
} > I meant something about scheduling.
} 
} Hmm, could you reveal?

This was proposed before we'd identified the problem with the signal
queue getting too deep, so may not be helpful any more; but the idea
is that since ^C returns you to the caller of recursive-edit, this
means precmd is not run.  So if you need to reset something that is
normally handled in precmd, you should include that part of precmd
in the wrapper, calling it after "zle .recursive-edit" returns.

} I thought about trapping INT when detected that
} in .recursive-edit, but directly that's "Ctrl-C problems" stuff

trap 'zle && zle .send-break' INT # might be helpful with localtraps

} > Either open a named pipe file (mknod p filename), or use the zsh/tcp
} > module as in the example that appears in the documentation.
} 
} That are rather heavy-implementation things

Is having a named pipe file really that much heavier than having the
shared text file plus that external flocking app to synchronize the
access to it?

Re: Where to start debugging zle recursive-edit? / Ctrl-C

[Sebastian Gniazdowski]

On 5 October 2016 at 20:31, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Oct 5, 12:27pm, Sebastian Gniazdowski wrote:
> } I thought about trapping INT when detected that
> } in .recursive-edit, but directly that's "Ctrl-C problems" stuff
>
> trap 'zle && zle .send-break' INT # might be helpful with localtraps

Thanks, was about to search for a way to interrupt recursive-edit.

> } > Either open a named pipe file (mknod p filename), or use the zsh/tcp
> } > module as in the example that appears in the documentation.
> }
> } That are rather heavy-implementation things
>
> Is having a named pipe file really that much heavier than having the
> shared text file plus that external flocking app to synchronize the
> access to it?

Binaries are cool if they are created using small, portable C code.
Compilation is fast, without "strcasestr() exists?" problems, and so
many such C programs out there, expecting them to compile everywhere
after I compiled util-linux/flock on FreeBSD and OS X. Tried to
convert program "highlight" to return region_highlight and it was much
worse because it's large C++ code without configure script. Thinking
about what's actually heavy and what's not made me realize that I
might refuse a good solution, after all it's only one *period_passed
function that needs to be called. Will check this soon.

Best regards,
Sebastian Gniazdowski

Re: Where to start debugging zle recursive-edit? / Ctrl-C

[Bart Schaefer]

On Sep 30,  1:28pm, Sebastian Gniazdowski wrote:
} 
} The stack trace is at the end.

Is this a trace from a place where you sometimes get a core dump or
unexpected exit?  Because I don't see indication in the stack trace
that menu selection is active, for example.

Re: Where to start debugging zle recursive-edit? / Ctrl-C

[Sebastian Gniazdowski]

On 30 September 2016 at 23:45, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Sep 30,  1:28pm, Sebastian Gniazdowski wrote:
> }
> } The stack trace is at the end.
>
> Is this a trace from a place where you sometimes get a core dump or
> unexpected exit?  Because I don't see indication in the stack trace
> that menu selection is active, for example.

No, no luck with the crashes. It's a stack trace after Ctrl-C when in
.recursiveedit, i.e. when the Ctrl-R widget was invoked. The stressors
were commented out in zshrc. BTW. I was running multiple jobs because
on IRC one user said that he gets crashes when multiple jobs in
background. Didn't ask of ZSH_VERSION. Tomorrow will try with Zconvey
disabled, the Ctrl-C problems appeared after additional complex-char
mappings (e.g. cursors) so there should be something to examine.

Best regards,
Sebastian Gniazdowski