[BUG] Ctrl-C stops working after process substitution

[BUG] Ctrl-C stops working after process substitution

[Sebastian Gniazdowski]

Hello,
to reproduce, copy-paste following line:

cat < <( echo this is a test )

After it executes (after pressing enter, if copying it didn't work),
press Up cursor and try hitting Ctrl-C to cancel a command. Nothing
will happen, Ctrl-C will have no effect. To dig-out of that state,
Ctrl-D, then Ctrl-C, and then Ctrl-C will work.

Zsh-5.1.1-dev-0, HEAD, checked also zsh 5.0.0 and 5.0.8.

Where to debug this, in Zsh source? Maybe someone has an idea what can
be wrong? It seems to be nasty hidden bug, but who knows, maybe an
immediate fix is possible.

-- 
Sebastian Gniazdowski
News: https://twitter.com/ZdharmaI
IRC: https://kiwiirc.com/client/chat.freenode.net:+6697/#zplugin

Re: [BUG] Ctrl-C stops working after process substitution

[Daniel Shahaf]

Sebastian Gniazdowski wrote on Wed, 04 Jul 2018 13:47 +0200:
> After it executes (after pressing enter, if copying it didn't work),
> press Up cursor and try hitting Ctrl-C to cancel a command.

Works for me in master.

Re: [BUG] Ctrl-C stops working after process substitution

[Sebastian Gniazdowski]

On 4 July 2018 at 14:09, Daniel Shahaf <d.s@daniel.shahaf.name> wrote:
> Works for me in master.

True, here is code that yields Ctrl-C problem even in zsh -f:

For zsh -f:

noop() { IFS='' read line; MYFD="$1"; zle -F "$1"; exec {MYFD}<&-; };
exec {MYFD}< <( echo a test ); zle -F -w $MYFD noop

After executing, try Up cursor to obtain some command at prompt (or
just enter "abcd") and press Ctrl-C. First will be ignored, second one
will work. This differs from non-zsh-f behavior where any Ctrl-C will
be ignored. However in zsh -f situation, what's very interesting is
that the second-Ctrl-C behavior occurs for any number of following
commands.

-- 
Sebastian Gniazdowski
News: https://twitter.com/ZdharmaI
IRC: https://kiwiirc.com/client/chat.freenode.net:+6697/#zplugin

Re: [BUG] Ctrl-C stops working after process substitution

[Peter Stephenson]

On Wed, 4 Jul 2018 19:07:17 +0200
Sebastian Gniazdowski <sgniazdowski@gmail.com> wrote:
> On 4 July 2018 at 14:09, Daniel Shahaf <d.s@daniel.shahaf.name> wrote:

> > Works for me in master.  
> 
> True, here is code that yields Ctrl-C problem even in zsh -f:
> 
> For zsh -f:
> 
> noop() { IFS='' read line; MYFD="$1"; zle -F "$1"; exec {MYFD}<&-; };
> exec {MYFD}< <( echo a test ); zle -F -w $MYFD noop
> 
> After executing, try Up cursor to obtain some command at prompt (or
> just enter "abcd") and press Ctrl-C. First will be ignored, second one
> will work. This differs from non-zsh-f behavior where any Ctrl-C will
> be ignored. However in zsh -f situation, what's very interesting is
> that the second-Ctrl-C behavior occurs for any number of following
> commands.

My first guess about this is that it's to do with propagation
of errors and interrupts back from the widget.

The -F handling is in the loops beginning

		for (i = 0; i < lnwatch; i++) {

in zle_main.c.  At the end there's a check for errflag and if
it's set the error bit ERRFLAG_ERROR is removed and we ignore that
FD until the read function returns.  There could be something
unhelpful in this area.  In particular we don't do anything with
the interrupt bit ERRFLAG_INT or a hard error ERRFLAG_HARD.

But that's just a guess.

pws

Re: [BUG] Ctrl-C stops working after process substitution

[Sebastian Gniazdowski]

On 5 July 2018 at 10:44, Peter Stephenson <p.stephenson@samsung.com> wrote:
> My first guess about this is that it's to do with propagation
> of errors and interrupts back from the widget.
>
> The -F handling is in the loops beginning
>
>                 for (i = 0; i < lnwatch; i++) {
>
> in zle_main.c.  At the end there's a check for errflag and if
> it's set the error bit ERRFLAG_ERROR is removed and we ignore that
> FD until the read function returns.  There could be something
> unhelpful in this area.  In particular we don't do anything with
> the interrupt bit ERRFLAG_INT or a hard error ERRFLAG_HARD.

I'm trying to debug it now. I noted, that the handler (noop) isn't
run. Tried to attach from within Zle:

noop() { print "I'm ran" >> /tmp/reply; IFS='' read line; MYFD="$1";
zle -F "$1"; exec {MYFD}<&-; };
exec {MYFD}< <( sleep 2; echo a test ); attach() { zle -F -w $MYFD
noop; }; zle -N attach; bindkey '^T' attach

But still cannot see the `print' message in /tmp/reply. This is weird
as this works from bigger script, maybe there's a mistake?

-- 
Sebastian Gniazdowski
News: https://twitter.com/ZdharmaI
IRC: https://kiwiirc.com/client/chat.freenode.net:+6697/#zplugin

Re: [BUG] Ctrl-C stops working after process substitution

[Sebastian Gniazdowski]

On 7 July 2018 at 15:03, Sebastian Gniazdowski <sgniazdowski@gmail.com> wrote:

> But still cannot see the `print' message in /tmp/reply. This is weird
> as this works from bigger script, maybe there's a mistake?

There was indeed a mistake – zle -F should obtain a widget, not a
function. Handler is called now.

-- 
Sebastian Gniazdowski
News: https://twitter.com/ZdharmaI
IRC: https://kiwiirc.com/client/chat.freenode.net:+6697/#zplugin

Re: [BUG] Ctrl-C stops working after process substitution

[Sebastian Gniazdowski]

On 5 July 2018 at 10:44, Peter Stephenson <p.stephenson@samsung.com> wrote:
At the end there's a check for errflag and if
> it's set the error bit ERRFLAG_ERROR is removed and we ignore that
> FD until the read function returns.  There could be something
> unhelpful in this area.  In particular we don't do anything with
> the interrupt bit ERRFLAG_INT or a hard error ERRFLAG_HARD.
>
> But that's just a guess.

I have an observation. I configure LLDB to pass SIGINT to program.
LLDB shows this table:

NAME         PASS   STOP   NOTIFY
===========  =====  =====  ======
SIGINT       true   true   true

So on Ctrl-C, it will be passed to program and program will be stopped
(i.e. debugger will stop program to allow stepping, etc.).

I then attach to proper PID of a Zsh instance. Then press Ctrl-C –
debugger activates, Zsh obtains signal. Ok.

Then I paste some text into Zsh so that Fast Syntax Highlighting runs
zle -F -w handlers that check if string is a path (to highlight this
string).

So far so good. Then I press Ctrl-C and (IMPORTANT) – nothing happens.
Even debugger doesn't activate. I can press Ctrl-C many times. Pasting
text that runs zle -F -w stuff leaves Zsh and debugger blind to
Ctrl-C. Isn't this a masking of interrupt? This would explain the main
problem – no reaction to Ctrl-C.

If I then use the known method to dig-out from this improper state –
Ctrl-D, then Ctrl-C starts working again – debugger notices it and
activates, program obtains it (zhandler is called).

So what could mask SIGINT as the result of pasting text and zle -F -w
invocation? There's a hint: pasting the text into command line causes
zhandler invocations (I have breakpoint in zhandler). I paste string
"NAME" (header of the table) – nothing happens. Press space, then
paste "NAME" again – now Fast Syntax Highlighting will run zle -F -w
because "NAME" occurs as program argument, and can potentially be a
path. Effect: two zhandler calls:

    frame #0: 0x000000010f8517d2 zsh`zhandler(sig=20) at signals.c:601
-> 601         last_signal = sig;
   602         signal_process(sig);

sig == 20, then some stuff is executed that isn't clear for me yet.

What can be happening? SIGINT (?) reaching zhandler two times, just
because I pasted text that ran zle -F -w?

I then go final minimal thing: start zsh -f and run:

noop() { print "I'm ran" >> /tmp/reply; IFS='' read line; MYFD="$1";
zle -F "$1"; exec {MYFD}<&-; };
exec {MYFD}< <( echo a test ); attach() { zle -F -w $MYFD noop; }; zle
-N noop; attach

And – zhandler is called! It's pure-empty, clear Zshell, no
fast-syntax-highlighting, so just pasting doesn't yield any debugger
reaction. But running it calls zhandler (sig==20) two times. I then
minimize further:

exec {MYFD}< <( echo a test )

This code starts zhandler 1 time (sig==20).

So my conclusion is: exec and/or <() invoke zdhandler in such a way,
that it masks SIGINT.

That said, full Zsh with Fast-Syntax-Highlighting is needed for the
effect of Ctrl-C blockage to occur, in zsh -f I can repeat zhandler
invocations, but Ctrl-C still works.

-- 
Sebastian Gniazdowski
News: https://twitter.com/ZdharmaI
IRC: https://kiwiirc.com/client/chat.freenode.net:+6697/#zplugin

Re: [BUG] Ctrl-C stops working after process substitution

[Sebastian Gniazdowski]

Signal 20 is CHLD. I deferred sending this signal by <( echo test;
sleep 15 ) – the `sleep' here. So zhandler() isn't called when exec <
<() runs. I then test pressing of Ctrl-C. But, again: INT is
apparently masked, as both debugger and Zsh don't react to Ctrl-C. So
it's not zhandler that masks INT (it's called ~15 seconds after I test
Ctrl-C).

So what can mask SIGINT?

On 7 July 2018 at 16:48, Sebastian Gniazdowski <sgniazdowski@gmail.com> wrote:
> On 5 July 2018 at 10:44, Peter Stephenson <p.stephenson@samsung.com> wrote:
> At the end there's a check for errflag and if
>> it's set the error bit ERRFLAG_ERROR is removed and we ignore that
>> FD until the read function returns.  There could be something
>> unhelpful in this area.  In particular we don't do anything with
>> the interrupt bit ERRFLAG_INT or a hard error ERRFLAG_HARD.
>>
>> But that's just a guess.
>
> I have an observation. I configure LLDB to pass SIGINT to program.
> LLDB shows this table:
>
> NAME         PASS   STOP   NOTIFY
> ===========  =====  =====  ======
> SIGINT       true   true   true
>
> So on Ctrl-C, it will be passed to program and program will be stopped
> (i.e. debugger will stop program to allow stepping, etc.).
>
> I then attach to proper PID of a Zsh instance. Then press Ctrl-C –
> debugger activates, Zsh obtains signal. Ok.
>
> Then I paste some text into Zsh so that Fast Syntax Highlighting runs
> zle -F -w handlers that check if string is a path (to highlight this
> string).
>
> So far so good. Then I press Ctrl-C and (IMPORTANT) – nothing happens.
> Even debugger doesn't activate. I can press Ctrl-C many times. Pasting
> text that runs zle -F -w stuff leaves Zsh and debugger blind to
> Ctrl-C. Isn't this a masking of interrupt? This would explain the main
> problem – no reaction to Ctrl-C.
>
> If I then use the known method to dig-out from this improper state –
> Ctrl-D, then Ctrl-C starts working again – debugger notices it and
> activates, program obtains it (zhandler is called).
>
> So what could mask SIGINT as the result of pasting text and zle -F -w
> invocation? There's a hint: pasting the text into command line causes
> zhandler invocations (I have breakpoint in zhandler). I paste string
> "NAME" (header of the table) – nothing happens. Press space, then
> paste "NAME" again – now Fast Syntax Highlighting will run zle -F -w
> because "NAME" occurs as program argument, and can potentially be a
> path. Effect: two zhandler calls:
>
>     frame #0: 0x000000010f8517d2 zsh`zhandler(sig=20) at signals.c:601
> -> 601         last_signal = sig;
>    602         signal_process(sig);
>
> sig == 20, then some stuff is executed that isn't clear for me yet.
>
> What can be happening? SIGINT (?) reaching zhandler two times, just
> because I pasted text that ran zle -F -w?
>
> I then go final minimal thing: start zsh -f and run:
>
> noop() { print "I'm ran" >> /tmp/reply; IFS='' read line; MYFD="$1";
> zle -F "$1"; exec {MYFD}<&-; };
> exec {MYFD}< <( echo a test ); attach() { zle -F -w $MYFD noop; }; zle
> -N noop; attach
>
> And – zhandler is called! It's pure-empty, clear Zshell, no
> fast-syntax-highlighting, so just pasting doesn't yield any debugger
> reaction. But running it calls zhandler (sig==20) two times. I then
> minimize further:
>
> exec {MYFD}< <( echo a test )
>
> This code starts zhandler 1 time (sig==20).
>
> So my conclusion is: exec and/or <() invoke zdhandler in such a way,
> that it masks SIGINT.
>
> That said, full Zsh with Fast-Syntax-Highlighting is needed for the
> effect of Ctrl-C blockage to occur, in zsh -f I can repeat zhandler
> invocations, but Ctrl-C still works.
>
> --
> Sebastian Gniazdowski
> News: https://twitter.com/ZdharmaI
> IRC: https://kiwiirc.com/client/chat.freenode.net:+6697/#zplugin



-- 
Sebastian Gniazdowski
News: https://twitter.com/ZdharmaI
IRC: https://kiwiirc.com/client/chat.freenode.net:+6697/#zplugin

Re: [BUG] Ctrl-C stops working after process substitution

[Peter Stephenson]

On Tue, 10 Jul 2018 17:06:42 +0200
Sebastian Gniazdowski <sgniazdowski@gmail.com> wrote:
> Signal 20 is CHLD. I deferred sending this signal by <( echo test;
> sleep 15 ) – the `sleep' here. So zhandler() isn't called when exec <
> <() runs. I then test pressing of Ctrl-C. But, again: INT is
> apparently masked, as both debugger and Zsh don't react to Ctrl-C. So
> it's not zhandler that masks INT (it's called ~15 seconds after I test
> Ctrl-C).
> 
> So what can mask SIGINT?

This is usually a call to holdintr().

pws

Re: [BUG] Ctrl-C stops working after process substitution

[Sebastian Gniazdowski]

On 10 July 2018 at 17:21, Peter Stephenson <p.stephenson@samsung.com> wrote:
> On Tue, 10 Jul 2018 17:06:42 +0200
> Sebastian Gniazdowski <sgniazdowski@gmail.com> wrote:
>> So what can mask SIGINT?
>
> This is usually a call to holdintr().

I tried setting breakpoint on holdintr and it wasn't called.

I had some realization that maybe child-process created by <() masks
SIGINT, and as it holds /dev/tty, this somehow leaks-out to the main
zsh-process, but this is naive theory, as currently I kill -9 the
child process instantly after reading needed data from it, so the
"leak" of SIGINT mask should at that moment be withdrawn. Who knows..


-- 
Sebastian Gniazdowski
News: https://twitter.com/ZdharmaI
IRC: https://kiwiirc.com/client/chat.freenode.net:+6697/#zplugin

Re: [BUG] Ctrl-C stops working after process substitution

[Sebastian Gniazdowski]

Managed to find a workaround (shown at the end of email).

====== Eliminations ======

I did series of eliminations. Conclusion: it's {MYFD}< construct that
is responsible for my problem.

1. Instead of exec {MYFD}< <(...) I did: myfunct {MYFD}< <(...).
Problem still occurred.

This eliminated exec. The function is:

myfunct() { echo "$PCFD" >> /tmp/reply; }

When I add "command" before `echo', the problem disappears! (i.e.
Ctrl-C works). It is a recurring argument, that Zsh needs just a
little push to get out of this Ctrl-C stall. Running a command and not
series of builtins seems to be enough. This yielded a workaround shown
at the end of email.

2. I commented-out zle -F -w invocation. Problem still occurred.

This eliminated zle -F itself and code in zle_main.c/raw_getbyte().

3. I did cat <(...) instead of exec {MYFD}< <(...), problem didn't occur.

This eliminated <(...) as the source of problem. Also did myfunct <(...).

====== Workaround ======

Translating 1. and the `command echo' inside myfunct() into
exec-variant code, I thought that the `command echo ...' should be
placed just after the exec. The actual code (disturbed by long
debugging) is now:

                         exec {PCFD}< <(-fast-highlight-check-path; sleep 5)
                         command sleep 0
                         FAST_HIGHLIGHT[path-queue]+=";$_start_pos $_end_pos;"
                         is-at-least 5.0.6 && __pos=1 || __pos=0
                         zle -F ${${__pos:#0}:+-w} "$PCFD"
fast-highlight-check-path-handler

Ctrl-C works this way. The conclusion from the eliminations is that
{MYFD}< construct is responsible for those Ctrl-C problems. Where in
the source to look for it?

-- 
Sebastian Gniazdowski
News: https://twitter.com/ZdharmaI
IRC: https://kiwiirc.com/client/chat.freenode.net:+6697/#zplugin

Re: [BUG] Ctrl-C stops working after process substitution

[Peter Stephenson]

On Wed, 11 Jul 2018 10:48:54 +0200
Sebastian Gniazdowski <sgniazdowski@gmail.com> wrote:
> Ctrl-C works this way. The conclusion from the eliminations is that
> {MYFD}< construct is responsible for those Ctrl-C problems. Where in
> the source to look for it?

The wordcode is associated with WC_REDIR_VARID.  For the functions that
actually set up the redirection to/from MYFD, look in exec.c and search
for references to "varid" --- in particular, in addfd() in the case
where the last argument varid is not NULL.  All the relevant
calls are from the section commented as "Do io redirections" in
execcmd_exec().

If this is a "{MYFD}< <(stuff)", then the process is started from
getpipe() in the same file --- this is a special case compared with
other uses of <(...) because we don't need a file name, just an fd,
so don't need named pipes or /dev/stuff.  This happens within
spawnpipes(), which is called immediately above the chunk that does the
FD management using the newly created fd.

pws