exit_function - strange behavior

exit_function - strange behavior

[Tycho Kirchner]

Dear zsh-maintainers,

first of all, I'm not a mail-subscriber but please respond to this email 
anyway ^_^

In the function doc for zshexit it states:
 > Executed at the point where the main shell is about to exit normally. 
This is not called by exiting subshells...

Having read that the following session yields a surprising result (at 
least for me):


debian-dell% echo $ZSH_VERSION # Running on Debian Buster
5.8.0.2-dev
debian-dell% __zshrc_exit(){ echo "__zshrc_exit in subshell 
$ZSH_SUBSHELL with $?"; }
debian-dell% zshexit_functions+=(__zshrc_exit)
debian-dell% ( echo foo )
foo
debian-dell% ( exit 123 )
__zshrc_exit in subshell 1 with 123  # <---
debian-dell%


On the other hand the exit function seems to be not called for the main 
shell, when a subshell called 'exit' immediately before:


debian-dell% zsh -f -c '__zshrc_exit(){ echo "__zshrc_exit in subshell 
$ZSH_SUBSHELL with $?"; };  zshexit_functions+=(__zshrc_exit);  (exit 123);'
__zshrc_exit in subshell 1 with 123
debian-dell%


In case that this is desired behavior, how would one execute custom code 
*only* on exit of the main-shell?


Thanks in advance
Kind regards
Tycho Kirchner

Re: exit_function - strange behavior

[Bart Schaefer]

On Mon, Oct 25, 2021 at 4:28 PM Tycho Kirchner <tychokirchner@mail.de> wrote:
>
> In the function doc for zshexit it states:
>  > Executed at the point where the main shell is about to exit normally.
> This is not called by exiting subshells...

Documentation mistake.  An explicit call to "exit" always triggers the
hook, the reference to "not called by exiting subshells" refers to the
subshell ending by finishing all its commands.

You can avoid that by having the subshell end with "return 123"
instead of "exit 123", except for some reason (possibly a bug) when
the -c option is used in which case return behaves like exit again.

> On the other hand the exit function seems to be not called for the main
> shell, when a subshell called 'exit' immediately before:

That's tricky ... it's because zsh optimizes away one level of shell
when there's nothing else to do after the subshell exits, so the
subshell "becomes" the main shell.

There's definitely a bug in this somewhere because if the last command
executed by the final subshell an anonymous function call:

zsh -fc '...; ( () { return 123 } )'

then the zshexit hook is NEVER called.

Re: exit_function - strange behavior

[Bart Schaefer]

On Mon, Oct 25, 2021 at 5:53 PM Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> There's definitely a bug in this somewhere because if the last command
> executed by the final subshell [is] an anonymous function call:
>
> zsh -fc '...; ( () { return 123 } )'
>
> then the zshexit hook is NEVER called.

When an anonymous function (or probably any function) calls "exit", we
pass through this bit of code in builtin.c bin_break():

5719             *
5720             * If we are already exiting... give this all up as
5721             * a bad job.
5722             */
5723            if (stopmsg || (zexit(0, ZEXIT_DEFERRED), !stopmsg)) {
5724            retflag = 1;
5725            breaks = loops;
5726            exit_pending = 1;
5727            exit_level = locallevel;
5728            exit_val = num;

With ZEXIT_DEFERRED, zexit() always bails out here:

5842     /* Positive shell_exiting means we have been here before */
5843     if (from_where == ZEXIT_DEFERRED ||
5844         (shell_exiting++ && from_where != ZEXIT_NORMAL))
5845         return;

We then eventually call zexit(exit_val, ZEXIT_NORMAL) from doshfunc().

If instead the function calls "return 123" and is the last command in
the subshell, we pass through this branch of execcmd_exec():

4239     if (forked) {
4240         /*
4241          * So what's going on here then?  Well, I'm glad you asked.
[...]
4263          */
4264         for (i = 0; i < 10; i++)
4265             if (fdtable[i] != FDT_UNUSED)
4266                 close(i);
4267         closem(FDT_UNUSED, 1);
4268         if (thisjob != -1)
4269             waitjobs();
4270         _realexit();
4271     }

The call to _realexit() bypasses the hook.  This is the expected
behavior for "falling off the end" of a subshell, rather than
explicitly "exit"-ing.

However, in
  zsh -fc '( () { return 123 } )'
the parent shell optimizes away the subshell and we arrive here:

3596             /*
3597              * If we are in a subshell environment anyway, say
we're forked,
3598              * even if we're actually not forked because we know the
3599              * subshell is exiting.  This ensures SHLVL reflects
the current
3600              * shell, and also optimises out any save/restore we'd need to
3601              * do if we were returning to the main shell.
3602              */
3603             if (type == WC_SUBSH)
3604                 forked = 1;

This should not happen when there is an exit trap or an exit hook, I
think?  However, I'm not following the comment reference to SHLVL --
why would it not reflect the correct thing?  I suppose the actual
correct thing is higher up the call stack, where we should not assert
that the subshell is exiting if the parent shell still has traps or
hooks to process, so that we really have forked here.

As to this:

> You can avoid that by having the subshell end with "return 123"
> instead of "exit 123", except for some reason (possibly a bug) when
> the -c option is used in which case return behaves like exit again.

bin_break() treats "return" as synonymous with "exit" here:

5683     case BIN_RETURN:
5684         if ((isset(INTERACTIVE) && isset(SHINSTDIN))
5685             || locallevel || sourcelevel) {
[...]
5699             return lastval;
5700         }
5701         zexit(num, ZEXIT_NORMAL);       /* else treat return as
logout/exit */

To Tycho's original question, this means there is no way to have an
exit hook or trap called exactly when the original shell exits.  You
can't forcibly finish a subshell before it "falls off the end" without
possibly invoking the trap and hook, and you can't even reliably test
$ZSH_SUBSHELL inside the hook, because the parent might optimize out a
fork without decrementing that.

[[ $sysparams[pid] = $$ ]] almost gets there, except that you can't
assure the hook itself won't be skipped by a function that calls exit.

Re: exit_function - strange behavior

[Tycho Kirchner]

Am 01.11.21 um 00:52 schrieb Bart Schaefer:
> On Mon, Oct 25, 2021 at 5:53 PM Bart Schaefer <schaefer@brasslantern.com> wrote:
>>
>> There's definitely a bug in this somewhere because if the last command
>> executed by the final subshell [is] an anonymous function call:
>>
>> zsh -fc '...; ( () { return 123 } )'
>>
>> then the zshexit hook is NEVER called.
> 
> When an anonymous function (or probably any function) calls "exit", we
> pass through this bit of code in builtin.c bin_break():
> 
> 5719             *
> 5720             * If we are already exiting... give this all up as
> 5721             * a bad job.
> 5722             */
> 5723            if (stopmsg || (zexit(0, ZEXIT_DEFERRED), !stopmsg)) {
> 5724            retflag = 1;
> 5725            breaks = loops;
> 5726            exit_pending = 1;
> 5727            exit_level = locallevel;
> 5728            exit_val = num;
> 
> With ZEXIT_DEFERRED, zexit() always bails out here:
> 
> 5842     /* Positive shell_exiting means we have been here before */
> 5843     if (from_where == ZEXIT_DEFERRED ||
> 5844         (shell_exiting++ && from_where != ZEXIT_NORMAL))
> 5845         return;
> 
> We then eventually call zexit(exit_val, ZEXIT_NORMAL) from doshfunc().
> 
> If instead the function calls "return 123" and is the last command in
> the subshell, we pass through this branch of execcmd_exec():
> 
> 4239     if (forked) {
> 4240         /*
> 4241          * So what's going on here then?  Well, I'm glad you asked.
> [...]
> 4263          */
> 4264         for (i = 0; i < 10; i++)
> 4265             if (fdtable[i] != FDT_UNUSED)
> 4266                 close(i);
> 4267         closem(FDT_UNUSED, 1);
> 4268         if (thisjob != -1)
> 4269             waitjobs();
> 4270         _realexit();
> 4271     }
> 
> The call to _realexit() bypasses the hook.  This is the expected
> behavior for "falling off the end" of a subshell, rather than
> explicitly "exit"-ing.
> 
> However, in
>    zsh -fc '( () { return 123 } )'
> the parent shell optimizes away the subshell and we arrive here:
> 
> 3596             /*
> 3597              * If we are in a subshell environment anyway, say
> we're forked,
> 3598              * even if we're actually not forked because we know the
> 3599              * subshell is exiting.  This ensures SHLVL reflects
> the current
> 3600              * shell, and also optimises out any save/restore we'd need to
> 3601              * do if we were returning to the main shell.
> 3602              */
> 3603             if (type == WC_SUBSH)
> 3604                 forked = 1;
> 
> This should not happen when there is an exit trap or an exit hook, I
> think?  However, I'm not following the comment reference to SHLVL --
> why would it not reflect the correct thing?  I suppose the actual
> correct thing is higher up the call stack, where we should not assert
> that the subshell is exiting if the parent shell still has traps or
> hooks to process, so that we really have forked here.
> 
> As to this:
> 
>> You can avoid that by having the subshell end with "return 123"
>> instead of "exit 123", except for some reason (possibly a bug) when
>> the -c option is used in which case return behaves like exit again.
> 
> bin_break() treats "return" as synonymous with "exit" here:
> 
> 5683     case BIN_RETURN:
> 5684         if ((isset(INTERACTIVE) && isset(SHINSTDIN))
> 5685             || locallevel || sourcelevel) {
> [...]
> 5699             return lastval;
> 5700         }
> 5701         zexit(num, ZEXIT_NORMAL);       /* else treat return as
> logout/exit */
> 
> To Tycho's original question, this means there is no way to have an
> exit hook or trap called exactly when the original shell exits.  You
> can't forcibly finish a subshell before it "falls off the end" without
> possibly invoking the trap and hook, and you can't even reliably test
> $ZSH_SUBSHELL inside the hook, because the parent might optimize out a
> fork without decrementing that.
> 
> [[ $sysparams[pid] = $$ ]] almost gets there, except that you can't
> assure the hook itself won't be skipped by a function that calls exit.
> 

Bart, thanks for diving into this, I hope it can be fixed. I just wanted 
to point out that the documentation of $ZSH_SUBSHELL needs to be fixed 
as well:

ZSH_SUBSHELL

     Readonly integer. Initially zero, incremented each time the shell 
forks to create a subshell for executing code. Hence ‘(print 
$ZSH_SUBSHELL)’ and ‘print $(print $ZSH_SUBSHELL)’ output 1, while ‘( 
(print $ZSH_SUBSHELL) )’ outputs 2.

So $ZSH_SUBSHELL may also be incremented, if the shell did _not_ fork:

% zsh -fc 'zmodload zsh/system; echo parent_pid: $$;  echo 
parent_subshell $ZSH_SUBSHELL ; ( echo  sysparams_pid: $sysparams[pid]; 
read -d " " pid < /proc/self/stat; echo proc_pid:  $pid; echo 
child_subshell: $ZSH_SUBSHELL ; exit 123 )'
parent_pid: 20562
parent_subshell 0
sysparams_pid: 20562
proc_pid: 20562
child_subshell: 1

Anyway, I think it is ok for $ZSH_SUBSHELL to be inconsistent with 
$sysparams[pid] as the fork-optimization is an implementation detail and 
most users probably want to know whether being in enclosed '(  )'.