suspend (^Z) behavior while a function is running is unclear

suspend (^Z) behavior while a function is running is unclear

[Vincent Lefevre]

I've reported the following problem on

  http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=660630

The suspend (^Z) behavior while a function is running is unclear.
The documentation (man pages) should describe the behavior. There
may be a bug in zsh, as some other shells don't behave in the same
way.

Consider:

foo() { emacs "$@"; d=`date`; echo "$d"; }

and the following test under X (so that Emacs uses its own window,
or you can try with another X application):

1. Run foo.
2. Type ^Z in the terminal.
3. Type: fg [Return].
4. Quit Emacs.
5. Type: echo "$d" [Return].

With zsh, the whole function is suspended. The effect is that zsh is
forked (this is not documented, though can be guessed), thus $d will
not be set in the main shell (Step 5: nothing is displayed).

With bash, dash, mksh and pdksh, only the emacs process is suspended.
With ksh93, the ^Z is ignored until Emacs is quit, then the function
is suspended, with the same effects as in zsh. With posh, the whole
shell is suspended.

Versions of the packages:
  bash 4.2-1
  dash 0.5.7-2
  ksh 93u-1
  mksh 40.4-3
  pdksh 5.2.14-26
  posh 0.10.1
  zsh 4.3.15-1
  zsh-beta 4.3.15-dev-0+20120108-1

BTW, concerning zsh, if the behavior is regarded as correct, a fork
is still done even when the suspended command is the last one in the
function, e.g.

  foo() { emacs "$@"; }

Is the fork really necessary?

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <http://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: suspend (^Z) behavior while a function is running is unclear

[Peter Stephenson]

On Mon, 20 Feb 2012 14:02:59 +0100
Vincent Lefevre <vincent@vinc17.net> wrote:
> foo() { emacs "$@"; d=`date`; echo "$d"; }
> 
> and the following test under X (so that Emacs uses its own window,
> or you can try with another X application):
> 
> 1. Run foo.
> 2. Type ^Z in the terminal.
> 3. Type: fg [Return].
> 4. Quit Emacs.
> 5. Type: echo "$d" [Return].
> 
> With zsh, the whole function is suspended. The effect is that zsh is
> forked (this is not documented, though can be guessed), thus $d will
> not be set in the main shell (Step 5: nothing is displayed).

Yes, this is long-standing behaviour that's never been described.  I've
tried to document it.

> BTW, concerning zsh, if the behavior is regarded as correct, a fork
> is still done even when the suspended command is the last one in the
> function, e.g.
> 
>   foo() { emacs "$@"; }

There's certainly no good reason for trying to make a special case
here; it's complicated for very little gain.

> Is the fork really necessary?

If you're actually trying to do real, live job control (rather than some
internal approximation to it), yes, as that operates on processes, and
(in your example) the shell function is the top-level code you need to
suspend, regardless of any processes that may also have been started
from it. I can't tell you if it's necessary for what you thought you
were trying to do, if it wasn't job control.

Index: Doc/Zsh/jobs.yo
===================================================================
RCS file: /cvsroot/zsh/zsh/Doc/Zsh/jobs.yo,v
retrieving revision 1.5
diff -p -u -r1.5 jobs.yo
--- Doc/Zsh/jobs.yo	29 Mar 2006 19:25:15 -0000	1.5
+++ Doc/Zsh/jobs.yo	20 Feb 2012 15:00:32 -0000
@@ -37,6 +37,21 @@ when it is typed.
 
 A job being run in the background will suspend if it tries to read
 from the terminal.
+
+Note that if the job running in the foreground is a shell function,
+then suspending it will have the effect of causing the shell to fork.
+This is necessary to separate the function's state from that of the
+parent shell performing the job control, so that the latter can return
+to the command line prompt.  As a result, even if tt(fg) is
+used to continue the job the function will no longer be part of the
+parent shell, and any variables set by the function will not be visible
+in the parent shell.  Thus the behaviour is different from the case
+where the function was never suspended.  Zsh is is different from many
+other shells in this regard.
+
+The same behaviour is found when the shell is executing code as the
+right hand side of a pipeline, in order that the entire pipeline
+can be managed as a single job.
 cindex(background jobs, I/O)
 cindex(jobs, background, I/O)
 Background jobs are normally allowed to produce output,

-- 
Peter Stephenson <pws@csr.com>            Software Engineer
Tel: +44 (0)1223 692070                   Cambridge Silicon Radio Limited
Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, UK


Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom
More information can be found at www.csr.com. Follow CSR on Twitter at http://twitter.com/CSR_PLC and read our blog at www.csr.com/blog

Re: suspend (^Z) behavior while a function is running is unclear

[Vincent Lefevre]

On 2012-02-20 15:13:09 +0000, Peter Stephenson wrote:
> On Mon, 20 Feb 2012 14:02:59 +0100
> Vincent Lefevre <vincent@vinc17.net> wrote:
> > BTW, concerning zsh, if the behavior is regarded as correct, a fork
> > is still done even when the suspended command is the last one in the
> > function, e.g.
> > 
> >   foo() { emacs "$@"; }
> 
> There's certainly no good reason for trying to make a special case
> here; it's complicated for very little gain.
> 
> > Is the fork really necessary?
> 
> If you're actually trying to do real, live job control (rather than some
> internal approximation to it), yes, as that operates on processes, and
> (in your example) the shell function is the top-level code you need to
> suspend, regardless of any processes that may also have been started
> from it. I can't tell you if it's necessary for what you thought you
> were trying to do, if it wasn't job control.

Well, I have an emacs wrapper that does various things, then runs
emacs. Thus, when I suspend it or run it in background, there's a
spurious zsh process, which is sometimes a bit confusing. But to
avoid that, perhaps it's better to have a real script, not a zsh
function.

> Index: Doc/Zsh/jobs.yo
> ===================================================================
> RCS file: /cvsroot/zsh/zsh/Doc/Zsh/jobs.yo,v
> retrieving revision 1.5
> diff -p -u -r1.5 jobs.yo
> --- Doc/Zsh/jobs.yo	29 Mar 2006 19:25:15 -0000	1.5
> +++ Doc/Zsh/jobs.yo	20 Feb 2012 15:00:32 -0000
> @@ -37,6 +37,21 @@ when it is typed.
>  
>  A job being run in the background will suspend if it tries to read
>  from the terminal.
> +
> +Note that if the job running in the foreground is a shell function,
> +then suspending it will have the effect of causing the shell to fork.
> +This is necessary to separate the function's state from that of the
> +parent shell performing the job control, so that the latter can return
> +to the command line prompt.  As a result, even if tt(fg) is
> +used to continue the job the function will no longer be part of the
> +parent shell, and any variables set by the function will not be visible
> +in the parent shell.  Thus the behaviour is different from the case
> +where the function was never suspended.  Zsh is is different from many
                                                ^^^^^
duplicate "is"

> +other shells in this regard.
> +
> +The same behaviour is found when the shell is executing code as the
> +right hand side of a pipeline, in order that the entire pipeline
> +can be managed as a single job.
>  cindex(background jobs, I/O)
>  cindex(jobs, background, I/O)
>  Background jobs are normally allowed to produce output,

Now I'm thinking whether there should be an option to control that.
Indeed, some functions may fail to work correctly if they are suspended
in such a way. Some functions or { list } could be marked as not
backgroundable, e.g. by setting an option at the beginning, in which
case the ^Z could be ignored.

What happens if a TSTP signal is received while a completion function
is run or something like that?

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <http://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: suspend (^Z) behavior while a function is running is unclear

[Peter Stephenson]

On Mon, 20 Feb 2012 17:09:45 +0100
Vincent Lefevre <vincent@vinc17.net> wrote:
> Now I'm thinking whether there should be an option to control that.
> Indeed, some functions may fail to work correctly if they are suspended
> in such a way. Some functions or { list } could be marked as not
> backgroundable, e.g. by setting an option at the beginning, in which
> case the ^Z could be ignored.

Useful, but I'm not sure an option is the right way of doing this... I
should think you'd want to do something like:

{
  stty susp undef
  ...
} always {
  stty susp '^z'
}

which you could encapsulate as

nosuspend() {
  {
    stty susp undef
    "$@"
  } always {
    stty susp '^z'
  }
}
compdef _precommand nosuspend

except you can do better by saving and restoring the setting.

A library module for manipulating terminal settings using parameters
(which you can make local) might be an interesting project for someone
with nothing better to do.

(By the way, I'm wondering how many of the other shells simply execute
to the end of the parent function when they see the "emacs" process
suspended?)

-- 
Peter Stephenson <pws@csr.com>            Software Engineer
Tel: +44 (0)1223 692070                   Cambridge Silicon Radio Limited
Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, UK


Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom
More information can be found at www.csr.com. Follow CSR on Twitter at http://twitter.com/CSR_PLC and read our blog at www.csr.com/blog

Re: suspend (^Z) behavior while a function is running is unclear

[Bart Schaefer]

On Feb 20,  3:13pm, Peter Stephenson wrote:
> Subject: Re: suspend (^Z) behavior while a function is running is unclear
>
> +Note that if the job running in the foreground is a shell function,
> +then suspending it will have the effect of causing the shell to fork.
[...]
> +
> +The same behaviour is found when the shell is executing code as the
> +right hand side of a pipeline, in order that the entire pipeline
> +can be managed as a single job.

In fact it also happens when executing any "complex command" such as a
foreach loop:

torch% foreach foo (a b c) { sleep 5 }
zsh: suspended  for foo in a b c; do; sleep 5; done

On Feb 20,  4:35pm, Peter Stephenson wrote:
> Subject: Re: suspend (^Z) behavior while a function is running is unclear
>
> On Mon, 20 Feb 2012 17:09:45 +0100
> Vincent Lefevre <vincent@vinc17.net> wrote:
> > Now I'm thinking whether there should be an option to control that.
> > Indeed, some functions may fail to work correctly if they are suspended
> > in such a way. Some functions or { list } could be marked as not
> > backgroundable, e.g. by setting an option at the beginning, in which
> > case the ^Z could be ignored.

This is already partly in place:

2011-08-14  Barton E. Schaefer  <schaefer@zsh.org>

        * 29677: Src/exec.c, Src/signals.c, Src/zsh.h: flag jobs that are
        builtins running in the current shell, and if they control a
        pipeline, do not allow the external processes in that pipeline to
        become suspended when the foreground shell cannot suspend.

torch% foreach foo (a b c) { sleep 5 } | :  
zsh: job can't be suspended

Prior to 29677, ^Z here would have resulted in the loop becoming both
orphaned and stopped.  But here the foreach is already in a separate
process because of the way zsh manages pipelines.

> Useful, but I'm not sure an option is the right way of doing this... I
> should think you'd want to do something like:
> 
> {
>   stty susp undef
>   ...
> } always {
>   stty susp '^z'
> }

I'm not sure that's the right semantics.  Does one want to disable the
tty suspend character or just ingore the TSTP signal?  What if "emacs"
in the original example was replaced by running another shell?

> (By the way, I'm wondering how many of the other shells simply execute
> to the end of the parent function when they see the "emacs" process
> suspended?)

Preventing that behavior in a loop construct is I believe one of the
reasons zsh changed to behave the way it does.

(Another fine project from Wischnowsky Construction Co., as I recall.)

Re: suspend (^Z) behavior while a function is running is unclear

[Vincent Lefevre]

On 2012-02-20 09:41:00 -0800, Bart Schaefer wrote:
> On Feb 20,  4:35pm, Peter Stephenson wrote:
> > Subject: Re: suspend (^Z) behavior while a function is running is unclear
> >
> > On Mon, 20 Feb 2012 17:09:45 +0100
> > Vincent Lefevre <vincent@vinc17.net> wrote:
> > > Now I'm thinking whether there should be an option to control that.
> > > Indeed, some functions may fail to work correctly if they are suspended
> > > in such a way. Some functions or { list } could be marked as not
> > > backgroundable, e.g. by setting an option at the beginning, in which
> > > case the ^Z could be ignored.
> 
> This is already partly in place:
> 
> 2011-08-14  Barton E. Schaefer  <schaefer@zsh.org>
> 
>         * 29677: Src/exec.c, Src/signals.c, Src/zsh.h: flag jobs that are
>         builtins running in the current shell, and if they control a
>         pipeline, do not allow the external processes in that pipeline to
>         become suspended when the foreground shell cannot suspend.
> 
> torch% foreach foo (a b c) { sleep 5 } | :  
> zsh: job can't be suspended

However if the command outputs something, one loses the output.

> > Useful, but I'm not sure an option is the right way of doing this... I
> > should think you'd want to do something like:
> > 
> > {
> >   stty susp undef
> >   ...
> > } always {
> >   stty susp '^z'
> > }
> 
> I'm not sure that's the right semantics.  Does one want to disable the
> tty suspend character or just ingore the TSTP signal?  What if "emacs"
> in the original example was replaced by running another shell?

The ^Z should affect only the other shell. So, I think this is OK.
But with the encapsulation (to save typing), this is very limited.
For instance, one cannot use a complex command like:

  nosuspend for i in ...

or even use a function after "nosuspend".

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <http://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: suspend (^Z) behavior while a function is running is unclear

[Bart Schaefer]

On Feb 20,  7:33pm, Vincent Lefevre wrote:
} Subject: Re: suspend (^Z) behavior while a function is running is unclear
}
} > torch% foreach foo (a b c) { sleep 5 } | :  
} > zsh: job can't be suspended
} 
} However if the command outputs something, one loses the output.

I didn't really mean to imply that you should use this, just that some
of the plumbing is in place if we wanted to provide direct control.

On the other hand:

foreach foo (a b c) { print $foo; sleep 5 } | while { read -rE } { : }

(Using shortloops syntax to emphasize zsh-ness of this trick.)
 
} But with the encapsulation (to save typing), this is very limited.

As a hack, one could do:

    alias -g NOSUSP='|while { read -rE } { : }'

    foreach foo (a b c) { print $foo; sleep 5 } NOSUSP

This is similar to the "alias -g M='|more'" trick that I know a number
of people have used at one time or another.

Re: suspend (^Z) behavior while a function is running is unclear

[Vincent Lefevre]

On 2012-02-20 10:53:01 -0800, Bart Schaefer wrote:
> As a hack, one could do:
> 
>     alias -g NOSUSP='|while { read -rE } { : }'
> 
>     foreach foo (a b c) { print $foo; sleep 5 } NOSUSP

OK, I could do:

  alias Z='while { read -rE } { : }'

and use '|Z', which is shorter and should be safe.

But there are several problems:

1. The left-hand side is run in a subshell, so that side effects
   (e.g. setting a variable) are not taken into account. Wanting
   to disable TSTP was the main reason to avoid this problem!

2. One still has two processes (I wanted to avoid any additional
   process).

3. Buffering, e.g.: { echo -n a; sleep 2; echo b } |Z
   The 'a' isn't visible immediately.

4. The "while" terminates with a nonzero exit status (I could locally
   unsetopt PRINT_EXIT_VALUE, though).

5. I sometimes get a spurious additional newline.

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <http://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: suspend (^Z) behavior while a function is running is unclear

[Mikael Magnusson]

On 20 February 2012 19:53, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Feb 20,  7:33pm, Vincent Lefevre wrote:
> } Subject: Re: suspend (^Z) behavior while a function is running is unclear
> }
> } > torch% foreach foo (a b c) { sleep 5 } | :
> } > zsh: job can't be suspended
> }
> } However if the command outputs something, one loses the output.
>
> I didn't really mean to imply that you should use this, just that some
> of the plumbing is in place if we wanted to provide direct control.
>
> On the other hand:
>
> foreach foo (a b c) { print $foo; sleep 5 } | while { read -rE } { : }
>
> (Using shortloops syntax to emphasize zsh-ness of this trick.)

(This is actually not shortloops syntax.)

-- 
Mikael Magnusson

Re: suspend (^Z) behavior while a function is running is unclear

[Bart Schaefer]

On Feb 20,  8:48pm, Mikael Magnusson wrote:
} Subject: Re: suspend (^Z) behavior while a function is running is unclear
}
} On 20 February 2012 19:53, Bart Schaefer <schaefer@brasslantern.com> wrote:
} >
} > foreach foo (a b c) { print $foo; sleep 5 } | while { read -rE } { : }
} >
} > (Using shortloops syntax to emphasize zsh-ness of this trick.)
} 
} (This is actually not shortloops syntax.)

Well, OK: "alternative" loop syntax, if you want to be that persnickety.

(This is zsh-workers, though, so I doubt anyone is going to be confused.)

Hm, using braces instead of "end" isn't even documented for "foreach", so
I guess that's neither alternative nor shortloops.