Re: unable to wait on completed job [was: should $! give the pid of subshell?]

Re: unable to wait on completed job [was: should $! give the pid of subshell?]

[Peter Stephenson]

On Sun, 29 Nov 2009 21:14:36 -0800
Bart Schaefer <schaefer@brasslantern.com> wrote:
> [schaefer@torch]$ (sleep 10; exit 19) &
> [1] 1680
> [schaefer@torch]$ pid=$!
> [schaefer@torch]$ sleep 12; wait $pid
> [1]+  Exit 19                 ( sleep 10; exit 19 )
> bash: wait: pid 1680 is not a child of this shell
>>..
> Apparently POSIX.2 requires "wait" be able to access the exit status
> of an asynchronous job until the next asynchronous job is started.
> Zsh is only able to access that status until the next job (whether
> synchronous or not) completes; the completion of "sleep 12" clears
> the previously-completed "sleep 10 &" from the job table, so "wait"
> is no longer able to see it.

(Moved to zsh-workers.)

If it's only an issue for $! (lastpid internally), I think it's fairly
straightforward to fix.  I tried to make it work even if a new process with
the same PID came along later.

No idea about anything to do with SIGTTOU, however, that's outside my
range.

Index: Src/exec.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/exec.c,v
retrieving revision 1.172
diff -u -r1.172 exec.c
--- Src/exec.c	22 Sep 2009 09:17:10 -0000	1.172
+++ Src/exec.c	30 Nov 2009 18:34:17 -0000
@@ -2727,6 +2727,8 @@
 #endif
 	    if (how & Z_ASYNC) {
 		lastpid = (zlong) pid;
+		/* indicate it's possible to set status for lastpid */
+		lastpid_status = -2L;
 	    } else if (!jobtab[thisjob].stty_in_env && varspc) {
 		/* search for STTY=... */
 		Wordcode p = varspc;
Index: Src/init.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/init.c,v
retrieving revision 1.106
diff -u -r1.106 init.c
--- Src/init.c	12 Jul 2009 15:10:07 -0000	1.106
+++ Src/init.c	30 Nov 2009 18:34:18 -0000
@@ -913,6 +913,7 @@
     bufstack = znewlinklist();
     hsubl = hsubr = NULL;
     lastpid = 0;
+    lastpid_status = -1L;
     bshin = SHIN ? fdopen(SHIN, "r") : stdin;
     if (isset(SHINSTDIN) && !SHIN && unset(INTERACTIVE)) {
 #ifdef _IONBF
Index: Src/jobs.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/jobs.c,v
retrieving revision 1.73
diff -u -r1.73 jobs.c
--- Src/jobs.c	30 Sep 2009 20:32:49 -0000	1.73
+++ Src/jobs.c	30 Nov 2009 18:34:18 -0000
@@ -104,6 +104,15 @@
 /**/
 int numpipestats, pipestats[MAX_PIPESTATS];
 
+/*
+ * The status associated with the process lastpid.
+ * -1 if not set and no associated lastpid
+ * -2 if lastpid is set and status isn't yet
+ * else the value returned by wait().
+ */
+/**/
+long lastpid_status;
+
 /* Diff two timevals for elapsed-time computations */
 
 /**/
@@ -1109,6 +1118,14 @@
 {
     Process pn, *pnlist;
 
+    if (pid == lastpid && lastpid_status != -2L) {
+	/*
+	 * The status for the previous lastpid is invalid.
+	 * Presumably process numbers have wrapped.
+	 */
+	lastpid_status = -1L;
+    }
+
     DPUTS(thisjob == -1, "No valid job in addproc.");
     pn = (Process) zshcalloc(sizeof *pn);
     pn->pid = pid;
@@ -1845,6 +1862,8 @@
 		retval = waitforpid(pid, 1);
 		if (!retval)
 		    retval = lastval2;
+	    } else if (pid == lastpid && lastpid_status >= 0L) {
+		retval = (int)lastpid_status;
 	    } else {
 		zwarnnam(name, "pid %d is not a child of this shell", pid);
 		/* presumably lastval2 doesn't tell us a heck of a lot? */
Index: Src/signals.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/signals.c,v
retrieving revision 1.56
diff -u -r1.56 signals.c
--- Src/signals.c	20 Jul 2009 04:38:56 -0000	1.56
+++ Src/signals.c	30 Nov 2009 18:34:18 -0000
@@ -530,6 +530,7 @@
 	     * Find the process and job containing this pid and
 	     * update it.
 	     */
+	    pn = NULL;
 	    if (findproc(pid, &jn, &pn, 0)) {
 #if defined(HAVE_WAIT3) && defined(HAVE_GETRUSAGE)
 		struct timezone dummy_tz;
@@ -551,6 +552,15 @@
 		 */
 		get_usage();
 	    }
+	    /*
+	     * Remember the status associated with $!, so we can
+	     * wait for it even if it's exited.  This value is
+	     * only used if we can't find the PID in the job table,
+	     * so it doesn't matter that the value we save here isn't
+	     * useful until the process has exited.
+	     */
+	    if (pn != NULL && pid == lastpid && lastpid_status != -1L)
+		lastpid_status = lastval2;
 	}
         break;
  


-- 
Peter Stephenson <pws@csr.com>            Software Engineer
Tel: +44 (0)1223 692070                   Cambridge Silicon Radio Limited
Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, UK


Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom

Re: unable to wait on completed job [was: should $! give the pid of subshell?]

[Bart Schaefer]

On Nov 30,  6:37pm, Peter Stephenson wrote:
}
} If it's only an issue for $! (lastpid internally), I think it's fairly
} straightforward to fix. I tried to make it work even if a new process
} with the same PID came along later.

This looks reasonable, but I wonder if we should make it conditional
on POSIX_JOBS ?

} No idea about anything to do with SIGTTOU, however, that's outside my
} range.

SIGTTOU is only a side-effect.  The doc for POSIX_JOBS says:

    ... When the option is set, the MONITOR option and job control
    remain active in the subshell ... 

What's happening is that the subshell is not the tty process group
leader, so when MONITOR tries to print the job number and PID (and
later the exit status), the TTY driver stops it because it doesn't
appear to be the foreground job.

What's odder is that "stty -tostop" doesn't prevent this.  Something is
effectively reasserting that setting before the subshell gets going.

Re: unable to wait on completed job [was: should $! give the pid of subshell?]

[Peter Stephenson]

Bart Schaefer wrote:
> On Nov 30,  6:37pm, Peter Stephenson wrote:
> }
> } If it's only an issue for $! (lastpid internally), I think it's fairly
> } straightforward to fix. I tried to make it work even if a new process
> } with the same PID came along later.
> 
> This looks reasonable, but I wonder if we should make it conditional
> on POSIX_JOBS ?

I'm not sure what that gains, apart from an extra error message; is
there a case where it's important that exited background jobs can't be
waited for?  I can see one oddity, which is you can wait for it multiple
times without an error (and with the same return status); I could fix
that but I see bash does the same.

-- 
Peter Stephenson <pws@csr.com>            Software Engineer
Tel: +44 (0)1223 692070                   Cambridge Silicon Radio Limited
Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, UK


Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom

Re: unable to wait on completed job [was: should $! give the pid of subshell?]

[Peter Stephenson]

On Tue, 01 Dec 2009 09:52:54 +0000
Peter Stephenson <pws@csr.com> wrote:
> Bart Schaefer wrote:
> > On Nov 30,  6:37pm, Peter Stephenson wrote:
> > }
> > } If it's only an issue for $! (lastpid internally), I think it's fairly
> > } straightforward to fix. I tried to make it work even if a new process
> > } with the same PID came along later.
> > 
> > This looks reasonable, but I wonder if we should make it conditional
> > on POSIX_JOBS ?
> 
> I'm not sure what that gains, apart from an extra error message; is
> there a case where it's important that exited background jobs can't be
> waited for?

I don't know either.  I've committed it checking for POSIXJOBS only in
wait, so you can set that just for the wait command and it will work.  We
probably need some documentation.

There's a race here if process numbers are being used up megafast, but it's
unlikely enough I haven't bothered mentioning it.

Index: Doc/Zsh/options.yo
===================================================================
RCS file: /cvsroot/zsh/zsh/Doc/Zsh/options.yo,v
retrieving revision 1.86
diff -u -r1.86 options.yo
--- Doc/Zsh/options.yo	21 Jul 2009 09:35:20 -0000	1.86
+++ Doc/Zsh/options.yo	2 Dec 2009 09:51:56 -0000
@@ -1324,6 +1324,11 @@
 shell is saved for output within a subshell (for example, within a
 pipeline).  When the option is set, the output of tt(jobs) is empty
 until a job is started within the subshell.
+
+When the option is set, it becomes possible to use the tt(wait) builtin to
+wait for the last job started in the background (as given by tt($!)) even
+if that job has already exited.  This works even if the option is turned
+on temporarily around the use of the tt(wait) builtin.
 )
 enditem()
 

-- 
Peter Stephenson <pws@csr.com>            Software Engineer
Tel: +44 (0)1223 692070                   Cambridge Silicon Radio Limited
Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, UK


Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom