should $! give the pid of subshell?

should $! give the pid of subshell?

[Greg Klanderman]

Hi,

If I do

% ( cd directory && command ) &
% echo $!

should I get the pid of 'command' or of the subshell?

In bash, I get the pid of a subshell, but in zsh, there does
not even seem to be a subshell, and I get the pid of 'command'.

thanks,
Greg

Re: should $! give the pid of subshell?

[Peter Stephenson]

Greg Klanderman wrote:
> If I do
> 
> % ( cd directory && command ) &
> % echo $!
> 
> should I get the pid of 'command' or of the subshell?

The latter, but actually they're the same here.  zsh "exec"s the final
command since it notices the subshell isn't needed any more so there's
no point in forking.  All you're guaranteed is that the stuff in ( ) is
executed as part of a subshell, you're not guaranteed the shell forks
again to run "command".

-- 
Peter Stephenson <pws@csr.com>            Software Engineer
Tel: +44 (0)1223 692070                   Cambridge Silicon Radio Limited
Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, UK


Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom

unable to wait on completed job [was: should $! give the pid of subshell?]

[Greg Klanderman]

>>>>> On November 25, 2009 Peter Stephenson <pws@csr.com> wrote:

> zsh "exec"s the final
> command since it notices the subshell isn't needed any more so there's
> no point in forking.

Oh cool, I didn't realize zsh did tail recursion..

OK, so let me change the question a bit.  If I run this script:

  #!/bin/zsh -f
  sleep 10 &
  pid=$!
  sleep 12
  wait $pid

I get the error:

| zshbgtest.zsh:wait:12: pid 19673 is not a child of this shell

Using bash or ksh, I can successfully wait on the completed job.

Shouldn't I be able to wait on a job that has completed in zsh?

This is zsh 4.3.10-dev-1, patchlevel 1.4816 if it matters, though
4.2.0 and 4.2.6 behave the same.

thanks,
Greg

Re: unable to wait on completed job [was: should $! give the pid of subshell?]

[Bart Schaefer]

On Nov 29, 11:04pm, Greg Klanderman wrote:
} Subject: unable to wait on completed job [was: should $! give the pid of s
} 
} Shouldn't I be able to wait on a job that has completed in zsh?

I'd be inclined to respond "only if you set the POSIX_JOBS option," but
that doesn't appear to suffice either (and in fact has a bug).

In any case this is one of those places where zsh does not claim full
compatibility with POSIX.

} Using bash or ksh, I can successfully wait on the completed job.

Hmm.  [The extra "exit 19" below is so I could check whether bash
correctly reports the status of the waited-for PID.]

[schaefer@torch]$ (sleep 10; exit 19) &
[1] 1680
[schaefer@torch]$ pid=$!
[schaefer@torch]$ sleep 12; wait $pid
[1]+  Exit 19                 ( sleep 10; exit 19 )
bash: wait: pid 1680 is not a child of this shell
[schaefer@torch]$ echo $BASH_VERSION
3.00.15(1)-release

However, "wait" works as Greg describes in bash version 3.2.25(1)-release.
(More on this below [*].)

}   #!/bin/zsh -f
}   sleep 10 &
}   pid=$!
}   sleep 12
}   wait $pid

(The bug I mentioned:  If I run those commands in a subshell of an
interactive shell with POSIX_JOBS set, the subshell gets hit with a
SIGTTOU signal and the parent shell ends up hung waiting for it.)

Apparently POSIX.2 requires "wait" be able to access the exit status
of an asynchronous job until the next asynchronous job is started.
Zsh is only able to access that status until the next job (whether
synchronous or not) completes; the completion of "sleep 12" clears
the previously-completed "sleep 10 &" from the job table, so "wait"
is no longer able to see it.

"Fixing" this may be a fairly convoluted change to the already somewhat
baroque job table management code.

[*] The CHANGES file with bash-3.2 appears to claim this was changed a
lot longer ago than bash-3.0; some possibly-relevant entries:

----------
This document details the changes between this version, bash-3.1-rc1,
and the previous version, bash-3.1-beta1.

...

e.  In Posix mode, after `wait' is called to wait for a particular process
    explicitly, that process is removed from the list of processes known to
    the shell, and subsequent attempts to wait for it return errors.

----------
This document details the changes between this version, bash-2.03-beta1,
and the previous version, bash-2.03-alpha.

...

j.  Some changes were made to the job reaping code when the shell is not
    interactive, so the shell will retain exit statuses longer for examination
    by `wait'.

----------
This document details the changes between this version, bash-2.02-alpha1,
and the previous version, bash-2.01.1-release.

...

q.  A bug was fixed so that the job containing the last asynchronous
    process is not removed from the job table until a `wait' is executed
    for that process or another asynchronous process is started.  This
    satisfies a POSIX.2 requirement.

----------

Re: unable to wait on completed job

[Greg Klanderman]

Thank you Bart for the detailed investigation.  I can work around the
problem for now.  I remembered you mentioning zargs the other day, and
went to look how it handles parallel execution but it's not too smart
about managing jobs.

> In any case this is one of those places where zsh does not claim full
> compatibility with POSIX.

> "Fixing" this may be a fairly convoluted change to the already somewhat
> baroque job table management code.

So is this something you think needs to get fixed, would be nice to
fix, or not worth it?  Seems like you really need this if you want to
be able to manage parallel job execution.

Would also be nice if there were a way to test job status without
actually waiting.  I'm currently using 'kill -0'..  but maybe I can
use the jobstates parameter from the zsh/parameter module, will look
into that.

thanks,
Greg

Re: unable to wait on completed job

[Bart Schaefer]

On Nov 30, 10:36am, Greg Klanderman wrote:
}
} > "Fixing" this may be a fairly convoluted change to the already somewhat
} > baroque job table management code.
} 
} So is this something you think needs to get fixed, would be nice to
} fix, or not worth it?

Some input from PWS would help, as he's more familiar with the job
table code.  I'd classify it as "would be nice" because we've gotten
along for almost 20 years without anyone even noticing.

Note that AFAICT, even with the operation as supported by bash, you are
only guaranteed access to the exit status of the most recently started
asynchronous job; so it's not much help in handling multiple parallel
tasks.  If you don't care about the exit status, you can do

    ! kill -0 $pid || wait $pid

to get the necessary effect.

Re: unable to wait on completed job

[Greg Klanderman]

>>>>> On November 30, 2009 Bart Schaefer <schaefer@brasslantern.com> wrote:

> Note that AFAICT, even with the operation as supported by bash, you are
> only guaranteed access to the exit status of the most recently started
> asynchronous job; so it's not much help in handling multiple parallel
> tasks.

Yeah that's not all that useful.. conceptually at least it seems you
could keep the job table entry around with the exit status for
completed jobs until the job/pid is waited for or until 'jobs' is run
maybe?

> If you don't care about the exit status

"Don't care" is a little strong; more like "can deal"..

thanks,
Greg

Re: unable to wait on completed job

[Greg Klanderman]

>>>>> On November 30, 2009 Bart Schaefer <schaefer@brasslantern.com> wrote:

> Note that AFAICT, even with the operation as supported by bash, you are
> only guaranteed access to the exit status of the most recently started
> asynchronous job;

FWIW, my testing seems to indicate that bash maintains the status of
completed jobs until wait'ed on.  I've tested two jobs, and regardless
of the order in which they complete relative to the order they were
started, and also the order wait'ed on, wait seems to give the right
exit status.  Running 'jobs' does not seem to clear the status of
completed jobs.

Greg