Re: Parallel processing

[Bart Schaefer <schaefer@brasslantern.com>]

On Sat, Mar 26, 2022 at 11:10 AM Philippe Troin <phil@fifi.org> wrote:
>
> Collecting background jobs' exit status is discussed in the manual,
> under the POSIX_JOBS option:
>
>    In  previous  versions  of the shell, it was necessary to enable
>    POSIX_JOBS in order for the builtin command wait to  return  the
>    status of  background jobs that had already exited.  This is no
>    longer the case.
>
> Setting/unsetting POSIX_JOBS does not make any difference.

% (sleep 1; exit 13) & PID=$!; wait; wait $PID; echo "wait: $?"
[1] 64029
[1]  + exit 13    ( sleep 1; exit 13; )
wait: 13

The "wait" command with no arguments does not return a status for any
job, it just hangs until there are no jobs left, and then returns 0.
This is consistent with e.g. bash.  So when the doc says "The exit
status from this command is that of the job waited for" it should be
more precise about what "the job" means.  You have to identify in the
argument list the job(s) whose status you want.

> There does not seem to be a way to retrieve the exit
> status of a command as soon as SIGCHLD is trapped:

Indeed ... the trap handlers are called when the signal arrives (so
the child process has exited) but before the job has actually been [C
library level] waited-for, so the internal tables that are used to
track the exit status of those children have not been updated.  The
job search code invoked by the "wait" builtin therefore can't find the
job.

> Anyways, zargs is not doing a stellar job currently with collecting
> exit statuses from commands ran in parallel:

# Everything has to be in a subshell just in case of backgrounding jobs,
# so that we don't unintentionally "wait" for jobs of the parent shell.

Hmm ... zargs uses
  wait ${${jobstates[(R)running:*]/#*:/}/%=*/}
to wait for all the backgrounded jobs that it started.  (This causes a
segfault in the most recent git checkout if zargs itself is a subshell
job.)  However, that "wait" returns the exit status of only one of
those jobs.  There might be something more that could be done now, to
pick up the status of the rest ... but I'm reluctant to mess with that
while the segfault is unfixed.

>    % zargs -n 4 -P 2 -- 0 1 -- zsh -c 'sleep $1 ; exit $1 ' -; echo $?
>    123

This is explained in the comments in zargs:

# Like xargs, zargs exits with the following status:
#   0 if it succeeds
#   123 if any invocation of the command exited with status 1-125
#   124 if the command exited with status 255
#   125 if the command is killed by a signal
#   126 if the command cannot be run
#   127 if the command is not found
#   1 if some other error occurred.