wait()ing for a job

wait()ing for a job

[meino.cramer]

Hi,

I want to write a zsh-script to start several rsync-jobs in background
and 'wait' for each of them to end. I need the return code of each
of them.

>From the zsh manpage about the wait builtin:

      wait [ job ... ]
              Wait for the specified jobs or processes.  If job is not given then  all  currently
              active  child processes are waited for.  Each job can be either a job specification
              or the process ID of a job in the job table.  The exit status from this command  is
              that of the job waited for.

The last sentence confuses me:
'wait' seems to be able to wait not only for one but for several jobs.
The last sentence says that the return code of 'wait' is that of
*that* (not "those") job waited for.

I need all return codes -- if one rsync fails, I got a problem.

What is it, what I don't correctly understand from the manpage -- what
does wait according to the return codes of several jobs?

Best regards,
mcc

Re: wait()ing for a job

[Bart Schaefer]

On Jul 5,  7:07am, meino.cramer@gmx.de wrote:
} 
} I want to write a zsh-script to start several rsync-jobs in background
} and 'wait' for each of them to end. I need the return code of each
} of them.

[...]

} I need all return codes -- if one rsync fails, I got a problem.
} 
} What is it, what I don't correctly understand from the manpage -- what
} does wait according to the return codes of several jobs?

Unfortunately you can't accomplish what you need with only "&" and "wait".

The wait builtin always returns the status for exactly one job, either
the first one among those you specified as the arguments, or the last one
out of all possible jobs.  If several jobs exit "too close together" then
the return status from some of them may be lost, because the shell will
handle them asynchronously while "wait" is not active.  See e.g. comments
in Functions/Misc/zargs relating to the --max-procs option.

One way to achieve what you want is to wrap each rsync job in a subshell
which finishes with "print $?".  This will cause the subshell to block
until something reads its output, so that you can be sure of associating
the correct exit status with each rsync.  If you combine this with the
"... {var}>&1" syntax, you can dynamically assign descriptor numbers to
each subshell and either monitor them with "zselect" or poll them with
"read -t 0 <&$var".  I don't have time right now to write up an example
of this but I may be able to do so later if you can't work it out.

Re: wait()ing for a job

[Bart Schaefer]

On Jul 5, 10:32am, Bart Schaefer wrote:
}
} One way to achieve what you want is to wrap each rsync job in a subshell
} which finishes with "print $?".

Here's a basic formula:

  concurrent_rsync() {
    local fd rsync_args rsync_status
    local -A rsync_jobs

    for rsync_args in "$@"
    do
      exec {fd}<<(rsync $=rsync_args ; print $?)
      rsync_jobs[$fd]=$rsync_args
      unset fd
    done

    while (( $#rsync_jobs ))
    do
      for fd in ${(k)rsync_jobs}
      do
	if read -t 0 rsync_status <&$fd
	then
	  if (( rsync_status ))
	  then print "FAILED ($rsync_status): rsync $rsync_jobs[$fd]"
	  else print "SUCCEEDED: rsync $rsync_jobs[$fd]"
	  fi
	  noglob unset rsync_jobs[$fd]
	fi
      done
    done
  }

Note this will infinite-loop if any of the <<(rsync ...) substitutions
fails to print something to stdout, so additional defensive programming
is desirable.

Run it like:

    concurrent_rsync '-avuz this there:that' '-bz whither:what where'

etc.