long pipelines and coprocesses

long pipelines and coprocesses

[Oliver Kiddle]

In a shell script I'm writing, I've got an enormous pipe of commands and
I've used the following construct as one of the commands in the pipe:

( ( [ "$dogrep" ] && grep "$1" ) || cat - ) | \

Does anyone know of a more efficient way of doing this (i.e. without the
cat -) other than having an outer if statement and repeating the whole
pipeline of commands.

What do I have to do to get coprocesses working in zsh scripts. They
aren't the best documented feature.

I have the following test script:

-----
while read file; do
  rm $file
done |&

:>testfile
ls
print -p testfile
ls
-----

It seems to work fine in ksh but in zsh it just hangs. I've tried it on
both AIX and IRIX.

Thanks

Oliver Kiddle

long pipelines and coprocesses

[Bart Schaefer]

Oliver Kiddle writes:
 > ( ( [ "$dogrep" ] && grep "$1" ) || cat - ) | \
 > 
 > Does anyone know of a more efficient way of doing this (i.e. without the
 > cat -) other than having an outer if statement and repeating the whole
 > pipeline of commands.

Sure.

function tail_of_the_pipeline { blah | blah | blah }
[[ whatever ]] &&
    { grep $1 | tail_of_the_pipeline "$@" } ||
    tail_of_the_pipeline "$@"

Unlike other shells that fork the right-hand-side of pipelines, zsh forks
the left sides, so parameter changes that occur in tail_of_the_pipeline
can affect the current shell.  It's amazingly difficult to write bash
scripts that accomplish the equivalent.

All of the locals declared in the outer function/script are visible as
"globals" inside tail_of_the_pipeline, so as long as you remeber to declare
them (and not to unnecessarily redeclare them inside tail_of_the_pipeline)
you should just be able to move whatever code you've already written into
the new function, otherwise unchanged.

Of course, I'm not sure that's really what you mean, as in this example the
tail of the pipeline will run a second time if the grep fails.  (In your
original example, the "cat -" will run if the grep fails.)  You probably
don't want to use the && and || syntax at all here; rather,

      if [[ whatever ]]
      then grep $1 | tail_of_the_pipeline "$@"
      else tail_of_the_pipeline "$@"
      fi

 > What do I have to do to get coprocesses working in zsh scripts. They
 > aren't the best documented feature.

I think the coprocess documentation has been improved in 3.1.5, but it
probably could still use a lot of work.

 > I have the following test script:
 > 
 > -----
 > while read file; do
 >   rm $file
 > done |&

Zsh doesn't use the |& syntax for creating coprocesses; that syntax is
already taken as a shorthand for 2>&1|.  This is probably the most
significant incompatibility between ksh and zsh-emulating-ksh.  What you
want is

    coproc { while read file; do rm $file; done }

There are also some other different things about zsh's coproc; for example,
you can't use the same file descriptor tricks to close the input or output
of a zsh coprocess that you might use in ksh.  (This may have changed a
little in 3.1.5, I've forgotten.)