coproc tutorial (Re: questions)

["Bart Schaefer" <schaefer@candle.brasslantern.com>]

On Oct 3,  1:30am, Chris Hansen wrote:
} Subject: questions
}
} 1.  I'm currently using 3.0.5, from the tone of the list it sounds
} like most of you would recommend an upgrade to 3.1.6, how stable is
} it?

It's reasonably robust (as in, doesn't crash or horribly misinterpret any
commands you give it) for normal use.  There are a lot of minor bugs in
the more obscure bits of the code, though, so the rate of patching is very
high at this time.

There's also recently been a 3.0.6 release, and shortly will be a 3.0.7
to knock off a few bugs found since.  3.0.6 is mostly bugfixes to 3.0.5
plus a couple of things for forwards-compatibility with 3.1.6.

} what's different?(readers digest version, promise to RTFM later)

The best way to get this is to have a look at the Etc/NEWS file from the
distribution.  I was hoping it was available somewhere on www.zsh.org,
but it doesn't appear to be.

} 2.  I'd like to use the lovely command line environment in other
} programs the same way you use ile or fep, is there a way to do that?

That's been asked many times, but the line editor behavior is so tightly
tied to the rest of the shell that although you can have the shell with
no line editor, it's been more than anyone wants to tackle to try to have
the line editor without the shell.  That's even more true now that most
of the completion system is written as shell functions.
 
} 3.  Could someone show me an example using coproc, the otherwise excellent
} documentation is a bit weak in that area.

You start a coproc with

	coproc command

The command has to be prepared to read from stdin and/or write to stdout,
or it isn't of much use as a coproc.  Generally speaking, the command also
should not be one that uses buffered writes on its output, or you may end
up waiting for output that never appears.

After it's running, you have several choices:

Write to the coproc with "print -p ..."
Read from the coproc with "read -p ..."
Redirect output to the coproc with "othercommand >&p"
Redirect input from the coproc with "othercommand <&p"

So here's a very simple example of a coproc that converts all the text
you send it into upper case ("zsh% " is the shell prompt):

    zsh% coproc while read line; do print -r -- "$line:u"; done

Note that you can put an entire control structure into a coproc; it works
just like putting "&" at the end.  The coproc shows up in the job table
as a background job; you can bring it into the foreground, kill it, etc.
In fact, it's a no-op to put an "&" at the end of a "coproc ...", because
zsh is going to background the job already.

With that coproc running, I can say

    zsh% print -p foo ; read -ep
    FOO

(Using "read -e" means to immediately echo what was just read.)

About that output buffering thing:  You might wonder why I didn't use:

    zsh% coproc tr a-z A-Z
    zsh% print -p foo ; read -ep

It's because of the output buffering done by "tr".  The "print -p foo" is
happily consumed by "tr", but it doesn't produce any output until it has
either processed a whole buffer-full of bytes (usually 1024) or until it
has seen end-of-file on its input and is about to exit.  So "read -ep"
sits there forever, waiting for "tr", which is also sitting there forever
waiting for someone to send it some more bytes.

Which brings us to an oddity about zsh's coproc:  It sees end-of-file on
its input only when a new coproc is started.  In other shells, using the
equivalent of the "othercommand >&p" redirection causes the shell to
discard its own copy of the coproc descriptor, so the coprocess gets an
EOF when "othercommand" closes its output (exits).  Zsh, however, keeps
the coproc descriptor open so that you can repeatedly direct new output
to the same coprocess.  But there can only be one magic "p" descriptor,
so when you issue a new "coproc ..." command, zsh finally does close its
copy of the descriptor.  (Some "othercommand" may still have it open.)

One idiom for closing off a coproc's input and output is to use:

    zsh% coproc exit

That starts a new coproc (which immediately exits), causing the input and
output of the old coproc (if any) to be shut down.  Some coprocesses --
the "while" loop I used as an example is one of them -- don't notice when
their input and output are closed, and won't stop when you do this, so
you may still need to explicitly kill them off.  This is a VERY important
detail if you are in the habit of using "setopt no_hup".

Now a word about input buffering:  In my example, I sent a line to the
coprocess with "print -p foo ;" leaving the "print" in the foreground.
That's because I know for a fact that the coprocess will consume one line
of input (the "while read line" loop) before producing any output at all,
so I'm sure that "print" will finish successfully.  Some other coprocess
might read only a few bytes before stopping to do some other work, in
which case my "print" would block and "read -ep" might never run.  It's
more usual, therefore, to send input to the coprocess from a background
job:

    zsh% cat /etc/termcap >&p &

(I picked /etc/termcap because it's usually a huge file, so that command
will almost certainly block if not backgrounded.)

A final oddity (and maybe this should even be considered a bug):  You may
think from reading the above that you can build up your own pipelines by
chaining "coproc" together like this:

    coproc tail
    coproc head >&p

That appears to say "start `tail' as a coproc, and then start `head' as a
new coproc with its output connected to the input of the old coproc."
However, that doesn't work; zsh recreates the coproc descriptors before
processing the redirection, so what "coproc head >&p" actually does is run
"head" with its output connected back to its own input.  This is a good
way to create either deadlock or an extremely CPU-intensive loop, so I
don't recommend doing it.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com