questions

questions

[Chris Hansen]

Hello,

I'm new to the list, I've been playing with zsh off and on for about a
year, love it.

3 questions:

1.  I'm currently using 3.0.5, from the tone of the list it sounds
like most of you would recommend an upgrade to 3.1.6, how stable is
it?  what's different?(readers digest version, promise to RTFM later)

2.  I'd like to use the lovely command line environment in other
programs the same way you use ile or fep, is there a way to do that?

3.  Could someone show me an example using coproc, the otherwise excellent
documentation is a bit weak in that area.

Thanks in advance,
Chris

Re: questions

[Stefan Berglund]

On Sun, Oct 03, 1999 at 01:30:33AM -0700, Chris Hansen wrote:
> 1.  I'm currently using 3.0.5, from the tone of the list it sounds
> like most of you would recommend an upgrade to 3.1.6, how stable is
> it?  what's different?(readers digest version, promise to RTFM later)

I have had no problems with 3.1.6 besides that it sometimes doesn't
react when I start typing. This is cured with a Ctrl C, I'm running the
vi mode. I had problems with 3.0.5 regarding terminal resizing, that
was my main reason to upgrade to 3.1.x.
IMHO the main difference is the new completion system using shell
functions. This is more general and powerful. I have not noticed a
speed penalty as one might suspect.
Which leads to my question: is there a way for a normal user to supply
completion widgets overriding and complementing the globally installed
ones?

-- 
Stefan Berglund                       | sbl@dd.chalmers.se

Re: questions

[Bart Schaefer]

On Oct 3,  1:10pm, Stefan Berglund wrote:
} Subject: Re: questions
}
} I had problems with 3.0.5 regarding terminal resizing, that
} was my main reason to upgrade to 3.1.x.

Amusingly enough, 3.1.x was (before 3.1.6) using the old 3.0.4 terminal
sizing code.  So you upgraded in order to downgrade.

One of the things I did before releasing 3.0.6 was to introduce sanity
into that terminal sizing code and bring 3.0.6 and 3.1.6 into closer
alignment in that regard, so neither version should have serious trouble
at this point -- or if either does, they both will.

} Which leads to my question: is there a way for a normal user to supply
} completion widgets overriding and complementing the globally installed
} ones?

Of course.  This is detailed in the "completion system" section of the
3.1.6 manual ("man zshcompsys" if you aren't using the info pages).  That
has been expanded a bit since the 3.1.6 announcement, so you might want
to pick up the latest release from http://www.ifh.de/~pws/computing/ to
get a more complete picture.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

coproc tutorial (Re: questions)

[Bart Schaefer]

On Oct 3,  1:30am, Chris Hansen wrote:
} Subject: questions
}
} 1.  I'm currently using 3.0.5, from the tone of the list it sounds
} like most of you would recommend an upgrade to 3.1.6, how stable is
} it?

It's reasonably robust (as in, doesn't crash or horribly misinterpret any
commands you give it) for normal use.  There are a lot of minor bugs in
the more obscure bits of the code, though, so the rate of patching is very
high at this time.

There's also recently been a 3.0.6 release, and shortly will be a 3.0.7
to knock off a few bugs found since.  3.0.6 is mostly bugfixes to 3.0.5
plus a couple of things for forwards-compatibility with 3.1.6.

} what's different?(readers digest version, promise to RTFM later)

The best way to get this is to have a look at the Etc/NEWS file from the
distribution.  I was hoping it was available somewhere on www.zsh.org,
but it doesn't appear to be.

} 2.  I'd like to use the lovely command line environment in other
} programs the same way you use ile or fep, is there a way to do that?

That's been asked many times, but the line editor behavior is so tightly
tied to the rest of the shell that although you can have the shell with
no line editor, it's been more than anyone wants to tackle to try to have
the line editor without the shell.  That's even more true now that most
of the completion system is written as shell functions.
 
} 3.  Could someone show me an example using coproc, the otherwise excellent
} documentation is a bit weak in that area.

You start a coproc with

	coproc command

The command has to be prepared to read from stdin and/or write to stdout,
or it isn't of much use as a coproc.  Generally speaking, the command also
should not be one that uses buffered writes on its output, or you may end
up waiting for output that never appears.

After it's running, you have several choices:

Write to the coproc with "print -p ..."
Read from the coproc with "read -p ..."
Redirect output to the coproc with "othercommand >&p"
Redirect input from the coproc with "othercommand <&p"

So here's a very simple example of a coproc that converts all the text
you send it into upper case ("zsh% " is the shell prompt):

    zsh% coproc while read line; do print -r -- "$line:u"; done

Note that you can put an entire control structure into a coproc; it works
just like putting "&" at the end.  The coproc shows up in the job table
as a background job; you can bring it into the foreground, kill it, etc.
In fact, it's a no-op to put an "&" at the end of a "coproc ...", because
zsh is going to background the job already.

With that coproc running, I can say

    zsh% print -p foo ; read -ep
    FOO

(Using "read -e" means to immediately echo what was just read.)

About that output buffering thing:  You might wonder why I didn't use:

    zsh% coproc tr a-z A-Z
    zsh% print -p foo ; read -ep

It's because of the output buffering done by "tr".  The "print -p foo" is
happily consumed by "tr", but it doesn't produce any output until it has
either processed a whole buffer-full of bytes (usually 1024) or until it
has seen end-of-file on its input and is about to exit.  So "read -ep"
sits there forever, waiting for "tr", which is also sitting there forever
waiting for someone to send it some more bytes.

Which brings us to an oddity about zsh's coproc:  It sees end-of-file on
its input only when a new coproc is started.  In other shells, using the
equivalent of the "othercommand >&p" redirection causes the shell to
discard its own copy of the coproc descriptor, so the coprocess gets an
EOF when "othercommand" closes its output (exits).  Zsh, however, keeps
the coproc descriptor open so that you can repeatedly direct new output
to the same coprocess.  But there can only be one magic "p" descriptor,
so when you issue a new "coproc ..." command, zsh finally does close its
copy of the descriptor.  (Some "othercommand" may still have it open.)

One idiom for closing off a coproc's input and output is to use:

    zsh% coproc exit

That starts a new coproc (which immediately exits), causing the input and
output of the old coproc (if any) to be shut down.  Some coprocesses --
the "while" loop I used as an example is one of them -- don't notice when
their input and output are closed, and won't stop when you do this, so
you may still need to explicitly kill them off.  This is a VERY important
detail if you are in the habit of using "setopt no_hup".

Now a word about input buffering:  In my example, I sent a line to the
coprocess with "print -p foo ;" leaving the "print" in the foreground.
That's because I know for a fact that the coprocess will consume one line
of input (the "while read line" loop) before producing any output at all,
so I'm sure that "print" will finish successfully.  Some other coprocess
might read only a few bytes before stopping to do some other work, in
which case my "print" would block and "read -ep" might never run.  It's
more usual, therefore, to send input to the coprocess from a background
job:

    zsh% cat /etc/termcap >&p &

(I picked /etc/termcap because it's usually a huge file, so that command
will almost certainly block if not backgrounded.)

A final oddity (and maybe this should even be considered a bug):  You may
think from reading the above that you can build up your own pipelines by
chaining "coproc" together like this:

    coproc tail
    coproc head >&p

That appears to say "start `tail' as a coproc, and then start `head' as a
new coproc with its output connected to the input of the old coproc."
However, that doesn't work; zsh recreates the coproc descriptors before
processing the redirection, so what "coproc head >&p" actually does is run
"head" with its output connected back to its own input.  This is a good
way to create either deadlock or an extremely CPU-intensive loop, so I
don't recommend doing it.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Re: coproc tutorial (Re: questions)

[Chris Hansen]

I now understan how to coproc, when is it actually the best way?

In the example tr would obviously work better.

What have you used it for?

Chris

On Sun, 3 Oct 1999, Bart Schaefer wrote:

> On Oct 3,  1:30am, Chris Hansen wrote:
> } Subject: questions
> }
> } 1.  I'm currently using 3.0.5, from the tone of the list it sounds
> } like most of you would recommend an upgrade to 3.1.6, how stable is
> } it?
> 
> It's reasonably robust (as in, doesn't crash or horribly misinterpret any
> commands you give it) for normal use.  There are a lot of minor bugs in
> the more obscure bits of the code, though, so the rate of patching is very
> high at this time.
> 
> There's also recently been a 3.0.6 release, and shortly will be a 3.0.7
> to knock off a few bugs found since.  3.0.6 is mostly bugfixes to 3.0.5
> plus a couple of things for forwards-compatibility with 3.1.6.
> 
> } what's different?(readers digest version, promise to RTFM later)
> 
> The best way to get this is to have a look at the Etc/NEWS file from the
> distribution.  I was hoping it was available somewhere on www.zsh.org,
> but it doesn't appear to be.
> 
> } 2.  I'd like to use the lovely command line environment in other
> } programs the same way you use ile or fep, is there a way to do that?
> 
> That's been asked many times, but the line editor behavior is so tightly
> tied to the rest of the shell that although you can have the shell with
> no line editor, it's been more than anyone wants to tackle to try to have
> the line editor without the shell.  That's even more true now that most
> of the completion system is written as shell functions.
>  
> } 3.  Could someone show me an example using coproc, the otherwise excellent
> } documentation is a bit weak in that area.
> 
> You start a coproc with
> 
> 	coproc command
> 
> The command has to be prepared to read from stdin and/or write to stdout,
> or it isn't of much use as a coproc.  Generally speaking, the command also
> should not be one that uses buffered writes on its output, or you may end
> up waiting for output that never appears.
> 
> After it's running, you have several choices:
> 
> Write to the coproc with "print -p ..."
> Read from the coproc with "read -p ..."
> Redirect output to the coproc with "othercommand >&p"
> Redirect input from the coproc with "othercommand <&p"
> 
> So here's a very simple example of a coproc that converts all the text
> you send it into upper case ("zsh% " is the shell prompt):
> 
>     zsh% coproc while read line; do print -r -- "$line:u"; done
> 
> Note that you can put an entire control structure into a coproc; it works
> just like putting "&" at the end.  The coproc shows up in the job table
> as a background job; you can bring it into the foreground, kill it, etc.
> In fact, it's a no-op to put an "&" at the end of a "coproc ...", because
> zsh is going to background the job already.
> 
> With that coproc running, I can say
> 
>     zsh% print -p foo ; read -ep
>     FOO
> 
> (Using "read -e" means to immediately echo what was just read.)
> 
> About that output buffering thing:  You might wonder why I didn't use:
> 
>     zsh% coproc tr a-z A-Z
>     zsh% print -p foo ; read -ep
> 
> It's because of the output buffering done by "tr".  The "print -p foo" is
> happily consumed by "tr", but it doesn't produce any output until it has
> either processed a whole buffer-full of bytes (usually 1024) or until it
> has seen end-of-file on its input and is about to exit.  So "read -ep"
> sits there forever, waiting for "tr", which is also sitting there forever
> waiting for someone to send it some more bytes.
> 
> Which brings us to an oddity about zsh's coproc:  It sees end-of-file on
> its input only when a new coproc is started.  In other shells, using the
> equivalent of the "othercommand >&p" redirection causes the shell to
> discard its own copy of the coproc descriptor, so the coprocess gets an
> EOF when "othercommand" closes its output (exits).  Zsh, however, keeps
> the coproc descriptor open so that you can repeatedly direct new output
> to the same coprocess.  But there can only be one magic "p" descriptor,
> so when you issue a new "coproc ..." command, zsh finally does close its
> copy of the descriptor.  (Some "othercommand" may still have it open.)
> 
> One idiom for closing off a coproc's input and output is to use:
> 
>     zsh% coproc exit
> 
> That starts a new coproc (which immediately exits), causing the input and
> output of the old coproc (if any) to be shut down.  Some coprocesses --
> the "while" loop I used as an example is one of them -- don't notice when
> their input and output are closed, and won't stop when you do this, so
> you may still need to explicitly kill them off.  This is a VERY important
> detail if you are in the habit of using "setopt no_hup".
> 
> Now a word about input buffering:  In my example, I sent a line to the
> coprocess with "print -p foo ;" leaving the "print" in the foreground.
> That's because I know for a fact that the coprocess will consume one line
> of input (the "while read line" loop) before producing any output at all,
> so I'm sure that "print" will finish successfully.  Some other coprocess
> might read only a few bytes before stopping to do some other work, in
> which case my "print" would block and "read -ep" might never run.  It's
> more usual, therefore, to send input to the coprocess from a background
> job:
> 
>     zsh% cat /etc/termcap >&p &
> 
> (I picked /etc/termcap because it's usually a huge file, so that command
> will almost certainly block if not backgrounded.)
> 
> A final oddity (and maybe this should even be considered a bug):  You may
> think from reading the above that you can build up your own pipelines by
> chaining "coproc" together like this:
> 
>     coproc tail
>     coproc head >&p
> 
> That appears to say "start `tail' as a coproc, and then start `head' as a
> new coproc with its output connected to the input of the old coproc."
> However, that doesn't work; zsh recreates the coproc descriptors before
> processing the redirection, so what "coproc head >&p" actually does is run
> "head" with its output connected back to its own input.  This is a good
> way to create either deadlock or an extremely CPU-intensive loop, so I
> don't recommend doing it.
> 
> -- 
> Bart Schaefer                                 Brass Lantern Enterprises
> http://www.well.com/user/barts              http://www.brasslantern.com
> 

coproc

[Chris Hansen]

Is there a way to write to the keyboard of a coprocess?

If not, how hard would this feature be to add?

Chris

Re: coproc tutorial (Re: questions)

[Bart Schaefer]

On Oct 3,  2:08pm, Chris Hansen wrote:
} Subject: Re: coproc tutorial (Re: questions)
}
} I now understan how to coproc, when is it actually the best way?
} What have you used it for?

To be perfectly honest, I've never used a coproc for anything except
figuring out how to explain what coprocs do.

I suspect they're rather more useful in other ksh-like shells because of
the handling of piping into shell constructs.  In zsh, a "while ...; do"
loop at the right-hand-end of a pipeline is run in the current shell 
(unless backgrounded), so you can set and export parameters and so on 
and they remain set when the loop finishes.  Zsh is unique in this; in
other shells, such a loop would be run in a subshell.

But with a coproc, you can start the left side of the "pipeline" in the
coprocess and then use redirection for the input of the loop, and thus
you get the loop run in the current shell.

Coprocs are also useful if you ever have two commands that each want to
consume the other's output:

    coproc command1
    command2 <&p >&p

I've never had reason to do that, but I suppose it would be good if you
were testing a pair of network protocol daemons.

The only other reason I can think of for using a coprocess would be to
avoid using an "eval".  Something like:

    if condition1; then
        coproc command1
    elif condition2; then
        coproc command2
    else
        coproc command3
    fi
    # ... do a bunch of other setup for command4, then ...
    command4 <&p

On Oct 3,  7:01pm, Chris Hansen wrote:
} Subject: coproc
}
} Is there a way to write to the keyboard of a coprocess?
} If not, how hard would this feature be to add?

I can't answer that, as I'm completely mystified as to what "the keyboard
of a coprocess" means.  A coprocess is a background job; to the extent
that it has "a keyboard" it would be the same one as the shell that had
started it.  Can you give an example of what you're trying to accomplish?

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Re: coproc tutorial (Re: questions)

[Chris Hansen]

On Mon, 4 Oct 1999, Bart Schaefer wrote:

> On Oct 3,  2:08pm, Chris Hansen wrote:
> } Subject: Re: coproc tutorial (Re: questions)
> }
> } I now understan how to coproc, when is it actually the best way?

<<SNIP>>

> }
> } Is there a way to write to the keyboard of a coprocess?
> } If not, how hard would this feature be to add?
> 
> I can't answer that, as I'm completely mystified as to what "the keyboard
> of a coprocess" means.  A coprocess is a background job; to the extent
> that it has "a keyboard" it would be the same one as the shell that had
> started it.  Can you give an example of what you're trying to accomplish?
> 

Every tty/pty has a keyboard.  I don't know if that is true of
coprocesses/background processes.  What I was trying to do was write a
script that would allow me to use all of zsh's command line magic inside
of other programs.  I had some nice results with bc, because it will take
things from standard in.  I didn't do near as well with nslookup because
it insists on taking commands from the keyboard.

I can echo, print and pipe to stdin on the coproc, I can pipe to the
screen of the next terminal, but I'm beginning to suspect it can't be
done. (which is what you told me in the first place)

The code looked something like this:

#!/bin/zsh -i

coproc /usr/bin/nslookup

while true; do
	TEST=">"
	NSLOUT=" "
	while [[ $TEST != $NSLOUT ]] ; do
		print -n $NSLOUT
		read -pk NSLOUT
	done
	read 'QUERY?>'
	print -p $QUERY
done


Thanks
Chris

Re: coproc tutorial (Re: questions)

[Bart Schaefer]

On Oct 3, 10:21pm, Chris Hansen wrote:
} Subject: Re: coproc tutorial (Re: questions)
}
} What I was trying to do was write a script that would allow me to use
} all of zsh's command line magic inside of other programs. I had some
} nice results with bc, because it will take things from standard in.
} I didn't do near as well with nslookup because it insists on taking
} commands from the keyboard.

Aha.  Well, when I tried your script here, the problem with it was not
nslookup reading from the keyboard; it was good old output buffering,
just as I originally warned about ...  nslookup gets the first query
that your script sends it and computes the answer; but because standard
*output* is not a terminal, it doesn't flush its buffer, and "read -pk"
hangs the next time around your loop.

This is typical stdio-library behavior:  Output is block-buffered in
1024-byte (or 4096-byte or 8192-byte) chunks when the output is to a
pipe or file, and line-buffered or unbuffered when to a terminal.  By
putting nslookup in the coprocess, you put it into block-buffered mode.

You can avoid this by getting yourself a little program that opens a
pty and runs nslookup there; "expect" can do it, and there are several
others whose names I've forgotten.

But in any case you need to do a lot more than just call "read" to get
zsh's editor involved in all of this.  See

    http://www.zsh.org/cgi-bin/mla/redirect?USERNUMBER=1670

and good luck ...

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com