Re: ZSpeak! Only Zsh can do this!

Re: ZSpeak! Only Zsh can do this!

[Sven Wischnowsky]

Mario Lang wrote:

> # * Modify the completion system so that menu-completion inserted
> #   text gets spoken.

Hi!

This could get me immensely interested.  While developing the new
completion system we were discussing ways to get at the strings for
the matches from shell code.  Support for this kind of stuff is
certainly a good reason to add that (after the 4.0 release, though).

And I think the completion system can offer other things that might be 
helpful.  Having it tell you the descriptions it can generate for
example (describing the types of matches generated in a certain
context), having it read the completions and descriptions in menu
selection (`option -o is for ...') and so on.

I've long wanted to learn more about how handicapped people work with
computers.  Are there any interesting places on the net I could look
at for more information? (Google would certainly tell me several
thousand sites but I wouldn't know which are the `good' ones...)

Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

Re: ZSpeak! Only Zsh can do this!

[Mario Lang]

Sven Wischnowsky <wischnow@informatik.hu-berlin.de> writes:

> Mario Lang wrote:
> 
> > # * Modify the completion system so that menu-completion inserted
> > #   text gets spoken.
> 
> This could get me immensely interested.  While developing the new
> completion system we were discussing ways to get at the strings for
> the matches from shell code.  Support for this kind of stuff is
> certainly a good reason to add that (after the 4.0 release, though).
OK. this clarifies the question. It would be nice to have this.
Perhaps for other things to (such as explain-on, which could display/speak
hints while typing).

I suspect that I have to wait with my other features till this
is a bit clarified because completion gets written to stdout,
and when I want to capture stdout, Ill come into a lot of garbage problems I think?

Another really nice to have(tm) feature would be stdout buffering.
What I mean is that the shell should have an array (much like history)
which contains the outputs (stdout and stderr separated)
from the last commands.
This would create alot of possibilities, including the one
of extracting parts of the output in a command.
something like
% ps aux
...
% kill `grep myprog <(print $stdout[1])|awk '{print $2;}'`
would be possible. I know this is an extreme example. But I am
convinced that this feature would find its uses.
Obviously speech support would benefit ALOT.

> 
> And I think the completion system can offer other things that might be 
> helpful.  Having it tell you the descriptions it can generate for
> example (describing the types of matches generated in a certain
> context), having it read the completions and descriptions in menu
> selection (`option -o is for ...') and so on.
You got it. Thats what I want to do.

> I've long wanted to learn more about how handicapped people work with
> computers.  Are there any interesting places on the net I could look
> at for more information?

Hmm, you have to be a bit more specific. The topic is very large.
at least http://www.leb.net/blinux is the center of blind Linux users.


-- 
CYa,
   Mario <mlang@delysid.org>
Homepage(s): http://delysid.org | http://piss.at/

UNIX was half a billion (500000000) seconds old on
Tue Nov  5 00:53:20 1985 GMT (measuring since the time(2) epoch).
		-- Andy Tannenbaum

Re: ZSpeak! Only Zsh can do this!

[Bart Schaefer]

On Mar 21,  6:18pm, Mario Lang wrote:
} Subject: Re: ZSpeak! Only Zsh can do this!
}
} Another really nice to have(tm) feature would be stdout buffering.
} What I mean is that the shell should have an array (much like history)
} which contains the outputs (stdout and stderr separated)
} from the last commands.

This amounts, approximately, to implementing the "less" pager in the
zsh C code.  (The old, original "less" before it grew several million
bells and whistles, when it was mainly just a clever way of buffering
output so you could scroll backwards and forwards through it.)

The difficulty with doing this is that a surprisingly large number of
commands do pay attention to whether their standard output is a pipe
or a file or a terminal.  For the shell to capture the outputs means
that the output of every command is _always_ a pipe.  It simply does
not work (see my remarks on the problems of using multios with `exec').

However ... it might be possible to construct something based on the
zpty module, where stdout and stderr would (unless redirected) each go
to a separate pty, and both ptys would be managed by a front-end that
slurped up the output from each as it appeared.  That such a thing is
possible is evidenced by the `screen' tty windowing program, which
actually might be a better starting point than zsh for such a project.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Re: ZSpeak! Only Zsh can do this!

[Bart Schaefer]

On Mar 22,  6:52am (that's GMT), I wrote:
} Subject: Re: ZSpeak! Only Zsh can do this!
}
} On Mar 21,  6:18pm, Mario Lang wrote:
} } Subject: Re: ZSpeak! Only Zsh can do this!
} }
} } Another really nice to have(tm) feature would be stdout buffering.
} } What I mean is that the shell should have an array (much like history)
} } which contains the outputs (stdout and stderr separated)
} } from the last commands.
} 
} However ... it might be possible to construct something based on the
} zpty module, where stdout and stderr would (unless redirected) each go
} to a separate pty, and both ptys would be managed by a front-end that
} slurped up the output from each as it appeared.

This seemed like such an interesting idea that I tried it.

It should be pretty obvious how to modify this to do something else with
the captured output, e.g., write it to a speech synthesizer instead of
stuffing it into an array parameter ...

This may also need to be tweaked in the event that a command does not
begin producing output within one second; "zpty -r -t" is a rather blunt
instrument with which to be doing precision work.


#!/bin/zsh

# zplitty -- run a command with stdout and stderr directed to separate ttys,
# from which that output can be captured for later reference.
#
# Usage:
#    zplitty command args ...
#
# Output from the command is captured in the arrays $stdout and $stderr,
# one line per array element.  Beware of running commands that produce
# voluminous output; "zplitty yes" will consume all available memory
# unless interrupted. 
#
# zplitty can be used in a pipeline or with redirected I/O, but depending
# on its location in the pipeline it may be run in a subshell, in which
# case the parent shell is not able to make use of the captured output.
#
# zplitty requires the zsh/zpty module and uses the function zttyrun to
# set up two ptys, ztdout and ztderr, to which output can be redirected
# by referencing $ztdout and $ztderr.  Be careful not to redirect too
# much output to a pty unless something is reading it with "zpty -r".
#
# The command zttystop closes these two ptys and discards the captured
# output; zttyrun must be run again before a new zplitty can be started.
# The stdout or stderr arrays can be reset at any time without zttystop
# simply by assigning an empty array as usual.
#
# Note that some interactive commands (examples are zsh, vim, and less)
# insist that their standard input and output be the same device.  Zsh
# attempts to re-open its standard input for this, while vim and less do
# the same with their standard error.  This causes "zplitty zsh" to stop
# (SIGTTOU) and "zplitty vim" or "zplitty less" to hang (reading from
# $ztderr, whence they'll never get any input).  Some other interactive
# programs may (mis)behave in similar ways; use zplitty with caution.

zmodload -i zsh/zpty || return

zttywait () {
    stty onlret -ocrnl -onlcr	# May need to adjust this for your OS
    tty
    read -e
}

zttyget() {
    setopt localoptions extendedglob noshwordsplit
    zpty $1 zttywait
    zpty -r $1 $1
    : ${(P)1::=${(P)1%%[[:space:]]#}}
}

zttyrun() {
    setopt localoptions noshwordsplit
    typeset -g ztdout='' ztderr=''
    typeset -g -a stdout stderr
    zttyget ztdout
    zttyget ztderr
}

zttystop() {
    unset stdout stderr ztdout ztderr
    zpty -w ztdout exit &&
    zpty -rt ztdout &&
    zpty -d ztdout
    zpty -w ztderr exit &&
    zpty -rt ztderr &&
    zpty -d ztderr
}

zplitty() {
    setopt localoptions noshwordsplit nonotify noksharrays
    local out err
    ${@:?} > $ztdout 2> $ztderr &
    sleep 1	# Let the background command get started
    while zpty -r -t ztdout out || zpty -r -t ztderr err
    do
	(( $+out )) && {
	    stdout[$#stdout+1]=$out
	    print -Rn $out
	}
	(( $+err )) && {
	    stderr[$#stderr+1]=$err
	    print -u2 -Rn $err
	}
	unset out err
    done
}

zttyrun
[[ -o kshautoload ]] || zplitty "$@"

# end of zplitty

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

ZSpeak! Only Zsh can do this!

[Mario Lang]

Hi guys.

First of all, I want to introduce myself:
I am a fanatic linux user/developer since 1997
and recently (its a shame that it took so long) I discovered
Zsh and I want to say: You guys did a great job!

As I am a blind Linux user I began to wonder if ZLE (and other things)
could be extended to support Speech output.
This could be nice for various reasons:
1. For blind/visually impaired persons who want to get
a simple speech output from cli without much setup efforts.
2. Perhaps for mobile devices with are limited in space
for a large comprehensive speech output system.

As I said, I am fairly new to Zsh and shell programming in general.
But I couldn't resist to try my best, and here
is how far I got.

But it is far from being finished.
I wanted to post this here for various reasons:
1. Perhaps someone finds a way to do something much
   better than I currently do.
2. Perhaps someone can find some solutions for my TODO
   list.

OK, here we go. Please mail me with every (minor|major) suggestion.
I can use everything. Especially to learn nice Zsh techniques.


#
# Zspeak - Zsh Function to get speech feedback.
#
# by Mario Lang <lang@zid.tu-graz.ac.at>
#
# Currently only ZLE (the line editor) is supported.
# TODO:
# * Find a nice way to add/remove keybindings in speech-(on|off)
# * Find a way to make the prefix key-sequence configurable (^[v)
# * Add self-insert replacement functions to speak typed chars
#   * Add parameter and keybinding to control character echo
#   * Word-echo?? (magic-space??)
# * Add replacement function for up/down to speak new BUFFER content
# * Modify the completion system so that menu-completion inserted
#   text gets spoken.
# * Find a way to read stdout/stderr of commands too! is exec my friend?
#   If this can be done in a nice way, all kinds of non-interactive
#   console programs can be used eye-free with Zsh and a soft/hard-speechsynth.
# * Add speech rate parameter (and keybinding to change it)
# 
# Comments:
#   I am fairly new to Zsh and all its specialities.
#  If you find anything in this code which could be done
#  better, please tell me. I am eager to find the best way
#  to do this. Especially am I interested in ways
#  to get the last TODO item to work (stdout and stderr review).
#
#  This file should be called speech-on and placed
#  in fpath.
#  autoload it, and execute speech-on
#

speech-on () {
    zle -N speech-on
    zle -N speech-off
    zle -N speak-buffer
    zle -N speak-current-character
    zle -N speak-help
    zle -N forward-char speak-new-position
    zle -N backward-char speak-new-position
    #speech-load-spell-list
    bindkey "^[vb" speak-buffer
    bindkey "^[vc" speak-current-char
    bindkey "^[vh" speak-help
    bindkey "^[vo" speech-off
}

speech-off () {
    unset zspeak_spell
    bindkey "^[vo" speech-on
}

# BUG: Doesnt read in the spell list?? Any Zmagician?
speech-load-spell-list () {
    typeset -A zspeak_spell
    [[ -z $ZSPEAK_SPELL_LIST ]] && [[ -s "~/.zsh/spell.list" ]] \
				&& ZSPEAK_SPELL_LIST='~/.zsh/spell.list'
    if [[ -s "$ZSPEAK_SPELL_LIST" ]]; then
        zspeak_spell=(${$(<${ZSPEAK_SPELL_LIST})})
    else
        # Some fallback defaults
        zspeak_spell=(. dot , comma ; colon / slash _ underscore - dash)
    fi
}

speak-string () {
    # Here goes the compatibility code. All synth specific
    # decissions when outputting text should be done here.
    case $ZSPEAK_SYNTH in
	festival)
	    [[ -x `which festival` ]] && print "$@"|festival --tts
	    ;;
        speechd)
	    [[ -e "/dev/speech" ]] && print "$@">/dev/speech
	    ;;
	*)
    esac
}

spell-char () {
    speak-string $zspeak_spell[${1}]
}

# All following functions use the above as a backend
# So all speech synthesizer specific tweaks should go above this
# separator, and all Zsh/ZLE/whatever specific modifications below.

speak-buffer () {
    speak-string $BUFFER
}

speak-current-character () {
    spell-char $BUFFER["$CURSOR"]
}

speak-help () {
    speak-string `bindkey|grep "^\"^\[v"|sed 's/\"\(.*\)\" \(.*\)/\1 invokes \2. /'|sed 's/^^\[\([a-zA-Z0-9]\)/Alt \1 plus /'`
}

speak-new-position () {
    zle .$WIDGET "$@"
    speak-current-character
}

[[ -o kshautoload ]] || speech-on "$@"
 
-- 
CYa,
   Mario <mlang@delysid.org>
Homepage(s): http://delysid.org | http://piss.at/

Theory is gray, but the golden tree of life is green.
		-- Goethe

Re: ZSpeak! Only Zsh can do this!

[Peter Stephenson]

> First of all, I want to introduce myself:
> I am a fanatic linux user/developer since 1997
> and recently (its a shame that it took so long) I discovered
> Zsh and I want to say: You guys did a great job!

Thank you.  Don't look at the code too closely :-/.
> # BUG: Doesnt read in the spell list?? Any Zmagician?
> speech-load-spell-list () {
>     typeset -A zspeak_spell
>     [[ -z $ZSPEAK_SPELL_LIST ]] && [[ -s "~/.zsh/spell.list" ]] \
> 				&& ZSPEAK_SPELL_LIST='~/.zsh/spell.list'

I think you want
    [[ -z $ZSPEAK_SPELL_LIST ]] && [[ -s ~/.zsh/spell.list ]] \
				&& ZSPEAK_SPELL_LIST=~/.zsh/spell.list

since the `~' doesn't get expanded if it's quoted.

>     if [[ -s "$ZSPEAK_SPELL_LIST" ]]; then
>         zspeak_spell=(${$(<${ZSPEAK_SPELL_LIST})})

You don't need the outermost ${...}.  I think it's harmless here.

    if [[ -s "$ZSPEAK_SPELL_LIST" ]]; then
        zspeak_spell=($(<${ZSPEAK_SPELL_LIST}))

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Unit 300, Science Park, Milton Road,
Cambridge, CB4 0XL, UK                          Tel: +44 (0)1223 392070

Re: ZSpeak! Only Zsh can do this!

[Bart Schaefer]

On Mar 16,  3:21pm, Mario Lang wrote:
} Subject: ZSpeak! Only Zsh can do this!
} 
} OK, here we go. Please mail me with every (minor|major) suggestion.
} I can use everything. Especially to learn nice Zsh techniques.
} 
} # * Find a nice way to add/remove keybindings in speech-(on|off)

Finding a nice way to remove keybindings is sort of a general topic of
discussion right now.

For the rebindings of forward-char and backward-char, look at the way
it's done in Functions/Zle/predict-on (in the predict-off function).

} # * Find a way to make the prefix key-sequence configurable (^[v)

That should be as easy as stashing it in a parameter.  Or you could use
the zstyle mechanism.

} # * Modify the completion system so that menu-completion inserted
} #   text gets spoken.

Hrm, that one probably requires C source code changes.

} # * Find a way to read stdout/stderr of commands too! is exec my friend?

Yes, of course!

Let me jump ahead a bit, though:

} speak-string () {
}     # Here goes the compatibility code. All synth specific
}     # decissions when outputting text should be done here.
}     case $ZSPEAK_SYNTH in
} 	festival)
} 	    [[ -x `which festival` ]] && print "$@"|festival --tts
} 	    ;;
}         speechd)
} 	    [[ -e "/dev/speech" ]] && print "$@">/dev/speech
} 	    ;;
} 	*)
}     esac
} }

I'm not familiar with how "festival" works, but if it can accept a series
of lines and speak each one as it sees the newline, you can use a feature
of zsh's coproc mechanism to emulate /dev/speech.

Rather than test for the existence of festival every time speak-string is
called, you set it up in advance like this:

    if [[ $ZSPEAK_SYNTH == festival && -x `which festival` ]]
    then
      coproc festival --tts
      speak-string () {
        print -p "$@"
      }
    elif [[ [[ $ZSPEAK_SYNTH == speechd && -e /dev/speech ]]
      speak-string () {
        print "$@" > /dev/speech
      }
    else
      speak-string () { }
    fi

Now "print -p" sends its output to the speech synthesizer, as does any
command whose output is redirected with ">&p".  Unlike ksh coprocesses,
zsh holds open the coproc descriptors even when redirection has been
done, so as long as festival doesn't exit, you can redirect the output
of every command through it.

Of course this may have unwanted side-effects if festival holds open a
sound device, thereby preventing other sound processes from using it.
However, I'll proceed on the assumption that's not the case.  Also, it
means you can't use "coproc" for anything else, but in my experience
hardly anyone ever uses zsh coprocesses in the first place.

With the coproc in place, if you want to, you can take advantage of
"multios" like so:

    setopt multios
    exec 2>&1 2>&p	# Send stderr to both stdout and synthesizer
    exec >&2		# Send stdout to the two places stderr goes

Now everything output by any command that isn't explicitly redirected
somewhere else, will be both written to the terminal (or wherever fd 1
was originally pointing) and spoken.

Replace >&p with >/dev/speech in the first exec if using speechd, but
again the caveat about holding the device open may apply.

Note that some commands, such as "less", may attempt to use fd 2 for
interactive input when their standard input is a pipe.  In that case
the above trick of multi-redirecting stderr works very badly.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net