PATCH: enhanced word widgets

PATCH: enhanced word widgets

[Peter Stephenson]

I've always been unsatisfied by zle's handling of words; I don't like
the current $WORDCHARS mechanism very much.  I've come up with a more
flexible widget based system.  There's a simple front-end to it,
select-word-style, which hides much of the complexity, but you can do
lots of clever things with styles if you want.

I've provided replacements for eight basic word functions (the
replacements have `-match' tacked on the end).  They don't include
insert-last-word and its relatives since they've never used the normal
system for words.  With some enhancements to history (I've always wanted
to be able to get history lines directly via an array) that ought to be
possible, too.

Some simple examples:

  autoload -U
  select-word-style bash

loads the functions over the default set and makes them behave in bash
fashion (only alphanumerics are word characters).

  zle -N select-word-style
  bindkey '^x^w' select-word-style

allows you to set one of the word styles for all functions with a few
keystrokes.  There's an enhancement to read-from-minibuffer to make this
easy to use.

  zstyle ':zle:*word*-shell' word-style shell

Any future definitions of word widgets using the supplied functions
under names which end in `-shell' automatically use shell command
arguments as words.  For example,

  zle -N transpose-words-shell transpose-words-match

is covered.

I would propose removing the existing bash word functions, since this
gives more coverage more simply (the first example).  However, there is
a counterargument that the bash functions are much simpler and therefore
more likely to work.

One bit where I'm quite likely to have screwed up is in the
implementation of the word-chars and word-class styles --- quoting the
style values to fit into actual character classes is pretty hairy.
Connoisseurs of shell obscurity will wish to peruse
match-words-by-style.

Otherwise, feel free to test the numeric prefix and kill ring handling
to see if I've broken anything there.  Note that the kill widgets
require my fix for the embarrassing bugs in the $killring zle parameter.

By the way, I think something is a bit screwy at the moment in the
handling of cursor positions in undo.  I don't think this is related to
the use of widget functions --- setting $BUFFER etc. uses the same
underlying mechanism for keeping track of undo's and undon'ts as builtin
widgets.

Index: Doc/Zsh/contrib.yo
===================================================================
RCS file: /cvsroot/zsh/zsh/Doc/Zsh/contrib.yo,v
retrieving revision 1.25
diff -u -r1.25 contrib.yo
--- Doc/Zsh/contrib.yo	3 Feb 2003 11:06:02 -0000	1.25
+++ Doc/Zsh/contrib.yo	28 Mar 2003 11:06:43 -0000
@@ -362,29 +362,141 @@
 with a key sequence.  Suggested bindings are described below.
 
 startitem()
-tindex(bash-forward-word)
-tindex(bash-backward-word)
-tindex(bash-kill-word)
-tindex(bash-backward-kill-word)
-tindex(bash-transpose-words)
-tindex(bash-up-case-word)
-tindex(bash-down-case-word)
-xitem(tt(bash-forward-word), tt(bash-backward-word))
-xitem(tt(bash-kill-word), tt(bash-backward-kill-word))
-xitem(tt(bash-up-case-word), tt(bash-down-case-word))
-item(tt(bash-transpose-words))(
-These work similarly to the corresponding builtin zle functions without the
-`tt(bash-)' prefix, but a word is considered to consist of alphanumeric
-characters only.  If you wish to replace your existing bindings with these
-four widgets, the following is sufficient:
-
-example(for widget in kill-word backward-kill-word \ 
-forward-word backward-word \ 
-up-case-word down-case-word \ 
-transpose-words; do 
-  autoload bash-$widget 
-  zle -N $widget bash-$widget
-done)
+item(bash-style word functions)(
+If you are looking for functions to implement moving over and editing
+words in the manner of bash, where only alphanumeric characters are
+considered word characters, you can use the functions described in
+the next section.  The following is sufficient:
+
+example(autoload -U select-word-style
+select-word-style bash)
+
+)
+tindex(forward-word-match)
+tindex(backward-word-match)
+tindex(kill-word-match)
+tindex(backward-kill-word-match)
+tindex(transpose-words-match)
+tindex(capitalize-word-match)
+tindex(up-case-word-match)
+tindex(down-case-word-match)
+tindex(select-word-style)
+tindex(match-word-by-style)
+xitem(tt(forward-word-match), tt(backward-word-match))
+xitem(tt(kill-word-match), tt(backward-kill-word-match))
+xitem(tt(transpose-words-match), tt(capitalize-word-match))
+xitem(tt(up-case-word-match), tt(down-case-word-match))
+item(tt(select-word-style), tt(match-word-by-style))(
+The eight `tt(-match)' functions are drop-in replacements for the
+builtin widgets without the suffix.  By default they behave in a similar
+way.  However, by the use of styles and the function tt(select-word-style),
+the way words are matched can be altered.
+
+The simplest way of configuring the functions is to use
+tt(select-word-style), which can either be called as a normal function with
+the appropriate argument, or invoked as a user-defined widget that will
+prompt for the first character of the word style to be used.  The first
+time it is invoked, the eight tt(-match) functions will automatically
+replace the builtin versions, so they do not need to be loaded explicitly.
+
+The word styles available are as follows.  Only the first character
+is examined.
+
+startitem()
+item(tt(bash))(
+Word characters are alphanumeric characters only.
+)
+item(tt(normal))(
+As in normal shell operation:  word characters are alphanumeric characters
+plus any characters present in the string given by the parameter
+tt($WORDCHARS).
+)
+item(tt(shell))(
+Words are complete shell command arguments, possibly including complete
+quoted strings, or any tokens special to the shell.
+)
+item(tt(whitespace))(
+Words are any set of characters delimited by whitespace.
+)
+item(tt(default))(
+Restore the default settings; this is usually the same as `tt(normal)'.
+)
+enditem()
+
+More control can be obtained using the tt(zstyle) command, as described in
+ifzman(zmanref(zshmodules))\
+ifnzman(noderef(The zsh/zutil Module)).  Each style is looked up in the
+context tt(:zle:)var(widget) where var(widget) is the name of the
+user-defined widget, not the name of the function implementing it, so in
+the case of the definitions supplied by tt(select-word-style) the
+appropriate contexts are tt(:zle:forward-word), and so on.  The function
+tt(select-word-style) itself always defines styles for the context
+`tt(:zle:*)' which can be overridden by more specific (longer) patterns as
+well as explicit contexts.
+
+The style tt(word-style) specifies the rules to use.  This may have the
+following values.
+
+startitem()
+item(tt(normal))(
+Use the standard shell rules, i.e. alphanumerics and tt($WORDCHARS), unless
+overridden by the styles tt(word-chars) or tt(word-class).
+)
+item(tt(specified))(
+Similar to tt(normal), but em(only) the specified characters, and not also
+alphanumerics, are considered word characters.
+)
+item(tt(unspecified))(
+The negation of specified.  The given characters are those which will
+em(not) be considered part of a word.
+)
+item(tt(shell))(
+Words are obtained by using the syntactic rules for generating shell
+command arguments.  In addition, special tokens which are never command
+arguments such as `tt(())' are also treated as words.
+)
+item(tt(whitespace))(
+Words are whitespace-delimited strings of characters.
+)
+enditem()
+
+The first three of those styles usually use tt($WORDCHARS), but the value
+in the parameter can be overridden by the style tt(word-chars), which works
+in exactly the same way as tt($WORDCHARS).  In addition, the style
+tt(word-class) uses character class syntax to group characters and takes
+precedence over tt(word-chars) if both are set.  The tt(word-class) style
+does not include the surrounding brackets of the character class; for
+example, `tt(-:[:alnum:])' is a valid tt(word-class) to include all
+alphanumerics plus the characters `tt(-)' and `tt(:)'.  Be careful
+including `tt(])', `tt(^)' and `tt(-)' as these are special inside
+character classes.
+
+The final style is tt(skip-chars).  This is mostly useful for
+tt(transpose-words) and similar functions.  If set, it gives a count of
+characters starting at the cursor position which will not be considered
+part of the word and are treated as space, regardless of what they actually
+are.  For example, if
+
+example(zstyle ':zle:transpose-words' skip-chars 1)
+
+has been set, and tt(transpose-words-match) is called with the cursor on
+the var(X) of tt(foo)var(X)tt(bar), where var(X) can be any character, then
+the resulting expression is tt(bar)var(X)tt(foo).
+
+The word matching and all the handling of tt(zstyle) settings is actually
+implemented by the function tt(match-word-by-style).  This can be used to
+create new user-defined widgets.  The calling function should set the local
+parameter tt(curcontext) to tt(:zle:)var(widget), create the local
+parameter tt(matched_words) and call tt(match-word-by-style) with no
+arguments.  On return, tt(matched_words) will be set to an array with the
+elements: (1) the start of the line (2) the word before the cursor (3) any
+non-word characters between that word and the cursor (4) any non-word
+character at the cursor position plus any remaining non-word characters
+before the next word, including all characters specified by the
+tt(skip-chars) style, (5) the word at or following the cursor (6) any
+non-word characters following that word (7) the remainder of the line.  Any
+of the elements may be an empty string; the calling function should test
+for this to decide whether it can perform its function.
 )
 tindex(copy-earlier-word)
 item(tt(copy-earlier-word))(
@@ -600,6 +712,11 @@
 by a keyboard break (typically tt(^G)), the function returns status 1
 and tt($REPLY) is not set.  If an argument is supplied to the function
 it is taken as a prompt, otherwise `tt(? )' is used.
+
+One option is available: `tt(-k) var(num)' specifies that var(num)
+characters are to be read instead of a whole line.  The line editor is not
+invoked recursively in this case.  Note that unlike the tt(read) builtin
+var(num) must be given; there is no default.
 
 The name is a slight misnomer, as in fact the shell's own minibuffer is
 not used.  Hence it is still possible to call tt(executed-named-cmd) and
Index: Functions/Zle/backward-kill-word-match
===================================================================
RCS file: Functions/Zle/backward-kill-word-match
diff -N Functions/Zle/backward-kill-word-match
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ Functions/Zle/backward-kill-word-match	28 Mar 2003 11:06:43 -0000
@@ -0,0 +1,36 @@
+emulate -L zsh
+setopt extendedglob
+
+autoload match-words-by-style
+
+local curcontext=":zle:$WIDGET" word done
+local -a matched_words
+integer count=${NUMERIC:-1}
+
+if (( count < 0 )); then
+    (( NUMERIC = -count ))
+    zle ${WIDGET##backward-}
+    return
+fi
+
+while (( count-- )); do
+
+    match-words-by-style
+
+    word="$matched_words[2]$matched_words[3]"
+
+    if [[ -n $word ]]; then
+	if [[ -n $done || $LASTWIDGET = *kill* ]]; then
+	    CUTBUFFER="$word$CUTBUFFER"
+	else
+	    killring=("$CUTBUFFER" "${(@)killring[1,-2]}")
+	    CUTBUFFER=$word
+	fi
+	LBUFFER=$matched_words[1]
+    else
+	return 1
+    fi
+    done=1
+done
+
+return 0
Index: Functions/Zle/backward-word-match
===================================================================
RCS file: Functions/Zle/backward-word-match
diff -N Functions/Zle/backward-word-match
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ Functions/Zle/backward-word-match	28 Mar 2003 11:06:43 -0000
@@ -0,0 +1,29 @@
+emulate -L zsh
+setopt extendedglob
+
+autoload match-words-by-style
+
+local curcontext=":zle:$WIDGET" word
+local -a matched_words
+integer count=${NUMERIC:-1}
+
+if (( count < 0 )); then
+    (( NUMERIC = - count ))
+    zle ${WIDGET/backward/forward}
+    return
+fi
+
+while (( count-- )); do
+
+    match-words-by-style
+
+    word=$matched_words[2]$matched_words[3]
+
+    if [[ -n $word ]]; then
+	(( CURSOR -= ${#word} ))
+    else
+	return 1
+    fi
+done
+
+return 0
Index: Functions/Zle/capitalize-word-match
===================================================================
RCS file: Functions/Zle/capitalize-word-match
diff -N Functions/Zle/capitalize-word-match
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ Functions/Zle/capitalize-word-match	28 Mar 2003 11:06:43 -0000
@@ -0,0 +1,23 @@
+emulate -L zsh
+setopt extendedglob
+
+autoload match-words-by-style
+
+local curcontext=":zle:$WIDGET" word
+local -a matched_words
+integer count=${NUMERIC:-1}
+
+while (( count-- > 0 )); do
+    match-words-by-style
+
+    word=${(j..)matched_words[4,5]}
+
+    if [[ -n $word ]]; then
+	LBUFFER+=${(C)word}
+	RBUFFER=${(j..)matched_words[6,7]}
+    else
+	return 1
+    fi
+done
+
+return 0
Index: Functions/Zle/down-case-word-match
===================================================================
RCS file: Functions/Zle/down-case-word-match
diff -N Functions/Zle/down-case-word-match
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ Functions/Zle/down-case-word-match	28 Mar 2003 11:06:43 -0000
@@ -0,0 +1,23 @@
+emulate -L zsh
+setopt extendedglob
+
+autoload match-words-by-style
+
+local curcontext=":zle:$WIDGET" word
+local -a matched_words
+integer count=${NUMERIC:-1}
+
+while (( count-- > 0 )); do
+    match-words-by-style
+
+    word=${(j..)matched_words[4,5]}
+
+    if [[ -n word ]]; then
+	LBUFFER+=${(L)word}
+	RBUFFER=${(j..)matched_words[6,7]}
+    else
+	return 1
+    fi
+done
+
+return 0
Index: Functions/Zle/forward-word-match
===================================================================
RCS file: Functions/Zle/forward-word-match
diff -N Functions/Zle/forward-word-match
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ Functions/Zle/forward-word-match	28 Mar 2003 11:06:43 -0000
@@ -0,0 +1,39 @@
+emulate -L zsh
+setopt extendedglob
+
+autoload match-words-by-style
+
+local curcontext=":zle:$WIDGET" word
+local -a matched_words
+integer count=${NUMERIC:-1}
+
+if (( count < 0 )); then
+    (( NUMERIC = -count ))
+    zle ${WIDGET/forward/backward}
+    return
+fi
+
+while (( count-- )); do
+
+    match-words-by-style
+
+    # For some reason forward-word doesn't work like the other word
+    # word commnds; it skips whitespace only after any matched word
+    # characters.
+
+    if [[ -n $matched_words[4] ]]; then
+        # just skip the whitespace
+	word=$matched_words[4]
+    else
+        # skip the word and trailing whitespace
+	word=$matched_words[5]$matched_words[6]
+    fi
+
+    if [[ -n $word ]]; then
+	(( CURSOR += ${#word} ))
+    else
+	return 1
+    fi
+done
+
+return 0
Index: Functions/Zle/kill-word-match
===================================================================
RCS file: Functions/Zle/kill-word-match
diff -N Functions/Zle/kill-word-match
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ Functions/Zle/kill-word-match	28 Mar 2003 11:06:43 -0000
@@ -0,0 +1,36 @@
+emulate -L zsh
+setopt extendedglob
+
+autoload match-words-by-style
+
+local curcontext=":zle:$WIDGET" word done
+local -a matched_words
+integer count=${NUMERIC:-1}
+
+if (( count < 0 )); then
+    (( NUMERIC = -count ))
+    zle backward-$WIDGET
+    return
+fi
+
+while (( count-- )); do
+
+    match-words-by-style
+
+    word="${(j..)matched_words[4,5]}"
+
+    if [[ -n $word ]]; then
+	if [[ -n $done || $LASTWIDGET = *kill* ]]; then
+	    CUTBUFFER="$CUTBUFFER$word"
+	else
+	    killring=("$CUTBUFFER" "${(@)killring[1,-2]}")
+	    CUTBUFFER=$word
+	fi
+	RBUFFER=$matched_words[6]
+    else
+	return 1
+    fi
+    done=1
+done
+
+return 0
Index: Functions/Zle/match-words-by-style
===================================================================
RCS file: Functions/Zle/match-words-by-style
diff -N Functions/Zle/match-words-by-style
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ Functions/Zle/match-words-by-style	28 Mar 2003 11:06:43 -0000
@@ -0,0 +1,167 @@
+# Match words by the style given below.  The matching depends on the
+# cursor position.  The matched_words array is set to the matched portions
+# separately.  These look like:
+#    <stuff-at-start> <word-before-cursor> <whitespace-before-cursor>
+#    <whitespace-after-cursor> <word-after-cursor> <whitespace-after-word>
+#    <stuff-at-end>
+# where the cursor position is always after the third item and `after'
+# is to be interpreted as `after or on'.  Some
+# of the array elements will be empty; this depends on the style.
+# For example
+#    foo bar  rod stick
+#            ^
+# with the cursor where indicated whill with typical settings produce the
+# elements `foo ', `bar', ` ', ` ', `rod', ` ' and `stick'.
+#
+# The style word-style can be set to indicate what a word is.
+# The three possibilities are:
+#
+#  shell	Words are shell words, i.e. elements of a command line.
+#  whitespace	Words are space delimited words; only space or tab characters
+#               are considered to terminated a word.
+#  normal       (the default): the usual zle logic is applied, with all
+#		alphanumeric characters plus any characters in $WORDCHARS
+#		considered parts of a word.  The style word-chars overrides
+#		the parameter.  (Any currently undefined value will be
+#		treated as `normal', but this should not be relied upon.)
+#  specified    Similar to normal, except that only the words given
+#               in the string (and not also alphanumeric characters)
+#               are to be considerd parts of words.
+#  unspecified  The negation of `specified': the characters given
+#               are those that aren't to be considered parts of a word.
+#               They should probably include white space.
+#
+# In the case of the `normal' or `(un)specified', more control on the
+# behaviour can be obtained by setting the style `word-chars' for the
+# current context.  The value is used to override $WORDCHARS locally.
+# Hence,
+#   zstyle ':zle:transpose-words*' word-style normal
+#   zstyle ':zle:transpose-words*' word-chars ''
+# will force bash-style word recognition, i.e only alphanumeric characters
+# are considerd parts of a word.  It is up to the function which calls
+# match-words-by-style to set the context in the variable curcontext,
+# else a default context will be used (not recommended).
+#
+# You can override the use of word-chars with the style word-class.
+# This specifies the same information, but as a character class.
+# The surrounding square brackets shouldn't be given, but anything
+# which can appear inside is allowed.  For example,
+#   zstyle ':zle:*' word-class '-:[:alnum:]'
+# is valid.  Note the usual care with `]' , `^' and `-' must be taken if
+# they need to appear as individual characters rather than for grouping.
+#
+# The final style is `skip-chars'.  This is an integer; that many
+# characters counting the one under the cursor will be treated as
+# whitespace regardless and added to the front of the fourth element of
+# matched_words.  The default is zero, i.e. the character under the cursor
+# will appear in <whitespace-after-cursor> if it is whitespace, else in
+# <word-after-cursor>.  This style is mostly useful for forcing
+# transposition to ignore the current character.
+
+
+emulate -L zsh
+setopt extendedglob
+
+local wordstyle spacepat wordpat1 wordpat2 opt charskip
+local match mbegin mend pat1 pat2 word1 word2 ws1 ws2 ws3 skip
+local MATCH MBEGIN MEND
+
+if [[ -z $curcontext ]]; then
+    local curcontext=:zle:match-words-by-style
+fi
+
+zstyle -s $curcontext word-style wordstyle
+zstyle -s $curcontext skip-chars skip
+[[ -z $skip ]] && skip=0
+
+case $wordstyle in
+  (shell) local bufwords
+	  # This splits the line into words as the shell understands them.
+	  bufwords=(${(z)LBUFFER})
+	  # Work around bug: if stripping quotes failed, a bogus
+	  # space is appended.  Not a good test, since this may
+	  # be a quoted space, but it's hard to get right.
+	  wordpat1=${bufwords[-1]}
+	  if [[ ${wordpat1[-1]} = ' ' ]]; then
+	    wordpat1=${(q)wordpat1[1,-2]}
+	  else
+	    wordpat1="${(q)wordpat1}"
+	  fi
+
+	  # Take substring of RBUFFER to skip over $skip characters
+	  # from the cursor position.
+	  bufwords=(${(z)RBUFFER[1+$skip,-1]})
+	  # Work around bug again.
+	  wordpat2=${bufwords[1]}
+	  if [[ ${wordpat2[-1]} = ' ' ]]
+	  then
+	    wordpat2=${(q)wordpat2[1,-2]}
+	  else
+	    wordpat2="${(q)wordpat2}"
+	  fi
+	  spacepat='[[:space:]]#'
+	  ;;
+  (*space) spacepat='[[:space:]]#'
+           wordpat1='[^[:space:]]##'
+	   wordpat2=$wordpat1
+	   ;;
+  (*) local wc
+      # See if there is a character class.
+      if zstyle -s $curcontext word-class wc; then
+	  # Treat as a character class: do minimal quoting.
+	  wc=${wc//(#m)[\'\"\`\$\(\)\^]/\\$MATCH}
+      else
+          # See if there is a local version of $WORDCHARS.
+	  zstyle -s $curcontext word-chars wc ||
+	  wc=$WORDCHARS
+	  if [[ $wc = (#b)(?*)-(*) ]]; then
+              # We need to bring any `-' to the front to avoid confusing
+              # character classes... we get away with `]' since in zsh
+              # this isn't a pattern character if it's quoted.
+	      wc=-$match[1]$match[2]
+	  fi
+	  wc="${(q)wc}"
+      fi
+      # Quote $wc where necessary, because we don't want those
+      # characters to be considered as pattern characters later on.
+      if [[ $wordstyle = *specified ]]; then
+        if [[ $wordstyle != un* ]]; then
+	  # The given set of characters are the word characters, nothing else
+	  wordpat1="[${wc}]##"
+	  # anything else is a space.
+	  spacepat="[^${wc}]#"
+	else
+	  # The other way round.
+	  wordpat1="[^${wc}]##"
+	  spacepat="[${wc}]#"
+    	fi
+      else
+        # Normal: similar, but add alphanumerics.
+	wordpat1="[${wc}[:alnum:]]##"
+	spacepat="[^${wc}[:alnum:]]#"
+      fi
+      wordpat2=$wordpat1
+      ;;
+esac
+
+# The eval makes any special characters in the parameters active.
+# In particular, we need the surrounding `[' s to be `real'.
+# This is why we quoted the wordpats in the `shell' option, where
+# they have to be treated as literal strings at this point.
+match=()
+eval pat1='${LBUFFER%%(#b)('${wordpat1}')('${spacepat}')}'
+word1=$match[1]
+ws1=$match[2]
+
+match=()
+charskip=
+repeat $skip charskip+=\?
+
+eval pat2='${RBUFFER##(#b)('${charskip}${spacepat}')('\
+${wordpat2}')('${spacepat}')}'
+
+ws2=$match[1]
+word2=$match[2]
+ws3=$match[3]
+
+matched_words=("$pat1" "$word1" "$ws1" "$ws2" "$word2" "$ws3" "$pat2")
Index: Functions/Zle/read-from-minibuffer
===================================================================
RCS file: /cvsroot/zsh/zsh/Functions/Zle/read-from-minibuffer,v
retrieving revision 1.1
diff -u -r1.1 read-from-minibuffer
--- Functions/Zle/read-from-minibuffer	3 Feb 2003 11:06:03 -0000	1.1
+++ Functions/Zle/read-from-minibuffer	28 Mar 2003 11:06:43 -0000
@@ -1,3 +1,22 @@
+emulate -L zsh
+setopt extendedglob
+
+local opt keys
+integer stat
+
+while getopts "k:" opt; do
+    case $opt in
+	(k)
+	keys=$OPTARG
+	;;
+
+	(*)
+	return 1
+	;;
+    esac
+done
+(( OPTIND > 1 )) && shift $(( OPTIND - 1 ))
+
 local savelbuffer=$LBUFFER saverbuffer=$RBUFFER
 local savepredisplay=$PREDISPLAY savepostdisplay=$POSTDISPLAY
 
@@ -7,10 +26,15 @@
 ${1:-? }"
 POSTDISPLAY=
 
-zle recursive-edit
-integer stat=$?
-
-(( stat )) || REPLY=$BUFFER
+if [[ -n $keys ]]; then
+    zle -R
+    read -k $keys
+    stat=$?
+else
+    zle recursive-edit
+    stat=$?
+    (( stat )) || REPLY=$BUFFER
+fi
 
 LBUFFER=$savelbuffer
 RBUFFER=$saverbuffer
Index: Functions/Zle/select-word-style
===================================================================
RCS file: Functions/Zle/select-word-style
diff -N Functions/Zle/select-word-style
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ Functions/Zle/select-word-style	28 Mar 2003 11:06:43 -0000
@@ -0,0 +1,88 @@
+emulate -L zsh
+setopt extendedglob
+
+local -a word_functions
+
+word_functions=(backward-kill-word backward-word
+    capitalize-word down-case-word
+    forward-word kill-word
+    transpose-words up-case-word)
+
+[[ -z $1 ]] && autoload read-from-minibuffer
+
+local REPLY detail f
+
+if ! zle -l $word_functions[1]; then
+    for f in $word_functions; do
+	autoload -U $f-match
+	zle -N $f $f-match
+    done
+fi
+
+
+while true; do
+
+    if [[ -n $WIDGET && -z $1 ]]; then
+	read-from-minibuffer -k1 "Word styles (hit return for more detail):
+(b)ash (n)ormal (s)hell (w)hitespace (N)one (A)bort
+${detail}? " || return 1
+    else
+	REPLY=$1
+    fi
+
+    detail=
+
+    case $REPLY in
+	(b*)
+	# bash style
+	zstyle ':zle:*' word-style standard
+	zstyle ':zle:*' word-chars ''
+	;;
+
+	(n*)
+	# normal zsh style
+	zstyle ':zle:*' word-style standard
+	zstyle ':zle:*' word-chars "$WORDCHARS"
+	;;
+
+	(s*)
+	# shell command arguments or special tokens
+	zstyle ':zle:*' word-style shell
+	;;
+
+	(w*)
+	# whitespace-delimited
+	zstyle ':zle:*' word-style space
+	;;
+
+	(d*)
+	# default: could also return widgets to builtins here
+	zstyle -d ':zle:*' word-style
+	zstyle -d ':zle:*' word-chars
+	;;
+
+	(q*)
+	# quit without setting
+	return 1
+	;;
+
+	(*)
+	detail="\
+(b)ash:       Word characters are alphanumerics only
+(n)ormal:     Word characters are alphanumerics plus \$WORDCHARS
+(s)hell:      Words are command arguments using shell syntax
+(w)hitespace: Words are whitespace-delimited
+(d)efault:    Use default, no special handling (usually same as \`n')
+(q)uit:       Quit without setting a new style
+"
+	if [[ -z $WIDGET || -n $1 ]]; then
+	    print "Usage: $0 word-style
+where word-style is one of the characters in parentheses:
+$detail" >&2
+	    return 1
+	fi
+	continue
+	;;
+    esac
+    return
+done
Index: Functions/Zle/transpose-words-match
===================================================================
RCS file: Functions/Zle/transpose-words-match
diff -N Functions/Zle/transpose-words-match
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ Functions/Zle/transpose-words-match	28 Mar 2003 11:06:43 -0000
@@ -0,0 +1,31 @@
+# Transpose words, matching the words using match-words-by-style, q.v.
+# The group of word characters preceeding the cursor (not necessarily
+# immediately) are transposed with the group of word characters following
+# the cursor (again, not necessarily immediately).
+#
+# Note the style skip-chars, used in the context of the current widget.
+# This gives a number of character starting from the cursor position
+# which are never considered part of a word and hence are always left
+# alone.  The default is 0 and typically the only useful alternative
+# is one.  This would have the effect that `fooXbar' with the cursor
+# on X would be turned into `barXfoo' with the cursor still on the X,
+# regardless of what the character X is.
+
+autoload match-words-by-style
+
+local curcontext=":zle:$WIDGET" skip
+local -a matched_words
+integer count=${NUMERIC:-1}
+
+while (( count-- > 0 )); do
+    match-words-by-style
+
+    [[ -z "$matched_words[2]$matched_words[5]" ]] && return 1
+
+    LBUFFER="$matched_words[1]$matched_words[5]${(j..)matched_words[3,4]}\
+$matched_words[2]"
+    RBUFFER="${(j..)matched_words[6,7]}"
+
+done
+
+return 0
Index: Functions/Zle/up-case-word-match
===================================================================
RCS file: Functions/Zle/up-case-word-match
diff -N Functions/Zle/up-case-word-match
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ Functions/Zle/up-case-word-match	28 Mar 2003 11:06:43 -0000
@@ -0,0 +1,23 @@
+emulate -L zsh
+setopt extendedglob
+
+autoload match-words-by-style
+
+local curcontext=":zle:$WIDGET" word
+local -a matched_words
+integer count=${NUMERIC:-1}
+
+while (( count-- > 0 )); do
+    match-words-by-style
+
+    word=${(j..)matched_words[4,5]}
+
+    if [[ -n $word ]]; then
+	LBUFFER+=${(U)word}
+	RBUFFER=${(j..)matched_words[6,7]}
+    else
+	return 1
+    fi
+done
+
+return 0

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Science Park, Milton Road,
Cambridge, CB4 0WH, UK                          Tel: +44 (0)1223 692070


**********************************************************************
The information transmitted is intended only for the person or
entity to which it is addressed and may contain confidential 
and/or privileged material. 
Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is 
prohibited.  
If you received this in error, please contact the sender and 
delete the material from any computer.
**********************************************************************

Re: PATCH: enhanced word widgets

[Bart Schaefer]

On Mar 28, 11:26am, Peter Stephenson wrote:
}
} I've always been unsatisfied by zle's handling of words; I don't like
} the current $WORDCHARS mechanism very much.  I've come up with a more
} flexible widget based system.

I'd been working on something like that off-and-on, though I haven't got
as far as adding styles to it.  Maybe you can give an example of how to
do with your widgets what I have been trying to do with mine:

I want the definition of a "word" to be context-sensitive.  For example:
If the cursor is within a shell word that contains a "/" character, then
I want "ZLE words" to be pathname components (and transpose-words should
transpose around the nearest "/" either under the cursor or to the left);
but if the cursor is between two shell words, then I want "words" to be
shell words, e.g., pathnames including the slashes.

} By the way, I think something is a bit screwy at the moment in the
} handling of cursor positions in undo.

This has been an issue for a long time.  Undo almost never leaves the
cursor where it was before you "did," unless you were at the end of the
buffer the whole time.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Re: PATCH: enhanced word widgets

[Bart Schaefer]

On Mar 28, 11:26am, Peter Stephenson wrote:
}
} flexible widget based system.  There's a simple front-end to it,
} select-word-style

Point of order ... select-word-style is not a widget, so it should not
be in the Functions/Zle directory in the distribution.  I think it should
be in Functions/Misc instead.

I have a hard time deciding about match-words-by-style ... it's not a
widget, but it's not callable except from a widget.  However, it most
likely ought to be Functions/Misc as well.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Re: PATCH: enhanced word widgets

[Peter Stephenson]

"Bart Schaefer" wrote:
> Point of order ... select-word-style is not a widget

Yes it is, read the documentation.

> I have a hard time deciding about match-words-by-style ... it's not a
> widget, but it's not callable except from a widget.  However, it most
> likely ought to be Functions/Misc as well.

I disagree.  There's no rule that Functions/Zle contains only widgets,
just that it contains functions relevant to Zle.  If you look in
config.modules, it lists functions associated with particular modules by
directory so you can choose to include them or not.  If those start
being split up between directories things are going to get very
confusing.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Science Park, Milton Road,
Cambridge, CB4 0WH, UK                          Tel: +44 (0)1223 692070


**********************************************************************
The information transmitted is intended only for the person or
entity to which it is addressed and may contain confidential 
and/or privileged material. 
Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is 
prohibited.  
If you received this in error, please contact the sender and 
delete the material from any computer.
**********************************************************************

Re: PATCH: enhanced word widgets

[Peter Stephenson]

"Bart Schaefer" wrote:
> I want the definition of a "word" to be context-sensitive.  For example:
> If the cursor is within a shell word that contains a "/" character, then
> I want "ZLE words" to be pathname components (and transpose-words should
> transpose around the nearest "/" either under the cursor or to the left);
> but if the cursor is between two shell words, then I want "words" to be
> shell words, e.g., pathnames including the slashes.

That's a possibly enhancement to match-words-by-style.  There is already
code in the `shell' word-style case to look at where the cursor is in
terms of shell words.  This can be copied or moved earlier and used
based on another style (eg. word-context, form to be decided).  Then we
can combine the result of that test with the value of the style in some
way, and use that to generate a new value of curcontext for looking up
the standard styles (word-style, word-chars, word-class).

Or something.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Science Park, Milton Road,
Cambridge, CB4 0WH, UK                          Tel: +44 (0)1223 692070


**********************************************************************
The information transmitted is intended only for the person or
entity to which it is addressed and may contain confidential 
and/or privileged material. 
Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is 
prohibited.  
If you received this in error, please contact the sender and 
delete the material from any computer.
**********************************************************************

Re: PATCH: enhanced word widgets

[Zefram]

Bart Schaefer wrote:
>I want the definition of a "word" to be context-sensitive.  For example:
>If the cursor is within a shell word that contains a "/" character, then
>I want "ZLE words" to be pathname components (and transpose-words should
>transpose around the nearest "/" either under the cursor or to the left);
>but if the cursor is between two shell words, then I want "words" to be
>shell words, e.g., pathnames including the slashes.

This sounds a lot like Emacs' concept of sexprs -- roughly, a sexpr is
an HLL expression, so sexprs can be nested.  sexprs are also, roughly,
the level at which we want to do completion.  Can this concept be usefully
factored out?

-zefram

Re: PATCH: enhanced word widgets

[Peter Stephenson]

"Bart Schaefer" wrote:
> I want the definition of a "word" to be context-sensitive.  For example:
> If the cursor is within a shell word that contains a "/" character, then
> I want "ZLE words" to be pathname components (and transpose-words should
> transpose around the nearest "/" either under the cursor or to the left);
> but if the cursor is between two shell words, then I want "words" to be
> shell words, e.g., pathnames including the slashes.

OK, I think this does pretty much what you want.  (Luckily I was working
on a scatternet scheduling problem, so I was easily diverted.)  Zefram
is quite right but his point leads to a complete rewrite of the parser
rather than a quick ten-minute rewrite of the shell function...

No documentation yet and it won't be committed until we decide what the
neatest way of doing this is.

Set the style word-context to pairs of words: a pattern, and a
subcontext.  The function then tests against the patterns in order.

- If the cursor is in a command argument the pattern will be tested
  against the word (with any quotes stripped, so foo, \foo, 'foo' "foo"
  will all give you foo to match against --- I can remove this, but it
  seemed more convenient with it).
- If the cursor is in whitespace between words, it will be tested
  against the whitespace character under the cursor.
- If it is at the end of the buffer, it will be tested against the empty
  string.
- For reasons best known to the lexical analyser, the pattern is tested
  against `;' if you are at the end of a line when there are more to
  follow (unfortunately due to the unquoting this looks the same as a
  real semicolon in quotes).  I suppose I could sensibly trap this
  before the unquoting and turn it into a newline.

If a pattern matches, the corresponding subcontext is appended (with the
traditional colon) to the end of the current context.

I tried this out with the context array:

zstyle ':zle:*' word-context "[[:space:]]" whitespace "*/*" \
    filename "" end "*" other

(though all you need for the suggested use are the `filename' element
plus a suitable default and `other' is completely redundant (it's an
insignificant other)) and

zstyle ':zle:transpose-words:whitespace' word-style shell
zstyle ':zle:transpose-words:filename' word-style normal
zstyle ':zle:transpose-words:filename' word-chars ''

so if you are in a filename --- the character under the cursor is part
of a word which includes a slash --- you get bash-style behaviour, if
you are in whitespace, you get shell-word behaviour, and if you are in
any other word you get the default behaviour.


Index: Functions/Zle/match-words-by-style
===================================================================
RCS file: /cvsroot/zsh/zsh/Functions/Zle/match-words-by-style,v
retrieving revision 1.1
diff -u -r1.1 match-words-by-style
--- Functions/Zle/match-words-by-style	28 Mar 2003 11:34:30 -0000	1.1
+++ Functions/Zle/match-words-by-style	28 Mar 2003 18:46:56 -0000
@@ -62,12 +62,47 @@
 emulate -L zsh
 setopt extendedglob
 
-local wordstyle spacepat wordpat1 wordpat2 opt charskip
+local wordstyle spacepat wordpat1 wordpat2 charskip
 local match mbegin mend pat1 pat2 word1 word2 ws1 ws2 ws3 skip
-local MATCH MBEGIN MEND
+local MATCH MBEGIN MEND pattern subcontext
+local -a wordcontext
 
 if [[ -z $curcontext ]]; then
     local curcontext=:zle:match-words-by-style
+fi
+
+if zstyle -a $curcontext word-context wordcontext; then
+    # Pretend line so far is a complete set of words...
+    bufwords=(${(z)LBUFFER})
+    word1=${#bufwords}
+    # Word either before or including cursor
+    pat1=$bufwords[-1]
+    # If we get more words by adding in the cursor, we're at the
+    # start of the next word.
+    wordpat1="$LBUFFER$RBUFFER[1]"
+    bufwords=(${(z)wordpat1})
+    word2=${#bufwords}
+    pat2=$bufwords[-1]
+    bufwords=(${(z)BUFFER})
+    if (( word1 != word2 )); then
+	# At start of word2
+	wordpat2=${(Q)bufwords[$word2]}
+    elif [[ ${#bufwords[$word1]} > ${#pat1} ]]; then
+	# Word is longer when including whole buffer.
+	# How beautiful life is now you're in the word.
+	# Strip quotes in order to test actual argument
+	# (except we haven't done expansion on it).
+	wordpat2=${(Q)bufwords[$word1]}
+    else
+	# We are not in the word.  Just use character at cursor position.
+	wordpat2=$RBUFFER[1]
+    fi
+    for pattern subcontext in "${wordcontext[@]}"; do
+	if [[ $wordpat2 = ${~pattern} ]]; then
+	    curcontext+=":$subcontext"
+	    break
+	fi
+    done
 fi
 
 zstyle -s $curcontext word-style wordstyle

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Science Park, Milton Road,
Cambridge, CB4 0WH, UK                          Tel: +44 (0)1223 692070


**********************************************************************
The information transmitted is intended only for the person or
entity to which it is addressed and may contain confidential 
and/or privileged material. 
Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is 
prohibited.  
If you received this in error, please contact the sender and 
delete the material from any computer.
**********************************************************************

Re: PATCH: enhanced word widgets

[Miciah Dashiel Butler Masters]

In article <25439.1048850787@csr.com>, Peter Stephenson wrote:
...
> diff -N Functions/Zle/select-word-style
...
> +(b)ash (n)ormal (s)hell (w)hitespace (N)one (A)bort
...
> +(b)ash:       Word characters are alphanumerics only
> +(n)ormal:     Word characters are alphanumerics plus \$WORDCHARS
> +(s)hell:      Words are command arguments using shell syntax
> +(w)hitespace: Words are whitespace-delimited
> +(d)efault:    Use default, no special handling (usually same as \`n')
> +(q)uit:       Quit without setting a new style
...

The short list has '(N)one' instead of '(d)efault' and '(A)bort' instead
of '(q)uit'. Neither is default as the capitalisation suggests.

 -- Miciah <miciah@myrealbox.com>

Re: PATCH: enhanced word widgets

[Peter Stephenson]

Miciah Dashiel Butler Masters wrote:
> The short list has '(N)one' instead of '(d)efault' and '(A)bort' instead
> of '(q)uit'. Neither is default as the capitalisation suggests.

Thanks, I'll pick this up when we sort out context-sensitivity and
possibly one or two other minor things.  I changed the keys half-way
through writing so as not to need uppercase.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Science Park, Milton Road,
Cambridge, CB4 0WH, UK                          Tel: +44 (0)1223 692070


**********************************************************************
The information transmitted is intended only for the person or
entity to which it is addressed and may contain confidential 
and/or privileged material. 
Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is 
prohibited.  
If you received this in error, please contact the sender and 
delete the material from any computer.
**********************************************************************