subversion and programmable completion

subversion and programmable completion

[Clint Adams]

The zsh completion function for subversion currently parses the output
of svn help <subcommand>, attempting to complete based on the contents
of the usage: and Valid options: sections.

How does the subversion team feel about providing this and additional
information in a more-machine-parseable format?  I imagine that that
would lead to the bash completion becoming more intelligent as well.

Re: subversion and programmable completion

[kfogel]

Clint Adams <clint@zsh.org> writes:
> The zsh completion function for subversion currently parses the output
> of svn help <subcommand>, attempting to complete based on the contents
> of the usage: and Valid options: sections.
> 
> How does the subversion team feel about providing this and additional
> information in a more-machine-parseable format?  I imagine that that
> would lead to the bash completion becoming more intelligent as well.

What's not machine parseable about the current output?

Re: subversion and programmable completion

[Sebastien Cevey]

On Sun, Jul 20, 2003 at 12:27:23PM -0400, Clint Adams wrote:

> I imagine that that would lead to the bash completion becoming more
> intelligent as well.

The bash completion does not use the content of 'svn help'
AFAIK. (we're talking about the script in tools/client-side/ right?).

I wanted to work on a more powerful completion, but I was spending
most of my time learning to code in bash :)  I've not totally given
up, just trying to create some time to work on that ...

I had not thought of using svn help output to feed the completion
though, that sounds quite a neat idea. Are you willing to work on it
(you seemed to use zsh rather than bash though) ? Or do you know
existing examples of such bash scripts that could be a basis for an
svn one ?


-- 
Sebastien Cevey <seb@cine7.net>
Cine7.Net  -  Milcis.Net  -  ProgramPlay.Org
Jabber: theefer@albus.homelinux.net - ICQ: 48895760

" Rosebud... "
Orson Welles (Charles F. Kane) [ Citizen Kane ]

Re: subversion and programmable completion

[Clint Adams]

> What's not machine parseable about the current output?

The zsh completion function basically does the following:

To find out what subcommands are available, run 'svn help',
and grab everything between 'Available subcommands:' and
'Subversion is a tool for revision control.'  Subcommands
listed in parentheses are aliases to the word before the
parentheses.

For subcommands, run 'svn help <subcommand>', grab the
'usage:' line, check for the presence of 'URL' or 'PATH' on
that line, and try to complete URLs or files and directories, 
respectively, as well as the options listed after 'Valid options:'
(where options in brackets are aliases to the options outside).

There is a similar process for svnadmin.

There are several problems with this scheme:

It doesn't handle multiple usage possibilities.  For example,
usage: switch URL [PATH]   or
       switch --relocate FROM TO [PATH ... ]

usage: 1. diff [-r N[:M]] [--old OLD-TGT] [--new NEW-TGT] [PATH ...]
       2. diff -r N:M URL
       3. diff [-r N[:M]] URL1[@N] URL2[@M]

This could be solved with some more intelligent parsing.

It doesn't handle know whether or not PATH refers to files in the
repository (as in svn diff), files not yet in the repository (as in
svn add), or something else.  This could be solved with some
hardcoding in the completion function.

It doesn't know what 'arg' refers to in '-r arg', '--username arg',
'--password arg', and so on.  This could be solved with some hardcoding
in the completion function.

If the subversion help output ever is altered so that the heuristics
fail, completion will break.


None of this is a problem (other than lack of man-hours), but if svn were
to actively participate in its own command completion, I think it would be
more efficient and less error-prone.

Re: subversion and programmable completion

[Clint Adams]

> The bash completion does not use the content of 'svn help'
> AFAIK. (we're talking about the script in tools/client-side/ right?).

The only bash completion for subversion that I've seen is
/etc/bash_completion.d/subversion in the Debian subversion package,
which I assume is that to which you refer.  The zsh function was written
without hardcoding any subcommands or options.

> I had not thought of using svn help output to feed the completion
> though, that sounds quite a neat idea. Are you willing to work on it
> (you seemed to use zsh rather than bash though) ? Or do you know
> existing examples of such bash scripts that could be a basis for an
> svn one ?

I seem to recall Adam Heath doing some work on emulating zsh completion
functions in bash.  Perhaps he could encapsulate the zsh functions
somehow.

[PATCH] bash_completion: improvement, fixes and tests [was: Re: subversion and programmable completion]

[Julian Foad]

[-- Attachment #1: Type: text/plain, Size: 2772 bytes --]

Sebastien Cevey wrote:
> 
> The bash completion does not use the content of 'svn help'
> AFAIK. (we're talking about the script in tools/client-side/ right?).
...
> I had not thought of using svn help output to feed the completion
> though, that sounds quite a neat idea. Are you willing to work on it
> (you seemed to use zsh rather than bash though) ? Or do you know
> existing examples of such bash scripts that could be a basis for an
> svn one ?

Here is a step towards that.

After I fixed a trivial bug in bash_completion and added recognition of "--arg=value", I wrote a test script that (among other tests) checks the completions against "svn help".  That showed me lots more bugs.  So here are the fixes and the test script (which is not run automatically by "make check").

Note: the test script still fails for one reason: "--version" is described by "svn help" as an option to "svn help".  That is wrong: it doesn't work like that, though it is sort of internally handled that way.  "--version" and "--help" and "-h" are (should be) options to "svn" (without a subcommand).  I regard this as a bug in "svn", not in "bash_completion" or "bash_completion_test".

I could adjust bash_completion and/or bash_completion_test to overlook this anomoly if you really want.  It should be marked "XFAIL" but that's not easy to do.  This is what it says:

~/src/subversion/tools/client-side> ./bash_completion_test ./bash_completion
Checking general completion
Checking list of subcommands
Checking list of options for each subcommand
FAIL: completions for "svn ? -" != options accepted
          (completions: --quiet -q )
          (svn accepts: --quiet --version -q )
FAIL: completions for "svn h -" != options accepted
          (completions: --quiet -q )
          (svn accepts: --quiet --version -q )
FAIL: completions for "svn help -" != options accepted
          (completions: --quiet -q )
          (svn accepts: --quiet --version -q )
Checking rejection of synonyms
FAILURE: at least one bash_completion test failed.


Is this acceptable the way it is, given that the test script is only to be run manually by someone who is interested in bash_completion?

- Julian


Log message:
Improve and fix bash_completion.
Add a script that tests bash_completion, including checking the completion
results against the output of "svn help [...]".

* tools/client-side/bash_completion
  (_svn) Recognise "--arg=value" as an alternative to "--arg value" (when
         excluding options or synonyms that are already present).
         Fix the list of subcommands and options accepted as a first argument.
         Fix the lists of options accepted by various subcommands.

* tools/client-side/bash_completion_test
  New file: a shell script that tests bash_completion.



[-- Attachment #2: bash_completion.diff --]
[-- Type: text/plain, Size: 3441 bytes --]

Index: tools/client-side/bash_completion
===================================================================
--- tools/client-side/bash_completion	(revision 6521)
+++ tools/client-side/bash_completion	(working copy)
@@ -9,26 +9,26 @@
 
 _svn()
 {
-	local cur cmds cmdOpts pOpts mOpts rOpts qOpts nOpts optsParam opt
+	local cur cmds cmdOpts pOpts mOpts rOpts qOpts nOpts optsParam opt optBase
 
 	COMPREPLY=()
 	cur=${COMP_WORDS[COMP_CWORD]}
 
 	cmds='add cat checkout co cleanup commit ci copy cp delete del \
-	      remove rm diff di export import info h help list ls log merge \
+	      remove rm diff di export import info ? h help list ls log merge \
 	      mkdir move mv rename ren propdel pdel propedit pedit pe propget \
 	      pget pg proplist plist pl propset pset ps revert resolve status \
-	      stat st switch sw update up --version'
+	      stat st switch sw update up'
 
 	if [[ $COMP_CWORD -eq 1 ]] ; then
-		COMPREPLY=( $( compgen -W "$cmds" -- $cur ) )
+		COMPREPLY=( $( compgen -W "$cmds -h --help --version" -- $cur ) )
 		return 0
 	else
 		# if we're completing for 'svn help' or 'svn h', then just 
 		# complete on any valid svn command
 		case ${COMP_WORDS[1]} in
 		help|h|\?)
-			COMPREPLY=( $( compgen -W "$cmds" -- $cur ) )
+			COMPREPLY=( $( compgen -W "$cmds -q --quiet" -- $cur ) )
 			return 0
 			;;
 		*)
@@ -62,7 +62,7 @@
 		cmdOpts="$qOpts"
 		;;
 	add)
-		cmdOpts="--targets -$nOpts $qOpts"
+		cmdOpts="--targets $nOpts $qOpts"
 		;;
 	cat)
 		cmdOpts="$rOpts $pOpts"
@@ -89,7 +89,7 @@
 		cmdOpts="$rOpts $qOpts $pOpts --force"
 		;;
 	import)
-		cmdOpts="$mOpts $qOpts $nOpts --targets --editor-cmd $pOpts"
+		cmdOpts="$mOpts $qOpts $nOpts --editor-cmd $pOpts"
 		;; 
 	info)
 		cmdOpts="--targets -R --recursive"
@@ -99,7 +99,7 @@
 		;;
 	log)
 		cmdOpts="$rOpts -v --verbose --targets $pOpts --strict \
-		         --incremental --xml"
+		         --incremental --xml $qOpts"
 		;;
 	merge)
 		cmdOpts="$rOpts $nOpts $qOpts --force --dry-run --diff3-cmd \
@@ -115,7 +115,8 @@
 		cmdOpts="$qOpts -R --recursive $rOpts --revprop $pOpts"
 		;;
 	propedit|pedit|pe)
-		cmdOpts="$rOpts --revprop --encoding --editor-cmd $pOpts"
+		cmdOpts="$rOpts --revprop --encoding --editor-cmd $pOpts \
+		         --force"
 		;;
 	propget|pget|pg)
 		cmdOpts="-R --recursive $rOpts --revprop --strict $pOpts"
@@ -126,7 +127,7 @@
 		;;
 	propset|pset|ps)
 		cmdOpts="-F --file $qOpts --targets -R --recursive --revprop \
-		         --encoding $pOpts"
+		         --encoding $pOpts $rOpts --force"
 		;;
 	revert)
 		cmdOpts="--targets -R --recursive $qOpts"
@@ -139,10 +140,10 @@
 		         --no-ignore"
 		;;
 	switch|sw)
-		cmdOpts="--relocate $rOpts $nOpts $qOpts $pOpts"
+		cmdOpts="--relocate $rOpts $nOpts $qOpts $pOpts --diff3-cmd"
 		;;
 	update|up)
-		cmdOpts="$rOpts $nOpts $qOpts $pOpts"
+		cmdOpts="$rOpts $nOpts $qOpts $pOpts --diff3-cmd"
 		;;
 	*)
 		;;
@@ -151,11 +152,17 @@
 	# take out options already given
 	for (( i=2; i<=$COMP_CWORD-1; ++i )) ; do
 		opt=${COMP_WORDS[$i]}
+
+		case $opt in
+		--*)    optBase=${opt/=*/} ;;
+		-*)     optBase=${opt} ;;
+		esac
+
 		cmdOpts=" $cmdOpts "
-		cmdOpts=${cmdOpts/ ${opt} / }
+		cmdOpts=${cmdOpts/ ${optBase} / }
 
 		# take out alternatives
-		case $opt in
+		case $optBase in
 		-v)              cmdOpts=${cmdOpts/ --verbose / } ;;
 		--verbose)       cmdOpts=${cmdOpts/ -v / } ;;
 		-N)              cmdOpts=${cmdOpts/ --non-recursive / } ;;

[-- Attachment #3: bash_completion_test --]
[-- Type: text/plain, Size: 4139 bytes --]

#!/bin/bash
# Checks that the "_svn" function defined in the specified "bash_completion"
# script produces appropriate lists of completions for various incomplete svn
# command lines.

if [ ! -f "$1" ] || [ "$2" ]; then
  echo "Usage: bash_completion_test BASH_COMPLETION_PATHNAME"
  echo "Tests the specified \"bash_completion\" script,"
  echo "including checking it against the \"svn\" program found in the current PATH."
  exit 1
fi

set -e  # Exit on error
shopt -s extglob

# Execute the script which is to be tested.
. "$1"

# From the given incomplete svn command, print a space-separated list of
# possible completions of the last argument (or of an empty first argument
# if no subcommand is given).
# Usage: get_svn_completions [SVN-SUBCOMMAND [SVN-OPTION...]]
get_svn_completions() {
  COMP_WORDS=(svn "$@")
  if [ $# == 0 ]; then
    COMP_CWORD=1
  else
    COMP_CWORD=$#
  fi
  _svn
  echo -n "${COMPREPLY[*]}"
}

# Print a failure message, record the failure, and return "false".
# Usage: fail MESSAGE
fail() {
  PREFIX="FAIL: "
  for LINE in "$@"; do
    echo "$PREFIX$LINE"
    PREFIX="      "
  done
  TESTS_FAILED=1
  false
}

# Check that EXPECTED-WORD is among the completions of the last word in
# SVN-COMMAND.  SVN-COMMAND is a single argument to this function, split
# into multiple arguments when passed to "get_svn_completions()".
# Usage: includes SVN-COMMAND EXPECTED-WORD
includes() {
  COMPLETIONS=`get_svn_completions $1`
  if [[ "$2" != @(${COMPLETIONS// /|}) ]]; then
    fail "completions of \"svn $1\" should include \"$2\"" \
      "(completions: $COMPLETIONS)"
  fi
}

excludes() {
  COMPLETIONS=`get_svn_completions $1`
  if [[ "$2" == @(${COMPLETIONS// /|}) ]]; then
    fail "completions of \"svn $1\" should exclude \"$2\"" \
      "(completions: $COMPLETIONS)"
  fi
}

# Print the valid subcommands for "svn", one per line, sorted.
# Usage: get_svn_subcommands
get_svn_subcommands() {
  svn help |
    # Find the relevant lines;
    # remove brackets and commas; put each word on its own line.
    sed -n -e '1,/^Available subcommands:$/d;/^$/q' \
           -e 's/[ )]//g;s/[(,]/\n/g;p' |
    sort
}

# Print the valid option switches for "svn SUBCMD", one per line, sorted.
# Usage: get_svn_options SUBCMD
get_svn_options() {
  { svn help "$1" |
      # Find the relevant lines;
      # remove brackets, "arg" and description; put each word on its own line.
      sed -n -e '1,/^Valid options:$/d;/^  -/!d' \
             -e 's/\( arg\)* * : .*//;s/[] ]//g;s/[[]/\n/g;p'
    # The following options are always accepted but not listed in the help,
    # nor currently offered as completions.
    #echo "-h"
    #echo "--help"
  } | sort
  
}


# The tests.
set +e  # Do not exit on error
TESTS_FAILED=

echo "Checking general completion"
includes "he" "help"
includes "" "?"
includes "" "h"
includes "" "help"
includes "" "-h"
includes "" "--help"
includes "" "--version"

echo "Checking list of subcommands"
HELP_SUBCMDS=`get_svn_subcommands | tr "\n" " "`
COMPLETION_SUBCMDS=`get_svn_completions | tr " " "\n" | grep -v "^-" | sort | tr "\n" " "`
if [ "$HELP_SUBCMDS" != "$COMPLETION_SUBCMDS" ]; then
  fail "non-option completions for \"svn \" != subcommands accepted" \
       "    (non-o. cmpl.: $COMPLETION_SUBCMDS)" \
       "    (svn accepts:  $HELP_SUBCMDS)"
fi

echo "Checking list of options for each subcommand"
for SUBCMD in $HELP_SUBCMDS; do
  HELP_OPTIONS=`get_svn_options $SUBCMD | tr "\n" " "`
  COMPLETION_OPTIONS=`get_svn_completions $SUBCMD - | tr " " "\n" | sort | tr "\n" " "`
  if [ "$HELP_OPTIONS" != "$COMPLETION_OPTIONS" ]; then
    fail "completions for \"svn $SUBCMD -\" != options accepted" \
	 "    (completions: $COMPLETION_OPTIONS)" \
         "    (svn accepts: $HELP_OPTIONS)"
  fi
done

echo "Checking rejection of synonyms"
excludes "diff -x -u -" "-x"
excludes "diff -x -u --e" "--extensions"
excludes "diff --extensions -u -" "--extensions"
excludes "diff --extensions -u -" "-x"
excludes "diff --extensions=-u -" "-x"

if [ $TESTS_FAILED ]; then
  echo "FAILURE: at least one bash_completion test failed."
else
  echo "All bash_completion tests passed."
fi

Re: [PATCH] bash_completion: improvement, fixes and tests [was: Re: subversion and programmable completion]

[Philip Martin]

Julian Foad <julianfoad@btopenworld.com> writes:

> # Print the valid subcommands for "svn", one per line, sorted.
> # Usage: get_svn_subcommands
> get_svn_subcommands() {
>   svn help |
>     # Find the relevant lines;
>     # remove brackets and commas; put each word on its own line.
>     sed -n -e '1,/^Available subcommands:$/d;/^$/q' \
>            -e 's/[ )]//g;s/[(,]/\n/g;p' |
>     sort
> }
[...]
> echo "Checking list of subcommands"
> HELP_SUBCMDS=`get_svn_subcommands | tr "\n" " "`

At this point "echo $HELP_SUBCMDS" gives

add cat checkoutnco cleanup commitnci copyncp deletendelnremovenrm diffndi export helpn?nh import info listnls log merge mkdir movenmvnrenamenren propdelnpdel propeditnpeditnpe propgetnpgetnpg proplistnplistnpl propsetnpsetnps resolve revert statusnstatnst switchnsw updatenup

$ sed -V | head -1
GNU sed version 3.02
$ echo 'x(y' | sed 's/[(,]/\n/g'
xny


-- 
Philip Martin

Re: [PATCH] bash_completion: improvement, fixes and tests [was: Re: subversion and programmable completion]

[Julian Foad]

[-- Attachment #1: Type: text/plain, Size: 2323 bytes --]

Philip Martin wrote:
> Julian Foad <julianfoad@btopenworld.com> writes:
> 
>>    # Find the relevant lines;
>>    # remove brackets and commas; put each word on its own line.
>>    sed -n -e '1,/^Available subcommands:$/d;/^$/q' \
>>           -e 's/[ )]//g;s/[(,]/\n/g;p' |
> 
> $ sed -V | head -1
> GNU sed version 3.02
> $ echo 'x(y' | sed 's/[(,]/\n/g'
> xny

Oh dear.  I don't know why you get that.

~> bash --version
GNU bash, version 2.05b.0(1)-release (i586-suse-linux)
~> sed --version | head -1
GNU sed version 3.02.80
~> echo 'x(y' | sed 's/[(,]/\n/g'
x
y

I get the same under bash, ash-0.2, tcsh-6.12 and zsh-4.0.6 so I don't suppose it's she shell stripping the backslash.  But just to check, try

~> echo 's/[(,]/\n/g' | hexdump -C
00000000  73 2f 5b 28 2c 5d 2f 5c  6e 2f 67 0a              |s/[(,]/\n/g.|

Work-arounds: these all work for me; I'd be interested to know which work for you and which don't.

1. Use "tr" instead of "sed"; unlike sed, its escape sequences are explicitly listed in its man page:
~> tr --version | head -1
tr (textutils) 2.1
~> echo 'x(y' | tr '(,' '\n'

2. insert a literal newline character after the backslash, instead of 'n'; this is explicitly listed in the manual page:
~> echo 'x(y' | sed 's/[(,]/\
/g'

3. Use numeric escape sequences; these are not mentioned in the manual but work for me:
~> echo 'x(y' | sed 's/[(,]/\o012/g'
~> echo 'x(y' | sed 's/[(,]/\d010/g'
~> echo 'x(y' | sed 's/[(,]/\x0A/g'


Attached is a version of bash_completion_test using "tr", which I think is the safest solution.  And to keep the patch together, the original bash_completion.diff is also attached and the log message is repeated here:

Log message:
[[[
Improve and fix bash_completion.
Add a script that tests bash_completion, including checking the completion
results against the output of "svn help [...]".

* tools/client-side/bash_completion
 (_svn) Recognise "--arg=value" as an alternative to "--arg value" (when
        excluding options or synonyms that are already present).
        Fix the list of subcommands and options accepted as a first argument.
        Fix the lists of options accepted by various subcommands.

* tools/client-side/bash_completion_test
 New file: a shell script that tests bash_completion.
]]]

Let me know if there are further problems.

- Julian


[-- Attachment #2: bash_completion.diff --]
[-- Type: text/plain, Size: 3441 bytes --]

Index: tools/client-side/bash_completion
===================================================================
--- tools/client-side/bash_completion	(revision 6521)
+++ tools/client-side/bash_completion	(working copy)
@@ -9,26 +9,26 @@
 
 _svn()
 {
-	local cur cmds cmdOpts pOpts mOpts rOpts qOpts nOpts optsParam opt
+	local cur cmds cmdOpts pOpts mOpts rOpts qOpts nOpts optsParam opt optBase
 
 	COMPREPLY=()
 	cur=${COMP_WORDS[COMP_CWORD]}
 
 	cmds='add cat checkout co cleanup commit ci copy cp delete del \
-	      remove rm diff di export import info h help list ls log merge \
+	      remove rm diff di export import info ? h help list ls log merge \
 	      mkdir move mv rename ren propdel pdel propedit pedit pe propget \
 	      pget pg proplist plist pl propset pset ps revert resolve status \
-	      stat st switch sw update up --version'
+	      stat st switch sw update up'
 
 	if [[ $COMP_CWORD -eq 1 ]] ; then
-		COMPREPLY=( $( compgen -W "$cmds" -- $cur ) )
+		COMPREPLY=( $( compgen -W "$cmds -h --help --version" -- $cur ) )
 		return 0
 	else
 		# if we're completing for 'svn help' or 'svn h', then just 
 		# complete on any valid svn command
 		case ${COMP_WORDS[1]} in
 		help|h|\?)
-			COMPREPLY=( $( compgen -W "$cmds" -- $cur ) )
+			COMPREPLY=( $( compgen -W "$cmds -q --quiet" -- $cur ) )
 			return 0
 			;;
 		*)
@@ -62,7 +62,7 @@
 		cmdOpts="$qOpts"
 		;;
 	add)
-		cmdOpts="--targets -$nOpts $qOpts"
+		cmdOpts="--targets $nOpts $qOpts"
 		;;
 	cat)
 		cmdOpts="$rOpts $pOpts"
@@ -89,7 +89,7 @@
 		cmdOpts="$rOpts $qOpts $pOpts --force"
 		;;
 	import)
-		cmdOpts="$mOpts $qOpts $nOpts --targets --editor-cmd $pOpts"
+		cmdOpts="$mOpts $qOpts $nOpts --editor-cmd $pOpts"
 		;; 
 	info)
 		cmdOpts="--targets -R --recursive"
@@ -99,7 +99,7 @@
 		;;
 	log)
 		cmdOpts="$rOpts -v --verbose --targets $pOpts --strict \
-		         --incremental --xml"
+		         --incremental --xml $qOpts"
 		;;
 	merge)
 		cmdOpts="$rOpts $nOpts $qOpts --force --dry-run --diff3-cmd \
@@ -115,7 +115,8 @@
 		cmdOpts="$qOpts -R --recursive $rOpts --revprop $pOpts"
 		;;
 	propedit|pedit|pe)
-		cmdOpts="$rOpts --revprop --encoding --editor-cmd $pOpts"
+		cmdOpts="$rOpts --revprop --encoding --editor-cmd $pOpts \
+		         --force"
 		;;
 	propget|pget|pg)
 		cmdOpts="-R --recursive $rOpts --revprop --strict $pOpts"
@@ -126,7 +127,7 @@
 		;;
 	propset|pset|ps)
 		cmdOpts="-F --file $qOpts --targets -R --recursive --revprop \
-		         --encoding $pOpts"
+		         --encoding $pOpts $rOpts --force"
 		;;
 	revert)
 		cmdOpts="--targets -R --recursive $qOpts"
@@ -139,10 +140,10 @@
 		         --no-ignore"
 		;;
 	switch|sw)
-		cmdOpts="--relocate $rOpts $nOpts $qOpts $pOpts"
+		cmdOpts="--relocate $rOpts $nOpts $qOpts $pOpts --diff3-cmd"
 		;;
 	update|up)
-		cmdOpts="$rOpts $nOpts $qOpts $pOpts"
+		cmdOpts="$rOpts $nOpts $qOpts $pOpts --diff3-cmd"
 		;;
 	*)
 		;;
@@ -151,11 +152,17 @@
 	# take out options already given
 	for (( i=2; i<=$COMP_CWORD-1; ++i )) ; do
 		opt=${COMP_WORDS[$i]}
+
+		case $opt in
+		--*)    optBase=${opt/=*/} ;;
+		-*)     optBase=${opt} ;;
+		esac
+
 		cmdOpts=" $cmdOpts "
-		cmdOpts=${cmdOpts/ ${opt} / }
+		cmdOpts=${cmdOpts/ ${optBase} / }
 
 		# take out alternatives
-		case $opt in
+		case $optBase in
 		-v)              cmdOpts=${cmdOpts/ --verbose / } ;;
 		--verbose)       cmdOpts=${cmdOpts/ -v / } ;;
 		-N)              cmdOpts=${cmdOpts/ --non-recursive / } ;;

[-- Attachment #3: bash_completion_test --]
[-- Type: text/plain, Size: 4150 bytes --]

#!/bin/bash
# Checks that the "_svn" function defined in the specified "bash_completion"
# script produces appropriate lists of completions for various incomplete svn
# command lines.

if [ ! -f "$1" ] || [ "$2" ]; then
  echo "Usage: bash_completion_test BASH_COMPLETION_PATHNAME"
  echo "Tests the specified \"bash_completion\" script,"
  echo "including checking it against the \"svn\" program found in the current PATH."
  exit 1
fi

set -e  # Exit on error
shopt -s extglob

# Execute the script which is to be tested.
. "$1"

# From the given incomplete svn command, print a space-separated list of
# possible completions of the last argument (or of an empty first argument
# if no subcommand is given).
# Usage: get_svn_completions [SVN-SUBCOMMAND [SVN-OPTION...]]
get_svn_completions() {
  COMP_WORDS=(svn "$@")
  if [ $# == 0 ]; then
    COMP_CWORD=1
  else
    COMP_CWORD=$#
  fi
  _svn
  echo -n "${COMPREPLY[*]}"
}

# Print a failure message, record the failure, and return "false".
# Usage: fail MESSAGE
fail() {
  PREFIX="FAIL: "
  for LINE in "$@"; do
    echo "$PREFIX$LINE"
    PREFIX="      "
  done
  TESTS_FAILED=1
  false
}

# Check that EXPECTED-WORD is among the completions of the last word in
# SVN-COMMAND.  SVN-COMMAND is a single argument to this function, split
# into multiple arguments when passed to "get_svn_completions()".
# Usage: includes SVN-COMMAND EXPECTED-WORD
includes() {
  COMPLETIONS=`get_svn_completions $1`
  if [[ "$2" != @(${COMPLETIONS// /|}) ]]; then
    fail "completions of \"svn $1\" should include \"$2\"" \
      "(completions: $COMPLETIONS)"
  fi
}

excludes() {
  COMPLETIONS=`get_svn_completions $1`
  if [[ "$2" == @(${COMPLETIONS// /|}) ]]; then
    fail "completions of \"svn $1\" should exclude \"$2\"" \
      "(completions: $COMPLETIONS)"
  fi
}

# Print the valid subcommands for "svn", one per line, sorted.
# Usage: get_svn_subcommands
get_svn_subcommands() {
  svn help |
    # Find the relevant lines.
    sed -n -e '1,/^Available subcommands:$/d;/^$/q;p' |
    # Remove brackets and commas; put each word on its own line.
    tr -d ' )' | tr '(,' '\n' |
    sort
}

# Print the valid option switches for "svn SUBCMD", one per line, sorted.
# Usage: get_svn_options SUBCMD
get_svn_options() {
  { svn help "$1" |
      # Find the relevant lines; remove "arg" and description.
      sed -n -e '1,/^Valid options:$/d;/^  -/!d' \
             -e 's/\( arg\)* * : .*//;p' |
      # Remove brackets; put each word on its own line.
      tr -d '] ' | tr '[' '\n'
    # The following options are always accepted but not listed in the help,
    # nor currently offered as completions.
    #echo "-h"
    #echo "--help"
  } | sort
  
}


# The tests.
set +e  # Do not exit on error
TESTS_FAILED=

echo "Checking general completion"
includes "he" "help"
includes "" "?"
includes "" "h"
includes "" "help"
includes "" "-h"
includes "" "--help"
includes "" "--version"

echo "Checking list of subcommands"
HELP_SUBCMDS=`get_svn_subcommands | tr "\n" " "`
COMPLETION_SUBCMDS=`get_svn_completions | tr " " "\n" | grep -v "^-" | sort | tr "\n" " "`
if [ "$HELP_SUBCMDS" != "$COMPLETION_SUBCMDS" ]; then
  fail "non-option completions for \"svn \" != subcommands accepted" \
       "    (non-o. cmpl.: $COMPLETION_SUBCMDS)" \
       "    (svn accepts:  $HELP_SUBCMDS)"
fi

echo "Checking list of options for each subcommand"
for SUBCMD in $HELP_SUBCMDS; do
  HELP_OPTIONS=`get_svn_options $SUBCMD | tr "\n" " "`
  COMPLETION_OPTIONS=`get_svn_completions $SUBCMD - | tr " " "\n" | sort | tr "\n" " "`
  if [ "$HELP_OPTIONS" != "$COMPLETION_OPTIONS" ]; then
    fail "completions for \"svn $SUBCMD -\" != options accepted" \
	 "    (completions: $COMPLETION_OPTIONS)" \
         "    (svn accepts: $HELP_OPTIONS)"
  fi
done

echo "Checking rejection of synonyms"
excludes "diff -x -u -" "-x"
excludes "diff -x -u --e" "--extensions"
excludes "diff --extensions -u -" "--extensions"
excludes "diff --extensions -u -" "-x"
excludes "diff --extensions=-u -" "-x"

if [ $TESTS_FAILED ]; then
  echo "FAILURE: at least one bash_completion test failed."
else
  echo "All bash_completion tests passed."
fi

Re: [PATCH] bash_completion: improvement, fixes and tests [was: Re: subversion and programmable completion]

[Philip Martin]

Julian Foad <julianfoad@btopenworld.com> writes:

> But just to > check, try
>
> ~> echo 's/[(,]/\n/g' | hexdump -C
> 00000000  73 2f 5b 28 2c 5d 2f 5c  6e 2f 67 0a              |s/[(,]/\n/g.|

$ echo 's/[(,]/\n/g' | hexdump -C
00000000  73 2f 5b 28 2c 5d 2f 5c  6e 2f 67 0a              |s/[(,]/\n/g.|
0000000c

> Work-arounds: these all work for me; I'd be interested to know which
> work for you and which don't.

Results below are for sed and tr from the stable Debian distribution.
Looking at the package documentation, the sed info pages and faq both
cover s/// but are worded slightly differently!

> 1. Use "tr" instead of "sed"; unlike sed, its escape sequences are
>    explicitly listed in its man page:
> ~> tr --version | head -1
> tr (textutils) 2.1
> ~> echo 'x(y' | tr '(,' '\n'

$ echo 'x(y' | tr '(,' '\n'
x
y

> 2. insert a literal newline character after the backslash, instead
>    of 'n'; this is explicitly listed in the manual page:
> ~> echo 'x(y' | sed 's/[(,]/\
> /g'

$ echo 'x(y' | sed 's/[(,]/\
/g'
x
y

> 3. Use numeric escape sequences; these are not mentioned in the
>    manual but work for me:
> ~> echo 'x(y' | sed 's/[(,]/\o012/g'

$ echo 'x(y' | sed 's/[(,]/\o012/g'
xo012y

> ~> echo 'x(y' | sed 's/[(,]/\d010/g'

$ echo 'x(y' | sed 's/[(,]/\d010/g'
xd010y

> ~> echo 'x(y' | sed 's/[(,]/\x0A/g'

$ echo 'x(y' | sed 's/[(,]/\x0A/g'
xx0Ay

> Attached is a version of bash_completion_test using "tr", which I
> think is the safest solution.  And to keep the patch together, the
> original bash_completion.diff is also attached and the log message
> is repeated here:

Thanks!  I checked in a slightly modified version in r6557.

-- 
Philip Martin

Re: [PATCH] bash_completion: improvement, fixes and tests [was: Re: subversion and programmable completion]

[Roman Neuhauser]

# julianfoad@btopenworld.com / 2003-07-24 02:29:41 +0100:
> Philip Martin wrote:
> >Julian Foad <julianfoad@btopenworld.com> writes:
> >
> >>   # Find the relevant lines;
> >>   # remove brackets and commas; put each word on its own line.
> >>   sed -n -e '1,/^Available subcommands:$/d;/^$/q' \
> >>          -e 's/[ )]//g;s/[(,]/\n/g;p' |
> >
> >$ sed -V | head -1
> >GNU sed version 3.02
> >$ echo 'x(y' | sed 's/[(,]/\n/g'
> >xny
> 
> Oh dear.  I don't know why you get that.

    because that's what it's supposed to do. read the sed description in
    SUSv3. (I'd quote the two paragraphs here but I fear breaking the
    ToS...) basically:

    * \n is not a valid replacement metacharacter
    * behavior of \X where X is not one of &, <digit>, <delimiter> or
      <newline> is undefined
    * if you want to embed a newline in the replacement, do it this way:

    sed 's/BRE/first part\
    second part/'

    Julian, looks like your sed is broken.
    I know I'm coming awfully late, but wanted to make this clear.

-- 
The From: header's been munged to get rid of unwanted cc's.
My real address: neuhauser@bellavista.cz.