Unexpected foo==bar errors

Unexpected foo==bar errors

[Mikael Magnusson]

% foo==bar
zsh: bar not found

This happens even with globassign and magicequalsubst turned off.

A comparison,

% foo='*bar'
% baz=$~foo
% echo $baz
*bar

% foo='=bar'
% baz=$~foo
zsh: bar not found

And how I happened upon this:
% tar tf =(foo) <tab>
_tar:70:  not found


I always thought of EQUALS expansion as a form of globbing; should it
be enabled in these contexts even though regular globbing isn't?
(
_tar:70 is
tf=${~words[3]}
)

-- 
Mikael Magnusson

Re: Unexpected foo==bar errors

[Peter Stephenson]

On Sun, 7 Feb 2016 16:57:06 +0100
Mikael Magnusson <mikachu@gmail.com> wrote:
> % foo==bar
> zsh: bar not found
> 
> This happens even with globassign and magicequalsubst turned off.

The option is called "equals".  "magicequalsubst" is now redundant; it
allowed

typeset foo=<whatever>

to exapnd <whatever>, which would include =bar if "equals" was also set.
It's redundant because the reserved word interface does that anyway, and
in other shells the equivalent of the non-reserved-word interface
doesn't do it either.

pws

Re: Unexpected foo==bar errors

[Bart Schaefer]

On Feb 7,  4:57pm, Mikael Magnusson wrote:
}
} I always thought of EQUALS expansion as a form of globbing; should it
} be enabled in these contexts even though regular globbing isn't?

It's expansion, like with a leading tilde, not globbing, so GLOB_ASSIGN
doesn't apply here at all.

MAGIC_EQUAL_SUBST has never controlled expansion occurring in parameter
assignments, it only controls whether expansion occurs in normal command
arguments that LOOK LIKE assignments.

torch% bar==echo
torch% print $bar
/bin/echo
torch% print foo==echo
foo==echo
torch% setopt magicequalsubst
torch% print foo==echo       
foo=/bin/echo
torch% setopt no_equals
torch% print foo==echo    
foo==echo
torch% bar==echo
torch% print $bar
=echo
torch% 

} _tar:70 is
} tf=${~words[3]}

Ooh, that's a fun one: (foo) is being taken as glob qualifiers, which is
a larger problem in this case than the expansion of "=".  Then because
globassign is off, the equals expansion (on the empty string) is done,
but the qualifiers are never applied to the result.

If you "setopt shfileexpansion" so that equals applies after globbing
instead of before, the assignment behaves more sensibly, but:

The only use of $tf as far as I can see is to do lookups in the cache.
The apparent intent is to canonicalize the cache name to avoid "tar -tf"
of the same tar file more than once.  That optimization may not be worth
the buggy attempt at expansion, and caching the result of =(command) may
not be the best idea in the first place.

I'm not sure exactly what fix to suggest.

Re: Unexpected foo==bar errors

[Mikael Magnusson]

On Sun, Feb 7, 2016 at 8:16 PM, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Feb 7,  4:57pm, Mikael Magnusson wrote:
> }
> } I always thought of EQUALS expansion as a form of globbing; should it
> } be enabled in these contexts even though regular globbing isn't?
>
> It's expansion, like with a leading tilde, not globbing, so GLOB_ASSIGN
> doesn't apply here at all.

Ah yes, you're right; I also get errors if I do
% tar tf ~blurg <tab>
_tar:70: no such user or named directory: blurg

> MAGIC_EQUAL_SUBST has never controlled expansion occurring in parameter
> assignments, it only controls whether expansion occurs in normal command
> arguments that LOOK LIKE assignments.
[...]
> } _tar:70 is
> } tf=${~words[3]}
>
> Ooh, that's a fun one: (foo) is being taken as glob qualifiers, which is
> a larger problem in this case than the expansion of "=".  Then because
> globassign is off, the equals expansion (on the empty string) is done,
> but the qualifiers are never applied to the result.
>
> If you "setopt shfileexpansion" so that equals applies after globbing
> instead of before, the assignment behaves more sensibly, but:
>
> The only use of $tf as far as I can see is to do lookups in the cache.
> The apparent intent is to canonicalize the cache name to avoid "tar -tf"
> of the same tar file more than once.  That optimization may not be worth
> the buggy attempt at expansion, and caching the result of =(command) may
> not be the best idea in the first place.

Presumably at some point the given filename has to be interpreted so
that it can give any completions at all, even if it's not cached? I
didn't look closely at the rest of the completer though.

> I'm not sure exactly what fix to suggest.

I wonder if there's anywhere in the completion system that depends on
EQUALS being set... But maybe we should just 2> /dev/null the
assignment since tilde expansion also prints errors. (_zattr and _zip
also have this issue, presumably there are other places as well).

-- 
Mikael Magnusson

Re: Unexpected foo==bar errors

[Bart Schaefer]

On Feb 7,  7:06pm, Peter Stephenson wrote:
} Subject: Re: Unexpected foo==bar errors
}
} The option is called "equals".  "magicequalsubst" is now redundant; it
} allowed
} 
} typeset foo=<whatever>
} 
} to exapnd <whatever>

Hmm, as far as I can tell "typeset" always applied expansion in that
case, even when it was only a builtin; magicequalsubst is there for
other commands that take long options like --file=path or arguments
that have key=value format.

It's certainly not redundant.

Re: Unexpected foo==bar errors

[Peter Stephenson]

On Sun, 07 Feb 2016 23:09:19 -0800
Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Feb 7,  7:06pm, Peter Stephenson wrote:
> } Subject: Re: Unexpected foo==bar errors
> }
> } The option is called "equals".  "magicequalsubst" is now redundant; it
> } allowed
> } 
> } typeset foo=<whatever>
> } 
> } to exapnd <whatever>
> 
> Hmm, as far as I can tell "typeset" always applied expansion in that
> case, even when it was only a builtin; magicequalsubst is there for
> other commands that take long options like --file=path or arguments
> that have key=value format.
> 
> It's certainly not redundant.

You're right about "magicequalsubst".

I was attempting to describe "kshtypeset", but actually that only
applies to word splitting, not to expansion immediately after the "=".
I'd forgotten the extent to which the full assignment behaviour had
crept up by stealth.

Apart from that...

pws

Re: Unexpected foo==bar errors

[Bart Schaefer]

On Feb 8,  4:52am, Mikael Magnusson wrote:
}
} On Sun, Feb 7, 2016 at 8:16 PM, Bart Schaefer <schaefer@brasslantern.com> wrote:
} [...]
} > } _tar:70 is
} > } tf=${~words[3]}
} >
} > Ooh, that's a fun one: (foo) is being taken as glob qualifiers, which is
} > a larger problem in this case than the expansion of "=".  Then because
} > globassign is off, the equals expansion (on the empty string) is done,
} > but the qualifiers are never applied to the result.

One potentially open question here is whether the value should be parsed
as a glob (qualifiers recognized) when no globbing is possible.

} > The only use of $tf as far as I can see is to do lookups in the cache.
} 
} Presumably at some point the given filename has to be interpreted so
} that it can give any completions at all, even if it's not cached?

On closer inspection yes, I'm wrong about this, $tf is passed to "tar -tf"
to list the contents of the tar file even if not found in the cache.

} I wonder if there's anywhere in the completion system that depends on
} EQUALS being set... But maybe we should just 2> /dev/null the
} assignment since tilde expansion also prints errors. (_zattr and _zip
} also have this issue, presumably there are other places as well).

Redirecting error would still leave $tf empty.  But there's no way to
cause process substitution [that is, <(...) >(...) and =(...) syntax]
to be applied to the result of parameter expansion, except by using
"eval" -- and of course globbing won't happen without GLOB_ASSIGN, so
the only reason for tf=${~words[3]} (and the other assignments in _tar
from lines 66-82) is to do tilde and equals expansion.

Re: Unexpected foo==bar errors

[Mikael Magnusson]

On Mon, Feb 8, 2016 at 7:45 PM, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Feb 8,  4:52am, Mikael Magnusson wrote:
> } > The only use of $tf as far as I can see is to do lookups in the cache.
> }
> } Presumably at some point the given filename has to be interpreted so
> } that it can give any completions at all, even if it's not cached?
>
> On closer inspection yes, I'm wrong about this, $tf is passed to "tar -tf"
> to list the contents of the tar file even if not found in the cache.
>
> } I wonder if there's anywhere in the completion system that depends on
> } EQUALS being set... But maybe we should just 2> /dev/null the
> } assignment since tilde expansion also prints errors. (_zattr and _zip
> } also have this issue, presumably there are other places as well).
>
> Redirecting error would still leave $tf empty.  But there's no way to
> cause process substitution [that is, <(...) >(...) and =(...) syntax]
> to be applied to the result of parameter expansion, except by using
> "eval" -- and of course globbing won't happen without GLOB_ASSIGN, so
> the only reason for tf=${~words[3]} (and the other assignments in _tar
> from lines 66-82) is to do tilde and equals expansion.

Well, considering that something simple as foo*.tar (expanding only to
one file) also leaves $tf empty, I'd say it's better to leave $tf
empty for any weird command substitutions as well, but suppress the
error messages.

-- 
Mikael Magnusson

Failed process substitution on x=$~y (Re: Unexpected foo==bar errors)

[Bart Schaefer]

On Feb 8,  8:16pm, Mikael Magnusson wrote:
} Subject: Re: Unexpected foo==bar errors
}
} On Mon, Feb 8, 2016 at 7:45 PM, Bart Schaefer <schaefer@brasslantern.com> wrote:
} > On Feb 8,  4:52am, Mikael Magnusson wrote:
} > } I wonder if there's anywhere in the completion system that depends on
} > } EQUALS being set... But maybe we should just 2> /dev/null the
} > } assignment since tilde expansion also prints errors. (_zattr and _zip
} > } also have this issue, presumably there are other places as well).

Looks like stderr is already redirected in _zattr.

} Well, considering that something simple as foo*.tar (expanding only to
} one file) also leaves $tf empty, I'd say it's better to leave $tf
} empty for any weird command substitutions as well, but suppress the
} error messages.

The following all use ${~param} in places where an error could occur if
the value looks like a process substitution:

   Completion/Base/Utility/_arguments
   Completion/Base/Widget/_correct_filename
   Completion/Base/Widget/_most_recent_file
   Completion/Unix/Command/_devtodo
   Completion/Unix/Command/_make
   Completion/Unix/Command/_ssh
   Completion/Unix/Command/_tar
   Completion/Unix/Command/_zip
   Completion/Unix/Type/_canonical_paths

If for some reason one used a process substitution to locate the muttrc
file, Completion/Unix/Type/_mailboxes might also have that issue, but I
think that's wildly unlikely.

Some of these might warrant a closer look to see if it's a larger issue
to proceed with the resulting empty string.


diff --git a/Completion/Base/Utility/_arguments b/Completion/Base/Utility/_arguments
index 87fb20e..687c0c4 100644
--- a/Completion/Base/Utility/_arguments
+++ b/Completion/Base/Utility/_arguments
@@ -36,7 +36,7 @@ if (( long )); then
 
   tmpargv=( "${(@)argv[1,long-1]}" )  # optspec's before --, if any
 
-  name=${~words[1]}
+  name=${~words[1]} 2>/dev/null
   [[ "$name" = [^/]*/* ]] && name="$PWD/$name"
 
   name="_args_cache_${name}"
diff --git a/Completion/Base/Widget/_correct_filename b/Completion/Base/Widget/_correct_filename
index 7431a48..3150ffc 100644
--- a/Completion/Base/Widget/_correct_filename
+++ b/Completion/Base/Widget/_correct_filename
@@ -28,7 +28,7 @@ fi
 
 if [[ $file = \~*/* ]]; then
   tilde=${file%%/*}
-  etilde=${~tilde}
+  etilde=${~tilde} 2>/dev/null
   file=${file/#$tilde/$etilde}
 fi
 
diff --git a/Completion/Base/Widget/_most_recent_file b/Completion/Base/Widget/_most_recent_file
index 68d1c91..e72cf5e 100644
--- a/Completion/Base/Widget/_most_recent_file
+++ b/Completion/Base/Widget/_most_recent_file
@@ -11,7 +11,7 @@
 local file tilde etilde
 if [[ $PREFIX = \~*/* ]]; then
   tilde=${PREFIX%%/*}
-  etilde=${~tilde}
+  etilde=${~tilde} 2>/dev/null
   # PREFIX and SUFFIX have full command line quoting in, but we want
   # any globbing characters which are quoted to stay quoted.
   eval "file=($PREFIX*$SUFFIX(om[${NUMERIC:-1}]N))"
diff --git a/Completion/Unix/Command/_devtodo b/Completion/Unix/Command/_devtodo
index 2800f4a..dbc64f8 100644
--- a/Completion/Unix/Command/_devtodo
+++ b/Completion/Unix/Command/_devtodo
@@ -9,7 +9,7 @@ typeset expl
 
 for ((i=2; i <= $#words; i++)) {
 	if [[ $words[$i] == '--database' ]]; then
-		todo_opts+=(--database ${~words[$((++i))]})
+		todo_opts+=(--database ${~words[$((++i))]}) 2>/dev/null
 	elif [[ $words[$i] == '--sort' ]]; then
 		todo_opts+=(--sort ${words[$((++i))]})
 	fi
diff --git a/Completion/Unix/Command/_make b/Completion/Unix/Command/_make
index 48befa7..c3f3961 100644
--- a/Completion/Unix/Command/_make
+++ b/Completion/Unix/Command/_make
@@ -129,7 +129,7 @@ _make-findBasedir () {
   do
     if [[ $@[index] == -C ]]
     then
-      file=${~@[index+1]};
+      file=${~@[index+1]} 2>/dev/null
       if [[ -z $file ]]
       then
         # make returns with an error if an empty arg is given
diff --git a/Completion/Unix/Command/_ssh b/Completion/Unix/Command/_ssh
index c6ede9e..1f8f62c 100644
--- a/Completion/Unix/Command/_ssh
+++ b/Completion/Unix/Command/_ssh
@@ -622,7 +622,7 @@ _ssh_hosts () {
       ${opt_args[-l]:+"users=${opt_args[-l]:q}"} hosts "$@" && return
   fi
   if (( ind = ${words[(I)-F]} )); then
-    config=${~words[ind+1]}
+    config=${~words[ind+1]} 2>/dev/null
   else
     config="$HOME/.ssh/config"
   fi
diff --git a/Completion/Unix/Command/_tar b/Completion/Unix/Command/_tar
index 1e99ac0..4a24048 100644
--- a/Completion/Unix/Command/_tar
+++ b/Completion/Unix/Command/_tar
@@ -79,7 +79,7 @@ else
     tf=${~words[tmp+1]}
     _tar_cmd="f$_tar_cmd"
   fi
-fi
+fi 2>/dev/null
 
 # See if we should use a path prefix.  We have to use eval as the dir can
 # be any unevaluated thing which appears on the command line, including a
diff --git a/Completion/Unix/Command/_zip b/Completion/Unix/Command/_zip
index 171daf0..1040fa9 100644
--- a/Completion/Unix/Command/_zip
+++ b/Completion/Unix/Command/_zip
@@ -120,7 +120,7 @@ case $state in
 	zipfile=$testfile.ZIP
       else
 	return 1
-      fi
+      fi 2>/dev/null
       if [[ $zipfile !=  $_zip_cache_name ]]; then
 	_zip_cache_name="$zipfile"
 	_zip_cache_list=( ${(f)"$(zipinfo -1 $_zip_cache_name)"} )
diff --git a/Completion/Unix/Type/_canonical_paths b/Completion/Unix/Type/_canonical_paths
index d23b913..e4a725b 100644
--- a/Completion/Unix/Type/_canonical_paths
+++ b/Completion/Unix/Type/_canonical_paths
@@ -61,7 +61,7 @@ _canonical_paths_get_canonical_path() {
 _canonical_paths_add_paths () {
   local origpref=$1 expref rltrim curpref canpref subdir
   [[ $2 != add ]] && matches=()
-  expref=${~origpref}
+  expref=${~origpref} 2>/dev/null
   [[ $origpref == (|*/). ]] && rltrim=.
   curpref=${${expref%$rltrim}:-./}
   if zstat $curpref >&/dev/null; then