BUG: Shell builtin `which` prints non-existent commands to stdout

BUG: Shell builtin `which` prints non-existent commands to stdout

[Klaus Alexander Seistrup]

TL;DR: When `which` does not find a command it prints the error to 
standard output instead of standard error. Exit code is set correctly, 
though, and error messages about ‘bad options’ are sent to standard 
error as expected.

In a nutshell:

```sh
$ echo $SHELL              # can you tell us a little about yourself?
/usr/bin/zsh               # ✓
$ type which               # please tell us what you know about `which`
which is a shell builtin   # ✓
$ which vi                 # can you find `vi`?
/usr/bin/vi                # ✓
$ echo $?                  # show us the exit code
0                          # ✓
$ which nxcmd              # tell me about a non-existent command
nxcmd not found            # ✓
$ echo $?                  # and let us see the exit code
1                          # ✓
$ which nxcmd 2>/dev/null  # this command ought to keep `which` silent, but …
nxcmd not found            # ✗
$ which nxcmd >/dev/null   # … the error message is sent to stdout
$                          # ✗
$ which -T                 # let's try an unrecognized option
which: bad option: -T      # ✓
$ which -T 2>/dev/null     # redirect the same to stderr
$                          # ✓
```

Expected behaviour, as illustrated by GNU `which`:

```sh
$ /usr/bin/which --version | head -1
GNU which v2.21, Copyright (C) 1999 - 2015 Carlo Wood.
$ /usr/bin/which nxcmd 2>/dev/null
$ echo $?
1
```

Interim solution:

```sh
# ~/.zshrc
alias which='command which'
```

OS, architecture and zsh version:

```sh
$ uname -a
Linux nakamu 4.18.9 #1 SMP PREEMPT Wed Sep 19 21:19:17 UTC 2018 x86_64 GNU/Linux
$ echo $ZSH_VERSION
5.6.2
```

Cheers,

-- 
Klaus Alexander Seistrup
https://klaus.seistrup.dk/

Re: BUG: Shell builtin `which` prints non-existent commands to stdout

[Peter Stephenson]

On Mon, 24 Sep 2018 10:00:31 +0200
Klaus Alexander Seistrup <klaus@seistrup.dk> wrote:
> TL;DR: When `which` does not find a command it prints the error to 
> standard output instead of standard error. Exit code is set correctly, 
> though, and error messages about ‘bad options’ are sent to standard 
> error as expected.

This is not a bug and has been discussed numerous times, e.g. the thread
that starts at.

http://www.zsh.org/mla/workers//2015/msg02253.html

(official zsh-workers message number 36330).

We definitely don't wish to change this now.

pws

Re: BUG: Shell builtin `which` prints non-existent commands to stdout

[Peter Stephenson]

On Mon, 24 Sep 2018 11:22:18 +0100
Peter Stephenson <p.stephenson@samsung.com> wrote:
> On Mon, 24 Sep 2018 10:00:31 +0200
> Klaus Alexander Seistrup <klaus@seistrup.dk> wrote:
> > TL;DR: When `which` does not find a command it prints the error to 
> > standard output instead of standard error. Exit code is set correctly, 
> > though, and error messages about ‘bad options’ are sent to standard 
> > error as expected.  
> 
> This is not a bug and has been discussed numerous times, e.g. the thread
> that starts at.
> 
> http://www.zsh.org/mla/workers//2015/msg02253.html
> 
> (official zsh-workers message number 36330).
> 
> We definitely don't wish to change this now.

(Having said that, I don't really care about things pushed
behind POSIX_BUILTINS...  Despite previous suggestions, I don't
think changing default zsh behaviour at this stage is the right
way out.  I buy Bart's argument that expecting which to have
no output is a bit weird.)

pws

Re: BUG: Shell builtin `which` prints non-existent commands to stdout

[Klaus Alexander Seistrup]

Peter Stephenson wrote:

> This is not a bug and has been discussed numerous times

Thanks, I hadn't see that. I'll just keep the `which` alias, then.

Cheers,

-- 
Klaus Alexander Seistrup
https://klaus.seistrup.dk/

Re: BUG: Shell builtin `which` prints non-existent commands to stdout

[Peter Stephenson]

On Mon, 24 Sep 2018 14:29:33 +0200
Klaus Alexander Seistrup <klaus@seistrup.dk> wrote:
> Peter Stephenson wrote:
> 
> > This is not a bug and has been discussed numerous times  
> 
> Thanks, I hadn't see that. I'll just keep the `which` alias, then.

Might be useful to make the issue more visible...

diff --git a/Etc/FAQ.yo b/Etc/FAQ.yo
index a3dfc6c..e15e024 100644
--- a/Etc/FAQ.yo
+++ b/Etc/FAQ.yo
@@ -127,6 +127,7 @@ Chapter 3:  How to get various things to work
 3.26. Why is my output duplicated with `tt(foo 2>&1 >foo.out | bar)'?
 3.27. What are these `^' and `~' pattern characters, anyway?
 3.28. How do I edit the input buffer in $EDITOR?
+3.29. Why does `which' output for missing commands go to stdout?
 
 Chapter 4:  The mysteries of completion
 4.1. What is completion?
@@ -1964,6 +1965,34 @@ label(328)
   quitting the editor will only return to zsh's command-line editing mode.
 
 
+sect(Why does `which' output for missing commands go to stdout?)
+
+  The issue is that if you run:
+  verb(
+    which non-existent-command
+  )
+  the error message goes, unusually, to standard output rather than
+  to standard error.  Other shells send this message to standard error,
+  as they would if the command was about to be executed but could not be
+  found.
+
+  The original reason for this is that this behaviour is inherited
+  from the C shell (csh), where `tt(which)' itself orignated.  So
+  it has been in zsh a very long time, and it is now a feature.
+  (It would be possible to change this in emulation modes; however.
+  so far this possibility has been seen has more of an additional
+  confusion than a help.)
+
+  If you want some further rationalisation, which may be what the C
+  shell designers had in mind, you might note that `tt(which)' is
+  designed as a way of outputing information about a command.  So
+  `this command can be found in ...' and `this command can't be found'
+  are both bits of information here, unlike the case where the command
+  is to be executed.  So although it differs from other Bourne-style
+  shells it is in fact self-consistent.  Note that the status does
+  reflect the fact the command can't be found.
+
+
 chapter(The mysteries of completion)
 
 

[PATCH] zshbuiltins(1): Document 'which''s "not found is not an error" behaviour.

[Daniel Shahaf]

---
 Doc/Zsh/builtins.yo | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/Doc/Zsh/builtins.yo b/Doc/Zsh/builtins.yo
index 76b593fd2..0141305b4 100644
--- a/Doc/Zsh/builtins.yo
+++ b/Doc/Zsh/builtins.yo
@@ -2373,6 +2373,12 @@ item(tt(whence) [ tt(-vcwfpamsS) ] [ tt(-x) var(num) ] var(name) ...)(
 For each var(name), indicate how it would be interpreted if used as a
 command name.
 
+If var(name) is not an alias, built-in command, external command, shell
+function, hashed command, or a reserved word, the exit status shall be
+non-zero, and DASH()- if tt(-v), tt(-c), or tt(-w) was passed DASH()- a message
+will be written to standard output.  (This is different from other shells that
+write that message to standard error.)
+
 tt(whence) is most useful when var(name) is only the last path component
 of a command, i.e. does not include a `tt(/)'; in particular, pattern
 matching only succeeds if just the non-directory component of the command is

Re: BUG: Shell builtin `which` prints non-existent commands to stdout

[Stephane Chazelas]

2018-09-24 13:51:24 +0100, Peter Stephenson:
> On Mon, 24 Sep 2018 14:29:33 +0200
[...]
> +  The original reason for this is that this behaviour is inherited
> +  from the C shell (csh), where `tt(which)' itself orignated.  So
> +  it has been in zsh a very long time, and it is now a feature.
> +  (It would be possible to change this in emulation modes; however.
> +  so far this possibility has been seen has more of an additional
> +  confusion than a help.)
[...]

csh had no "which" (tcsh has a which builtin), but there was a
"which" csh script added to 3BSD 1980 that was looking up
commands in $PATH and in the aliases defined through ~/.cshrc.
https://github.com/dspinellis/unix-history-repo/blob/BSD-3/usr/ucb/which

There's no easy way to write something to stderr in csh, so it's
not surprising that that script didn't.

In any case, the "command not found" can be seen as the normal
output of which, it is *the* information that it is returning
for that command.

zsh is not the only shell for which "type not-a-command" doesn't
output "command not found" on stderr. The Bourne shell, pdksh
and derivatives and ash-based shells do as well.

-- 
Stephane

Re: BUG: Shell builtin `which` prints non-existent commands to stdout

[Stephane Chazelas]

2018-09-24 13:51:24 +0100, Peter Stephenson:
> On Mon, 24 Sep 2018 14:29:33 +0200
[...]
> +  The issue is that if you run:
> +  verb(
> +    which non-existent-command
> +  )
> +  the error message goes, unusually, to standard output rather than
> +  to standard error.  Other shells send this message to standard error,
> +  as they would if the command was about to be executed but could not be
> +  found.
[...]

About "other shells" above, AFAIK, tcsh is the only other shell
that has a "which" builtin and it does also send "command not
found" to stdout. So zsh behaves like every other shells in that
regard.

To phrase my last comment differently, there are also a variety
of "which" command implementations outside of any shell, the
first of which (which appeared in 3BSD) was primarily intended
for csh users (as it read your ~/.cshrc for aliases) and was
written as a csh script that did also send the "command not
found" to stdout.

That's also the one still found on many commercial Unices.

There is also a GNU "which" command (which tries to generalise
what that csh script did for other shells) and sends the
command-not-found to stderr.

Debian has a POSIX sh which script that doesn't output any
command-not-found error and doesn't look-up aliases of any
shell (it writes the usage message on stdout followed by the
"illegal option" on stderr).

BSDs which has now been merged with whereis and been rewritten in
C and no longer looks up your csh aliases and like the Debian
script doesn't output any command-not-found error.

See also
https://unix.stackexchange.com/questions/85249/why-not-use-which-what-to-use-then

-- 
Stephane

Re: BUG: Shell builtin `which` prints non-existent commands to stdout

[Stephane Chazelas]

2018-09-24 23:18:20 +0100, Stephane Chazelas:
> 2018-09-24 13:51:24 +0100, Peter Stephenson:
> > On Mon, 24 Sep 2018 14:29:33 +0200
> [...]
> > +  The original reason for this is that this behaviour is inherited
> > +  from the C shell (csh), where `tt(which)' itself orignated.  So
> > +  it has been in zsh a very long time, and it is now a feature.
> > +  (It would be possible to change this in emulation modes; however.
> > +  so far this possibility has been seen has more of an additional
> > +  confusion than a help.)
> [...]
> 
> csh had no "which" (tcsh has a which builtin), but there was a
> "which" csh script added to 3BSD 1980 that was looking up
> commands in $PATH and in the aliases defined through ~/.cshrc.
> https://github.com/dspinellis/unix-history-repo/blob/BSD-3/usr/ucb/which
[...]

Sorry, correction: "which" was added to csh in 4.4BSD (1991) and
various other BSDs that had already forked by then later (NetBSD
in 1994 for instance, and FreeBSD has switched to tcsh IIRC),
though csh on some systems like Solaris are based on older
versions and still don't have "which" builtin.

-- 
Stephane

Re: BUG: Shell builtin `which` prints non-existent commands to stdout

[Peter Stephenson]

On Mon, 24 Sep 2018 23:32:53 +0100
Stephane Chazelas <stephane.chazelas@gmail.com> wrote:
> About "other shells" above, AFAIK, tcsh is the only other shell
> that has a "which" builtin and it does also send "command not
> found" to stdout. So zsh behaves like every other shells in that
> regard.

Thanks, I've updated the text.

pws

diff --git a/Etc/FAQ.yo b/Etc/FAQ.yo
index 9f634d1..81d7628 100644
--- a/Etc/FAQ.yo
+++ b/Etc/FAQ.yo
@@ -1976,21 +1976,22 @@ sect(Why does `which' output for missing commands go to stdout?)
   as they would if the command was about to be executed but could not be
   found.
 
-  The original reason for this is that this behaviour is inherited
-  from the C shell (csh), where `tt(which)' itself originated.  So
-  it has been in zsh a very long time, and it is now a feature.
-  (It would be possible to change this in emulation modes; however.
-  so far this possibility has been seen has more of an additional
-  confusion than a help.)
-
-  If you want some further rationalisation, which may be what the C
-  shell designers had in mind, you might note that `tt(which)' is
-  designed as a way of outputting information about a command.  So
-  `this command can be found in ...' and `this command can't be found'
-  are both bits of information here, unlike the case where the command
-  is to be executed.  So although it differs from other Bourne-style
-  shells it is in fact self-consistent.  Note that the exit status does
-  reflect the fact the command can't be found.
+  The original reason for this is that this behaviour is inherited from
+  previous versions of `tt(which)', a builtin in tcsh (an adaptation of
+  the C Shell with better editing) and also a separate script.  Other
+  shells had equivalent commands, `tt(whence)' and `tt(type), that zsh
+  has also adopted.  So in fact this has always been a feature of
+  `tt(which)'.  (It would be possible to change this in emulation modes;
+  however.  so far this possibility has been seen has more of an
+  additional confusion than a help.)
+
+  If you want some further rationalisation, you might note that
+  `tt(which)' is designed as a way of outputting information about a
+  command.  So `this command can be found in ...' and `this command
+  can't be found' are both bits of information here, unlike the case
+  where the command is to be executed.  So although it differs from
+  other Bourne-style shells it is in fact self-consistent.  Note that
+  the exit status does reflect the fact the command can't be found.
 
 
 chapter(The mysteries of completion)