Case Sensitivity Bug?

Case Sensitivity Bug?

[Sean B. Palmer]

zsh is doing something very strange on OS X:

$ zsh -c 'echo test; which head'
test
/opt/local/bin/head

$ zsh -c 'date; which head'
Tue Dec 16 13:57:00 GMT 2008
/usr/bin/head

The problem centres around the case-sensitivity of HFS+, the OS X
filesystem. I have two head binaries, one called "head", the standard
unix head, and one called "HEAD", part of perl's lwp library:

$ which head
/usr/bin/head

$ which HEAD
/opt/local/bin/HEAD

As you can see from the echo and date commands, my problem is that
invoking echo first in the shell (or invoking nothing at all) causes
which to work in a case insensitive manner. When date is invoked first
instead, which works in a case sensitive manner.

Why is this so? And how does zsh manage to implement a case sensitive
which at all? Neither sh and bash on OS X have a case sensitive which.
I'm using OS X 10.4.11 (Tiger).

The #zsh channel on Freenode suggested I try using -x for extra
diagnostics, but that didn't help. Someone else asked me to check
$PATH to see if it was modified before or after any command; it is
not. In fact, the whole ENV seems static, so I don't think it's any
ENV related problem.

Some more information that might help you to help me:

$ zsh -c 'echo $ZSH_VERSION'
4.2.3

$ zsh -c 'which head'
/opt/local/bin/head

$ zsh -c 'which HEAD'
/opt/local/bin/HEAD

$ zsh -c 'time echo test; which head'
test
/opt/local/bin/head

$ zsh -c 'time date; which head'
Tue Dec 16 14:02:17 GMT 2008
real    0.008s
user    0.001s
sys     0.004s
cpu     64%
/usr/bin/head

$ zsh -c 'cd ..; which head'
/opt/local/bin/head

$ zsh -c 'ls -ald .; which head'
drwxr-xr-x   196 sbp  sbp  6664 Dec 15 21:36 .
/usr/bin/head

The pattern seems to be that any builtin invocation before which means
that which behaves case insensitively. When a non-builtin is invoked,
then which behaves case sensitively. The sensitivity only depends on
the very first invocation:

$ zsh -c 'which head; date; which head'
/opt/local/bin/head
Tue Dec 16 14:04:52 GMT 2008
/opt/local/bin/head

The context of this bug report is the following thread on MacPorts:

https://trac.macports.org/ticket/17620

As you can see, there's some debate there as to what is *right*;
whether which should or should not be case sensitive on OS X. The main
question for me is how which manages to be case sensitive at all,
since obviously I want to continue using both my head and HEAD
binaries in the normal manner, as I have been doing.

(My .zshrc includes a call to "date", and I don't tend to use any
other shell, so I've never noticed the case sensitivity problem before
until it came up with python_select in this MacPorts bug.)

In other words, if zsh can be case sensitive on OS X then so,
presumably, can MacPorts using the same mechanism. This will fix my
python_select bug more satisfactorily than having to remove the "HEAD"
binary.

Thanks,

-- 
Sean B. Palmer, http://inamidst.com/sbp/

Re: Case Sensitivity Bug?

[Peter Stephenson]

"Sean B. Palmer" wrote:
> zsh is doing something very strange on OS X:
> 
> $ zsh -c 'echo test; which head'
> test
> /opt/local/bin/head
> 
> $ zsh -c 'date; which head'
> Tue Dec 16 13:57:00 GMT 2008
> /usr/bin/head
> 
> The problem centres around the case-sensitivity of HFS+, the OS X
> filesystem. I have two head binaries, one called "head", the standard
> unix head, and one called "HEAD", part of perl's lwp library:
> 
> $ which head
> /usr/bin/head
> 
> $ which HEAD
> /opt/local/bin/HEAD
> 
> As you can see from the echo and date commands, my problem is that
> invoking echo first in the shell (or invoking nothing at all) causes
> which to work in a case insensitive manner. When date is invoked first
> instead, which works in a case sensitive manner.
> 
> Why is this so?

I think it's down to command hashing.  In the second case "date" is an
external command and zsh will fill its hash of external commands as it
searches the path to find date.  The which command then relies on this.
In this case the shell is comparing the command you searched for against
an internal string, which is case sensitive.  A corollary is likely to
be that command completion is case sensitive by default (though
completion is powerful enough for you to fix this fairly generally),
since this uses the hash, too.

In the first case, there's no hash; "which" does a blind search of the
path for <path_element>/head by trying to access the file and
/opt/local/bin/HEAD matches because the comparison is done in the file
system driver.

This is inconsistent, so regardless of what the right answer for case
sensitivity is, this isn't it.  Command hashing isn't really appropriate
if searching is done case insensitively, at least not without a flag
saying the command is stored on a case insensitive system, which the
shell doesn't know.  Hence the easiest fix (without actually looking at
the code) is likely to be to make the code fully case insensitive by
adding a string comparison in the case where which searches the path
itself.

> And how does zsh manage to implement a case sensitive
> which at all?

Your file system is case preserving as well as case insensitive, so a
file name read from the disk will have a certain format which the shell
will use when it does string comparisons.

> Neither sh and bash on OS X have a case sensitive which.
> I'm using OS X 10.4.11 (Tiger).

So they're not doing that.  There's an argument that command hashing is
unnecessary on modern systems anyway, although the way it's wired into
the completion system means this is a little different in zsh from most
shells.


A trivial way of guaranteeing that "which" is case insensitive is to
use pattern matching.  This would remain even if in future we guaranteed
that the default string match was case sensitive.

  % setopt extendedglob
  % which -m '(#i)head'
  /usr/bin/HEAD
  /usr/bin/head

(here on Fedora 8, hence file system is actually case sensitive).

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070

Re: Case Sensitivity Bug?

[Sean B. Palmer]

On Tue, Dec 16, 2008 at 2:40 PM, Peter Stephenson <pws@csr.com> wrote:

> I think it's down to command hashing.

If that were so, wouldn't this work?

$ zsh -c 'hash -r; which head'
/opt/local/bin/head

-- 
Sean B. Palmer, http://inamidst.com/sbp/

Re: Case Sensitivity Bug?

[Peter Stephenson]

"Sean B. Palmer" wrote:
> On Tue, Dec 16, 2008 at 2:40 PM, Peter Stephenson <pws@csr.com> wrote:
> 
> > I think it's down to command hashing.
> 
> If that were so, wouldn't this work?
> 
> $ zsh -c 'hash -r; which head'
> /opt/local/bin/head

I've completely lost track of what you mean by "work", but to answer the
implied question about the behaviour of "hash -r" instead:  all it does
is empty the hash table so it will be built up again in the usual way.
It doesn't refill the table again at that point.  Try "hash -rf".

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070