bug in patterns used for filename expansion (e.g. a[b/c]d)

bug in patterns used for filename expansion (e.g. a[b/c]d)

[Vincent Lefevre]

zsh 4.3.4 isn't POSIX-compliant concerning filename expansion and
unmatched [. POSIX[*] says:

  For example, the pattern "a[b/c]d" does not match such pathnames as
  abd or a/d. It only matches a pathname of literally a[b/c]d.

[*] http://www.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_13_03

But with zsh 4.3.4:

prunille% ls a[b/c]d
zsh: no matches found: a[b/c]d

while the file exists (no problem with bash, dash or ksh93). For more
information, see the following thread:

https://www.opengroup.org/sophocles/show_archive.tpl?source=L&listname=austin-group-l&first=1&pagesize=80&searchstring=Defect+in+XCU+2.13.3&zone=G

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)

Re: bug in patterns used for filename expansion (e.g. a[b/c]d)

[Peter Stephenson]

Vincent Lefevre wrote:
> zsh 4.3.4 isn't POSIX-compliant concerning filename expansion and
> unmatched [. POSIX[*] says:
> 
>   For example, the pattern "a[b/c]d" does not match such pathnames as
>   abd or a/d. It only matches a pathname of literally a[b/c]d.
> 
> [*] http://www.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#t
> ag_02_13_03
> 
> But with zsh 4.3.4:
> 
> prunille% ls a[b/c]d
> zsh: no matches found: a[b/c]d

As I said on the Austin group list, this isn't a bug.  It's controlled
by the option NOMATCH; you'll find that if zsh is in compatibility mode
that option is unset.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070

Re: bug in patterns used for filename expansion (e.g. a[b/c]d)

[Vincent Lefevre]

On 2007-10-09 13:59:44 +0100, Peter Stephenson wrote:
> As I said on the Austin group list, this isn't a bug.  It's controlled
> by the option NOMATCH; you'll find that if zsh is in compatibility mode
> that option is unset.

OK, but these compatibility modes are not really clear, e.g. which sh
and which ksh? And what about POSIX?

In particular:

vin:~> mkdir 'a[b'
vin:~> touch 'a[b'/file

vin:~> ksh --version
zsh 4.3.4 (i686-pc-linux-gnu)
vin:~> ksh
$ ls a[b/*
ls: a[b/*: No such file or directory

vin:~> ksh93
$ ls a[b/*
> ksh93: syntax error: `[' unmatched

vin:~> ksh -c 'ls a[b/*'
ls: a[b/*: No such file or directory
vin:~> ksh93 -c 'ls a[b/*'
ls: a[b/*[: No such file or directory

vin:~> touch 'a[b/*'
vin:~> ksh93 -c 'ls a[b/*'
ls: a[b/*[: No such file or directory
vin:~> ksh -c 'ls a[b/*'
a[b/*

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)

Re: bug in patterns used for filename expansion (e.g. a[b/c]d)

[Peter Stephenson]

Vincent Lefevre wrote:
> On 2007-10-09 13:59:44 +0100, Peter Stephenson wrote:
> > As I said on the Austin group list, this isn't a bug.  It's controlled
> > by the option NOMATCH; you'll find that if zsh is in compatibility mode
> > that option is unset.
> 
> OK, but these compatibility modes are not really clear

Yes, I tend to agree with that, although it's partly because the targets
are moving and we haven't tried very hard to keep up.

> e.g. which sh and which ksh? And what about POSIX?

"sh" usually means POSIX with a leaning towards traditional
behaviour where that's not in conflict with the spec.  Anything in "sh"
mode that's definitely in violation of POSIX can be regarded as a bug.
(Whether it's going to get fixed is another matter, as with the long
standing word-splitting niggle that Wayne eventually fixed.)

"ksh" really means ksh88 and it probably ought to be documented as
such.  There have been vague plans to have more recent ksh features but
they need architectural changes.

pws

Re: bug in patterns used for filename expansion (e.g. a[b/c]d)

[Bart Schaefer]

On Oct 9,  4:12pm, Vincent Lefevre wrote:
} Subject: Re: bug in patterns used for filename expansion (e.g. a[b/c]d)
}
} On 2007-10-09 13:59:44 +0100, Peter Stephenson wrote:
} > As I said on the Austin group list, this isn't a bug.  It's controlled
} > by the option NOMATCH; you'll find that if zsh is in compatibility mode
} > that option is unset.
} 
} OK, but these compatibility modes are not really clear, e.g. which sh
} and which ksh? And what about POSIX?

The intent is that starting zsh as "sh" provides full POSIX compatility.

However, starting zsh as "ksh" has never promised to be 100% compatible
with any version of ksh; it just means that most of the behavior becomes
ksh-like.  Similarly "emulate csh" makes no promise that zsh will be a
drop-in replacement for csh, and even when started as "sh" there is no
specific other-thing-that-calls-itself-"sh" that you can point at to say
"zsh works like that."

That said, the thing zsh-as-ksh is most like is ksh88.
 
} vin:~> ksh --version
} zsh 4.3.4 (i686-pc-linux-gnu)
} vin:~> ksh
} $ ls a[b/*
} ls: a[b/*: No such file or directory
} 
} vin:~> ksh93
} $ ls a[b/*
} > ksh93: syntax error: `[' unmatched

This is exactly the difference between bash and ksh as mentioned on
the austin-group, is it not?

} vin:~> ksh -c 'ls a[b/*'
} ls: a[b/*: No such file or directory
} vin:~> ksh93 -c 'ls a[b/*'
} ls: a[b/*[: No such file or directory

Where'd that extra '[' come from?  I'd say this is a ksh bug.

} vin:~> touch 'a[b/*'
} vin:~> ksh93 -c 'ls a[b/*'
} ls: a[b/*[: No such file or directory
} vin:~> ksh -c 'ls a[b/*'
} a[b/*

Again modulo the extra-bracket bug, this is the bash v. ksh thing.

Re: bug in patterns used for filename expansion (e.g. a[b/c]d)

[Vincent Lefevre]

On 2007-10-09 08:24:13 -0700, Bart Schaefer wrote:
> That said, the thing zsh-as-ksh is most like is ksh88.

I think that the manual should say that, because it seems that the
provided AT&T ksh is generally ksh93 (e.g. under Debian and Mac OS X).

>  
> } vin:~> ksh --version
> } zsh 4.3.4 (i686-pc-linux-gnu)
> } vin:~> ksh
> } $ ls a[b/*
> } ls: a[b/*: No such file or directory
> } 
> } vin:~> ksh93
> } $ ls a[b/*
> } > ksh93: syntax error: `[' unmatched
> 
> This is exactly the difference between bash and ksh as mentioned on
> the austin-group, is it not?

No, Geoff Clare said in the austin-group is:

$ ls a[b/*
a[b/file

in both cases. This is also what I obtain with AT&T ksh93 under Mac OS X.
So, only zsh, when run as ksh, is different. Is this a bug in zsh or does
ksh88 behave like that? (I don't have ksh88, so I can't test.)

> } vin:~> ksh -c 'ls a[b/*'
> } ls: a[b/*: No such file or directory
> } vin:~> ksh93 -c 'ls a[b/*'
> } ls: a[b/*[: No such file or directory
> 
> Where'd that extra '[' come from?  I'd say this is a ksh bug.

Yes, this is strange and ksh93 under Mac OS X behaves differently, so
I assume that this is a bug in Debian's ksh package.

> 
> } vin:~> touch 'a[b/*'
> } vin:~> ksh93 -c 'ls a[b/*'
> } ls: a[b/*[: No such file or directory
> } vin:~> ksh -c 'ls a[b/*'
> } a[b/*
> 
> Again modulo the extra-bracket bug, this is the bash v. ksh thing.

No:

vin:~> ll a\[b
-rw-r--r-- 1 vlefevre vlefevre 0 2007-10-09 16:10:36 *
-rw-r--r-- 1 vlefevre vlefevre 0 2007-10-09 16:25:17 *[
-rw-r--r-- 1 vlefevre vlefevre 0 2007-10-09 16:04:18 file
vin:~> bash -c 'ls a[b/*'
a[b/*  a[b/*[  a[b/file

and with ksh from Mac OS X (sh (AT&T Labs Research) 1993-12-28 p),
I get the same behavior, while zsh gives:

vin:~> ksh -c 'ls a[b/*'
a[b/*

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)

Re: bug in patterns used for filename expansion (e.g. a[b/c]d)

[Stephane Chazelas]

On Tue, Oct 09, 2007 at 01:59:44PM +0100, Peter Stephenson wrote:
> Vincent Lefevre wrote:
> > zsh 4.3.4 isn't POSIX-compliant concerning filename expansion and
> > unmatched [. POSIX[*] says:
> > 
> >   For example, the pattern "a[b/c]d" does not match such pathnames as
> >   abd or a/d. It only matches a pathname of literally a[b/c]d.
> > 
> > [*] http://www.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#t
> > ag_02_13_03
> > 
> > But with zsh 4.3.4:
> > 
> > prunille% ls a[b/c]d
> > zsh: no matches found: a[b/c]d
> 
> As I said on the Austin group list, this isn't a bug.  It's controlled
> by the option NOMATCH; you'll find that if zsh is in compatibility mode
> that option is unset.
[...]

On the other hand:

$ mkdir -p 'a[b/c]d' abd
$ zsh -c 'echo a[b/c]d'
abd
$ ARGV0=sh zsh -c 'echo a[b/c]d'
abd

According to POSIX, it should not match abd.

But I'd agree that POSIX requirement looks a bit silly.

Cheers,
Stéphane

Re: bug in patterns used for filename expansion (e.g. a[b/c]d)

[Bart Schaefer]

On Oct 9,  4:00pm, Stephane Chazelas wrote:
}
} $ mkdir -p 'a[b/c]d' abd
} $ zsh -c 'echo a[b/c]d'
} abd

Thanks for coming up with that example; I had a suspicion that this was
a problem, but I wasn't awake enough yet to demonstrate it.

A related question (which I ought to ask on austin-group) is:

case $foo in
(a[b/c]d) print 'is this pattern valid?'
esac