Unreadable directories aren't included in glob expansion

Unreadable directories aren't included in glob expansion

[Daniel Shahaf]

Consider:

% mkdir foo
% mkdir -m 000 bar 
% echo */ 
foo/
% echo *(/)
bar foo
% 

Shouldn't the expansion of «*/» include «bar/»?

The shell knows, in that situation, that ./bar is a directory; «test -d bar»
and «zstat -or bar» both confirm this.

dash/bash/ksh all include «bar/» in that expansion.  

strace shows a probable cause:
.
    access("buster/.", F_OK)                = -1 EACCES (Permission denied)

I've discussed this on IRC with Eric, but I'd rather let him explain his view
in his own words.

Cheers,

Daniel

P.S. Test case:

diff --git a/Test/D02glob.ztst b/Test/D02glob.ztst
index 5638e1255..1c1663a48 100644
--- a/Test/D02glob.ztst
+++ b/Test/D02glob.ztst
@@ -728,3 +728,10 @@
 >does/not/exist
 >not/exist
 >exist
+
+ mkdir -m 000 glob.tmp/secret1
+ (setopt nullglob
+  echo glob.tmp/*/ glob.tmp/*(/)
+  )
+-f:unreadable directories can be globbed
+>glob.tmp/secret1/ glob.tmp/secret1

Re: Unreadable directories aren't included in glob expansion

[Peter Stephenson]

On Thu, 2020-01-09 at 15:41 +0000, Daniel Shahaf wrote:
> Consider:
> 
> % mkdir foo
> % mkdir -m 000 bar 
> % echo */ 
> foo/
> % echo *(/)
> bar foo
> % 
> 
> Shouldn't the expansion of «*/» include «bar/»?

Certainly, */ and *(/) have always been inconsistent and I don't know of
any fundamental reason why that needs to be.

At a practical level, the difference may have arisen because a / not in
a glob qualifier is usually present in order to select files within the
directory, which won't work if the directory isn't readable.  A glob
qualifier, on the other hand, always applies to the final path segment.

pws

Re: Unreadable directories aren't included in glob expansion

[Bart Schaefer]

[-- Attachment #1: Type: text/plain, Size: 520 bytes --]

On Thu, Jan 9, 2020, 7:52 AM Peter Stephenson <p.stephenson@samsung.com>
wrote:

>
> At a practical level, the difference may have arisen because a / not in
> a glob qualifier is usually present in order to select files within the
> directory, which won't work if the directory isn't readable.  A glob
> qualifier, on the other hand, always applies to the final path segment.
>


I think this is exactly it.  foo/<TAB> is like asking to select from among
the matches of foo/* and if foo is not readable, there are none.

Re: Unreadable directories aren't included in glob expansion

[Daniel Shahaf]

Bart Schaefer wrote on Thu, 09 Jan 2020 16:10 +00:00:
> On Thu, Jan 9, 2020, 7:52 AM Peter Stephenson <p.stephenson@samsung.com>
> wrote:
> 
> >
> > At a practical level, the difference may have arisen because a / not in
> > a glob qualifier is usually present in order to select files within the
> > directory, which won't work if the directory isn't readable.  A glob
> > qualifier, on the other hand, always applies to the final path segment.
> >
> 
> 
> I think this is exactly it.  foo/<TAB> is like asking to select from among
> the matches of foo/* and if foo is not readable, there are none.

But I'm not doing completion; I'm just echoing the result of the glob, as in «sudo ls */».

Re: Unreadable directories aren't included in glob expansion

[Stephane Chazelas]

[re-post to zsh-users. Sorry I read my MLs on gmane and always
forget that zsh-users and zsh-workers are cross-posted]

2020-01-09 15:51:41 +0000, Peter Stephenson:
> On Thu, 2020-01-09 at 15:41 +0000, Daniel Shahaf wrote:
> > Consider:
> > 
> > % mkdir foo
> > % mkdir -m 000 bar 
> > % echo */ 
> > foo/
> > % echo *(/)
> > bar foo
> > % 
> > 
> > Shouldn't the expansion of «*/» include «bar/»?
> 
> Certainly, */ and *(/) have always been inconsistent and I don't know of
> any fundamental reason why that needs to be.
[...]

*/ in all shells also includes symlinks to directories. So if
only for that reason, it has to be different from *(/).

Now, zsh seems to be the only shell where */ excludes the
non-*searchable* (it does include the non-readable ones provided
you have search access).

It might make it the only  POSIX compliant shell as I have a
vague recollection it's meant to give the same result as */.,
but that could very  well be a bad recollection, and if that was
a POSIX requirement, that's probably one that should change.

IMO, */ should give the same list as *(-/) (with some /
appended), that is expand to the files that can be determined to
be of type "directory" after symlink resolution (determined from
the result of stat(), not lstat()) for consistency with other
shells.

-- 
Stephane

Re: Unreadable directories aren't included in glob expansion

[Mikael Magnusson]

On 1/9/20, Stephane Chazelas <stephane.chazelas@gmail.com> wrote:
>
> IMO, */ should give the same list as *(-/) (with some /
> appended)

FWIW, this can be spelled shorter as *(-/M)

-- 
Mikael Magnusson

Re: Unreadable directories aren't included in glob expansion

[Daniel Shahaf]

Stephane Chazelas wrote on Thu, Jan 09, 2020 at 17:22:22 +0000:
> Now, zsh seems to be the only shell where */ excludes the
> non-*searchable* (it does include the non-readable ones provided
> you have search access).
> 
> It might make it the only  POSIX compliant shell as I have a
> vague recollection it's meant to give the same result as */.,
> but that could very  well be a bad recollection, and if that was
> a POSIX requirement, that's probably one that should change.

So there isn't any need to make the bugfix conditional on some POSIX_* option
being unset, right?

> IMO, */ should give the same list as *(-/) (with some /
> appended), that is expand to the files that can be determined to
> be of type "directory" after symlink resolution (determined from
> the result of stat(), not lstat()) for consistency with other
> shells.

Cheers,

Daniel

Re: Unreadable directories aren't included in glob expansion

[Stephane Chazelas]

2020-01-09 17:22:22 +0000, Stephane Chazelas:
[...]
> Now, zsh seems to be the only shell where */ excludes the
> non-*searchable* (it does include the non-readable ones provided
> you have search access).
[...]

And conversely, it doesn't include readable dirs if they are not
searchable which can be even more surprising:

$ ls -l
total 2
drwxrwxr-x 2 chazelas chazelas 2 Jan  9 17:07 1/
dr--r--r-- 2 chazelas chazelas 3 Jan  9 17:17 2/
drwxrwxr-x 2 chazelas chazelas 2 Jan  9 17:07 3/
$ echo */
1/ 3/
$ echo */*
2/file

-- 
Stephane

Re: Unreadable directories aren't included in glob expansion

[Daniel Shahaf]

Stephane Chazelas wrote on Fri, Jan 10, 2020 at 07:27:53 +0000:
> 2020-01-09 17:22:22 +0000, Stephane Chazelas:
> [...]
> > Now, zsh seems to be the only shell where */ excludes the
> > non-*searchable* (it does include the non-readable ones provided
> > you have search access).
> [...]
> 
> And conversely, it doesn't include readable dirs if they are not
> searchable which can be even more surprising:
> 
> $ ls -l
> total 2
> drwxrwxr-x 2 chazelas chazelas 2 Jan  9 17:07 1/
> dr--r--r-- 2 chazelas chazelas 3 Jan  9 17:17 2/
> drwxrwxr-x 2 chazelas chazelas 2 Jan  9 17:07 3/
> $ echo */
> 1/ 3/
> $ echo */*
> 2/file

Thanks, here's an updated test:

diff --git a/Test/D02glob.ztst b/Test/D02glob.ztst
index 5638e1255..22c663260 100644
--- a/Test/D02glob.ztst
+++ b/Test/D02glob.ztst
@@ -728,3 +728,16 @@
 >does/not/exist
 >not/exist
 >exist
+
+ mkdir -m 000 glob.tmp/secret-d000
+ mkdir -m 111 glob.tmp/secret-d111
+ mkdir -m 444 glob.tmp/secret-d444
+ for 1 in 000 111 444 ; do ln -s secret-d$1 glob.tmp/secret-s$1; done
+ print -rC 2 -- glob.tmp/secret-*/ glob.tmp/secret-*(-/)
+-f:unreadable directories can be globbed (users/24619, users/24626)
+>glob.tmp/secret-d000/  glob.tmp/secret-d000
+>glob.tmp/secret-d111/  glob.tmp/secret-d111
+>glob.tmp/secret-d444/  glob.tmp/secret-d444
+>glob.tmp/secret-s000/  glob.tmp/secret-s000
+>glob.tmp/secret-s111/  glob.tmp/secret-s111
+>glob.tmp/secret-s444/  glob.tmp/secret-s444

It passes if the mkdir's are done with the default permissions.