`test -x` returns wrong results for directories

`test -x` returns wrong results for directories

[Ethan Gardener]

in rc-httpd, i rely on `test -x` to check if a directory is searchable. this works in plan9port, 9base, inferno (with root from host fs), gnu coreutils, and freebsd. it doesn't work in 9front, nor in labs plan 9. (the labs version tested was a live-cd from 2010.)

term% test -x static ; echo $status
test 13436: false
term% ls -ld static
d-rwxrwxr-x M 24 ethan adm 0 Mar 15 14:58 static
term% man 1 test | grep -- -x
          -x file    True if the file exists and has execute permis-
term% 

the word 'wrong' in the subject may be a little strong, but there doesn't seem to be an alternative test for searchable directories other than mucking about with ls -l | sed or awk. test(1) has nothing to say on the matter.

for anyone using rc-httpd, the intended logic in rc-httpd/handlers/dir-index is this:
	if(! test -r $full_path -a -x $full_path) {
a suitable workaround is to remove ` -a -x $full_path`.
9front's version of rc-http has a workaround which may break when test is fixed.
/rc/bin/rc-httpd/handlers/dir-index:/-x
	if(! test -r $full_path -x $full_path){
in plan 9 and gnu coreutils, this effectively replaces the logical and (-a) with a logical or. freebsd reports an error, 'unexpected operator'.

Re: [9fans] `test -x` returns wrong results for directories

[Richard Miller]

Looks to me like access(2) is not doing the right thing for directory
execute (=search) permission.

Re: [9fans] `test -x` returns wrong results for directories

[Ethan Gardener]

On Fri, Jun 5, 2020, at 8:22 PM, Richard Miller wrote:
> Looks to me like access(2) is not doing the right thing for directory
> execute (=search) permission.

thanks for the tip. access is a very simple function. it doesn't do the right thing, but there's a reason:

     BUGS
          Since file permissions are checked by the server and group
          information is not known to the client, access must open the
          file to check permissions.  (It calls stat(2) to check sim-
          ple existence.)

it's open() which is failing. i suppose it should.

if the open fails, maybe access should stat the file, and if it's a directory, try dirread(2). or maybe just opening it for reading will work. i don't know, i'm new to this bit of plan 9 & i haven't slept.

Re: [9fans] `test -x` returns wrong results for directories

[Alexander Kapshuk]

On Sat, Jun 6, 2020 at 9:38 AM Ethan Gardener <eekee57@fastmail.fm> wrote:
>
> On Fri, Jun 5, 2020, at 8:22 PM, Richard Miller wrote:
> > Looks to me like access(2) is not doing the right thing for directory
> > execute (=search) permission.
>
> thanks for the tip. access is a very simple function. it doesn't do the right thing, but there's a reason:
>
>      BUGS
>           Since file permissions are checked by the server and group
>           information is not known to the client, access must open the
>           file to check permissions.  (It calls stat(2) to check sim-
>           ple existence.)
>
> it's open() which is failing. i suppose it should.
>
> if the open fails, maybe access should stat the file, and if it's a directory, try dirread(2). or maybe just opening it for reading will work. i don't know, i'm new to this bit of plan 9 & i haven't slept.
>

In sysopen, there is a call to namec:
/sys/src/9/port/sysfile.c:272
c = namec((char*)arg[0], Aopen, arg[1], 0);

Namec would return an error when called to open a directory with
execute permissions:
/sys/src/9/port/chan.c:1453,1454
if(amode == Aopen && (omode&3) == OEXEC && (c->qid.type&QTDIR))
        error("cannot exec directory");

But I'm not sure if this is where the problem actually is, as I haven't seen
the 'cannot exec directory' message when running 'test -x dir' in
acid:

cpu% test -x ch7
cpu% echo $status
test 528: false

cpu% acid -l truss /bin/test
/bin/test:386 plan 9 executable
/sys/lib/acid/port
/sys/lib/acid/386
/sys/lib/acid/truss
acid: progargs = "-x ch7"
acid: new()
acid: truss()
open("ch7", 3)
return value: -1
open("#c/pid", 0)
return value: 3
pread(3, 0xdfffef08, 20, 4294967295)
return value: 12
data: "        533 "
close(3)
return value: 0
533: breakpoint _exits+0x5 INTB $0x40

Re: [9fans] `test -x` returns wrong results for directories

[Charles Forsyth]

[-- Attachment #1: Type: text/plain, Size: 3031 bytes --]

execute permission on files, meaning here non-directories, is a special
variant of read. a file with mode 0111 can be opened with OEXEC and read(2)
will work as well as exec(2),
but can't be opened with OREAD, because it's not got any of 0444 set. bits
0111 distinguish a file with contents that are intended to be executed once
read from files with only 0444 that do not contain executable content.
you wouldn't want every readable file to be executable (especially if
you've used systems that didn't have that distinction).
on the other hand, in a distributed file system, the client needs the
contents of the file to run it (whether code or #!script) so it needs to be
able to read files with just OEXEC.
I suppose the rule could have been that it would need mode 5 (r+x) to make
clear that the file was also readable, but it isn't.

that OEXEC allows reading isn't true for a directory because exec means
"search", so if it's mode 0111 (say) you can chdir into it but not read the
names within it.
if you know a name of a file in that directory, though, you can still open
that. that's entirely enforced by the server.

as the bug in access(2) suggests, only the server knows whether access
should be granted, and the open call gets it to do that,
but it doesn't work for OEXEC for directories as others have noted. perhaps
stat+chdir is the most accurate test, since you need x (search) permission
to walk(5) into a directory,
but the caller won't thank you for the chdir (and there's no easy or
certain way back), and ... that restriction isn't enforced by fossil or
ramfs. (ramfs wrongly allows you to read a directory that's mode 0.)

probably the best thing is just to ignore the owner/group/other
distinction, and if the open(...OEXEC) fails, dirstat it, and if it's a
directory with any of 0111 set, it's fine (a little better than now).



On Sat, Jun 6, 2020 at 7:38 AM Ethan Gardener <eekee57@fastmail.fm> wrote:

> On Fri, Jun 5, 2020, at 8:22 PM, Richard Miller wrote:
> > Looks to me like access(2) is not doing the right thing for directory
> > execute (=search) permission.
>
> thanks for the tip. access is a very simple function. it doesn't do the
> right thing, but there's a reason:
>
>      BUGS
>           Since file permissions are checked by the server and group
>           information is not known to the client, access must open the
>           file to check permissions.  (It calls stat(2) to check sim-
>           ple existence.)
>
> it's open() which is failing. i suppose it should.
>
> if the open fails, maybe access should stat the file, and if it's a
> directory, try dirread(2). or maybe just opening it for reading will work.
> i don't know, i'm new to this bit of plan 9 & i haven't slept.
>
> ------------------------------------------
> 9fans: 9fans
> Permalink:
> https://9fans.topicbox.com/groups/9fans/Tdd7a9b1b32d01f54-M6c82b233d9b0cabf21ca7512
> Delivery options: https://9fans.topicbox.com/groups/9fans/subscription
>

[-- Attachment #2: Type: text/html, Size: 3840 bytes --]

Re: [9fans] `test -x` returns wrong results for directories

[Ethan Gardener]

On Sat, Jun 6, 2020, at 2:25 PM, Charles Forsyth wrote:
> execute permission on files, meaning here non-directories, is a special variant of read. a file with mode 0111 can be opened with OEXEC and read(2) will work as well as exec(2),
> but can't be opened with OREAD, because it's not got any of 0444 set. bits 0111 distinguish a file with contents that are intended to be executed once read from files with only 0444 that do not contain executable content.
> you wouldn't want every readable file to be executable (especially if you've used systems that didn't have that distinction).
> on the other hand, in a distributed file system, the client needs the contents of the file to run it (whether code or #!script) so it needs to be able to read files with just OEXEC.
> I suppose the rule could have been that it would need mode 5 (r+x) to make clear that the file was also readable, but it isn't.
> 
> that OEXEC allows reading isn't true for a directory because exec means "search", so if it's mode 0111 (say) you can chdir into it but not read the names within it.
> if you know a name of a file in that directory, though, you can still open that. that's entirely enforced by the server.
> 
> as the bug in access(2) suggests, only the server knows whether access should be granted, and the open call gets it to do that,
> but it doesn't work for OEXEC for directories as others have noted. perhaps stat+chdir is the most accurate test, since you need x (search) permission to walk(5) into a directory,
> but the caller won't thank you for the chdir (and there's no easy or certain way back), and ... that restriction isn't enforced by fossil or ramfs. (ramfs wrongly allows you to read a directory that's mode 0.)
> 
> probably the best thing is just to ignore the owner/group/other distinction, and if the open(...OEXEC) fails, dirstat it, and if it's a directory with any of 0111 set, it's fine (a little better than now).

thanks for the analysis, charles. the dirstat you suggest wouldn't do any good for my case because rc-httpd runs as user none. the common problem it's trying to catch is a directory which isn't world-readable & world-searchable. 770 750 and 700 are common permissions. perhaps i should have rc-httpd just run the commands and test their status rather than trying to test ahead of time, but this would somewhat spoil the neat and simple design.

Re: [9fans] `test -x` returns wrong results for directories

[ori]

> it's open() which is failing. i suppose it should.
> 
> if the open fails, maybe access should stat the file, and if it's a
> directory, try dirread(2). or maybe just opening it for reading will
> work. i don't know, i'm new to this bit of plan 9 & i haven't slept.

This is a bit subtle, though -- if you want
to check if *you* can traverse the directory,
then simply checking for an execute bit isn't
enough -- you need to check that you belong to
a group that has the execute bit.

On a related note, execute permission seems
to behave strangely in general. You can create
a directory and list it:


Even without execute permission it's listable.
Makes sense.

	cpu% mkdir -p d/a/b
	cpu% touch d/x
	cpu% chmod -x d
	cpu% ls -ld d
	d-rw-r--r-- M 81 ori ori 0 Jun  5 07:53 d
	cpu% ls d
	d/a
	d/x

As expected, it can't be traversed:

	cpu% ls d/x
	ls: d/x: 'd/x' does not exist

But, you can cd into it:

	cpu% cd d

And list '.':

	cpu% ls
	a
	x
	cpu% ls .
	a
	x

Can't traverse it to list subdirectories,
though:

	ls a
	ls: a: 'a' access permission denied
	cd a
	Can't cd a: 'a' access permission denied
	cpu% cd ..
	Can't cd ..: '..' access permission denied

So, cd'ing into a directory withut +x leads
to an inescapabler trap.

Perhaps 'cd' should prevent entering that
directory.

Re: [9fans] `test -x` returns wrong results for directories

[Ethan Gardener]

On Sat, Jun 6, 2020, at 10:13 PM, ori@eigenstate.org wrote:
> > it's open() which is failing. i suppose it should.
> > 
> > if the open fails, maybe access should stat the file, and if it's a
> > directory, try dirread(2). or maybe just opening it for reading will
> > work. i don't know, i'm new to this bit of plan 9 & i haven't slept.
> 
> This is a bit subtle, though -- if you want
> to check if *you* can traverse the directory,
> then simply checking for an execute bit isn't
> enough -- you need to check that you belong to
> a group that has the execute bit.
> 
> On a related note, execute permission seems
> to behave strangely in general. You can create
> a directory and list it:
> 
> 
> Even without execute permission it's listable.
> Makes sense.

sarcasm? i've been confused about search permission since my earliest linux use where i experimented with permissions to see what, exactly, they did. it made no sense then, even without plan 9 leaving permission checks up to the individual filesystems. (this would have been linux 2.0.0)

> 
> 	cpu% mkdir -p d/a/b
> 	cpu% touch d/x
> 	cpu% chmod -x d
> 	cpu% ls -ld d
> 	d-rw-r--r-- M 81 ori ori 0 Jun  5 07:53 d
> 	cpu% ls d
> 	d/a
> 	d/x
> 
> As expected, it can't be traversed:
> 
> 	cpu% ls d/x
> 	ls: d/x: 'd/x' does not exist
> 
> But, you can cd into it:
> 
> 	cpu% cd d
> 
> And list '.':
> 
> 	cpu% ls
> 	a
> 	x
> 	cpu% ls .
> 	a
> 	x
> 
> Can't traverse it to list subdirectories,
> though:
> 
> 	ls a
> 	ls: a: 'a' access permission denied
> 	cd a
> 	Can't cd a: 'a' access permission denied
> 	cpu% cd ..
> 	Can't cd ..: '..' access permission denied
> 
> So, cd'ing into a directory withut +x leads
> to an inescapabler trap.
> 
> Perhaps 'cd' should prevent entering that
> directory.

thanks for your analysis too, although you didn't say what filesystem you're using. i had to laugh at that trap; doesn't software get ridiculous sometimes? but it made me think, because charles said fossil doesn't handle the execute bit properly. perhaps once plan 9 went open source & had a public contrib, someone made just such a trap in contrib and fossil's permission handling was changed to stop it happening again. regardless of the cause, fossil just moved up another notch in my estimation because directory search restriction is so broken.

i just checked linux 2.6.30. (old vm i had handy.) it's much like you describe except there is no trap. cd is indeed prevented and if you're in the directory when "search" permission is revoked, you can't list it but you can `cd ..`.

"search" is exactly the wrong word for what this bit does, because if you don't have "search" permission, the one thing you can still do is look at the names... which lets you search them! it's more like "permission to open the display case and touch the things inside" or "permission to step over the museum's rope barriers".

Re: [9fans] `test -x` returns wrong results for directories

[Richard Miller]

>> So, cd'ing into a directory withut +x leads
>> to an inescapabler trap.
> ...
> fossil just moved up another notch in my estimation because directory search restriction is so broken.

ori@eigenstate.org, what version of fossil were your experiments done on?
My observations are different:

term% mkdir -p d/a/b
term% touch d/x
term% chmod -x d
term% ls -ld d
d-rw-r--r-- M 12 miller miller 0 Jun  7 16:38 d
term% cd d
term% cd ..
term% ls d
d/a
d/x
term% 

No trap that I can see here.

Re: [9fans] `test -x` returns wrong results for directories

[ori]

>>> So, cd'ing into a directory withut +x leads
>>> to an inescapabler trap.
>> ...
>> fossil just moved up another notch in my estimation because directory search restriction is so broken.
> 
> ori@eigenstate.org, what version of fossil were your experiments done on?
> My observations are different:
> 
> term% mkdir -p d/a/b
> term% touch d/x
> term% chmod -x d
> term% ls -ld d
> d-rw-r--r-- M 12 miller miller 0 Jun  7 16:38 d
> term% cd d
> term% cd ..
> term% ls d
> d/a
> d/x
> term% 
> 
> No trap that I can see here.

cwfs, but -- yeah, it's enforced inconsistently across
different file systems. (of course). 

Re: [9fans] `test -x` returns wrong results for directories

[Charles Forsyth]

[-- Attachment #1: Type: text/plain, Size: 1230 bytes --]

>
> "search" is exactly the wrong word for what this bit does, because if you
> don't have "search" permission, the one thing you can still do is look at
> the names.


in ramfs, but that's a bug that no-one had noticed

On Mon, Jun 8, 2020 at 3:05 AM <ori@eigenstate.org> wrote:

> >>> So, cd'ing into a directory withut +x leads
> >>> to an inescapabler trap.
> >> ...
> >> fossil just moved up another notch in my estimation because directory
> search restriction is so broken.
> >
> > ori@eigenstate.org, what version of fossil were your experiments done
> on?
> > My observations are different:
> >
> > term% mkdir -p d/a/b
> > term% touch d/x
> > term% chmod -x d
> > term% ls -ld d
> > d-rw-r--r-- M 12 miller miller 0 Jun  7 16:38 d
> > term% cd d
> > term% cd ..
> > term% ls d
> > d/a
> > d/x
> > term%
> >
> > No trap that I can see here.
>
> cwfs, but -- yeah, it's enforced inconsistently across
> different file systems. (of course).
>
>
> ------------------------------------------
> 9fans: 9fans
> Permalink:
> https://9fans.topicbox.com/groups/9fans/Tdd7a9b1b32d01f54-M9cf36356f4d9761339774af6
> Delivery options: https://9fans.topicbox.com/groups/9fans/subscription
>

[-- Attachment #2: Type: text/html, Size: 2148 bytes --]

Re: [9fans] `test -x` returns wrong results for directories

[Charles Forsyth]

[-- Attachment #1: Type: text/plain, Size: 1591 bytes --]

I see I'd misapplied the rule in walk(5) so fossil is fine. It has to do
with searching from . when you are there, which makes sense, not when
entering the directory from its parent,
so ignore that part of my earlier post.


On Mon, Jun 8, 2020 at 3:13 AM Charles Forsyth <charles.forsyth@gmail.com>
wrote:

> "search" is exactly the wrong word for what this bit does, because if you
>> don't have "search" permission, the one thing you can still do is look at
>> the names.
>
>
> in ramfs, but that's a bug that no-one had noticed
>
> On Mon, Jun 8, 2020 at 3:05 AM <ori@eigenstate.org> wrote:
>
>> >>> So, cd'ing into a directory withut +x leads
>> >>> to an inescapabler trap.
>> >> ...
>> >> fossil just moved up another notch in my estimation because directory
>> search restriction is so broken.
>> >
>> > ori@eigenstate.org, what version of fossil were your experiments done
>> on?
>> > My observations are different:
>> >
>> > term% mkdir -p d/a/b
>> > term% touch d/x
>> > term% chmod -x d
>> > term% ls -ld d
>> > d-rw-r--r-- M 12 miller miller 0 Jun  7 16:38 d
>> > term% cd d
>> > term% cd ..
>> > term% ls d
>> > d/a
>> > d/x
>> > term%
>> >
>> > No trap that I can see here.
>>
>> cwfs, but -- yeah, it's enforced inconsistently across
>> different file systems. (of course).
>>
>>
>> ------------------------------------------
>> 9fans: 9fans
>> Permalink:
>> https://9fans.topicbox.com/groups/9fans/Tdd7a9b1b32d01f54-M9cf36356f4d9761339774af6
>> Delivery options: https://9fans.topicbox.com/groups/9fans/subscription
>>
>

[-- Attachment #2: Type: text/html, Size: 2785 bytes --]

Re: [9fans] `test -x` returns wrong results for directories

[Ethan Gardener]

On Mon, Jun 8, 2020, at 3:13 AM, Charles Forsyth wrote:
>> "search" is exactly the wrong word for what this bit does, because if you don't have "search" permission, the one thing you can still do is look at the names.
> 
> in ramfs, but that's a bug that no-one had noticed 

oh it's the same in cwfs and in linux. if i remember right, (and i think i do,) it's always been this way in linux. (tried linux tmpfs recently, but i think permissions are enforced by the kernel 'over there', and i'm pretty sure it was the same on ext2 in linux 2.0.) i'd try some other unixes, but i have too much work to do today.