readdir(3): behavior on descriptors with O_SEARCH

readdir(3): behavior on descriptors with O_SEARCH

[Dmitry Selyutin]

[-- Attachment #1: Type: text/plain, Size: 2165 bytes --]

Hello everyone!

First of all, thank you for musl; it is an amazing tool and I really glad
I'm using it. I'm able to compile and run everything I write (of course if
POSIX standard applies).

Recently I've found some edge case where both musl-gcc and musl-clang
behave differently compared to usual gcc, clang and tcc, and I think the
reason lies in the musl library. The example code:

int const flags = (O_DIRECTORY | O_SEARCH);
int descriptor = open(path, flags);
DIR *handle = fdopendir(descriptor);
struct dirent *entry = readdir(handle);

To cut the long story short, any attempt to call readdir(3) on directory
handle obtained via fdopendir(3) returns NULL and sets the errno variable
to EBADF. This behavior arises only on descriptors opened with (O_DIRECTORY
| O_SEARCH) flags enabled; it goes away if O_SEARCH flag is removed.

So it seems that O_SEARCH is the reason; I thought that this flag tells
exactly "well, I'm going to use it for search only", which implies "well,
I'm going to use only readdir(3) to get information about files inside". Is
my interpretation correct?

I'm not really sure if it is a bug, since I suspect POSIX may allow open(3)
with (O_DIRECTORY | O_SEARCH) flags to behave in an implementation-defined
matter; it can be possible that file descriptors obtained via open(3) with
O_DIRECTORY flag set are guaranteed to work only with fchdir(3) and *at(3)
operations. However, if such behavior is intentional, it would be a good
idea (in my opinion) make fdopendir(3) return NULL (though it won't match
behavior e.g. for glibc).

I don't know if this behavior still matters since I doubt I use the latest
musl version; I couldn't find any relevant information about this behavior
though.

The musl version I'm using is 1.1.15; the code is compiled for x86_64. I'm
using the version provided by Arch repositories.

I don't have a stable Internet connection now, so I couldn't manage to take
a look at the corresponding code in musl; I'll do it as soon as possible
and create a patch if my assumptions are correct (and if the above behavior
still applies to musl).

Thank you for musl and patience (well, it was quite a long letter)!

[-- Attachment #2: Type: text/html, Size: 2444 bytes --]

Re: readdir(3): behavior on descriptors with O_SEARCH

[Markus Wichmann]

On Sat, Aug 27, 2016 at 09:23:50PM +0300, Dmitry Selyutin wrote:
> int const flags = (O_DIRECTORY | O_SEARCH);
> int descriptor = open(path, flags);
> DIR *handle = fdopendir(descriptor);
> struct dirent *entry = readdir(handle);
> 
> To cut the long story short, any attempt to call readdir(3) on directory
> handle obtained via fdopendir(3) returns NULL and sets the errno variable
> to EBADF. This behavior arises only on descriptors opened with (O_DIRECTORY
> | O_SEARCH) flags enabled; it goes away if O_SEARCH flag is removed.
> 

Try strace(1). That should tell you what glibc and musl are doing
differently. For instace, whether glibc removes O_SEARCH from the fd.

musl defines O_SEARCH to be equal to O_PATH. The manpage says that
O_PATH means the file isn't opened for reading. I guess if you do that
then getdents(2) will fail, which is what musl uses to implement
readdir(3).

I tried to do the same trace in glibc 2.19 (which is what Debian stable
is using right now), but to no avail: O_SEARCH isn't even mentioned
anywhere in that code. But its implementation of fdopendir(3) rejects
fds open only for writing. The readdir(3) implementation is, of course,
overcomplicated, but also seems to just call getdents(2). And then it
tries to pack the kernel structures into its own structures, probably
for ABI reasons. And people wonder why I dislike dynamic linking...

> So it seems that O_SEARCH is the reason; I thought that this flag tells
> exactly "well, I'm going to use it for search only", which implies "well,
> I'm going to use only readdir(3) to get information about files inside". Is
> my interpretation correct?
> 

My manpage doesn't know O_SEARCH, but it knows O_PATH, and then you're
wrong. It means "I'll only use this fd in *at() and fchdir() and
similar; this fd isn't open for reading."

> I'm not really sure if it is a bug, since I suspect POSIX may allow open(3)
> with (O_DIRECTORY | O_SEARCH) flags to behave in an implementation-defined
> matter; it can be possible that file descriptors obtained via open(3) with
> O_DIRECTORY flag set are guaranteed to work only with fchdir(3) and *at(3)
> operations. However, if such behavior is intentional, it would be a good
> idea (in my opinion) make fdopendir(3) return NULL (though it won't match
> behavior e.g. for glibc).
> 

POSIX doesn't know O_SEARCH or O_PATH, and thus mandates nothing about
their meaning.

Ciao,
Markus

Re: readdir(3): behavior on descriptors with O_SEARCH

[Dmitry Selyutin]

[-- Attachment #1: Type: text/plain, Size: 3418 bytes --]

Hi Markus,

thank you for your reply!

> POSIX doesn't know O_SEARCH or O_PATH, and thus mandates nothing about
> their meaning.
POSIX 2008 with 2013 corrigenda mentions both O_SEARCH and O_EXEC.
http://pubs.opengroup.org/onlinepubs/9699919799/functions/open.html

> Try strace(1). That should tell you what glibc and musl are doing
> differently. For instace, whether glibc removes O_SEARCH from the fd.
Thank you! I'll try it and report the results here.

FWIW it does not seem to be correct that O_SEARCH can be equal to O_PATH;
from what I gathered from manuals and various mailing lists discussions,
O_SEARCH may be equal to O_EXEC since each of these flags is valid either
for directory or file respectively.
28 авг. 2016 г. 10:11 пользователь "Markus Wichmann" <nullplan@gmx.net>
написал:

> On Sat, Aug 27, 2016 at 09:23:50PM +0300, Dmitry Selyutin wrote:
> > int const flags = (O_DIRECTORY | O_SEARCH);
> > int descriptor = open(path, flags);
> > DIR *handle = fdopendir(descriptor);
> > struct dirent *entry = readdir(handle);
> >
> > To cut the long story short, any attempt to call readdir(3) on directory
> > handle obtained via fdopendir(3) returns NULL and sets the errno variable
> > to EBADF. This behavior arises only on descriptors opened with
> (O_DIRECTORY
> > | O_SEARCH) flags enabled; it goes away if O_SEARCH flag is removed.
> >
>
> Try strace(1). That should tell you what glibc and musl are doing
> differently. For instace, whether glibc removes O_SEARCH from the fd.
>
> musl defines O_SEARCH to be equal to O_PATH. The manpage says that
> O_PATH means the file isn't opened for reading. I guess if you do that
> then getdents(2) will fail, which is what musl uses to implement
> readdir(3).
>
> I tried to do the same trace in glibc 2.19 (which is what Debian stable
> is using right now), but to no avail: O_SEARCH isn't even mentioned
> anywhere in that code. But its implementation of fdopendir(3) rejects
> fds open only for writing. The readdir(3) implementation is, of course,
> overcomplicated, but also seems to just call getdents(2). And then it
> tries to pack the kernel structures into its own structures, probably
> for ABI reasons. And people wonder why I dislike dynamic linking...
>
> > So it seems that O_SEARCH is the reason; I thought that this flag tells
> > exactly "well, I'm going to use it for search only", which implies "well,
> > I'm going to use only readdir(3) to get information about files inside".
> Is
> > my interpretation correct?
> >
>
> My manpage doesn't know O_SEARCH, but it knows O_PATH, and then you're
> wrong. It means "I'll only use this fd in *at() and fchdir() and
> similar; this fd isn't open for reading."
>
> > I'm not really sure if it is a bug, since I suspect POSIX may allow
> open(3)
> > with (O_DIRECTORY | O_SEARCH) flags to behave in an
> implementation-defined
> > matter; it can be possible that file descriptors obtained via open(3)
> with
> > O_DIRECTORY flag set are guaranteed to work only with fchdir(3) and
> *at(3)
> > operations. However, if such behavior is intentional, it would be a good
> > idea (in my opinion) make fdopendir(3) return NULL (though it won't match
> > behavior e.g. for glibc).
> >
>
> POSIX doesn't know O_SEARCH or O_PATH, and thus mandates nothing about
> their meaning.
>
> Ciao,
> Markus
>

[-- Attachment #2: Type: text/html, Size: 4068 bytes --]

Re: readdir(3): behavior on descriptors with O_SEARCH

[Dmitry Selyutin]

[-- Attachment #1: Type: text/plain, Size: 1205 bytes --]

I think the mistery is partially solved: glibc simply doesn't have
O_SEARCH, so the following code

#ifdef O_SEARCH
flags |= O_SEARCH
#endif

does not execute (so only O_DIRECTORY is being set). Dumb, had to check
twice, sorry for the noise.

However, the question if it is correct to define O_SEARCH to be equal to
O_PATH. From what I see, both O_SEARCH and O_EXEC have the same value as
O_PATH, but I'm not sure if this solution is technically correct.

However, I suspect that support for O_EXEC and O_SEARCH must be provided by
the kernel first, so until kernel implements such functionality, all talks
seem to be meaningless (unless someone wants to emulate such functionality
in the userspace). Anyway, I tend to think that any attempt to use O_SEARCH
until then should either return -1 on open(3), or NULL on fdopendir(3); I
really don't think that O_SEARCH shall mean the same as O_PATH.

But I'm by no means claim myself to be a POSIX expert; what do you think?
The information on O_SEARCH seems to be incomplete and even inconsistent,
so it may turn that my claims are incorrect. Any suggestions are welcome;
I'll take a more deep look at it in the evening.

And again, thank you for your help!

[-- Attachment #2: Type: text/html, Size: 1355 bytes --]

Re: readdir(3): behavior on descriptors with O_SEARCH

[Rich Felker]

On Sun, Aug 28, 2016 at 12:02:02PM +0300, Dmitry Selyutin wrote:
> I think the mistery is partially solved: glibc simply doesn't have
> O_SEARCH, so the following code
> 
> #ifdef O_SEARCH
> flags |= O_SEARCH
> #endif

If a program is doing this then trying to fdopendir/readdir, it's a
bug in the program. A directory opened with O_SEARCH is only usable
for search (attempting to access a file/directory in that directory
by name, using one of the *at functions) not for reading. Unix has
always distinguished search (+x) and read (+r) permission for
directories and O_SEARCH vs O_RDONLY is similar.

> However, the question if it is correct to define O_SEARCH to be equal to
> O_PATH. From what I see, both O_SEARCH and O_EXEC have the same value as
> O_PATH, but I'm not sure if this solution is technically correct.

It's not quite, but there are only a few minor technical differences,
and most of them can be papered over from userspace. We're not doing
all that yet. The only one I'm aware of that needs kernel help is
allowing search even if the user has lost permission (chmod -x)
between the time the directory was opened successfully for O_SEARCH
and the time of the *at function.

> However, I suspect that support for O_EXEC and O_SEARCH must be provided by
> the kernel first, so until kernel implements such functionality, all talks
> seem to be meaningless (unless someone wants to emulate such functionality
> in the userspace). Anyway, I tend to think that any attempt to use O_SEARCH
> until then should either return -1 on open(3), or NULL on fdopendir(3); I
> really don't think that O_SEARCH shall mean the same as O_PATH.

It's been an ongoing fight even trying to get the kernel to reserve a
bit number for them. It looks like the correct course of action (the
one that's compatible with their non-action) is going to be using
O_PATH|3 for both of them. This allows userspace open to process
O_SEARCH and O_EXEC slightly differently from O_PATH (O_NOFOLLOW has
different semantics) and the kernel will ignore the extra access mode
bits with O_PATH anyway.

> But I'm by no means claim myself to be a POSIX expert; what do you think?
> The information on O_SEARCH seems to be incomplete and even inconsistent,
> so it may turn that my claims are incorrect. Any suggestions are welcome;
> I'll take a more deep look at it in the evening.

I don't see what's incomplete or inconsistent about it.

Rich

Re: readdir(3): behavior on descriptors with O_SEARCH

[Dmitry Selyutin]

[-- Attachment #1: Type: text/plain, Size: 2168 bytes --]

Hi Rich,

thank you for the detailed answer!

> If a program is doing this then trying to fdopendir/readdir, it's a
> bug in the program. A directory opened with O_SEARCH is only usable
> for search (attempting to access a file/directory in that directory
> by name, using one of the *at functions) not for reading. Unix has
> always distinguished search (+x) and read (+r) permission for
> directories and O_SEARCH vs O_RDONLY is similar.
Well, that's probably the worst kind of bugs, since it was caused by
misinterpretation of the documentation.
That was written intentionally due to misunderstanding, so I really want to
clarify the situation here.
So does it mean that descriptors opened with O_SEARCH are usable only for
*at functions?
I've been under the impression that it's part of the O_PATH functionality,
not O_SEARCH.

> It's been an ongoing fight even trying to get the kernel to reserve a
> bit number for them. It looks like the correct course of action (the
> one that's compatible with their non-action) is going to be using
> O_PATH|3 for both of them. This allows userspace open to process
> O_SEARCH and O_EXEC slightly differently from O_PATH (O_NOFOLLOW has
> different semantics) and the kernel will ignore the extra access mode
> bits with O_PATH anyway.
At the same time, if I understand you correctly, you mean that O_PATH is a
different beast than O_SEARCH.
I've found a mail in the DragonFly mailing list, which also slightly
touches this topic[0].
They also propose to use value 3 as currently unused value; I don't know if
it is implemented yet.
FWIW, neither OpenBSD nor FreeBSD provide O_SEARCH flag; the latter
provides O_EXEC though.

I finally have a good access to the Internet so I managed to find a
detailed discussion on this topic[1].
Sorry guys (and especially you, Rich) that I didn't find it before; it
would have saved some questions earlier.

Rich, could you please elaborate on the exact semantics of O_SEARCH flag?
Again, thank you very much for your answer and for your patience!


[0] https://www.dragonflybsd.org/mailarchive/kernel/2009-08/msg00000.html
[1] https://sourceware.org/ml/libc-alpha/2013-08/msg00016.html

[-- Attachment #2: Type: text/html, Size: 2733 bytes --]

Re: readdir(3): behavior on descriptors with O_SEARCH

[Rich Felker]

On Sun, Aug 28, 2016 at 07:02:18PM +0300, Dmitry Selyutin wrote:
> Hi Rich,
> 
> thank you for the detailed answer!
> 
> > If a program is doing this then trying to fdopendir/readdir, it's a
> > bug in the program. A directory opened with O_SEARCH is only usable
> > for search (attempting to access a file/directory in that directory
> > by name, using one of the *at functions) not for reading. Unix has
> > always distinguished search (+x) and read (+r) permission for
> > directories and O_SEARCH vs O_RDONLY is similar.
> Well, that's probably the worst kind of bugs, since it was caused by
> misinterpretation of the documentation.
> That was written intentionally due to misunderstanding, so I really want to
> clarify the situation here.
> So does it mean that descriptors opened with O_SEARCH are usable only for
> *at functions?
> I've been under the impression that it's part of the O_PATH functionality,
> not O_SEARCH.

http://pubs.opengroup.org/onlinepubs/9699919799/functions/open.html

	O_RDONLY
	Open for reading only.

	...

	O_SEARCH
	Open directory for search only.

Note the word "only". Moreover it's clear from the permissions model
that you can't read from a directory opened for O_SEARCH, since
O_SEARCH does not require read permission; permission enforcement is
specific in the errors (and possibly elsewhere):

	[EACCESS]
	Search permission is denied on a component of the path prefix,
	or the file exists and the permissions specified by oflag are
	denied, or the file does not exist and write permission is
	denied for the parent directory of the file to be created, or
	O_TRUNC is specified and write permission is denied.

The relevant part of this text is "or the file exists and the
permissions specified by oflag are denied". If you could read from a
fd opened for O_SEARCH, you could bypass the check for "r" permission
at the fs level and read any directory to which you had "x"
permission. (In a sense you can do this anyway by brute-force search,
but that does not work when filenames in a +x-r directory are
sufficiently long and unpredictable, which is the case for secure uses
of +x-r.

> > It's been an ongoing fight even trying to get the kernel to reserve a
> > bit number for them. It looks like the correct course of action (the
> > one that's compatible with their non-action) is going to be using
> > O_PATH|3 for both of them. This allows userspace open to process
> > O_SEARCH and O_EXEC slightly differently from O_PATH (O_NOFOLLOW has
> > different semantics) and the kernel will ignore the extra access mode
> > bits with O_PATH anyway.
> At the same time, if I understand you correctly, you mean that O_PATH is a
> different beast than O_SEARCH.

Linux implemented O_PATH independently of POSIX O_SEARCH and O_EXEC,
not specifically with the intent to model them but rather to allow
more general things.

> I've found a mail in the DragonFly mailing list, which also slightly
> touches this topic[0].
> They also propose to use value 3 as currently unused value; I don't know if
> it is implemented yet.
> FWIW, neither OpenBSD nor FreeBSD provide O_SEARCH flag; the latter
> provides O_EXEC though.

Linux actually uses the value 3 for a wacky purpose: ioctl-only
opening of certain device nodes. Opening with mode 3 requires +rw
permissions. This was a huge waste of the value but I think they feel
obligated to keep it for compatibility.

> I finally have a good access to the Internet so I managed to find a
> detailed discussion on this topic[1].
> Sorry guys (and especially you, Rich) that I didn't find it before; it
> would have saved some questions earlier.
> 
> Rich, could you please elaborate on the exact semantics of O_SEARCH flag?
> Again, thank you very much for your answer and for your patience!

My understanding is that you can use an O_SEARCH fd for the *at
functions and in any place where an operation is just being performed
on the fd (like fstat, fchown, etc.) where the open modes are not
relavant (e.g. fchown does not use the mode the file was opened with
but checks permissions at the time of the fchown). You definitely
cannot use it for read or write operations.

Rich

Re: readdir(3): behavior on descriptors with O_SEARCH

[Dmitry Selyutin]

[-- Attachment #1: Type: text/plain, Size: 133 bytes --]

I think I got the idea; now I understand that the reported behavior is not
a bug. Thank you very much for explanations and patience!

[-- Attachment #2: Type: text/html, Size: 150 bytes --]