[musl] aio_close needed in dup2?

[musl] aio_close needed in dup2?

[Markus Wichmann]

Hi all,

I noticed something today: In close(), we call __aio_close() to both
prevent AIO from using invalid file descriptors and implement the
requirement that outstanding AIO be cancelled. But in dup2() and dup3(),
that doesn't happen.

POSIX only says that dup2() closes newfd if it already is a valid file
descriptor. While not explicitly stated, I can't really find a sensible
interpretation of that requirement that is different from "as if by way
of close()". POSIX has no concept of closing file descriptors in any
other way. And dup3() is an extension function, but I think most
programmers will understand it to be an extension of dup2(), so the same
argument applies there.

So, do we need to call __aio_close() in dup2() and dup3()?

Ciao,
Markus

Re: [musl] aio_close needed in dup2?

[Rich Felker]

On Thu, Oct 12, 2023 at 06:43:20PM +0200, Markus Wichmann wrote:
> Hi all,
> 
> I noticed something today: In close(), we call __aio_close() to both
> prevent AIO from using invalid file descriptors and implement the
> requirement that outstanding AIO be cancelled. But in dup2() and dup3(),
> that doesn't happen.
> 
> POSIX only says that dup2() closes newfd if it already is a valid file
> descriptor. While not explicitly stated, I can't really find a sensible
> interpretation of that requirement that is different from "as if by way
> of close()". POSIX has no concept of closing file descriptors in any
> other way. And dup3() is an extension function, but I think most
> programmers will understand it to be an extension of dup2(), so the same
> argument applies there.
> 
> So, do we need to call __aio_close() in dup2() and dup3()?

I'm not sure. Unlike close, which invalidates the fd and makes any
subsequent use by aio a use-after-close bug (extremely dangerous),
dup2/dup3 does not invalidate the fd. They change what it refers to.
While I'm not sure this is sanctioned by POSIX, it would be reasonable
to want to *atomically replace* an fd that aio is pending on, such
that the operations happen either on the previously-referenced open
file description or the new one (but never zero or both).

On the other hand, I don't see a safe way to implement __aio_close
semantics for dup2/dup3, because you can't know in advance whether
it's going to succeed, and once it succeeds, you can no longer do the
work __aio_close would have done. I think you'd have to juggle around
temporary fds to make it work in any reasonable way.

Most likely, it's just supposed to be UB to replace an fd with pending
aio operations on it using dup2/dup3. (BTW, dup3 is not just an
extension, it's POSIX-future, so all this applies to it too.)

Rich

Re: [musl] aio_close needed in dup2?

[Gabriel Ravier]

On 10/12/23 18:43, Rich Felker wrote:
> On Thu, Oct 12, 2023 at 06:43:20PM +0200, Markus Wichmann wrote:
>> Hi all,
>>
>> I noticed something today: In close(), we call __aio_close() to both
>> prevent AIO from using invalid file descriptors and implement the
>> requirement that outstanding AIO be cancelled. But in dup2() and dup3(),
>> that doesn't happen.
>>
>> POSIX only says that dup2() closes newfd if it already is a valid file
>> descriptor. While not explicitly stated, I can't really find a sensible
>> interpretation of that requirement that is different from "as if by way
>> of close()". POSIX has no concept of closing file descriptors in any
>> other way. And dup3() is an extension function, but I think most
>> programmers will understand it to be an extension of dup2(), so the same
>> argument applies there.
>>
>> So, do we need to call __aio_close() in dup2() and dup3()?
> I'm not sure. Unlike close, which invalidates the fd and makes any
> subsequent use by aio a use-after-close bug (extremely dangerous),
> dup2/dup3 does not invalidate the fd. They change what it refers to.
> While I'm not sure this is sanctioned by POSIX, it would be reasonable
> to want to *atomically replace* an fd that aio is pending on, such
> that the operations happen either on the previously-referenced open
> file description or the new one (but never zero or both).

I can confirm that current POSIX wants dup2 to be atomic:

 > All of the following functions shall be atomic with respect to each 
other in the effects specified in
POSIX.1-2008 when they operate on regular files or symbolic links:
 > [list that includes many functions operating on file descriptors, 
including dup2]
- POSIX-1.2008, 2.9.7 *Thread Interactions with Regular File Operations*

The latest draft has similar wording which also includes dup3, in 
particular:

 > Except where specified otherwise, all of the following functions 
shall be atomic with respect to each other in the effects specified in 
POSIX.1-202x when they operate on file descriptors that are open, or 
being opened, to files in the file hierarchy:
 > [list that includes many functions operating on file descriptors, 
including dup2 and dup3]
 > If two threads each call one of these functions, each call shall 
either see all of the specified effects of the other call, or none of 
them. The requirement on the close() function shall also apply whenever 
a file descriptor is successfully closed, however caused (for example, 
as a consequence of calling close(), calling dup2(), or of process 
termination).
- POSIX.1-202x

with the specification of `close` including:
 > When there is an outstanding cancelable asynchronous I/O operation 
against fildes when close() is called, that I/O operation may be 
canceled. An I/O operation that is not canceled completes as if the 
close() operation had not yet occurred. All operations that are not 
canceled shall complete as if the close() blocked until the operations 
completed. The close() operation itself need not block awaiting such I/O 
completion. Whether any I/O operation is canceled, and which I/O 
operation may be canceled upon close(), is implementation-defined.
- POSIX.1-202x

and nothing in the specification of dup3 or dup2 indicating that they 
are not atomic.

>
> On the other hand, I don't see a safe way to implement __aio_close
> semantics for dup2/dup3, because you can't know in advance whether
> it's going to succeed, and once it succeeds, you can no longer do the
> work __aio_close would have done. I think you'd have to juggle around
> temporary fds to make it work in any reasonable way.
>
> Most likely, it's just supposed to be UB to replace an fd with pending
> aio operations on it using dup2/dup3. (BTW, dup3 is not just an
> extension, it's POSIX-future, so all this applies to it too.)
>
> Rich