[musl] Feasibility of FD_CLOEXEC on all streams

[musl] Feasibility of FD_CLOEXEC on all streams

[Markus Wichmann]

Hi all,

I was recently reading the source code of popen(), and noticed that it
has to iterate over all open files to close all the open pipe FDs the
child might inherit. And that made me wonder:

1. Does POSIX allow for all FILE streams to have FD_CLOEXEC applied by
default?

2. Is that something we might wish to explore?

Number two I will just have to open to debate here on the list (and
let's be honest, Rich is going to be the one to have final say on the
matter).

As for number one, obviously ISO C isn't going to say anything on the
matter one way or the other, seeing as ISO C doesn't know about exec.
And POSIX has a chapter talking about the relationship between FDs and
streams, that says explicitly that after exec all streams are going to
be closed, no matter what FDs remain open.

I could find nothing condemning or condoning this approach. So it
appears to be a valid implementation choice.

To be clear, I am basically only talking about adding O_CLOEXEC to the
open() call in fopen(), and keeping the FD_CLOEXEC flag set on the pipe
FD in popen(). fdopen() would remain as is. That means that fopen() with
"e" in the mode string is still possible, only it does nothing other
than without the "e".

The technical benefits are minor, admittedly. The loop that closes all
pipe FDs in popen() could be removed. And that is mostly it. Programs
using fopen() that spawn subprocesses can no longer forget to close
those FDs, limiting FD leakage. Which usually is not a security problem,
but can be. But in most instances where it is, the program is buggy with
glibc, so the bug would need to be fixed on the application level
(programs cannot rely on this behavior). So on the advantages side, we
would be moving closer to "security-by-default".

Still, I don't foresee too many technical drawbacks, either. The only
case I can think of that would fail now is if a program were to open a
file with fopen(), and try to bestow the FD to a subprocess, and only
dup() it if it does not equal an expected value. E.g.

    FILE *f = fopen(...);
    ...
    if (fileno(f) != 3)
        dup2(fileno(f), 3);
    exec(program that does something with FD 3);

But I would expect such usage to be extremely rare.

Thoughts?

Ciao,
Markus

Re: [musl] Feasibility of FD_CLOEXEC on all streams

[Rich Felker]

On Sat, Dec 18, 2021 at 05:33:20PM +0100, Markus Wichmann wrote:
> Hi all,
> 
> I was recently reading the source code of popen(), and noticed that it
> has to iterate over all open files to close all the open pipe FDs the
> child might inherit. And that made me wonder:
> 
> 1. Does POSIX allow for all FILE streams to have FD_CLOEXEC applied by
> default?

No. Accessing fileno(f) is permissible subject to following the rules
for active handle:

https://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_05_01

and that entails being able to use them according to the rules for how
fds are inherited across exec.

Rich

Re: [musl] Feasibility of FD_CLOEXEC on all streams

[Rich Felker]

On Sat, Dec 18, 2021 at 12:14:15PM -0500, Rich Felker wrote:
> On Sat, Dec 18, 2021 at 05:33:20PM +0100, Markus Wichmann wrote:
> > Hi all,
> > 
> > I was recently reading the source code of popen(), and noticed that it
> > has to iterate over all open files to close all the open pipe FDs the
> > child might inherit. And that made me wonder:
> > 
> > 1. Does POSIX allow for all FILE streams to have FD_CLOEXEC applied by
> > default?
> 
> No. Accessing fileno(f) is permissible subject to following the rules
> for active handle:
> 
> https://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_05_01
> 
> and that entails being able to use them according to the rules for how
> fds are inherited across exec.

Also, the POSIX spec for fopen is rather explicit:

    "[CX] The file descriptor associated with the opened stream shall
    be allocated and opened as if by a call to open() with the
    following flags: ..."

https://pubs.opengroup.org/onlinepubs/9699919799/functions/fopen.html

Rich

Re: [musl] Feasibility of FD_CLOEXEC on all streams

[Alex Xu (Hello71)]

Excerpts from Rich Felker's message of December 18, 2021 12:26 pm:
> On Sat, Dec 18, 2021 at 12:14:15PM -0500, Rich Felker wrote:
>> On Sat, Dec 18, 2021 at 05:33:20PM +0100, Markus Wichmann wrote:
>> > Hi all,
>> > 
>> > I was recently reading the source code of popen(), and noticed that it
>> > has to iterate over all open files to close all the open pipe FDs the
>> > child might inherit. And that made me wonder:
>> > 
>> > 1. Does POSIX allow for all FILE streams to have FD_CLOEXEC applied by
>> > default?
>> 
>> No. Accessing fileno(f) is permissible subject to following the rules
>> for active handle:
>> 
>> https://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_05_01
>> 
>> and that entails being able to use them according to the rules for how
>> fds are inherited across exec.
> 
> Also, the POSIX spec for fopen is rather explicit:
> 
>     "[CX] The file descriptor associated with the opened stream shall
>     be allocated and opened as if by a call to open() with the
>     following flags: ..."
> 
> https://pubs.opengroup.org/onlinepubs/9699919799/functions/fopen.html
> 
> Rich
> 

Playing devil's advocate here, can't the implementation unset FD_CLOEXEC 
when fileno is called? This doesn't fix the latter issue, but if that's 
the only problem then I would argue that it can be sufficiently covered 
by the as-if rule. It also wouldn't fix the popen loop, but would still 
add some hardening for poorly written programs.

Cheers,
Alex.

Re: [musl] Feasibility of FD_CLOEXEC on all streams

[Rich Felker]

On Sun, Dec 19, 2021 at 09:54:32AM -0500, Alex Xu (Hello71) wrote:
> Excerpts from Rich Felker's message of December 18, 2021 12:26 pm:
> > On Sat, Dec 18, 2021 at 12:14:15PM -0500, Rich Felker wrote:
> >> On Sat, Dec 18, 2021 at 05:33:20PM +0100, Markus Wichmann wrote:
> >> > Hi all,
> >> > 
> >> > I was recently reading the source code of popen(), and noticed that it
> >> > has to iterate over all open files to close all the open pipe FDs the
> >> > child might inherit. And that made me wonder:
> >> > 
> >> > 1. Does POSIX allow for all FILE streams to have FD_CLOEXEC applied by
> >> > default?
> >> 
> >> No. Accessing fileno(f) is permissible subject to following the rules
> >> for active handle:
> >> 
> >> https://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_05_01
> >> 
> >> and that entails being able to use them according to the rules for how
> >> fds are inherited across exec.
> > 
> > Also, the POSIX spec for fopen is rather explicit:
> > 
> >     "[CX] The file descriptor associated with the opened stream shall
> >     be allocated and opened as if by a call to open() with the
> >     following flags: ..."
> > 
> > https://pubs.opengroup.org/onlinepubs/9699919799/functions/fopen.html
> 
> Playing devil's advocate here, can't the implementation unset FD_CLOEXEC 
> when fileno is called? This doesn't fix the latter issue, but if that's 
> the only problem then I would argue that it can be sufficiently covered 
> by the as-if rule.

No, because file descriptors are required to be assigned in
lowest-unused order and some really bad software could skip fileno and
just assume it got the fd number it wanted(*). In fact this is somewhat
common practice for the standard streams, albeit awful, and usually
with just plain open not fopen.

(*) One could argue that this is invalid usage, as library functions
are allowed to open and close file descriptors for their own internal
use as long as it's not visible to the application. However,
interpreted too loosely, that would effectively nullify the
lowest-unused requirement, so I would read that allowance with an
as-if rule, that the "lowest-unused" has to be assigned as if the set
of already-used fds was the same as at the time of call.

> It also wouldn't fix the popen loop, but would still 
> add some hardening for poorly written programs.

This "hardening" should be understood as avoiding a potential fd leak
in erroneous programs at the expense of *introducing use-after-close
bugs* in very-bad-style-but-correct programs. This does not seem like
a reasonable tradeoff at all.

Rich