Re: vfork replacement proposal

[Rich Felker <dalias@aerifal.cx>]

On Mon, Dec 31, 2012 at 03:34:17PM -0500, Rich Felker wrote:
> I've been looking for a viable replacement of the vfork usage in musl
> for a while, since it has two serious problems:
> 
> 1. strace is buggy and causes the parent and child to run
> simultaneously on the same stack under vfork when the process is being
> traced. Binaries which can crash or go crazy under strace are highly
> undesirable, even if the fault is with strace.
> 
> 2. While current compilers don't do this, the compiler is conceptually
> free to generate code that clobbers parts of the stack that still need
> to be used by the parent when it determines they are no longer needed
> in the child.
> 
> The affected functions are posix_spawn[p], system, and popen.
> 
> My new proposed design for these functions is:

I've implemented the new design and it seems to be working. After a
few more checks, I'll commit it and see if anybody can give it some
stress testing.

> 4. In the child, close the read end of the pipe and then shuffle file
> descriptors as needed (for setting up stdin/out for popen, or file
> actions for posix_spawn[p]), but with the added stipulations A-C:
> 
> A. Before closing or dup2'ing onto a file descriptor in file actions,
> check to see if it's occupied by the pipe fd, and if so, use fcntl
> F_DUPFD_CLOEXEC to move it to a new number first.
> 
> B. Before calling open in file actions, always use fcntl with
> F_DUPFD_CLOEXEC and close the original pipe fd, to ensure that the
> pipe is never occupying the otherwise-lowest-available fd number.

I was wrong about (B); the "open" file action does not assign the
lowest-available fd, but a caller-chosen fd. Thus, for our purposes,
it's just like close or dup2, targetting a known fd number. This means
the same logic can be used for all three operations, and it can be
based on dup() rather than F_DUPFD_CLOEXEC. Note that F_DUPFD_CLOEXEC
is actually not viable because it's missing on slightly-old kernels
(up through mid 2.6 series), but we don't need atomicity anyway since
this thread/process is fully under posix_spawn's control.

Also, I think it would be possible to abandon the "shuffling" logic
and compute in advance a safe fd number to put the pipe on. 

Finally, it seems posix_spawn will be sufficient as a backend for
implementing popen, wordexp, and system, so I just put all the logic
in posix_spawn itself rather than trying to design a more abstract API
with callbacks for the specific caller case.

Rich