Fixing set*id() rlimit hack

Fixing set*id() rlimit hack

[Rich Felker]

The last Alpine-local change I'm trying to resolve for release is that
they're patching out the hack in the multithreaded set*id() code that
temporarily sets RLIMIT_NPROC to infinity before making the set*id
syscalls and fails the operation if it can't change the limit. The
purpose of this code is to prevent the nasty situation, on old
(pre-3.1) kernels, where RLIMIT_NPROC is enforced at set*id() time and
can cause the operation to fail for some threads, leaving the process
in an inconsistent and dangerous state. The problem with the current
code is that, if you use setreuid() to swap real/effective uid to
"temporarily drop root", swapping back will fail as a result of
lacking permissions to re-raise RLIMIT_NPROC to infinity.

There is possibly an elaborate way to work around the problem by
retaining RLIMIT_NPROC at infinity whenever the destination uid
combination would permit regaining root, but this is complex
(especially with regards to synchronization and global state) and I'm
afraid that even without rlimits coming into play, there may be
spurious ways the set*id syscalls can fail on Linux, due to its poorly
designed handling of the storage for permissions on the kernel side.
So I think there's probably a need to add some additional checking for
inconsistent state with raise(SIGKILL) in the failure path.

Is it reasonable to just remove the rlimit hack at the same time? Or
should it be kept, perhaps just as a best-effort approach, i.e.
attempting to set it to infinity, but still trying to operation if it
can't be raised? Or if it's kept, the rlimit-raising code could be
activated only for kernels <3.1, but I really don't like hard-coded
version comparisons. (RHEL is likely to have a kernel that calls
itself 2.6 or even 2.4 but has this fix from 3.1 "backported"...)

Beyond the rlimit issue, there are other serious issues with
multithreaded set*id() which I'd like to fix, like the lack of
AS-safety, but I don't yet know how. These can probably be pushed back
to the next release cycle, though.

Rich