32-bit vdso syscall cancellation proposal

[Andy Lutomirski <luto@amacapital.net>]

Hi-

After chatting with Rich Felker on IRC, he's convinced me that the
32-bit vsyscall mechanism is annoying for syscall cancellation.  To
help, I propose we add two functions:

unsigned long __vdso_pending_vsyscall_eip(void *context);

If context is a signal context pointing to the AT_SYSINFO entry and if
that entry has *not* executed its syscall yet (or if it has but it's
about to resume it), then returns the EIP to which the vsyscall will
return.  Otherwise returns -1.

int __vdso_cancel_pending_vsyscall(void *context);

If not in a valid pending vsyscall, returns -EINVAL.  Otherwise
clobbers all vsyscall return registers except ESP, EIP, and EAX and
arranges for EAX to be -ENOSYS on return and for the vsyscall to be
skipped.  Returns 0.

(Musl would slightly prefer -EINTR, but that's harder to implement.
Musl would like the ability to *change* the return EIP even more, but
that's considerably harder, given that the location of the saved EIP
will vary depending on where we are in the vsyscall.)

Ingo, if I implement something like this, it'll come with a careful
test case.  I already have a test case that tests the CFI annotations
(by single-stepping through a syscall and asserting that it unwinds
correctly from every single SIGTRAP).  It'll also be on top of a
series that turns __kernel_vsyscall into a *single* function.

__vdso_pending_vsyscall_eip can't be reliably implemented using the
CFI annotations, because you can't tell which side of the syscall
you're on.  __vdso_cancel_pending_vsyscall could be, but that's quite
heavyweight, and implementing it directly will only be a couple lines
of code.

--Andy

-- 
Andy Lutomirski
AMA Capital Management, LLC