Compile-time flag to enable optional EINTR's?

Compile-time flag to enable optional EINTR's?

[Joseph C. Sible]

I know musl has reasons not to enable EINTR where it's optional (such
as c0ed5a20), but there are a lot of use cases where the lack of it
causes problems. As a compromise, would a patch to add a ./configure
flag (say --enable-optional-eintr) to change this behavior be
accepted?

Joseph C. Sible

Re: Compile-time flag to enable optional EINTR's?

[Rich Felker]

On Thu, Sep 06, 2018 at 09:26:19PM -0400, Joseph C. Sible wrote:
> I know musl has reasons not to enable EINTR where it's optional (such
> as c0ed5a20), but there are a lot of use cases where the lack of it
> causes problems. As a compromise, would a patch to add a ./configure
> flag (say --enable-optional-eintr) to change this behavior be
> accepted?

Not having configurable switches is a very conscious decision for
musl. The exponential complexity that results makes it nearly
impossible to manage testing, and was a big part of what happened to
uclibc. Instead we aim to support as wide a range of needs/use cases
as possible with a single configuration.

Can you discuss what you're trying to use EINTR for here? Most uses of
EINTR have fundamental race conditions -- if the signal arrives just
moments before the syscall you hoped to interrupt, it won't get
interrupted, and will block until some other event lets it proceed.

If we did want to bring back EINTR for sem_[timed]wait, I think the
right thing to do would be to look for a workaround for the underlying
kernel bug, or some way to detect it and avoid honoring the EINTRs
that happen on old buggy kernels. One thing that was once suggested,
but I didn't really like it, was ignoring (retrying on) EINTR for
sem_wait, where the caller may not be prepared for it to return
without having decremented the semaphore value, but honoring it
(failing) for sem_timedwait, where the caller has to be prepared for
the possibility of failure (ETIMEDOUT) anyway. I didn't do much
research into whether this would be conforming but I think it would.

Rich

Re: Compile-time flag to enable optional EINTR's?

[Joseph C. Sible]

> Can you discuss what you're trying to use EINTR for here?

It's for a Python program with several threads that are often waiting
on various locks, events, etc., all implemented by Python in terms of
POSIX semaphores. If a thread is just waiting for work to do, and I
trigger a graceful shutdown via a signal, I'd like it to give up
waiting for work immediately rather than holding up shutdown until it
times out doing nothing.

> Most uses of
> EINTR have fundamental race conditions -- if the signal arrives just
> moments before the syscall you hoped to interrupt, it won't get
> interrupted, and will block until some other event lets it proceed.

I acknowledge the race condition. It's too bad there's no "p" variant
of sem_timedwait(3) (thus of futex(2) as well) like there is of
pselect vs. select.

> If we did want to bring back EINTR for sem_[timed]wait, I think the
> right thing to do would be to look for a workaround for the underlying
> kernel bug, or some way to detect it and avoid honoring the EINTRs
> that happen on old buggy kernels. One thing that was once suggested,
> but I didn't really like it, was ignoring (retrying on) EINTR for
> sem_wait, where the caller may not be prepared for it to return
> without having decremented the semaphore value, but honoring it
> (failing) for sem_timedwait, where the caller has to be prepared for
> the possibility of failure (ETIMEDOUT) anyway. I didn't do much
> research into whether this would be conforming but I think it would.

Either one of these solutions sounds okay to me. I don't have any good
ideas on how to do the former, though.

Joseph C. Sible

Re: Compile-time flag to enable optional EINTR's?

[Rich Felker]

On Fri, Sep 07, 2018 at 12:09:33AM -0400, Joseph C. Sible wrote:
> > Can you discuss what you're trying to use EINTR for here?
> 
> It's for a Python program with several threads that are often waiting
> on various locks, events, etc., all implemented by Python in terms of
> POSIX semaphores.

I seem to recall Python's testsuite having a test asserting that POSIX
semaphores respond to interrupting signals, so it sounds like the
underlying problem is that Python is implementing some of its
primitives with a non-portable assumption. I'll take a look or see if
someone else from the community wants to.

Rich

Re: Compile-time flag to enable optional EINTR's?

[Joseph C. Sible]

On Fri, Sep 7, 2018 at 9:15 AM Rich Felker <dalias@libc.org> wrote:
> I seem to recall Python's testsuite having a test asserting that POSIX
> semaphores respond to interrupting signals, so it sounds like the
> underlying problem is that Python is implementing some of its
> primitives with a non-portable assumption. I'll take a look or see if
> someone else from the community wants to.

You're correct. I agree that they're in the wrong here. I opened
https://bugs.python.org/issue34004 about it a while ago. I'm not sure
what the best way is to fix it though. The only way I know of to get
rid of the race condition is pselect/ppoll/etc. (all way heavier than
futex), something nonportable like signalfd or eventfd, or having
another thread to do all the signal handling and wake up all the
futexes (and the associated complexity of making sure they never get
the two wakeup cases mixed up). Do you have an opinion as to the best
solution that could be done on their end?

Joseph C. Sible