Re: Explaining cond var destroy [Re: [musl] C threads, v3.0]

[Rich Felker <dalias@libc.org>]

On Wed, Aug 06, 2014 at 12:32:52PM +0200, Jens Gustedt wrote:
> > I think, and perhaps
> > other serious drawbacks too. If I remember correctly, musl's original
> > implementation looked something like the above, but I was also having
> > a hard time figuring out how to work requeue-to-mutex (which makes a
> > big deal to performance on big broadcasts) into it.
> 
> Look into pthread_cond_broadcast. I think this could do better
> bookkeeping. Currently it does its case analysis with a mildly
> complicated expression in preparing the arguments to the
> requeue-syscall.

I don't think that expression is the problem. What it's doing is very
simple: it's useless to wake any threads if the thread calling
pthread_cond_broadcast currently holds the mutex, since the woken
thread would immediately block again on the mutex. Instead, one thread
will be woken when the mutex is unlocked. This is ensured by "moving"
any remaining wait count from the cond var waiters to the mutex
waiters (so that the mutex unlock will cause a futex wake).

> I think we could be better off by doing a case
> analysis beforehand and update the different "waiters" variables
> accordingly *before* the syscall. So the targeted threads wouldn't
> have to do any bookkeeping when coming back from wait.

One problem with this is that being woken by a signal isn't the only
way a cond var wait can end. It can also end due to timeout or
cancellation, and in that case, there's no signaling thread to be
responsible for the bookkeeping, but if another thread calls
pthread_cond_broadcast at this point, it can legally destroy and free
the cond var, despite the timed-out/cancelled wait still having its
own bookkeeping to do.

The best possible solution would be if we could eliminate the need for
this bookkeeping entirely, but I don't see any easy way to do that.

Rich