Re: Explaining cond var destroy [Re: [musl] C threads, v3.0]

[Rich Felker <dalias@libc.org>]

On Wed, Aug 06, 2014 at 11:41:04AM +0200, Jens Gustedt wrote:
> Am Mittwoch, den 06.08.2014, 10:43 +0200 schrieb Jens Gustedt:
> > Am Dienstag, den 05.08.2014, 23:52 -0400 schrieb Rich Felker:
> > > If you think this is a bad idea, I'd be willing to hear alternate
> > > ideas. I'm not really happy with the cond var implementation (if
> > > nothing else, the sequence number thing is an ugly hack and not 100%
> > > robust, I think) and at some point I'd like to redesign it.
> > 
> > I am not sure about how to deal with this. The idea that destroy may
> > be blocking or even be doing a context switch to the kernel came as a
> > surprise to me. Maybe I would be happier if _c_destroy would be used
> > as a usage count and destroy would just spinlock on that would be more
> > straight.
> 
> After digging a bit deeper I think I would favor a solution where no
> waiter has to touch the condition object at all when coming back from
> the wait. Currently this is not the case, in the contrary it reads
> _c_seq and then does heavy maintenance in the unwait() call.
> 
> The return from a wait should just run into the the call to
> pthread_mutex_lock, which would be save since the call has access to
> that mutex from its arguments. All the bookkeeping should be done by
> pthread_cond_signal and pthread_cond_broadcast before they wake up
> anybody.
> 
> I am not sure that this would easily be possible with the current
> implementation, but I think that this would be ideal.

Yes, that would be my ideal too, but I'm not sure it's possible. Well,
not with an efficient implementation anyway. There's a trivial
implementation where pthread_cond_timedwait is basically just:

a_store(&c->_tid, self->tid);
pthread_mutex_unlock(mtx);
__timedwait(&c->tid, self->tid, ts, clock, ts, cleanup, mtx, 0);
pthread_mutex_lock(mtx);

This makes signal and broadcast operations utterly trivial (just set
c->tid to 0 and call __wake with 1 or INT_MAX as the argument), but it
has lots of spurious wakes under contention, I think, and perhaps
other serious drawbacks too. If I remember correctly, musl's original
implementation looked something like the above, but I was also having
a hard time figuring out how to work requeue-to-mutex (which makes a
big deal to performance on big broadcasts) into it.

One idea I've been playing with is possibly eliminating most or all
use of waiter counts in musl and using a pure potential-waiters-flag
approach. That's a whole separate topic I should start a thread on,
but to be brief, it's motivated by some pathological behavior of
waiter counts under rapid lock/unlock by the same thread where a
useless futex wake happens after each unlock if the waiter has not yet
gotten a chance to run. If eliminating waiter counts would make it
easier to do a good cond var implementation that would be another
reason to consider it.

Rich