Re: Explaining cond var destroy [Re: [musl] C threads, v3.0]

[Rich Felker <dalias@libc.org>]

On Tue, Aug 12, 2014 at 09:04:39AM +0200, Jens Gustedt wrote:
> > > Generally I think that the control structures should be as tight as
> > > possible, give provable properties in the mathematical sense. The
> > > interaction between user- and kernelland should be minimal, and we
> > > shouldn't provoque reactions of the kernel that concern threads (or
> > > even process) that are not really targetted. 
> > 
> > The former (provable properties) is definitely a goal we should not
> > deviate from. But I don't think the current spurious futex wakes
> > conflict with that goal.
> > 
> > The latter (not provoking reactions in untargetted threads) is a
> > desirable goal, but not if it conflicts with more important goals like
> > avoiding unnecessary allocation (actually, I don't think it's possible
> > to solve the problem with allocation; I think an additional layer of
> > allocation just makes it worse), fail-safety, performance, etc.
> 
> Did you have a chance to look into my recent implementation of C
> threads that I attached to my last post? In particular in
> cnd_broadcast you see the advantages, I think:
> 
>  - cnd doesn't have to do bookkeeping for the threads waiting on the
>    condition, the kernel bookkeeping is used for that
> 
>  - threads that had to go into futex wait only touch the temporary
>    structure and this only for the reference count
> 
>  - a tight spinlock clearly defines the ordering of the cnd_t
>    operations

Yes, I've spent a total of about 30-45 min reading it, so I have a
fairly good idea what it's doing, but my understanding surely has
gaps.

As far as I can tell, the only thing that's saving you from sending
futex wakes after free is that you're just using spinlocks. This is an
extremely expensive solution: While contention is rare, as soon as you
do hit contention, if there are many threads they all pile on and
start spinning, and the time to obtain a lock (and cpu time/energy
spent waiting) grows extremely high. And of course it becomes infinite
if you have any threads of differing priorities and the low-priority
thread has the lock...

Rich