Re: New optimized normal-type mutex?

[Alexander Monakov <amonakov@ispras.ru>]

[-- Attachment #1: Type: text/plain, Size: 2160 bytes --]

> I will try to discuss this below.

Thanks for the detailed response.  Misc. comments below.
 
> > > With so little work in the critical section, it does
> > > not make sense to me that you would use something like a normal-type futex-y
> > > mutex.  Even a call/return to grab it gives you some overhead.  I'd expect you
> > > would use a fully inlined spinlock acquisition/release around the memory copy.
> > 
> > No, spinlocks are completely unusable in a POSIX libc that implements
> > priorities. They will deadlock whenever a lower-priority thread gets
> > preempted by a higher-priority one while holding the lock.

Thanks Rich for pointing that out.

[...]
> Now let us try to figure out what happens in the case that started
> this new discussion, namely that there is so much contention such that
> the probability of an EAGAIN-failure of the mutex call increases.

As long as we're talking in terms of futexes, it's EWOULDBLOCK, not EAGAIN.
 
> In fact even if inside the critical section the application itself
> does nothing, the lock code first does the spinning. So regardless of
> the application, any thread does 100 spins (so some work) before it
> starts fighting to be put to sleep on the futex. None of the threads
> reaching that point, changes the value of of the counter, so all
> threads that reach the point and call futex_wait "simultaneously" know
> about the same actual value.
> 
> The value can only change by new threads entering the acquisition
> loop. There are no more such threads at a time than can be effectively
> be scheduled. These will all spend some time spinning, so there can't
> be a bad cascade of such threads that change the counter.

OK — initially I missed that spinning a bit before *each* futex_wait is going
to work well for you.

How do you imagine it to scale when the number of CPUs increases?  Naively, it
appears that the more CPUs are in contention, the more you would need to spin?
 
> In fact, there will certainly be a tradeoff between the number of
> spins and the overall waste in CPU time and bandwidth. I will
> experiment a bit with that.

Curious to hear what you find.

Thanks!
Alexander