Re: bug in pthread_cond_broadcast

[Szabolcs Nagy <nsz@port70.net>]

* Jens Gustedt <jens.gustedt@inria.fr> [2014-08-12 01:58:52 +0200]:
> thread_ret client(void *arg) {
>   unsigned * number = arg;
>   for (unsigned i = 0; i < phases; ++i) {
>     trace("thread %u in phase %u\n", *number, i);
>     mutex_lock(&mut[i]);
>     ++inside[i];
>     if (inside[i] == threads) {
>       trace("thread %u is last, signalling main\n", *number);
>       int ret = condition_signal(&cond_main);

the last client at the end of phase 0 wakes the main thread here

the main thead is waiting on cond_main using mut[0]

>       trace("thread %u is last, signalling main, %s\n", *number, errorstring(ret));
>     }
>     while (i == phase) {
>       tell("thread %u in phase %u (%u), waiting\n", *number, i, phase);
>       int ret = condition_wait(&cond_client, &mut[i]);
>       trace("thread %u in phase %u (%u), finished, %s\n", *number, i, phase, errorstring(ret));

the last client thread will wait here unlocking mut[0] so
the main thread can continue

the main thread broadcast wakes all clients while holding both
mut[0] and mut[1] then unlocks mut[0] and starts waiting on
cond_main using mut[1]

the awaken clients will go into the next phase locking mut[1]
and waiting on cond_client using mut[1]

however there might be still clients waiting on cond_client
using mut[0] (eg. the broadcast is not yet finished)

i see logs where one thread is already in phase 1 (using mut[1])
while another is not yet out of condition_wait (using mut[0]):

pthread_cond_smasher.c:120: thread 3 in phase 1 (1), waiting
pthread_cond_smasher.c:122: thread 6 in phase 0 (1), finished, No error information

"When a thread waits on a condition variable, having specified a particular
 mutex to either the pthread_cond_timedwait() or the pthread_cond_wait()
 operation, a dynamic binding is formed between that mutex and condition
 variable that remains in effect as long as at least one thread is blocked
 on the condition variable. During this time, the effect of an attempt by
 any thread to wait on that condition variable using a different mutex is
 undefined. "

so are all clients considered unblocked after a broadcast?

>     }
>     int ret = mutex_unlock(&mut[i]);
>     trace("thread %u in phase %u (%u), has unlocked mutex: %s\n", *number, i, phase, errorstring(ret));
>   }
>   return 0;
> }
> 
> 
> int main(void) {
>   tell("start up of main, using %s, library %s\n", VERSION, LIBRARY);
>   condition_init(&cond_client);
>   condition_init(&cond_main);
>   for (unsigned i = 0; i < phases; ++i) {
>     mutex_init(&mut[i]);
>   }
>   mutex_lock(&mut[0]);
> 
>   for (unsigned i = 0; i < threads; ++i) {
>     args[i] = i;
>     thread_create(&id[i], client, &args[i]);
>   }
> 
>   while (phase < phases) {
>     while (inside[phase] < threads) {
>       trace("main seeing %u threads in phase %u, waiting\n", inside[phase], phase);
>       int ret = condition_wait(&cond_main, &mut[phase]);
>       tell("main seeing %u threads in phase %u, %s\n", inside[phase], phase, errorstring(ret));
>     }
>     /* now we know that everybody is waiting inside, lock the next
>        mutex, if any, such that nobody can enter the next phase
>        without our permission. */
>     if (phase < phases-1)
>       mutex_lock(&mut[phase+1]);
>     /* Now signal all clients, update the phase count and release the
>        mutex they are waiting for. */
>     int ret = condition_broadcast(&cond_client);
>     trace("main has broadcast to %u: %s\n", phase, errorstring(ret));
>     ++phase;
>     ret = mutex_unlock(&mut[phase-1]);
>     trace("main has unlocked mutex %u: %s\n", phase-1, errorstring(ret));
>   }
> 
>