[PATCH v2] fix race condition in file locking

[PATCH v2] fix race condition in file locking

[Kaarle Ritvanen]

The condition occurs when
- thread #1 is holding the lock
- thread #2 is waiting for it on __futexwait
- thread #1 is about to release the lock and performs a_swap
- thread #3 enters the __lockfile function and manages to grab the lock
  before thread #1 calls __wake, resetting the MAYBE_WAITERS flag
- thread #1 calls __wake
- thread #2 wakes up but goes again to __futexwait as the lock is
  held by thread #3
- thread #3 releases the lock but does not call __wake as the
  MAYBE_WAITERS flag is not set

This condition results in thread #2 not being woken up. This patch fixes
the problem by making the woken up thread ensure that the flag is
properly set before going to sleep again.
---
 src/stdio/__lockfile.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/src/stdio/__lockfile.c b/src/stdio/__lockfile.c
index 2ff75d8a..0dcb2a42 100644
--- a/src/stdio/__lockfile.c
+++ b/src/stdio/__lockfile.c
@@ -8,13 +8,13 @@ int __lockfile(FILE *f)
 	int owner = f->lock, tid = __pthread_self()->tid;
 	if ((owner & ~MAYBE_WAITERS) == tid)
 		return 0;
-	for (;;) {
-		owner = a_cas(&f->lock, 0, tid);
-		if (!owner) return 1;
-		if (a_cas(&f->lock, owner, owner|MAYBE_WAITERS)==owner) break;
+	owner = a_cas(&f->lock, 0, tid);
+	if (!owner) return 1;
+	while ((owner = a_cas(&f->lock, 0, tid|MAYBE_WAITERS))) {
+		if ((owner & MAYBE_WAITERS) ||
+		    a_cas(&f->lock, owner, owner|MAYBE_WAITERS)==owner)
+			__futexwait(&f->lock, owner|MAYBE_WAITERS, 1);
 	}
-	while ((owner = a_cas(&f->lock, 0, tid|MAYBE_WAITERS)))
-		__futexwait(&f->lock, owner, 1);
 	return 1;
 }
 
-- 
2.14.4

Re: [PATCH v2] fix race condition in file locking

[Rich Felker]

On Tue, Sep 18, 2018 at 10:03:27AM +0300, Kaarle Ritvanen wrote:
> The condition occurs when
> - thread #1 is holding the lock
> - thread #2 is waiting for it on __futexwait
> - thread #1 is about to release the lock and performs a_swap
> - thread #3 enters the __lockfile function and manages to grab the lock
>   before thread #1 calls __wake, resetting the MAYBE_WAITERS flag
> - thread #1 calls __wake
> - thread #2 wakes up but goes again to __futexwait as the lock is
>   held by thread #3
> - thread #3 releases the lock but does not call __wake as the
>   MAYBE_WAITERS flag is not set
> 
> This condition results in thread #2 not being woken up. This patch fixes
> the problem by making the woken up thread ensure that the flag is
> properly set before going to sleep again.
> ---
>  src/stdio/__lockfile.c | 12 ++++++------
>  1 file changed, 6 insertions(+), 6 deletions(-)
> 
> diff --git a/src/stdio/__lockfile.c b/src/stdio/__lockfile.c
> index 2ff75d8a..0dcb2a42 100644
> --- a/src/stdio/__lockfile.c
> +++ b/src/stdio/__lockfile.c
> @@ -8,13 +8,13 @@ int __lockfile(FILE *f)
>  	int owner = f->lock, tid = __pthread_self()->tid;
>  	if ((owner & ~MAYBE_WAITERS) == tid)
>  		return 0;
> -	for (;;) {
> -		owner = a_cas(&f->lock, 0, tid);
> -		if (!owner) return 1;
> -		if (a_cas(&f->lock, owner, owner|MAYBE_WAITERS)==owner) break;
> +	owner = a_cas(&f->lock, 0, tid);
> +	if (!owner) return 1;
> +	while ((owner = a_cas(&f->lock, 0, tid|MAYBE_WAITERS))) {
> +		if ((owner & MAYBE_WAITERS) ||
> +		    a_cas(&f->lock, owner, owner|MAYBE_WAITERS)==owner)
> +			__futexwait(&f->lock, owner|MAYBE_WAITERS, 1);
>  	}
> -	while ((owner = a_cas(&f->lock, 0, tid|MAYBE_WAITERS)))
> -		__futexwait(&f->lock, owner, 1);
>  	return 1;
>  }
>  
> -- 
> 2.14.4

Thanks! I'm applying this with a note added to the commit message
about the source of the regression. I hit the same race today working
on a test case for something else, so I can confirm it occurs in
practice. Distros using musl 1.1.20 should apply this patch.

Rich