mailing list of musl libc
 help / color / mirror / code / Atom feed
* [musl] [PATCH] fix potential ref count overflow in sem_open()
@ 2022-09-03  0:19 Alexey Izbyshev
  2022-09-03  4:48 ` Rich Felker
  0 siblings, 1 reply; 5+ messages in thread
From: Alexey Izbyshev @ 2022-09-03  0:19 UTC (permalink / raw)
  To: musl

sem_open() attempts to avoid overflow on the future ref count increment
by computing the sum of all ref counts in semtab and checking that it's
not INT_MAX when semtab is locked for slot reservation.  This approach
suffers from a TOCTTOU: by the time semtab is re-locked after opening a
semaphore, the sum could have already been increased by a concurrent
sem_open(), so it will overflow on the increment. An individual ref
count can be overflowed as well if the call happened to open a duplicate
of the only other semaphore.

Moreover, since the overflow avoidance check admits a negative (i.e.
overflowed) sum, after this state is reached, an individual ref count
can be incremented until it reaches 1 again, and then sem_close() will
incorrectly free the semaphore, promoting what could be just a
signed-overflow UB to a use-after-free.

Fix the overflow check by accounting for the maximum possible number of
concurrent sem_open() calls that have already reserved a slot, but
haven't incremented the ref count yet.
---
Alternatively, we could use the number of currently reserved slots
instead of SEM_NSEMS_MAX, preserving the ability of individual ref
counts to reach INT_MAX in non-concurrent scenarios. I'm not sure
whether it matters, so I'm sending a smaller of the two fixes.

Alexey
---
 src/thread/sem_open.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/thread/sem_open.c b/src/thread/sem_open.c
index 0ad29de9..611a3f64 100644
--- a/src/thread/sem_open.c
+++ b/src/thread/sem_open.c
@@ -61,7 +61,7 @@ sem_t *sem_open(const char *name, int flags, ...)
 		if (!semtab[i].sem && slot < 0) slot = i;
 	}
 	/* Avoid possibility of overflow later */
-	if (cnt == INT_MAX || slot < 0) {
+	if (cnt > INT_MAX - SEM_NSEMS_MAX || slot < 0) {
 		errno = EMFILE;
 		UNLOCK(lock);
 		return SEM_FAILED;
-- 
2.37.2


^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2022-09-03 18:56 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-09-03  0:19 [musl] [PATCH] fix potential ref count overflow in sem_open() Alexey Izbyshev
2022-09-03  4:48 ` Rich Felker
2022-09-03 14:28   ` Alexey Izbyshev
2022-09-03 17:08     ` Rich Felker
2022-09-03 18:55       ` Alexey Izbyshev

Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/musl/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).