scandir qsort usage

scandir qsort usage

[Rich Felker]

The scandir fixes thread prompted me to notice that scandir is
invoking UB calling qsort with mismatched cmp function type. This was
unavoidable without fully duplicating qsort code in the past, but now
that we have qsort_r, it's fixable. If no one beats me to it I'll work
on a patch to fix this too.

Rich

[musl] [PATCH] scandir: fix qsort usage

[Luca Kellermann]

calling qsort() with a pointer to a function whose type is not
compatible with int(const void *, const void *) results in UB because
qsort() would call this function with an incompatible type.

avoid this by using qsort_r(). this is similar to how qsort() is
implemented on top of qsort_r().

the casts to void * in wrapper_cmp() are required because scandir()
takes int (*)(const struct dirent **, const struct dirent **) and not
int (*)(const struct dirent *const *, const struct dirent *const *).
---
Since you haven't posted a patch yet, I went ahead and fixed this. The
patch is based on the scandir() patch series I submitted in July.

 src/dirent/scandir.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/src/dirent/scandir.c b/src/dirent/scandir.c
index 9e4e623f..993e0c8b 100644
--- a/src/dirent/scandir.c
+++ b/src/dirent/scandir.c
@@ -7,9 +7,15 @@
 #include <errno.h>
 #include <stddef.h>
 
+typedef int (*cmpfun)(const struct dirent **, const struct dirent **);
+
+static int wrapper_cmp(const void *de1, const void *de2, void *cmp)
+{
+	return ((cmpfun)cmp)((void *)de1, (void *)de2);
+}
+
 int scandir(const char *path, struct dirent ***res,
-	int (*sel)(const struct dirent *),
-	int (*cmp)(const struct dirent **, const struct dirent **))
+	int (*sel)(const struct dirent *), cmpfun cmp)
 {
 	DIR *d;
 	struct dirent *de, **names=0, **tmp;
@@ -70,7 +76,7 @@ int scandir(const char *path, struct dirent ***res,
 	/* cmp() and caller must not observe that errno was set to 0. */
 	errno = old_errno;
 
-	if (cmp) qsort(names, cnt, sizeof *names, (int (*)(const void *, const void *))cmp);
+	if (cmp) __qsort_r(names, cnt, sizeof *names, wrapper_cmp, (void *)cmp);
 	*res = names;
 	return cnt;
 }