[musl] Potential bug in musl in nftw implementation

[musl] Potential bug in musl in nftw implementation

[Neeraj Sharma]

[-- Attachment #1: Type: text/plain, Size: 17251 bytes --]

Hi,

While evaluating fil-c [1], I noticed a strange issue with a sample C code
to list folders and files with sizes (code enclosed below). I used fil-c
[1]  with musl, but compared the output with stock clang-17, gcc and
noticed a difference in output on my system. After much investigation and
validation I have come to realize a potential issue with musl nftw
implementation. I have not come around finding the actual issue, but
reporting this nonetheless in case someone else knows about this or is
facing similar issues.

My setup
- musl-1.2.5.orig
- Linux 6.2.0-36-generic #37~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon Oct  9
15:34:04 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
- Ubuntu 22.04 LTS

```c
#define _XOPEN_SOURCE 500
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <ftw.h>
#include <sys/stat.h>
#include <dirent.h>

static off_t total_size = 0;
static off_t total_symbolic_links = 0;
static off_t total_regular_files = 0;
static off_t total_directories_cannot_be_read = 0;
static off_t total_directories_before_contents_visited = 0;
static off_t total_directories_visited = 0;
static off_t total_symbolic_link_naming_nonexisting_file = 0;
static off_t total_unstatable_file = 0;


#ifdef HAVE_LOCAL_NFTW
int nftw_local(const char *path, int (*fn)(const char *, const struct stat
*, int, struct FTW *), int fd_limit, int flags);
#endif

// Callback function for nftw()
static int process_entry(const char *fpath, const struct stat *sb,
                          int typeflag, struct FTW *ftwbuf) {
    if (typeflag == FTW_F) {
        // Regular file
        ++total_regular_files;
        printf("%*s├─ %s (%lld bytes)\n", ftwbuf->level * 2, "",
               fpath + ftwbuf->base, (long long)sb->st_size);
        total_size += sb->st_size;
    } else if (typeflag == FTW_D) {
        // Directory (preorder - before contents)
        ++total_directories_before_contents_visited;
        if (ftwbuf->level > 0) {
            printf("%*s📁 %s/\n", ftwbuf->level * 2, "",
                   fpath + ftwbuf->base);
        }
    } else if (typeflag == FTW_DNR) {
        // Directory cannot be read
        ++total_directories_cannot_be_read;
        fprintf(stderr, "Error: Cannot read %s\n", fpath);
    } else if (typeflag == FTW_SL) {
       // Symbolic link.
       ++total_symbolic_links;
    } else if (typeflag == FTW_DP) {
       // Directory, all subdirs have been visited.
       ++total_directories_visited;
    } else if (typeflag == FTW_SLN) {
       // Symbolic link naming non-existing file
       ++total_symbolic_link_naming_nonexisting_file;
    } else if (typeflag == FTW_NS) {
       // Unstatable file
       ++total_unstatable_file;
    } else {
        fprintf(stderr, "Unknown: typeflag = %d\n", typeflag);
    }
    return 0;  // Continue walking
}

int main(int argc, char *argv[]) {
    if (argc != 2) {
        fprintf(stderr, "Usage: %s <directory_path>\n", argv[0]);
        exit(EXIT_FAILURE);
    }

    const char *dirpath = argv[1];
    total_size = 0;

    printf("Scanning: %s\n\n", dirpath);

    // nftw flags:
    // FTW_PHYS: Don't follow symlinks
    // FTW_DEPTH: Visit files before directories (postorder)
    // 10: Max open file descriptors
#ifdef HAVE_LOCAL_NFTW
    if (nftw_local(dirpath, process_entry, 10, FTW_PHYS | FTW_DEPTH) == -1)
{
#else
    if (nftw(dirpath, process_entry, 10, FTW_PHYS | FTW_DEPTH) == -1) {
#endif
        perror("nftw_local");
        exit(EXIT_FAILURE);
    }

    printf("\n📊 Total size: %lld bytes (%.2f MB)\n",
           (long long)total_size, total_size / (1024.0 * 1024.0));

    printf("\nTotal symbolic links %lld", (long long)total_symbolic_links);
    printf("\nTotal regular files %lld", (long long)total_regular_files);
    printf("\nTotal directories cannot be read %lld", (long
long)total_directories_cannot_be_read);
    printf("\nTotal directories before contents visited %lld", (long
long)total_directories_before_contents_visited);
    printf("\nTotal directories visited %lld", (long
long)total_directories_visited);
    printf("\nTotal symbolic link naming non-existing file %lld", (long
long)total_symbolic_link_naming_nonexisting_file);
    printf("\nTotal unstatable file %lld", (long
long)total_unstatable_file);
    return 0;
}
```

Building musl itself

```bash
git clone https://github.com/richfelker/musl-cross-make
cd musl-cross-make
echo "TARGET=x86_64-linux-musl" >> config.mak
make -j16
make TARGET=x86_64-linux-musl install
```

Building the sample code with musl and stock clang-17 and gcc

```bash
/opt/sdb1/filc/musl-cross-make/output/bin/x86_64-linux-musl-gcc -static -o
dirworth_gcc_musl dirworth.c
clang-17 -o dirworth_clang17 dirworth.c
gcc -o dirworth_gcc dirworth.c
```

Running sample code for comparison

```bash
./dirworth_gcc_musl /opt/sdb1/filc/fil-c > out_dirworth_gcc_musl.log 2>&1
./dirworth_clang17 /opt/sdb1/filc/fil-c > out_dirworth_clang17.log 2>&1
./dirworth_gcc /opt/sdb1/filc/fil-c > out_dirworth_gcc.log 2>&1
tail -n8 out_dirworth_*.log
```

Output:

```
==> out_dirworth_gcc.log <==

Total symbolic links 678
Total regular files 670846
Total directories cannot be read 0
Total directories before contents visited 0
Total directories visited 55362
Total symbolic link naming non-existing file 0
Total unstatable file 0
==> out_dirworth_clang17.log <==

Total symbolic links 678
Total regular files 670846
Total directories cannot be read 0
Total directories before contents visited 0
Total directories visited 55362
Total symbolic link naming non-existing file 0
Total unstatable file 0
==> out_dirworth_gcc_musl.log <==

Total symbolic links 678
Total regular files 652585
Total directories cannot be read 0
Total directories before contents visited 0
Total directories visited 54104
Total symbolic link naming non-existing file 0
Total unstatable file 0
```

Base setup of `/opt/sdb1/filc/fil-c`. See
https://github.com/pizlonator/fil-c/issues/157

```bash
git clone https://github.com/pizlonator/fil-c
cd fil-c
./build_all.sh
```

I even went ahead and copied a local version of `nftw.c` from musl and
compiled with clang-17
and noticed an issue. Note: I have added few logs for my initial
investigation and flags at the top
to make clang-17 happy.

```c [file=nftw_local.c]
#define _XOPEN_SOURCE 500
#define _POSIX_C_SOURCE 200809L

#include <ftw.h>
#include <dirent.h>
#include <fcntl.h>
#include <sys/stat.h>
#include <errno.h>
#include <unistd.h>
#include <string.h>
#include <limits.h>
#include <pthread.h>
#include <stdio.h>

struct history
{
        struct history *chain;
        dev_t dev;
        ino_t ino;
        int level;
        int base;
};

#undef dirfd
#define dirfd(d) (*(int *)d)

static int do_nftw_local(char *path, int (*fn)(const char *, const struct
stat *, int, struct FTW *), int fd_limit, int flags, struct history *h)
{
        size_t l = strlen(path), j = l && path[l-1]=='/' ? l-1 : l;
        struct stat st;
        struct history new;
        int type;
        int r;
        int dfd;
        int err;
        struct FTW lev;

        st.st_dev = st.st_ino = 0;

    fprintf(stderr, ">> 0. path = %s\n", path);
        if ((flags & FTW_PHYS) ? lstat(path, &st) : stat(path, &st) < 0) {
                if (!(flags & FTW_PHYS) && errno==ENOENT && !lstat(path,
&st))
                        type = FTW_SLN;
                else if (errno != EACCES) {
            fprintf(stderr, ">> a. path = %s\n", path);
            return -1;
        }
                else type = FTW_NS;
        } else if (S_ISDIR(st.st_mode)) {
                if (flags & FTW_DEPTH) type = FTW_DP;
                else type = FTW_D;
        } else if (S_ISLNK(st.st_mode)) {
                if (flags & FTW_PHYS) type = FTW_SL;
                else type = FTW_SLN;
        } else {
                type = FTW_F;
        }

        if ((flags & FTW_MOUNT) && h && type != FTW_NS && st.st_dev !=
h->dev) {
        fprintf(stderr, ">> 1. path = %s\n", path);
                return 0;
    }

        new.chain = h;
        new.dev = st.st_dev;
        new.ino = st.st_ino;
        new.level = h ? h->level+1 : 0;
        new.base = j+1;

        lev.level = new.level;
        if (h) {
                lev.base = h->base;
        } else {
                size_t k;
                for (k=j; k && path[k]=='/'; k--);
                for (; k && path[k-1]!='/'; k--);
                lev.base = k;
        }

        if (type == FTW_D || type == FTW_DP) {
                dfd = open(path, O_RDONLY);
                err = errno;
                if (dfd < 0 && err == EACCES) type = FTW_DNR;
                if (!fd_limit) close(dfd);
        }

        if (!(flags & FTW_DEPTH) && (r=fn(path, &st, type, &lev))) {
        fprintf(stderr, ">> 2. path = %s\n", path);
                return r;
    }

        for (; h; h = h->chain)
                if (h->dev == st.st_dev && h->ino == st.st_ino) {
            fprintf(stderr, ">> 3. path = %s\n", path);
                        return 0;
        }

        if ((type == FTW_D || type == FTW_DP) && fd_limit) {
                if (dfd < 0) {
                        errno = err;
            fprintf(stderr, ">> 4. path = %s\n", path);
                        return -1;
                }
                DIR *d = fdopendir(dfd);
                if (d) {
                        struct dirent *de;
                        while ((de = readdir(d))) {
                                if (de->d_name[0] == '.'
                                 && (!de->d_name[1]
                                  || (de->d_name[1]=='.'
                                   && !de->d_name[2]))) continue;
                                if (strlen(de->d_name) >= PATH_MAX-l) {
                                        errno = ENAMETOOLONG;
                                        closedir(d);
                    fprintf(stderr, ">> 5. path = %s\n", path);
                                        return -1;
                                }
                                path[j]='/';
                                strcpy(path+j+1, de->d_name);
                                if ((r=do_nftw_local(path, fn, fd_limit-1,
flags, &new))) {
                                        closedir(d);
                    fprintf(stderr, ">> 6. path = %s, r = %d\n", path, r);
                                        return r;
                                }
                        }
                        closedir(d);
                } else {
                        close(dfd);
            fprintf(stderr, ">> 7. path = %s\n", path);
                        return -1;
                }
        }

        path[l] = 0;
        if ((flags & FTW_DEPTH) && (r=fn(path, &st, type, &lev))) {
        fprintf(stderr, ">> 8. path = %s, r = %d\n", path, r);
                return r;
    }

        return 0;
}

int nftw_local(const char *path, int (*fn)(const char *, const struct stat
*, int, struct FTW *), int fd_limit, int flags)
{
        int r, cs;
        size_t l;
        char pathbuf[PATH_MAX+1];

        if (fd_limit <= 0) return 0;

        l = strlen(path);
        if (l > PATH_MAX) {
                errno = ENAMETOOLONG;
                return -1;
        }
        memcpy(pathbuf, path, l+1);

        pthread_setcancelstate(PTHREAD_CANCEL_DISABLE, &cs);
        r = do_nftw_local(pathbuf, fn, fd_limit, flags, NULL);
        pthread_setcancelstate(cs, 0);
        return r;
}
```

Building and running local copy of nftw with clang-17

```bash
clang-17 -o dirworth_local_clang17 -DHAVE_LOCAL_NFTW dirworth.c nftw_local.c
./dirworth_local_clang17 /opt/sdb1/filc/fil-c >
out_dirworth_local_clang17.log 2>&1
```

Output from clang-17 with nftw implementation from musl.

```
==> out_dirworth_local_clang17.log <==

Total symbolic links 678
Total regular files 652585
Total directories cannot be read 0
Total directories before contents visited 0
Total directories visited 54104
Total symbolic link naming non-existing file 0
Total unstatable file 0
```

Diff of nftw_local.c (modified copy) with musl nftw.c

```diff
--- ../musl-cross-make/musl-1.2.5.orig/src/misc/nftw.c  2024-03-01
07:37:33.000000000 +0530
+++ nftw_local.c        2025-12-03 09:26:32.554116560 +0530
@@ -1,3 +1,6 @@
+#define _XOPEN_SOURCE 500
+#define _POSIX_C_SOURCE 200809L
+
 #include <ftw.h>
 #include <dirent.h>
 #include <fcntl.h>
@@ -7,6 +10,7 @@
 #include <string.h>
 #include <limits.h>
 #include <pthread.h>
+#include <stdio.h>

 struct history
 {
@@ -20,7 +24,7 @@
 #undef dirfd
 #define dirfd(d) (*(int *)d)

-static int do_nftw(char *path, int (*fn)(const char *, const struct stat
*, int, struct FTW *), int fd_limit, int flags, struct history *h)
+static int do_nftw_local(char *path, int (*fn)(const char *, const struct
stat *, int, struct FTW *), int fd_limit, int flags, struct history *h)
 {
        size_t l = strlen(path), j = l && path[l-1]=='/' ? l-1 : l;
        struct stat st;
@@ -33,10 +37,14 @@

        st.st_dev = st.st_ino = 0;

+    fprintf(stderr, ">> 0. path = %s\n", path);
        if ((flags & FTW_PHYS) ? lstat(path, &st) : stat(path, &st) < 0) {
                if (!(flags & FTW_PHYS) && errno==ENOENT && !lstat(path,
&st))
                        type = FTW_SLN;
-               else if (errno != EACCES) return -1;
+               else if (errno != EACCES) {
+            fprintf(stderr, ">> a. path = %s\n", path);
+            return -1;
+        }
                else type = FTW_NS;
        } else if (S_ISDIR(st.st_mode)) {
                if (flags & FTW_DEPTH) type = FTW_DP;
@@ -48,8 +56,10 @@
                type = FTW_F;
        }

-       if ((flags & FTW_MOUNT) && h && type != FTW_NS && st.st_dev !=
h->dev)
+       if ((flags & FTW_MOUNT) && h && type != FTW_NS && st.st_dev !=
h->dev) {
+        fprintf(stderr, ">> 1. path = %s\n", path);
                return 0;
+    }

        new.chain = h;
        new.dev = st.st_dev;
@@ -74,16 +84,21 @@
                if (!fd_limit) close(dfd);
        }

-       if (!(flags & FTW_DEPTH) && (r=fn(path, &st, type, &lev)))
+       if (!(flags & FTW_DEPTH) && (r=fn(path, &st, type, &lev))) {
+        fprintf(stderr, ">> 2. path = %s\n", path);
                return r;
+    }

        for (; h; h = h->chain)
-               if (h->dev == st.st_dev && h->ino == st.st_ino)
+               if (h->dev == st.st_dev && h->ino == st.st_ino) {
+            fprintf(stderr, ">> 3. path = %s\n", path);
                        return 0;
+        }

        if ((type == FTW_D || type == FTW_DP) && fd_limit) {
                if (dfd < 0) {
                        errno = err;
+            fprintf(stderr, ">> 4. path = %s\n", path);
                        return -1;
                }
                DIR *d = fdopendir(dfd);
@@ -97,30 +112,35 @@
                                if (strlen(de->d_name) >= PATH_MAX-l) {
                                        errno = ENAMETOOLONG;
                                        closedir(d);
+                    fprintf(stderr, ">> 5. path = %s\n", path);
                                        return -1;
                                }
                                path[j]='/';
                                strcpy(path+j+1, de->d_name);
-                               if ((r=do_nftw(path, fn, fd_limit-1, flags,
&new))) {
+                               if ((r=do_nftw_local(path, fn, fd_limit-1,
flags, &new))) {
                                        closedir(d);
+                    fprintf(stderr, ">> 6. path = %s, r = %d\n", path, r);
                                        return r;
                                }
                        }
                        closedir(d);
                } else {
                        close(dfd);
+            fprintf(stderr, ">> 7. path = %s\n", path);
                        return -1;
                }
        }

        path[l] = 0;
-       if ((flags & FTW_DEPTH) && (r=fn(path, &st, type, &lev)))
+       if ((flags & FTW_DEPTH) && (r=fn(path, &st, type, &lev))) {
+        fprintf(stderr, ">> 8. path = %s, r = %d\n", path, r);
                return r;
+    }

        return 0;
 }

-int nftw(const char *path, int (*fn)(const char *, const struct stat *,
int, struct FTW *), int fd_limit, int flags)
+int nftw_local(const char *path, int (*fn)(const char *, const struct stat
*, int, struct FTW *), int fd_limit, int flags)
 {
        int r, cs;
        size_t l;
@@ -136,7 +156,7 @@
        memcpy(pathbuf, path, l+1);

        pthread_setcancelstate(PTHREAD_CANCEL_DISABLE, &cs);
-       r = do_nftw(pathbuf, fn, fd_limit, flags, NULL);
+       r = do_nftw_local(pathbuf, fn, fd_limit, flags, NULL);
        pthread_setcancelstate(cs, 0);
        return r;
 }
```

Thanks and Regards,
Neeraj

[1]: https://fil-c.org/

[-- Attachment #2: Type: text/html, Size: 22217 bytes --]

[musl] Re: Potential bug in musl in nftw implementation

[Neeraj Sharma]

On Wed, Dec 3, 2025 at 11:21 AM Neeraj Sharma
<neerajsharma.live@gmail.com> wrote:
>
> While evaluating fil-c [1], I noticed a strange issue with a sample C code to list folders and files with sizes (code enclosed below). I used fil-c [1]  with musl, but compared the output with stock clang-17, gcc and noticed a difference in output on my system. After much investigation and validation I have come to realize a potential issue with musl nftw implementation. I have not come around finding the actual issue, but reporting this nonetheless in case someone else knows about this or is facing similar issues.
>

I have identified a bug in nftw.c implementation within musl. At least
to the extent I understand the functionality of nftw as described
within https://linux.die.net/man/3/nftw

Line 74 in `musl-1.2.5.orig/src/misc/nftw.c` is the issue. It closes
the file descriptor and leaves it there. Irrespective continuing
further without any other action seems incorrect anyways. This means
that further folder traversal is not done. I confirmed this analysis
by increasing the value of fd_limit in my calling code (dirworth.c)
and this time around the undiscovered file earlier was now discovered.

```
 70     if (type == FTW_D || type == FTW_DP) {
 71         dfd = open(path, O_RDONLY);
 72         err = errno;
 73         if (dfd < 0 && err == EACCES) type = FTW_DNR;
 74         if (!fd_limit) close(dfd);
 75     }
```

Modified dirworth.c code which worked for my folder depth.

```c
// this works because my fd_limit is higher, which was earlier 10
nftw(dirpath, process_entry, 20, FTW_PHYS | FTW_DEPTH
```

Regards,
Neeraj

Re: [musl] Re: Potential bug in musl in nftw implementation

[Rich Felker]

On Wed, Dec 03, 2025 at 01:04:05PM +0530, Neeraj Sharma wrote:
> On Wed, Dec 3, 2025 at 11:21 AM Neeraj Sharma
> <neerajsharma.live@gmail.com> wrote:
> >
> > While evaluating fil-c [1], I noticed a strange issue with a sample C code to list folders and files with sizes (code enclosed below). I used fil-c [1]  with musl, but compared the output with stock clang-17, gcc and noticed a difference in output on my system. After much investigation and validation I have come to realize a potential issue with musl nftw implementation. I have not come around finding the actual issue, but reporting this nonetheless in case someone else knows about this or is facing similar issues.
> >
> 
> I have identified a bug in nftw.c implementation within musl. At least
> to the extent I understand the functionality of nftw as described
> within https://linux.die.net/man/3/nftw
> 
> Line 74 in `musl-1.2.5.orig/src/misc/nftw.c` is the issue. It closes
> the file descriptor and leaves it there. Irrespective continuing
> further without any other action seems incorrect anyways. This means
> that further folder traversal is not done. I confirmed this analysis
> by increasing the value of fd_limit in my calling code (dirworth.c)
> and this time around the undiscovered file earlier was now discovered.
> 
> ```
>  70     if (type == FTW_D || type == FTW_DP) {
>  71         dfd = open(path, O_RDONLY);
>  72         err = errno;
>  73         if (dfd < 0 && err == EACCES) type = FTW_DNR;
>  74         if (!fd_limit) close(dfd);
>  75     }
> ```
> 
> Modified dirworth.c code which worked for my folder depth.
> 
> ```c
> // this works because my fd_limit is higher, which was earlier 10
> nftw(dirpath, process_entry, 20, FTW_PHYS | FTW_DEPTH
> ```

Are you saying you'd deem it a better behavior to return with an error
when it can't traverse within the fd limit, instead of skipping
descent past what it can do within the limit?

That's probably more correct. My understanding is that historically,
the limit was a depth limit, but POSIX repurposed it into a "fd limit"
with an expectation that implementations somehow do the traversal with
fewer fds than the depth if the limit is exceeded (or error out?).
Trying to actually do it seems error-prone (requires unbounded working
space that grows with size of a single directory, vs growth only with
depth for fds, and involves caching potentially-stale data) but maybe
just erroring so the caller knows to use a larger limit would be the
right thing to do here...?

Rich

Re: [musl] Re: Potential bug in musl in nftw implementation

[Neeraj Sharma]

On Wed, Dec 3, 2025 at 7:08 PM Rich Felker <dalias@libc.org> wrote:
>
> Are you saying you'd deem it a better behavior to return with an error
> when it can't traverse within the fd limit, instead of skipping
> descent past what it can do within the limit?
>
> That's probably more correct. My understanding is that historically,
> the limit was a depth limit, but POSIX repurposed it into a "fd limit"
> with an expectation that implementations somehow do the traversal with
> fewer fds than the depth if the limit is exceeded (or error out?).
> Trying to actually do it seems error-prone (requires unbounded working
> space that grows with size of a single directory, vs growth only with
> depth for fds, and involves caching potentially-stale data) but maybe
> just erroring so the caller knows to use a larger limit would be the
> right thing to do here...?

I would suggest aligning with common understanding across nix in this
case. This was the main reason for my confusion in the beginning.
Silently skipping or erroring out both seems unaligned with common
understanding as in [1], [2], [3]. The documentation in linux [1] is
more explicit about the functionality than IEEE [2] or opengroup [3]
in this case.

Quotes from [1] or linux man page ftw(3).

"To avoid using up all of the calling process's file descriptors,
nopenfd specifies the maximum number of directories that ftw() will
hold open simultaneously. When the search depth exceeds this, ftw()
will become slower because directories have to be closed and reopened.
ftw() uses at most one file descriptor for each level in the directory
tree."

"The function nftw() is the same as ftw(), except that it has one
additional argument, flags, and calls fn() with one more argument,
ftwbuf."

[1] https://linux.die.net/man/3/nftw
[2] IEEE Std 1003.1-2001/Cor 2-2004, item XSH/TC2/D6/6 Change Number:
XSH/TC2/D6/64 [XSH ERN 73] - p52,
https://pubs.opengroup.org/onlinepubs/7899949099/toc.pdf
[3] https://pubs.opengroup.org/onlinepubs/000095399/functions/nftw.html

Regards,
Neeraj

Re: [musl] Re: Potential bug in musl in nftw implementation

[Rich Felker]

On Wed, Dec 03, 2025 at 08:25:03PM +0530, Neeraj Sharma wrote:
> On Wed, Dec 3, 2025 at 7:08 PM Rich Felker <dalias@libc.org> wrote:
> >
> > Are you saying you'd deem it a better behavior to return with an error
> > when it can't traverse within the fd limit, instead of skipping
> > descent past what it can do within the limit?
> >
> > That's probably more correct. My understanding is that historically,
> > the limit was a depth limit, but POSIX repurposed it into a "fd limit"
> > with an expectation that implementations somehow do the traversal with
> > fewer fds than the depth if the limit is exceeded (or error out?).
> > Trying to actually do it seems error-prone (requires unbounded working
> > space that grows with size of a single directory, vs growth only with
> > depth for fds, and involves caching potentially-stale data) but maybe
> > just erroring so the caller knows to use a larger limit would be the
> > right thing to do here...?
> 
> I would suggest aligning with common understanding across nix in this
> case. This was the main reason for my confusion in the beginning.
> Silently skipping or erroring out both seems unaligned with common
> understanding as in [1], [2], [3]. The documentation in linux [1] is
> more explicit about the functionality than IEEE [2] or opengroup [3]
> in this case.
> 
> Quotes from [1] or linux man page ftw(3).
> 
> "To avoid using up all of the calling process's file descriptors,
> nopenfd specifies the maximum number of directories that ftw() will
> hold open simultaneously. When the search depth exceeds this, ftw()
> will become slower because directories have to be closed and reopened.
> ftw() uses at most one file descriptor for each level in the directory
> tree."

This sounds like a documentation bug. "Slower because directories have
to be closed and reopened" is not an accurate description of what
happens if you close directory fds to stay under a limit. The reality
is that the working space requirements become unbounded, because you
can't just close and reopen a directory and continue where you left
off. The seek address is only valid until the directory is closed, so
the only way to continue where you left off is to continue reading all
the remaining entries and store them in memory before closing it.

That's not to say we shouldn't make a change here (tho actually
buffering remaining files rather than just erroring would be some new
work to implement), but if folks are calling it with a low fd_limit
argument based on the above advice, that should be fixed too. There's
absolutely no reason for a limit below a few hundred (it's not
documented as thread-safe, so you can't be doing it concurrently in
the same process, so all you need to avoid is hitting process fd limit
which defaults to 1024).

Rich

Re: [musl] Re: Potential bug in musl in nftw implementation

[Lénárd Szolnoki]

On 03/12/2025 14:55, Neeraj Sharma wrote:
> On Wed, Dec 3, 2025 at 7:08 PM Rich Felker <dalias@libc.org> wrote:
>>
>> Are you saying you'd deem it a better behavior to return with an error
>> when it can't traverse within the fd limit, instead of skipping
>> descent past what it can do within the limit?
>>
>> That's probably more correct. My understanding is that historically,
>> the limit was a depth limit, but POSIX repurposed it into a "fd limit"
>> with an expectation that implementations somehow do the traversal with
>> fewer fds than the depth if the limit is exceeded (or error out?).
>> Trying to actually do it seems error-prone (requires unbounded working
>> space that grows with size of a single directory, vs growth only with
>> depth for fds, and involves caching potentially-stale data) but maybe
>> just erroring so the caller knows to use a larger limit would be the
>> right thing to do here...?
> 
> I would suggest aligning with common understanding across nix in this
> case. This was the main reason for my confusion in the beginning.
> Silently skipping or erroring out both seems unaligned with common
> understanding as in [1], [2], [3]. The documentation in linux [1] is
> more explicit about the functionality than IEEE [2] or opengroup [3]
> in this case.
> 
> Quotes from [1] or linux man page ftw(3).
> 
> "To avoid using up all of the calling process's file descriptors,
> nopenfd specifies the maximum number of directories that ftw() will
> hold open simultaneously. When the search depth exceeds this, ftw()
> will become slower because directories have to be closed and reopened.
> ftw() uses at most one file descriptor for each level in the directory
> tree."
> 
> "The function nftw() is the same as ftw(), except that it has one
> additional argument, flags, and calls fn() with one more argument,
> ftwbuf."

Notably the Linux man page also says this:

"As long as fn() returns 0, ftw() will continue either until it has traversed the entire 
tree, in which case it will return zero, or until it encounters an error (such as a 
malloc(3) failure), in which case it will return -1. "

So allocation failure is on the table, I assume with ENOMEM being reported as the error. 
Allocation error is merely an example here, otherwise the set of errors seem to be rather 
open-ended here. Arguably running out of working space in any way, or just refusing to 
work outside of the fd limit fits into this description.

The posix docs don't seem to admit ENOMEM or otherwise running out of working space being 
a possible error (apart from the callback setting that), which seems like a defect. The 
traversal algorithm inherently needs more than O(1) space, whether that is partially 
allocated in the kernel or not.

Lénárd

> 
> [1] https://linux.die.net/man/3/nftw
> [2] IEEE Std 1003.1-2001/Cor 2-2004, item XSH/TC2/D6/6 Change Number:
> XSH/TC2/D6/64 [XSH ERN 73] - p52,
> https://pubs.opengroup.org/onlinepubs/7899949099/toc.pdf
> [3] https://pubs.opengroup.org/onlinepubs/000095399/functions/nftw.html
> 
> Regards,
> Neeraj