Re: [musl] [PATCH v3 2/2] nftw: implement FTW_ACTIONRETVAL (GNU extension)

[Dominique MARTINET <dominique.martinet@atmark-techno.com>]

Sorry for the delay, I was waiting for some "official" feedback and
forgot about the patch as none came.

Rich (or others), is there anything we can do to convince you the
feature would be interesting?
I had sent you recap from debian codesearch back in April last year:
https://www.openwall.com/lists/musl/2021/04/12/10
but didn't hear any reply.

It'd be great to just hear a final "no unless it gets more widely used"
so I could give up and work on converting bpftools to not depend on
it -- it's just frustrating due to the lack of reply, and while adding
FTW_ACTIONRETVAL to musl is easy converting a user to something else is
not so straightforward so I was putting it off...

(Conversely, if there is interest please say so as well -- my patch had
a couple of feedbacks I would be happy to address, but I just never sent
a v2 because it looked like it wouldn't be accepted anyway from your
(lack of) reaction. I don't care if my version of this one gets in
ultimately, and can help either way.)


Ismael, comments below.


Ismael Luceno wrote on Mon, Jan 24, 2022 at 08:53:14PM +0100:
> > > +	if (!(flags & FTW_DEPTH) && (r=fn(path, &st, type, &lev))) {
> > > +		if ((flags & FTW_ACTIONRETVAL)) {
> > > +			if (r == FTW_STOP) return FTW_STOP;
> > > +			if (r == FTW_SKIP_SUBTREE) return 0;
> > > +			/* other values are saved for when returning */
> > 
> > Hm, I'd naively think you would want to return immediately the other
> > values as well, so the else below is wrong?
> > But I didn't take long enough to check what e.g. a SKIP_SIBLINGS would
> > mean here, the construction just looks a bit odd to me.
> 
> SKIP_SIBLINGS doesn't imply SKIP_SUBTREE, so must be saved and returned
> when finishing.

Hm, I was starting to agree with you and wondered how my patch worked,
but actually... According to how glibc behaves, it does:

----
# in musl source tree after patch
$ make -j6
make: Nothing to be done for 'all'.
$ cat ftw/ftw.c
#define _GNU_SOURCE
#include <ftw.h>
#include <string.h>
#include <stdio.h>

int cb(const char *path, const struct stat *sb, int typeflag, struct FTW *ftwbuf) {
        printf("got %s\n", path);
        if (strcmp(path, "./src") == 0)
                return FTW_SKIP_SUBTREE;
        if (strcmp(path, "./.git") == 0) {
                return FTW_SKIP_SIBLINGS;
        }
        return 0;
}

int main() {
        int rc = nftw(".", cb, 10, FTW_ACTIONRETVAL);
        printf("rc %d\n", rc);
        return 0;
}
$ gcc -o ftw/ftw_glibc ftw/ftw.c 
$ gcc -o ftw/ftw_musl ftw/ftw.c lib/libc.a
$ diff -u <( ./ftw/ftw_musl ) <( ./ftw/ftw_glibc )
--- /dev/fd/63	2022-03-09 13:22:55.566285599 +0900
+++ /dev/fd/62	2022-03-09 13:22:55.566285599 +0900
@@ -1,120 +1,3 @@
 got .
 got ./.git
-got ./.git/branches
-got ./.git/hooks
-got ./.git/hooks/applypatch-msg.sample
-got ./.git/hooks/commit-msg.sample
-got ./.git/hooks/fsmonitor-watchman.sample
...
-got ./.git/ORIG_HEAD
-got ./.git/index
-got ./.git/HEAD
 rc 0
----

so the glibc version of ftw skipped .git here when it only returned
FTW_SKIP_SIBLINGS.

You could argue that this is a bug in glibc, but since there is no spec
for FTW_ACTIONRETVAL I had chosen at the time to implement something
compatible bug-for-bug as "glibc is the reference".


> > >  	path[l] = 0;
> > > -	if ((flags & FTW_DEPTH) && (r=fn(path, &st, type, &lev)))
> > > -		return r;
> > > -
> > > -	return 0;
> > > +	if (flags & FTW_DEPTH) {
> > > +		r = fn(path, &st, type, &lev);
> > > +		/* ignore FTW_SKIP_SUBTREE (too late), the caller is broken */
> > > +		if ((flags & FTW_ACTIONRETVAL) && r == FTW_SKIP_SUBTREE)
> > > +			return 0;
> > 
> > IIRC the glibc implementation also ignores FTW_SKIP_SIBLINGS in that
> > case (nftw returns 0), I'm not sure how much of a 1-to-1 implementation
> > we want -- I had implemented my version through a black-box approach
> > with a client exercising all kind of different code paths as for a gnu
> > extension I'd assume glibc to be the reference.
> 
> FTW_SKIP_SUBTREE makes no sense with FTW_DEPTH, but FTW_SKIP_SIBLINGS
> works with both.

I agree it makes no sense, this was another bug-for-bug thing:

----
$ cat subtree.c 
#define _GNU_SOURCE
#include <ftw.h>
#include <string.h>
#include <stdio.h>

int cb(const char *path, const struct stat *sb, int typeflag, struct FTW *ftwbuf) {
	return FTW_SKIP_SIBLINGS;
}

int main() {
        int rc = nftw(".", cb, 10, FTW_ACTIONRETVAL);
        printf("rc %d\n", rc);
        return 0;
}
$ ./ftw_glibc 
rc 0
$ ./ftw_musl 
rc 3
----

My version at the time just reset the return value at the end if
FTW_ACTIONRETVAL was specified in flags, to conform to that.

-- 
Dominique