> > > Are you sure the "running under strace" has no differences in how the > test program is invoked aside from just using strace? I'm running strace & ltrace from the command prompt. For what it is worth, the process tree is a little weird on ChromeOS: [image: Screenshot 2022-02-14 at 13.58.03.png] > Rather than > running it via the ruby machinery, can you just test under plain > manual execution from the same shell instance for both? > > The previous email showed it running via manual execution from the same shell instance, invoked without and with ltrace. i.e. I have the same issue. > Can you report what Docker version you're using, and try executing > with Docker's seccomp sandboxing disabled? docker version Client: Docker Engine - Community Version: 20.10.12 API version: 1.41 Go version: go1.16.12 Git commit: e91ed57 Built: Mon Dec 13 11:45:41 2021 OS/Arch: linux/amd64 Context: default Experimental: true Server: Docker Engine - Community Engine: Version: 20.10.12 API version: 1.41 (minimum version 1.12) Go version: go1.16.12 Git commit: 459d0df Built: Mon Dec 13 11:44:05 2021 OS/Arch: linux/amd64 Experimental: false containerd: Version: 1.4.12 GitCommit: 7b11cfaabd73bb80907dd23182b9347b4245eb5d runc: Version: 1.0.2 GitCommit: v1.0.2-0-g52b36a2 docker-init: Version: 0.19.0 GitCommit: de40ad0 This is the docker command I used, which may disable the seccomp sandboxing? #!/bin/bash docker pull satmandu/crewbuild:alex-i686.m58 docker pull tonistiigi/binfmt docker run --privileged --rm tonistiigi/binfmt --install all docker run --security-opt seccomp=unconfined --platform linux/386 --cap-add SYS_PTRACE --rm -v $(pwd)/pkg_cache:/usr/local/tmp/packages -v $(pwd):/output -h $(hostname)-i686 -it satmandu/crewbuild:alex-i686.m58 /usr/local/bin/setarch i686 sudo -i -u chronos /usr/local/bin/bash -i ../musl_getaddrinfo_test google.com AF_INET: 142.250.80.46 AF_INET6: 2607:f8b0:4006:80b::200e Commands inside that docker invocation to test this: crew upgrade ; yes | crew install ltrace CREW_TESTING_REPO=https://github.com/satmandu/chromebrew.git CREW_TESTING_BRANCH=musl_testing CREW_TESTING=1 crew update crew upgrade cd /usr/local/tmp yes | crew build -k musl_getaddrinfo_test cd crew/musl_getaddrinfo_test.* ../musl_getaddrinfo_test google.com When I ran that, I got this: ../musl_getaddrinfo_test google.com AF_INET: 142.250.80.46 AF_INET6: 2607:f8b0:4006:80b::200e > This shouldn't happen, but > it's plausible that your old kernel has bugs where seccomp filtering > gets bypassed when the process is running under strace, thereby > working around a buggy seccomp filter in Docker. > > Is it possible there are seccomp issues with the old kernel that just weren't triggered by an older version of musl? > If you know how to use gdb, you could also try setting some > breakpoints to see what code is or isn't reached. > > This would be ideal. I'm still trying to get gdb built with musl on this older setup. The gdb I have built against glibc doesn't seem to be particularly helpful when I'm running it with this program built against the musl libc. Is there a static build of i686 gdb built against musl someone has available which might be helpful here? Satadru > > On Mon, Feb 7, 2022 at 4:02 PM Rich Felker wrote: > > > > > On Mon, Feb 07, 2022 at 02:19:05PM -0500, Satadru Pramanik wrote: > > > > The test programs are being run from... > > > > glibc 2.23 -> bash (crosh shell) > > > > crosh shell -> invokes ruby -> invokes bash to run the test programs. > > > > > > > > tcpdump on the router shows no network activity at all when running > > > > the test program with tcpdump -i any -vvv host (IP address) > > > > > > There's reliably no network traffic when you run the test program not > > > under strace? Is there any difference in how you're invoking it other > > > than strace not being there? I'm running out of possible explanations > > > unless there's some hidden details we don't know about. > > > > > > > When I run the test pogram with strace though I see this: > > > > 14:06:24.617860 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], > proto > > > UDP > > > > (17), length 56) > > > > 192.168.0.121.46846 > office.lan.53: [udp sum ok] 16051+ A? > > > google.com. > > > > (28) > > > > 14:06:24.622352 IP (tos 0x0, ttl 64, id 15884, offset 0, flags [DF], > > > proto > > > > UDP (17), length 72) > > > > office.lan.53 > 192.168.0.121.46846: [bad udp cksum 0x8210 -> > > > 0x7bc1!] > > > > 16051 q: A? google.com. 1/0/0 google.com. [1m32s] A 142.251.40.110 > (44) > > > > 14:06:24.688610 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], > proto > > > UDP > > > > (17), length 56) > > > > 192.168.0.121.42267 > office.lan.53: [udp sum ok] 35406+ A? > > > google.com. > > > > (28) > > > > 14:06:24.688931 IP (tos 0x0, ttl 64, id 15887, offset 0, flags [DF], > > > proto > > > > UDP (17), length 72) > > > > office.lan.53 > 192.168.0.121.42267: [bad udp cksum 0x8210 -> > > > 0x4209!] > > > > 35406 q: A? google.com. 1/0/0 google.com. [1m32s] A 142.251.40.110 > (44) > > > > 14:06:24.689018 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], > proto > > > UDP > > > > (17), length 56) > > > > 192.168.0.121.42267 > office.lan.53: [udp sum ok] 13657+ AAAA? > > > > google.com. (28) > > > > 14:06:24.689186 IP (tos 0x0, ttl 64, id 15888, offset 0, flags [DF], > > > proto > > > > UDP (17), length 84) > > > > office.lan.53 > 192.168.0.121.42267: [bad udp cksum 0x821c -> > > > 0xc77e!] > > > > 13657 q: AAAA? google.com. 1/0/0 google.com. [20s] AAAA > > > > 2607:f8b0:4006:80b::200e (56) > > > > > > > > On Sun, Feb 6, 2022 at 9:40 PM Rich Felker > wrote: > > > > > > > > > On Sun, Feb 06, 2022 at 08:29:16PM -0500, Satadru Pramanik wrote: > > > > > > Here are illustrative logs of output and strace logs. > > > > > > > > > > > > Note that while the musl toolchain is built in a container on a > much > > > more > > > > > > powerful machine, this "musl_getaddrinfo_test" app is built > locally > > > on > > > > > the > > > > > > machine with the 3.8 kernel. > > > > > > > > > > > > I ran the following to get the output on the smaller i686 machine > > > > > > immediately after the app is built. > > > > > > Apologies for the ruby code wrapping the shell commands. > > > > > > > > > > > > @musl_ver = `#{CREW_MUSL_PREFIX}/lib/libc.so 2>&1 >/dev/null > | > > > head > > > > > -2 > > > > > > | tail -1 | awk '{print $2}'`.chomp > > > > > > puts 'Testing the musl resolver to see if it can resolve > > > google.com: > > > > > > '.lightblue > > > > > > system "./musl_getaddrinfo_test google.com set_ai_family > 2>&1 > > > |tee > > > > > -a > > > > > > > /tmp/musl_#{@musl_ver}_getaddrinfo_test_google.com_set_ai_family.txt > > > " > > > > > > system "./musl_getaddrinfo_test google.com 2>&1 |tee -a > > > > > > /tmp/musl_#{@musl_ver}_getaddrinfo_test_google.com.txt" > > > > > > system "strace -o > > > > > > > > > > > > > > > /tmp/musl_#{@musl_ver}_getaddrinfo_test_google.com_set_ai_family_STRACE.txt > > > > > > ../musl_getaddrinfo_test google.com set_ai_family" > > > > > > system "strace -o > > > > > > /tmp/musl_#{@musl_ver}_getaddrinfo_test_google.com_STRACE.txt > > > > > > ../musl_getaddrinfo_test google.com" > > > > > > > > > > > > And here is the output for each run before running again via > strace. > > > Note > > > > > > how IPv6 addresses show up sporadically, and for 1.2.2 nothing > at all > > > > > shows > > > > > > up, but everything works fine according to the strace logs. > (Strace > > > is > > > > > > built against glibc 2.23.) > > > > > > > > > > > > ==> > > > > > > > > > > musl_1.2.0-git-17-g33338ebc_getaddrinfo_test_google.com_set_ai_family.txt > > > > > > <== > > > > > > AF_INET: 142.251.40.110 > > > > > > > > > > > > ==> musl_1.2.0-git-17-g33338ebc_getaddrinfo_test_google.com.txt > <== > > > > > > AF_INET: 142.251.40.110 > > > > > > > > > > > > ==> > > > > > > > > > > musl_1.2.0-git-39-g5cf1ac24_getaddrinfo_test_google.com_set_ai_family.txt > > > > > > <== > > > > > > AF_INET: 142.251.40.142 > > > > > > > > > > > > ==> musl_1.2.0-git-39-g5cf1ac24_getaddrinfo_test_google.com.txt > <== > > > > > > getaddrinfo: Try again > > > > > > > > > > > > ==> > > > > > > > > > > musl_1.2.0-git-40-g1b4e84c5_getaddrinfo_test_google.com_set_ai_family.txt > > > > > > <== > > > > > > AF_INET: 142.251.40.206 > > > > > > > > > > > > ==> musl_1.2.0-git-40-g1b4e84c5_getaddrinfo_test_google.com.txt > <== > > > > > > AF_INET6: 2607:f8b0:4006:81f::200e > > > > > > AF_INET: 142.251.40.206 > > > > > > > > > > > > ==> > > > > > > > > > > musl_1.2.0-git-6-g2f2348c9_getaddrinfo_test_google.com_set_ai_family.txt > > > > > <== > > > > > > AF_INET: 142.250.65.206 > > > > > > > > > > > > ==> musl_1.2.0-git-6-g2f2348c9_getaddrinfo_test_google.com.txt > <== > > > > > > AF_INET: 142.250.65.206 > > > > > > > > > > > > ==> musl_1.2.1_getaddrinfo_test_google.com_set_ai_family.txt <== > > > > > > AF_INET: 142.251.40.110 > > > > > > > > > > > > ==> musl_1.2.1_getaddrinfo_test_google.com.txt <== > > > > > > getaddrinfo: Try again > > > > > > > > > > > > ==> musl_1.2.2_getaddrinfo_test_google.com_set_ai_family.txt <== > > > > > > getaddrinfo: Try again > > > > > > > > > > > > ==> musl_1.2.2_getaddrinfo_test_google.com.txt <== > > > > > > getaddrinfo: Try again > > > > > > > > > > > > Regards, > > > > > > > > > > OK, I don't see anything in the strace suggesting a cause. The > kernel > > > > > version (or whether a container was used) present on the system > where > > > > > you built musl or the test programs should make no difference > > > > > whatsoever; musl has no build dependencies on the host kernel or > > > > > kernel headers or anything like that (and doesn't even need to be > > > > > built on a Linux host). > > > > > > > > > > A couple questions: > > > > > > > > > > Are the test programs on the i686 machine running under Docker or > any > > > > > other container environment? > > > > > > > > > > Can you tcpdump the traffic between the test program and the > dnsmasq > > > > > during a failing run, with verbose display of the packet contents > > > > > (-vvv or something like that)? > > > > > > > > > > I don't see any plausible explanation for the result varying > between > > > > > runs and with timing like this unless dnsmasq is doing something > > > > > odd/wrong. I thought it might be related to something blocking > time64 > > > > > syscalls but that doesn't seem to be the case -- according to the > > > > > strace logs they're getting ENOSYS as expected with fallback to the > > > > > legacy 32-bit clock_gettime etc. which is fine. > > > > > > > > > > >