Hi everyone, Sorry for bothering you, but we do have an Android specific question. When using the official WireGuard app on Android (v1.0.20200927), as well as our own, on certain device types we have issues reaching local devices in the network, even if we exclude private IPs. It works entirely fine on other device models though. I have tried on several different wireless networks & different subnets (192.168.0.1/24, 192.168.0.1/16, 10.220.0.1/16), as well as reaching various devices. So far, we have had issues on OnePlus HD1903 & Samsung Galaxy S10e (model number SM-G970F). Network I have three devices in the wireless network (10.220.1.1/16). 1. Samsung Galaxy S10e (10.220.0.2) 2. Macbook (10.220.0.1) 3. Lenovo thinkpad (10.220.1.7) On the S10e, I can not ping 10.220.0.1 nor 10.220.1.7 when connected. On the Macbook, I can ping 10.220.1.7 when connected. Configurations Macbook uses following config: [Interface] PrivateKey = XXX Address = 172.20.196.X/32, fd00:0:1337:cafe:1111:1111:X:1a60/128 DNS = 46.227.67.134, 192.165.9.158 [Peer] PublicKey = 0CvxLBss+WQyJqidXRUnd5iO9gMJwMFNPidg09d4Onk= AllowedIPs = ::/0, 0.0.0.0/5, 8.0.0.0/7, 11.0.0.0/8, 12.0.0.0/6, 16.0.0.0/4, 32.0.0.0/3, 64.0.0.0/2, 128.0.0.0/3, 160.0.0.0/5, 168.0.0.0/6, 172.0.0.0/12, 172.32.0.0/11, 172.64.0.0/10, 172.128.0.0/9, 173.0.0.0/8, 174.0.0.0/7, 176.0.0.0/4, 192.0.0.0/9, 192.128.0.0/11, 192.160.0.0/13, 192.169.0.0/16, 192.170.0.0/15, 192.172.0.0/14, 192.176.0.0/12, 192.192.0.0/10, 193.0.0.0/8, 194.0.0.0/7, 196.0.0.0/6, 200.0.0.0/5, 208.0.0.0/4, 46.227.67.134/32, 192.165.9.158/32 Endpoint = vpn90.prd.erfurt.ovpn.com:9929 Samsung Galaxy S10e uses the following config: [Interface] Address = X/32, fd00:0:1337:cafe:1111:1111:e827:X/128 DNS = 46.227.67.134, 192.165.9.158 PrivateKey = XXX [Peer] AllowedIPs = 0.0.0.0/5, 8.0.0.0/7, 11.0.0.0/8, 12.0.0.0/6, 16.0.0.0/4, 32.0.0.0/3, 64.0.0.0/2, 128.0.0.0/3, 160.0.0.0/5, 168.0.0.0/6, 172.0.0.0/12, 172.32.0.0/11, 172.64.0.0/10, 172.128.0.0/9, 173.0.0.0/8, 174.0.0.0/7, 176.0.0.0/4, 192.0.0.0/9, 192.128.0.0/11, 192.160.0.0/13, 192.169.0.0/16, 192.170.0.0/15, 192.172.0.0/14, 192.176.0.0/12, 192.192.0.0/10, 193.0.0.0/8, 194.0.0.0/7, 196.0.0.0/6, 200.0.0.0/5, 208.0.0.0/4, ::/0, 46.227.67.134/32, 192.165.9.158/32 Endpoint = vpn11.prd.kista.ovpn.com:9929 PublicKey = p4s2SuFgKgWnV56Uh3yCx6ZWnaVyFXegV9FQqv670So= Is this an issue you’re aware of? Is there anything else needed from me? Thanks, David Wibergh
Hi David, I haven't seen other reports like this before, so I'm not really sure off hand. Firstly, do the Macbook and the Thinkpad respond to pings in the first place? Modern macOS and Windows have built-in firewalls that usually prevent this. So make sure that the pings work without WireGuard part of the equation. If you've done this, and it works without WireGuard, and it doesn't work with WireGuard, then we can proceed assuming this is an issue with WireGuard. That config seems fine on a cursory glance. You mentioned that this only happens on certain phones. Which Android phones work as intended, and which do not? Which operating system versions are each of these running? The more general information about this that you can provide, the more we can narrow it down. Between Android releases, there have been subtle changes in their routing particulars, and between Android vendors, I've seen aggressive power management policies affecting WireGuard, and between Android configurations, I've seen newer features like DoH/DoT confusing the VPN subsystem too. And there may be other weird patterns and quirks too. If this really is a problem with "phone X but not phone Y," we'll need some more smells to find out what's going on. Alternatively, you can dump `ip route show table all` and `ip rule show` and `iptables-save` on each of the phones and see if you notice an obvious difference in the routing that netd sets up. That might not lead to a fix of the issue, but it might add more precision to why it's not working as intended. Jason
[-- Attachment #1: Type: text/plain, Size: 2637 bytes --] Hi, The Windows laptop that I'm usin as "destination ping" has the network profile set to private, so the S10e and Macbook are able to ping it. The phones that we've tested so far that work as intended are: - Pixel 2, Android 11 - Pixel 2XL, Android 11 - Meizu M5S, Android 6 (flyme 6.3.1) - Oneplus 2, LineageOS 17.1 - Sony Xperia XZ1, Android 9 The phone I have access to and does not work properly: - Samsung Galaxy S10e, Android 10. One UI 2.0 with android security patch level 1 February 2020. WireGuard on iOS with excluding private IPs is able to ping the Windows laptop. I'm attaching a photo of the Sony Xperia XZ1 & Samsung Galaxy S10e next to each other, displaying discoverable devices in the newly created wifi network to troubleshoot this further. Command output has also been attached from Sony Xperia XZ1 & Samsung Galaxy S10e. Thanks, David På 9 oktober 2020 kl. 14:22:01, Jason A. Donenfeld (jason@zx2c4.com) skrev: > Hi David, > > I haven't seen other reports like this before, so I'm not really sure off hand. > > Firstly, do the Macbook and the Thinkpad respond to pings in the first > place? Modern macOS and Windows have built-in firewalls that usually > prevent this. So make sure that the pings work without WireGuard part > of the equation. If you've done this, and it works without WireGuard, > and it doesn't work with WireGuard, then we can proceed assuming this > is an issue with WireGuard. > > That config seems fine on a cursory glance. You mentioned that this > only happens on certain phones. Which Android phones work as intended, > and which do not? Which operating system versions are each of these > running? The more general information about this that you can provide, > the more we can narrow it down. > > Between Android releases, there have been subtle changes in their > routing particulars, and between Android vendors, I've seen aggressive > power management policies affecting WireGuard, and between Android > configurations, I've seen newer features like DoH/DoT confusing the > VPN subsystem too. And there may be other weird patterns and quirks > too. If this really is a problem with "phone X but not phone Y," we'll > need some more smells to find out what's going on. > > Alternatively, you can dump `ip route show table all` and `ip rule > show` and `iptables-save` on each of the phones and see if you notice > an obvious difference in the routing that netd sets up. That might not > lead to a fix of the issue, but it might add more precision to why > it's not working as intended. > > Jason > [-- Attachment #2: sony.txt --] [-- Type: text/plain, Size: 5545 bytes --] $ ip route show all 10.220.0.0/24 dev wlan0 proto kernel scope link src 10.220.0.6 $ ip route show table all default via 10.220.0.1 dev wlan0 table 1026 proto static 10.220.0.0/24 dev wlan0 table 1026 proto static scope link 0.0.0.0/5 dev tun0 table 1028 proto static scope link 8.0.0.0/7 dev tun0 table 1028 proto static scope link 11.0.0.0/8 dev tun0 table 1028 proto static scope link 12.0.0.0/6 dev tun0 table 1028 proto static scope link 16.0.0.0/4 dev tun0 table 1028 proto static scope link 32.0.0.0/3 dev tun0 table 1028 proto static scope link 46.227.67.134 dev tun0 table 1028 proto static scope link 64.0.0.0/2 dev tun0 table 1028 proto static scope link 128.0.0.0/3 dev tun0 table 1028 proto static scope link 160.0.0.0/5 dev tun0 table 1028 proto static scope link 168.0.0.0/6 dev tun0 table 1028 proto static scope link 172.0.0.0/12 dev tun0 table 1028 proto static scope link 172.20.192.155 dev tun0 table 1028 proto static scope link 172.32.0.0/11 dev tun0 table 1028 proto static scope link 172.64.0.0/10 dev tun0 table 1028 proto static scope link 172.128.0.0/9 dev tun0 table 1028 proto static scope link 173.0.0.0/8 dev tun0 table 1028 proto static scope link 174.0.0.0/7 dev tun0 table 1028 proto static scope link 176.0.0.0/4 dev tun0 table 1028 proto static scope link 192.0.0.0/9 dev tun0 table 1028 proto static scope link 192.128.0.0/11 dev tun0 table 1028 proto static scope link 192.160.0.0/13 dev tun0 table 1028 proto static scope link 192.165.9.158 dev tun0 table 1028 proto static scope link 192.169.0.0/16 dev tun0 table 1028 proto static scope link 192.170.0.0/15 dev tun0 table 1028 proto static scope link 192.172.0.0/14 dev tun0 table 1028 proto static scope link 192.176.0.0/12 dev tun0 table 1028 proto static scope link 192.192.0.0/10 dev tun0 table 1028 proto static scope link 193.0.0.0/8 dev tun0 table 1028 proto static scope link 194.0.0.0/7 dev tun0 table 1028 proto static scope link 196.0.0.0/6 dev tun0 table 1028 proto static scope link 200.0.0.0/5 dev tun0 table 1028 proto static scope link 208.0.0.0/4 dev tun0 table 1028 proto static scope link default dev dummy0 table 1003 proto static scope link 10.220.0.0/24 dev wlan0 proto kernel scope link src 10.220.0.6 broadcast 10.220.0.0 dev wlan0 table local proto kernel scope link src 10.220.0.6 local 10.220.0.6 dev wlan0 table local proto kernel scope host src 10.220.0.6 broadcast 10.220.0.255 dev wlan0 table local proto kernel scope link src 10.220.0.6 broadcast 127.0.0.0 dev lo table local proto kernel scope link src 127.0.0.1 local 127.0.0.0/8 dev lo table local proto kernel scope host src 127.0.0.1 local 127.0.0.1 dev lo table local proto kernel scope host src 127.0.0.1 broadcast 127.255.255.255 dev lo table local proto kernel scope link src 127.0.0.1 local 172.20.192.155 dev tun0 table local proto kernel scope host src 172.20.192.155 fe80::/64 dev wlan0 table 1026 proto kernel metric 256 pref medium fe80::/64 dev wlan0 table 1026 proto static metric 1024 pref medium fd00:0:1337:cafe:1111:1111:c5a7:bdd0 dev tun0 table 1028 proto kernel metric 256 pref medium fd00:0:1337:cafe:1111:1111:c5a7:bdd0 dev tun0 table 1028 proto static metric 1024 pref medium default dev tun0 table 1028 proto static metric 1024 pref medium fe80::/64 dev dummy0 table 1003 proto kernel metric 256 pref medium default dev dummy0 table 1003 proto static metric 1024 pref medium fe80::/64 dev rmnet_data0 table 1009 proto kernel metric 256 mtu 2000 pref medium default via fe80::69a8:d914:8160:58e3 dev rmnet_data0 table 1009 proto ra metric 1024 expires 1329sec hoplimit 255 pref medium local ::1 dev lo table local proto unspec metric 0 pref medium local fd00:0:1337:cafe:1111:1111:c5a7:bdd0 dev lo table local proto unspec metric 0 pref medium local fe80::2a3f:69ff:fec8:740 dev lo table local proto unspec metric 0 pref medium local fe80::2c03:deff:fe32:d296 dev lo table local proto unspec metric 0 pref medium local fe80::cda1:1c6:bb03:6c0c dev lo table local proto unspec metric 0 pref medium ff00::/8 dev dummy0 table local metric 256 pref medium ff00::/8 dev rmnet_data0 table local metric 256 mtu 2000 pref medium ff00::/8 dev wlan0 table local metric 256 pref medium ff00::/8 dev tun0 table local metric 256 pref medium $ ip rule show 0: from all lookup local 10000: from all fwmark 0xc0000/0xd0000 lookup 99 10500: from all iif lo oif dummy0 uidrange 0-0 lookup 1003 10500: from all iif lo oif rmnet_data0 uidrange 0-0 lookup 1009 10500: from all iif lo oif wlan0 uidrange 0-0 lookup 1026 11000: from all iif tun0 lookup 97 12000: from all fwmark 0x0/0x20000 iif lo uidrange 0-99999 lookup 1028 12000: from all fwmark 0xc00ce/0xcffff lookup 1028 13000: from all fwmark 0x10063/0x1ffff iif lo lookup 97 13000: from all fwmark 0xd0001/0xdffff iif lo lookup 1009 13000: from all fwmark 0x100cd/0x1ffff iif lo lookup 1026 13000: from all fwmark 0x100ce/0x1ffff iif lo uidrange 0-99999 lookup 1028 13000: from all fwmark 0x100ce/0x1ffff iif lo uidrange 0-0 lookup 1028 14000: from all iif lo oif dummy0 lookup 1003 14000: from all fwmark 0xc0000/0xc0000 iif lo oif rmnet_data0 lookup 1009 14000: from all iif lo oif wlan0 lookup 1026 14000: from all iif lo oif tun0 uidrange 0-99999 lookup 1028 15000: from all fwmark 0x0/0x10000 lookup 99 16000: from all fwmark 0x0/0x10000 lookup 98 17000: from all fwmark 0x0/0x10000 lookup 97 19000: from all fwmark 0xcd/0x1ffff iif lo lookup 1026 21000: from all fwmark 0xce/0x1ffff lookup 1026 22000: from all fwmark 0x0/0xffff iif lo lookup 1026 32000: from all unreachable [-- Attachment #3: galaxy.txt --] [-- Type: text/plain, Size: 5084 bytes --] $ ip route show table all 0.0.0.0/5 dev tun0 table 1061 proto static scope link 8.0.0.0/7 dev tun0 table 1061 proto static scope link 11.0.0.0/8 dev tun0 table 1061 proto static scope link 12.0.0.0/6 dev tun0 table 1061 proto static scope link 16.0.0.0/4 dev tun0 table 1061 proto static scope link 32.0.0.0/3 dev tun0 table 1061 proto static scope link 46.227.67.134 dev tun0 table 1061 proto static scope link 64.0.0.0/2 dev tun0 table 1061 proto static scope link 128.0.0.0/3 dev tun0 table 1061 proto static scope link 160.0.0.0/5 dev tun0 table 1061 proto static scope link 168.0.0.0/6 dev tun0 table 1061 proto static scope link 172.0.0.0/12 dev tun0 table 1061 proto static scope link 172.27.240.16 dev tun0 table 1061 proto static scope link 172.32.0.0/11 dev tun0 table 1061 proto static scope link 172.64.0.0/10 dev tun0 table 1061 proto static scope link 172.128.0.0/9 dev tun0 table 1061 proto static scope link 173.0.0.0/8 dev tun0 table 1061 proto static scope link 174.0.0.0/7 dev tun0 table 1061 proto static scope link 176.0.0.0/4 dev tun0 table 1061 proto static scope link 192.0.0.0/9 dev tun0 table 1061 proto static scope link 192.128.0.0/11 dev tun0 table 1061 proto static scope link 192.160.0.0/13 dev tun0 table 1061 proto static scope link 192.165.9.158 dev tun0 table 1061 proto static scope link 192.169.0.0/16 dev tun0 table 1061 proto static scope link 192.170.0.0/15 dev tun0 table 1061 proto static scope link 192.172.0.0/14 dev tun0 table 1061 proto static scope link 192.176.0.0/12 dev tun0 table 1061 proto static scope link 192.192.0.0/10 dev tun0 table 1061 proto static scope link 193.0.0.0/8 dev tun0 table 1061 proto static scope link 194.0.0.0/7 dev tun0 table 1061 proto static scope link 196.0.0.0/6 dev tun0 table 1061 proto static scope link 200.0.0.0/5 dev tun0 table 1061 proto static scope link 208.0.0.0/4 dev tun0 table 1061 proto static scope link 90.132.128.59 via 10.220.0.1 dev wlan0 table 99 proto static default via 10.220.0.1 dev wlan0 table 1017 proto static 10.220.0.0/24 dev wlan0 table 1017 proto static scope link 10.220.0.0/24 dev wlan0 proto kernel scope link src 10.220.0.5 broadcast 10.220.0.0 dev wlan0 table local proto kernel scope link src 10.220.0.5 local 10.220.0.5 dev wlan0 table local proto kernel scope host src 10.220.0.5 broadcast 10.220.0.255 dev wlan0 table local proto kernel scope link src 10.220.0.5 broadcast 127.0.0.0 dev lo table local proto kernel scope link src 127.0.0.1 local 127.0.0.0/8 dev lo table local proto kernel scope host src 127.0.0.1 local 127.0.0.1 dev lo table local proto kernel scope host src 127.0.0.1 broadcast 127.255.255.255 dev lo table local proto kernel scope link src 127.0.0.1 local 172.27.240.16 dev tun0 table local proto kernel scope host src 172.27.240.16 fd00:0:1337:cafe:1111:1111:e827:475a dev tun0 table 1061 proto kernel metric 256 pref medium fd00:0:1337:cafe:1111:1111:e827:475a dev tun0 table 1061 proto static metric 1024 pref medium fe80::/64 dev tun0 table 1061 proto kernel metric 256 pref medium default dev tun0 table 1061 proto static metric 1024 pref medium fe80::/64 dev wlan0 table 1017 proto kernel metric 256 pref medium fe80::/64 dev wlan0 table 1017 proto static metric 1024 pref medium local ::1 dev lo table local proto kernel metric 0 pref medium local fd00:0:1337:cafe:1111:1111:e827:475a dev tun0 table local proto kernel metric 0 pref medium local fe80::2cd1:3bff:fece:5c6d dev wlan0 table local proto kernel metric 0 pref medium local fe80::95fb:ac18:144f:f287 dev tun0 table local proto kernel metric 0 pref medium ff00::/8 dev tun0 table local metric 256 pref medium ff00::/8 dev wlan0 table local metric 256 pref medium $ ip rule show 0:Â Â Â Â Â from all lookup local 10000:Â from all fwmark 0xc0000/0xd0000 lookup 99 10500:Â from all iif lo oif wlan0 uidrange 0-0 lookup 1017 11000:Â from all iif tun0 lookup 97 12000:Â from all fwmark 0xc0289/0xcffff lookup 1047 12000:Â from all fwmark 0xc02a3/0xcffff lookup 1058 12000:Â from all fwmark 0xc02a4/0xcffff lookup 1059 12000:Â from all fwmark 0x0/0x20000 iif lo uidrange 0-99999 lookup 1061 12000:Â from all fwmark 0xc02ab/0xcffff lookup 1061 13000:Â from all fwmark 0x10063/0x1ffff iif lo lookup 97 13000:Â from all fwmark 0x10289/0x1ffff iif lo uidrange 0-0 lookup 1047 13000:Â from all fwmark 0x102a3/0x1ffff iif lo uidrange 0-0 lookup 1058 13000:Â from all fwmark 0x102a4/0x1ffff iif lo uidrange 0-0 lookup 1059 13000:Â from all fwmark 0x102ab/0x1ffff iif lo uidrange 0-99999 lookup 1061 13000:Â from all fwmark 0x102ab/0x1ffff iif lo uidrange 0-0 lookup 1061 13000:Â from all fwmark 0x102ad/0x1ffff iif lo lookup 1017 14000:Â from all iif lo oif tun0 uidrange 0-99999 lookup 1061 14000:Â from all iif lo oif wlan0 lookup 1017 15000:Â from all fwmark 0x0/0x10000 lookup 99 16000:Â from all fwmark 0x0/0x10000 lookup 98 17000:Â from all fwmark 0x0/0x10000 lookup 97 19000:Â from all fwmark 0x2ad/0x1ffff iif lo lookup 1017 21000:Â from all fwmark 0x2ab/0xffff lookup 1017 22000:Â from all fwmark 0x0/0xffff iif lo lookup 1017 32000:Â from all unreachable [-- Attachment #4: Screenshot at Oct 09 15-40-55.png --] [-- Type: image/png, Size: 1659779 bytes --]
Hmm, from that command output, I'm still not quite sure. The most significant difference I can see from casual inspection is: Sony: 19000: from all fwmark 0xcd/0x1ffff iif lo lookup 1026 21000: from all fwmark 0xce/0x1ffff lookup 1026 Galaxy: 19000: from all fwmark 0x2ad/0x1ffff iif lo lookup 1017 21000: from all fwmark 0x2ab/0xffff lookup 1017 Is the galaxy missing a leading 0x1 in the mask of that second rule there? What do you get on each when you run `ip route get 10.220.0.4`? Same thing or is it different? I also noticed that on the Galaxy, there's a direct route to some telco server -- `90.132.128.59 via 10.220.0.1 dev wlan0 table 99 proto static`. I didn't see this direct route on the Sony. I doubt it's related to the above, but I've also never seen this before and wonder if it's some VoLTE/VoWIFI hand-over trick in recent Android networking stacks.
The command output is the same:
Sony
$ ip route get 10.220.0.4
10.220.0.4 dev wlan0 table 1026 src 10.220.0.6 uid 10356
cache
Galaxy S10e
$ ip route get 10.220.0.4
10.220.0.4 dev wlan0 table 1017 src 10.220.0.5 uid 10294
cache
We’ll try to get some of our users to run similar tests. We do have an S7 that we can dig up and do some testing on. If you want to go down the rabbit hole, but don’t have access to a galaxy device, I could ship the S10e to you.
Thanks,
David
På 9 oktober 2020 kl. 16:03:02, Jason A. Donenfeld (jason@zx2c4.com) skrev:
> Hmm, from that command output, I'm still not quite sure. The most
> significant difference I can see from casual inspection is:
>
> Sony:
> 19000: from all fwmark 0xcd/0x1ffff iif lo lookup 1026
> 21000: from all fwmark 0xce/0x1ffff lookup 1026
>
> Galaxy:
> 19000: from all fwmark 0x2ad/0x1ffff iif lo lookup 1017
> 21000: from all fwmark 0x2ab/0xffff lookup 1017
>
> Is the galaxy missing a leading 0x1 in the mask of that second rule there?
>
> What do you get on each when you run `ip route get 10.220.0.4`? Same
> thing or is it different?
>
> I also noticed that on the Galaxy, there's a direct route to some
> telco server -- `90.132.128.59 via 10.220.0.1 dev wlan0 table 99 proto
> static`. I didn't see this direct route on the Sony. I doubt it's
> related to the above, but I've also never seen this before and wonder
> if it's some VoLTE/VoWIFI hand-over trick in recent Android networking
> stacks.
>