From: Christoph Loesch <wireguard-mail@chil.at>
To: <wireguard@lists.zx2c4.com>
Subject: Re: Wireguard source ip selection issue with multi interfaces
Date: Tue, 23 Nov 2021 14:52:37 +0100 [thread overview]
Message-ID: <60b5d439-e117-f3c7-3a6b-8ce46bc48fc2@chil.at> (raw)
In-Reply-To: <CACu-5+0Z1JPCttbZpM13870N9Wce8K3V2ZRdhV6XFmVk4FJKxQ@mail.gmail.com>
Hi,
Is this related to my issue with using wrong source ip on outgoing connections?
(https://lists.zx2c4.com/pipermail/wireguard/2021-November/007307.html)
Which bug would that be? Would be interesting in more detail to understand the issue.
I'm having a hard time to reproduce (or somehow "fix") the issue I am facing, since this issue is on only 5 devices from over 20 devices where all are configured same.
The configuration or better all routes do not even have any metric defined so I am unsure what to add/remove to change the behaviour.
Also in my configuration there are multiple interfaces (br0,br1,eth0,eth1,etc..) as it is a router but there are no multiple default routes.
Comparing perfectly fine working devices to one of those five devices which have this issue I am not able to find a difference in IP-config or routes.
Even kernel and firmware versions are all same.
The only difference I see is while looking at tcpdump where a simple ping to the wireguard server's internal IP (172.27.0.1/24 in my case) originates with the public IP of the device.
If it is a known bug, it would be nice to know what (and if) I could try to fix and possibly confirm the bug. Or even better beeing able to reproduce the issue.
Thanks and kind regards,
Christoph
Am 19.11.2021 um 14:10 schrieb 曹煜:
> Hi,
> As Jason said in mailing list
> (https://lists.zx2c4.com/pipermail/wireguard/2017-November/002018.html)
> years ago, if wireguard reply the client with wrong src ip, then it is
> a bug. And now I'm trying to set up a minimal configuration.
>
> Server info:
>
> Ubuntu 20.04 in VMware with two Host-only network adapters:
>
> wacke@Ubuntu:/etc$ lsb_release -a
> No LSB modules are available.
> Distributor ID: Ubuntu
> Description: Ubuntu 20.04.3 LTS
> Release: 20.04
> Codename: focal
>
> wacke@Ubuntu:/etc$ uname -a
> Linux Ubuntu 5.14.8-051408-generic #202109280455 SMP Tue Sep 28
> 04:57:31 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
>
> wacke@Ubuntu:/etc$ sudo ip route show
> 169.254.0.0/16 dev ens33 scope link metric 1000
> 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
> 192.168.187.0/24 dev ens33 proto kernel scope link src 192.168.187.129
> metric 100
> 192.168.187.0/24 dev ens35 proto kernel scope link src 192.168.187.128
> metric 101
> 192.168.199.0/24 dev wg0 proto kernel scope link src 192.168.199.1
>
> wacke@Ubuntu:/etc$ sudo cat /etc/wireguard/wg0.conf
> [Interface]
> PrivateKey = oLLeTlvwWsfGoyQpDwZo0k7AsMf0LkkfycGPwFjamEA=
> ListenPort = 5999
>
> [Peer]
> PublicKey = JYvK2/FBbbSrDbkH29ejWejkBJS3ch/SFtyZwNf5O1c=
> AllowedIPs = 192.168.199.2/32
> PersistentKeepalive = 25
>
> [Peer]
> PublicKey = keIPXZFgvB79biV74kGc5R9vAHpzyVyQHci8KBSDIUw=
> AllowedIPs = 192.168.199.3/32
> PersistentKeepalive = 25
>
> wacke@Ubuntu:/etc$ sudo wg setconf wg0 /etc/wireguard/wg0.conf
> wacke@Ubuntu:/etc$ sudo ip address add dev wg0 192.168.199.1/24
> wacke@Ubuntu:/etc$ sudo ip link set up dev wg0
> wacke@Ubuntu:/etc$ sudo wg show
> interface: wg0
> public key: Vkdx0MhQkc9anLaUehR/Df1zuKjxceflxMCAeAaCWCo=
> private key: (hidden)
> listening port: 5999
>
> peer: JYvK2/FBbbSrDbkH29ejWejkBJS3ch/SFtyZwNf5O1c=
> endpoint: 192.168.187.1:35560
> allowed ips: 192.168.199.2/32
> transfer: 2.89 KiB received, 1.80 KiB sent
> persistent keepalive: every 25 seconds
>
> peer: keIPXZFgvB79biV74kGc5R9vAHpzyVyQHci8KBSDIUw=
> allowed ips: 192.168.199.3/32
> persistent keepalive: every 25 seconds
>
> Client info:
>
> HOME-Server:/NAS/Software/Software # uname -a
> Linux HOME-Server 5.14.14-2-default #1 SMP Thu Oct 21 05:05:03 UTC
> 2021 (2b5383f) x86_64 x86_64 x86_64 GNU/Linux
>
> HOME-Server:/NAS/Software/Software # cat wg0.conf
> [Interface]
> PrivateKey = 4Pw1cetxd9TdfH3TSab+9UcRBlHwZ1o/vmrUgkerZls=
>
> [Peer]
> PublicKey = Vkdx0MhQkc9anLaUehR/Df1zuKjxceflxMCAeAaCWCo=
> Endpoint = 192.168.187.128:5999
> AllowedIPs = 192.168.199.0/24
> PersistentKeepalive = 25
>
> HOME-Server:/NAS/Software/Software # ip link add wg0 type wireguard
> HOME-Server:/NAS/Software/Software # ip address add dev wg0 192.168.199.2/24
> HOME-Server:/NAS/Software/Software # wg setconf wg0 wg0.conf
> HOME-Server:/NAS/Software/Software # ip link set wg0 up
> HOME-Server:/NAS/Software/Software # wg show
> interface: wg0
> public key: JYvK2/FBbbSrDbkH29ejWejkBJS3ch/SFtyZwNf5O1c=
> private key: (hidden)
> listening port: 53058
>
> peer: Vkdx0MhQkc9anLaUehR/Df1zuKjxceflxMCAeAaCWCo=
> endpoint: 192.168.187.128:5999
> allowed ips: 192.168.199.0/24
> transfer: 0 B received, 59.26 KiB sent
> persistent keepalive: every 25 seconds
>
>
> tcpdump of client side:
>
> HOME-Server:/NAS/Software/Software # tcpdump -i any -vn "(dst port 5999)"
> tcpdump: data link type LINUX_SLL2
> tcpdump: listening on any, link-type LINUX_SLL2 (Linux cooked v2),
> snapshot length 262144 bytes
> 22:10:37.794301 vmnet1 Out IP (tos 0x88, ttl 64, id 54724, offset 0,
> flags [none], proto UDP (17), length 176)
> 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148
> 22:10:43.170306 vmnet1 Out IP (tos 0x88, ttl 64, id 55327, offset 0,
> flags [none], proto UDP (17), length 176)
> 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148
> 22:10:48.546281 vmnet1 Out IP (tos 0x88, ttl 64, id 56142, offset 0,
> flags [none], proto UDP (17), length 176)
> 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148
> 22:10:53.670343 vmnet1 Out IP (tos 0x88, ttl 64, id 57053, offset 0,
> flags [none], proto UDP (17), length 176)
> 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148
> 22:11:18.754308 vmnet1 Out IP (tos 0x88, ttl 64, id 58505, offset 0,
> flags [none], proto UDP (17), length 176)
> 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148
> 22:11:24.130286 vmnet1 Out IP (tos 0x88, ttl 64, id 58621, offset 0,
> flags [none], proto UDP (17), length 176)
> 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148
> 22:11:29.506305 vmnet1 Out IP (tos 0x88, ttl 64, id 58810, offset 0,
> flags [none], proto UDP (17), length 176)
> 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148
> 22:11:34.882326 vmnet1 Out IP (tos 0x88, ttl 64, id 59444, offset 0,
> flags [none], proto UDP (17), length 176)
> 192.168.187.1.53058 > 192.168.187.128.5999: UDP, length 148
> ^C
> 8 packets captured
> 8 packets received by filter
> 0 packets dropped by kernel
>
> tcpdump of server side:
>
> wacke@Ubuntu:~$ sudo tcpdump -i any -vn "(src port 5999)"
> tcpdump: listening on any, link-type LINUX_SLL (Linux cooked v1),
> capture size 262144 bytes
> 22:10:11.754862 IP (tos 0x88, ttl 64, id 22024, offset 0, flags
> [none], proto UDP (17), length 120)
> 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
> 22:10:16.874400 IP (tos 0x88, ttl 64, id 22399, offset 0, flags
> [none], proto UDP (17), length 120)
> 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
> 22:10:21.994433 IP (tos 0x88, ttl 64, id 23057, offset 0, flags
> [none], proto UDP (17), length 120)
> 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
> 22:10:27.370558 IP (tos 0x88, ttl 64, id 23912, offset 0, flags
> [none], proto UDP (17), length 120)
> 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
> 22:10:32.494605 IP (tos 0x88, ttl 64, id 24944, offset 0, flags
> [none], proto UDP (17), length 120)
> 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
> 22:10:37.610779 IP (tos 0x88, ttl 64, id 25327, offset 0, flags
> [none], proto UDP (17), length 120)
> 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
> 22:10:42.986834 IP (tos 0x88, ttl 64, id 25498, offset 0, flags
> [none], proto UDP (17), length 120)
> 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
> 22:10:48.362908 IP (tos 0x88, ttl 64, id 26521, offset 0, flags
> [none], proto UDP (17), length 120)
> 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
> 22:10:53.486942 IP (tos 0x88, ttl 64, id 27214, offset 0, flags
> [none], proto UDP (17), length 120)
> 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
> 22:11:18.571437 IP (tos 0x88, ttl 64, id 31112, offset 0, flags
> [none], proto UDP (17), length 120)
> 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
> 22:11:23.947460 IP (tos 0x88, ttl 64, id 31897, offset 0, flags
> [none], proto UDP (17), length 120)
> 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
> 22:11:29.323608 IP (tos 0x88, ttl 64, id 32715, offset 0, flags
> [none], proto UDP (17), length 120)
> 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
> 22:11:34.699676 IP (tos 0x88, ttl 64, id 33840, offset 0, flags
> [none], proto UDP (17), length 120)
> 192.168.187.129.5999 > 192.168.187.1.53058: UDP, length 92
> ^C
> 13 packets captured
> 13 packets received by filter
> 0 packets dropped by kernel
>
> As the tcpdump log showed, the wireguard server always chose the ip
> with lowest metric as source ip (192.168.187.129) to reply to the
> client (while the client tried to connect 192.168.187.128).
>
> Hope the info above can help to debug. Many thanks.
next prev parent reply other threads:[~2021-11-23 13:52 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-11-19 13:10 曹煜
2021-11-23 13:52 ` Christoph Loesch [this message]
2022-10-19 22:47 ` Source IP selection or multi-home or multi-interface issue - solved with source-based route - was: " Christoph Loesch
2022-10-19 22:56 ` Christoph Loesch
2022-10-21 13:11 ` Christoph Loesch
2022-10-21 13:25 ` Janne Johansson
2022-10-21 14:12 ` 曹煜
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=60b5d439-e117-f3c7-3a6b-8ce46bc48fc2@chil.at \
--to=wireguard-mail@chil.at \
--cc=wireguard@lists.zx2c4.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).