Development discussion of WireGuard
 help / color / mirror / Atom feed
* Source IP incorrect on multi homed systems
@ 2023-02-18 20:14 Nico Schottelius
       [not found] ` <CAHx9msc1cNV80YU7HRmQ9gsjSEiVZ=pb31aYqfP62hy8DeuGZA@mail.gmail.com>
  2023-02-19  0:45 ` Mike O'Connor
  0 siblings, 2 replies; 35+ messages in thread
From: Nico Schottelius @ 2023-02-18 20:14 UTC (permalink / raw)
  To: WireGuard mailing list


Dear group,

I was wondering how wireguard [Linux kernel] or wireguard-go [FreeBSD]
are supposed to decide which IP address to use for replying?

I have seen both on FreeBSD and Linux that wireguard seems to use the IP
address of the outgoing interface, i.e. the one with the route returning
to the sender. However in multi homed situations, this can be wrong,
let's take this example:

      19:57:24.607526 net1  In  IP 194.5.220.43.60770 > 147.78.195.254.51820: UDP, length 148
      19:57:24.608358 net2  Out IP 195.141.200.73.51820 > 194.5.220.43.60770: UDP, length 92

The initiator sends from 194.5.220.43 to the receiver 147.78.195.254.
Wireguard then replies with the source IP of 195.141.200.73 instead of
147.78.195.254.

As the node is multi homed, the packet might leave through any of its
uplinks and thus return with a random (unexpected) IP address and will
not pass NAT rules on firewalls and finally be dropped. F.i. in above
example the firewall drops the packet from 195.141.200.73, because there
is no session entry for that.

I have observed this behaviour both on Linux 6.1.11 as well as
wireguard-go 0.0.20220316_8,1 on FreeBSD and in both cases the
connection will break depending on which active interface is taken as
exit.

I would argue that wireguard should by default invert the IP
addresses, i.e. switch dst=src, src=dst and then reply with that,
instead of adapting an interface specific address, or is there a good
reason for the current behaviour?

Best regards,

Nico

--
Sustainable and modern Infrastructures by ungleich.ch

^ permalink raw reply	[flat|nested] 35+ messages in thread
* Re: Source IP incorrect on multi homed systems
@ 2023-02-20 11:09 Janne Johansson
  0 siblings, 0 replies; 35+ messages in thread
From: Janne Johansson @ 2023-02-20 11:09 UTC (permalink / raw)
  To: WireGuard mailing list

rewriting for the lists, managed to bold some pasted text and hence
get blocked due to html-mails not allowed on list.

Den sön 19 feb. 2023 kl 21:17 skrev Nico Schottelius
<nico.schottelius@ungleich.ch>:
> Janne Johansson <icepic.dz@gmail.com> writes:
> > *) https://en.wiktionary.org/wiki/Chesterton%27s_fence
>
> I am happy to have learned a new principle today, thanks for that.
>
> And to be sure that everyone is on the same page:
>
>     Wireguard should reply by default with the source address that
>     used to be the destination address, but at the moment wireguard is not
>     doing that at the moment.
>
> If anyone disagrees with above statement, please let me know.

I disagree, but perhaps only because that statement is slightly too short.

Let's assume I have two ISPs and hence a multihomed wg peer, with ip
A.x.x.x from isp A, and ip B.x.x.x from isp B. For some reason, this
box has a routing table that says "prefer link A to reach the
internet", but I set up client C to set up wireguard to B.1.2.3 and
client C sends it udp packet with src ip C and dest IP B.x.x.x. Since
UDP is stateless, the "response" from the multihomed server is created
"out of thin air" as a random UDP packet destined for C. We don't feel
it is unrelated to the previous received packet, but from the tcp
stack perspective it is.

The routing table now decides that interface A will be the awesomest
for sending UDP to C, and therefore creates a packet with source ip
A.x.x.x and dest ip C.x.x.x and sends it off. This surprise seems to
be the main issue in this thread. Perhaps we see this multihomed box
as slightly misconfigured as far as wireguard goes, perhaps it should
have posted A.x.x.x instead of B.x.x.x as the wg endpoint to the
client or whatever, but the facts remain.

Now, in your above statement you hope to get everyone to agree on,
this would need to also include "sending it back on interface B, to
the gw used by interface B to ISP B if there is one" or else isp A
might drop the packet as being sent from a "forged" address since it
looks like a fake source ip from isp As perspective. The routing
lookups - before any applied tricks - will look at destination IPs
only and make the decision based on that.

I think the proposed solution, while attractive at first glance, may
be trading one kind of "surprise" behaviour to another where the
interface B might be less useful than A which would explain why the
default route is set to use A. If you look at the many posts on the
internet over many years about "why udp source ip got chosen wrong on
multihomed boxes" you see answers like:
"You either bind(2) to each interface address and manage multiple
sockets, or let the kernel do the implicit source IP assignment with
INADDR_ANY. There is no other way."
( https://stackoverflow.com/questions/3062205/setting-the-source-ip-for-a-udp-socket
, not about vpns and lots older than wireguard)

What this means is that if you have a box where links and interfaces
come and go (usb wifi dongles, tethered cell phones..) then wireguard
now has to make a lot of extra work, trying to keep tabs on what
interfaces exist or not, instead of just binding to port 0 and letting
the kernel handle this by itself in the normal but to some, surprising
way for udp packets.

My gut feeling is that if you have a setup like this example
multihomed peer, you get to do some extra steps, which may include the
aforementioned firewall mark "tricks", use VRFs/Namespaced
interfaces/routing domains, add a specific route to client C ip over
link B, bind wg to a loopback interface or source-nat on outgoing wg
traffic or something along those lines in order to have a wg endpoint
on a less-preferred interface and not cause issues with
stateful-nat-gws at client C.

--
May the most significant bit of your life be positive.

^ permalink raw reply	[flat|nested] 35+ messages in thread

end of thread, other threads:[~2023-02-20 20:43 UTC | newest]

Thread overview: 35+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-02-18 20:14 Source IP incorrect on multi homed systems Nico Schottelius
     [not found] ` <CAHx9msc1cNV80YU7HRmQ9gsjSEiVZ=pb31aYqfP62hy8DeuGZA@mail.gmail.com>
2023-02-18 22:34   ` Nico Schottelius
2023-02-19  0:45 ` Mike O'Connor
2023-02-19  8:01   ` Nico Schottelius
2023-02-19  9:19     ` Mikma
2023-02-19 12:04       ` Nico Schottelius
2023-02-19 12:10     ` Nico Schottelius
2023-02-19 18:59       ` Peter Linder
     [not found]     ` <2ed829aaed9fec59ac2a9b32c4ce0a9005b8d8b850be81c81a226791855fe4eb@mu.id>
2023-02-19 12:13       ` Nico Schottelius
2023-02-19 14:39         ` Christoph Loesch
2023-02-19 16:32           ` David Kerr
2023-02-19 16:54             ` Sebastian Hyrvall
2023-02-19 18:04               ` Janne Johansson
2023-02-19 18:08                 ` Sebastian Hyrvall
2023-02-19 20:11                 ` Nico Schottelius
2023-02-19 17:05             ` tlhackque
2023-02-19 18:37               ` David Kerr
2023-02-19 18:52                 ` tlhackque
2023-02-19 18:42               ` tlhackque
2023-02-19 20:18                 ` Nico Schottelius
2023-02-19 20:42                   ` Roman Mamedov
2023-02-19 21:19                     ` Nico Schottelius
2023-02-19 22:06                       ` tlhackque
2023-02-19 22:42                       ` Src addr code review (Was: Source IP incorrect on multi homed systems) Daniel Gröber
2023-02-20  0:28                         ` 曹煜
2023-02-20 10:40                           ` Nico Schottelius
2023-02-20 11:21                             ` 曹煜
2023-02-20  9:47                         ` Nico Schottelius
2023-02-20 20:43                           ` dxld
2023-02-19 21:39                     ` Source IP incorrect on multi homed systems tlhackque
     [not found]               ` <CADGd2DoE6TCtCxxWL7JWyNW5+yy_Pe+9MNzHznbudMWLTXQreA@mail.gmail.com>
2023-02-19 18:30                 ` Fwd: " John Lauro
2023-02-19 22:28                 ` tlhackque
2023-02-20  0:58                   ` Luiz Angelo Daros de Luca
2023-02-19 20:02           ` Nico Schottelius
2023-02-20 11:09 Janne Johansson

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).