Fwd: Source IP incorrect on multi homed systems

[John Lauro <johnalauro@gmail.com>]

I think the ip route with src would work, but only as a short lived
work around.  The problem with it is if dealing with dynamic routes is
it could go a way when a link is down and then come back and the src
setting would be lost.  You would need the bgp software to add the
src.

UDP is connectionless.  Sending back out the same as it's coming in
isn't strictly the same.  The streams are not attached the same as
they would be with TCP on nginx or a reply with icmp. You should be
able to whitelist the udp port on the NAT devices, as it shouldn't use
state info.

I am not sure if you are attempting to do site to site or client to
server/site and which end has the NAT (or both).  What I do for site
to site is use a different port for each connection and have a
separate BGP connection for each possible connection (ie: different
one for different network providers).  Have a full mesh with 8 sites
and upto 3 providers per site.

That said, you probably have floating IPs on the client side, and
don't want to lock in a single IP on the multi-homed server side?  You
could nat the incoming IPs on the border from an internal IP and then
then lock to a single private IP on the wireguard server for in/out
and that border nat would force the reply back to the same gateway it
came in from.

I know, you don't want work arounds, just want to mention it's not the
same as comparing a single stream to something that handles routing
though it.  As you are doing bgp and redundant routes I assume you
also reset rp_filter on all nat/wireguard/routers so the routers will
allow packets to come from different sources.

On Sun, Feb 19, 2023 at 12:07 PM tlhackque <tlhackque@yahoo.com> wrote:
>
> FWIW, while clever, I don't think that iptables mark solves all cases.
> E.g., consider an interface with multiple addresses, where a packet
> comes in on a secondary address.  The proposed rule would send it out
> the right interface, but still with the wrong (primary) address picked
> from the interface...
>
> With IPv6 it's common to assign an address to a service rather than a
> host so services can move easily.  So multiple addresses per interface
> are the rule, not the exception.
>
> I do the same with IPv4 inside addresses, though these days public IPv4
> addresses are scarce enough that it's not common for public IPs.  It
> amounts to the same issue - the NAT tracking is stateful.
>
> Trying to work around this with routing seems like a maze of twisty
> passages - so I agree that the right solution is for WG to respond from
> the address that receives a packet.