Multiple Endpoints

Multiple Endpoints

[em12345]

Hi,

I'm wondering how to setup a WG client when having multiple DynDNS names
for a single WG server using dynamic IP. Using multiple DynDNS services
for the same host is a common pattern to work around unreliable DynDNS
services.
As far as I see, only a single endpoint can be given in the configuration.

If there are no plans to support multiple endpoints, I guess this has to
be done via a cron job, since I don't see any explicit hook which could
be used instead.

In order to implement such a script I would have the following
additional questions:

1.) Is "wg setconf" returning/succeeding before the client-server
connection could be established? If so, would it be possible to add an
additional timeout option which waits up to the timeout for establishing
the connection and exits non zero if it failed to do so?
2.) Is "ip link up wg0" already returning/succeeding before the
client-server connection could be established?
3.) What is the best way to determine if a WG client-server connection
is currently alive/working/established? E.g.: wg show wg0
latest-handshakes? I assume that "ip link show wg0" will not show such info.
4.) Is it possible to use "wg set" and/or "wg setconf" while the link is
already up?
5.) Is it possible, using "wg set wg0 endpoint" to only change the
endpoint of an existing peer? Or is it necessary to use "wg setconf"
which replaces every setting for the link?


Similar problems have to be resolved when having Road Warrior PCs, which
may by sometimes directly connected inside the LAN, in which case the
tunnel should not be used. So:
1.) block server endpoint (e.g. via firewall) for crypted traffic
originating from a LAN client
2.) drop (or don't add in the first place) routes into tunnel when no
connection could be established


Thanks

Emmanuel

Re: Multiple Endpoints

[Jason A. Donenfeld]

Hello,

Keep in mind that WireGuard's roaming property means that while the
two peers are communicating, they'll automatically be updating to each
others' latest IP addresses. One way to ensure that they _keep_
communicating is by using the PersistentKeepalive feature. This then
shifts the problem to "how do they start communicating", in which case
you can just use a little resolve,ping,resolve,ping loop on your
various dyndns services.

But, in case you want a different architecture, I'll directly answer
your questions:

- wg setconf/addconf/set can be run at any time, before or after the
link is up, and before or after peers are communicating. It returns
and succeeds immediately, leaving the actual negotiation to be done
whenever data needs to be sent.
- The same goes for `ip link up`, with the sole exception that `ip
link up` may fail if the UDP port is already in use by a different
program.
- The best way to determine if a wireguard link is up is if you can
send a ping through the tunnel.
- Your syntax doesn't make sense for endpoint setting. What you want
is: `wg set wg0 peer ABCDEFG... endpoint 1.2.3.4:1234`. So, yes, you
can individually set the endpoint of a peer.

Jason

Re: Multiple Endpoints

[em12345]

Hi Jason,

I'm not sure that I'm understanding the roaming feature in WireGuard.

>From your response it sounds like once a connection is established, then
the server can inform the client about a server IP change.

This would require PersistentKeepalive on "server" side. But assuming
the common case that the client sits behind a stateful firewall, how
would the server be able to inform the client about its IP change?

On a server IP change:

- the client still sends UDP packages to old server IP, which is useless

- the server (from its new IP) can send UDP packages to the still
remembered client IP (because of PersistentKeepalive). But my
understanding is that stateful firewalls will block UDP packages from
the new IP until the client has send an UDP to the new server IP.

So in such a scenario the roaming feature wouldn't help.

Thanks

Emmanuel



> Hello,
> 
> Keep in mind that WireGuard's roaming property means that while the
> two peers are communicating, they'll automatically be updating to each
> others' latest IP addresses. One way to ensure that they _keep_
> communicating is by using the PersistentKeepalive feature. This then
> shifts the problem to "how do they start communicating", in which case
> you can just use a little resolve,ping,resolve,ping loop on your
> various dyndns services.
> 
> But, in case you want a different architecture, I'll directly answer
> your questions:
> 
> - wg setconf/addconf/set can be run at any time, before or after the
> link is up, and before or after peers are communicating. It returns
> and succeeds immediately, leaving the actual negotiation to be done
> whenever data needs to be sent.
> - The same goes for `ip link up`, with the sole exception that `ip
> link up` may fail if the UDP port is already in use by a different
> program.
> - The best way to determine if a wireguard link is up is if you can
> send a ping through the tunnel.
> - Your syntax doesn't make sense for endpoint setting. What you want
> is: `wg set wg0 peer ABCDEFG... endpoint 1.2.3.4:1234`. So, yes, you
> can individually set the endpoint of a peer.
> 
> Jason
> 

Re: Multiple Endpoints

[Baptiste Jonglez]

[-- Attachment #1: Type: text/plain, Size: 2258 bytes --]

Hi,

On Sat, Jan 07, 2017 at 03:43:10PM +0100, em12345 wrote:
> I'm wondering how to setup a WG client when having multiple DynDNS names
> for a single WG server using dynamic IP. Using multiple DynDNS services
> for the same host is a common pattern to work around unreliable DynDNS
> services.
> As far as I see, only a single endpoint can be given in the configuration.

I am also interested in multiple endpoints support, and I am preparing a
proposal that I will send soon.

However, in your case, it would not really help.  Actually, even using a
single DynDNS name would not work correctly, because DNS resolution is
done only once by `wg`, when you configure your interface.  Afterwards,
the wireguard kernel module uses the IP address that was provided by `wg`,
not the hostname.  This is similar to the behaviour of iptables, and the
reason is simple: it would be really cumbersome to perform DNS resolution
from kernelspace.

So, if a client is connected to the server and the server changes its IP
address, the client will keep trying to use the old IP address forever.
You would need to destroy the wireguard interface on the client and
recreate it, so that `wg` configures the kernel module with the new IP
address associated with the hostname.

On Sat, Jan 07, 2017 at 05:45:35PM +0100, em12345 wrote:
> On a server IP change:
> 
> - the client still sends UDP packages to old server IP, which is useless
> 
> - the server (from its new IP) can send UDP packages to the still
> remembered client IP (because of PersistentKeepalive). But my
> understanding is that stateful firewalls will block UDP packages from
> the new IP until the client has send an UDP to the new server IP.

For a stateful firewall, this will likely be the case.  However, it would
work with a full-cone NAT:

    https://en.wikipedia.org/wiki/Network_address_translation#Full-cone_NAT

That being said, I'm not an expert on NAT, so I don't know which kind of
NAT is the most widespread.

> So in such a scenario the roaming feature wouldn't help.

You're right, in your case, you would need to setup port forwarding on
your client, so that wireguard on your client device can be reached from
any IP address.

Baptiste

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Multiple Endpoints

[Jörg Thalheim]

[-- Attachment #1.1: Type: text/plain, Size: 226 bytes --]

> I am also interested in multiple endpoints support, and I am preparing a
> proposal that I will send soon.
>

Does your proposal also includes support for different address family? (switch between ipv4/ipv6 and back)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 646 bytes --]

Re: Multiple Endpoints

[Baptiste Jonglez]

[-- Attachment #1: Type: text/plain, Size: 332 bytes --]

On Sun, Jan 08, 2017 at 03:39:07PM +0100, Jörg Thalheim wrote:
> > I am also interested in multiple endpoints support, and I am preparing a
> > proposal that I will send soon.
> >
> 
> Does your proposal also includes support for different address family? (switch between ipv4/ipv6 and back)

Yes, this is my main use case.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Multiple Endpoints

[Jason A. Donenfeld]

On Sat, Jan 7, 2017 at 5:45 PM, em12345 <em12345@web.de> wrote:
> This would require PersistentKeepalive on "server" side. But assuming
> the common case that the client sits behind a stateful firewall, how
> would the server be able to inform the client about its IP change?

Yes, the server would need the PersistentKeepalive; you're right.

> - the server (from its new IP) can send UDP packages to the still
> remembered client IP (because of PersistentKeepalive). But my
> understanding is that stateful firewalls will block UDP packages from
> the new IP until the client has send an UDP to the new server IP.

No, usually not. In most cases, the NAT mapping depends on the
client's local IP and sport/dport, but not on the remote dst IP.
Otherwise common NAT holepunching schemes like STUN and the example
holepuncher [1] wouldn't work. The new UDP packets will make it to the
client, in fact.

[1] https://git.zx2c4.com/WireGuard/tree/contrib/examples/nat-hole-punching/README

Re: Multiple Endpoints

[Jason A. Donenfeld]

On Sun, Jan 8, 2017 at 3:12 PM, Baptiste Jonglez
<baptiste@bitsofnetworks.org> wrote:
> I am also interested in multiple endpoints support, and I am preparing a
> proposal that I will send soon.

Cool. Any details? Probably best to discuss it casually before putting
too much work into it.
Have you read that Mosh multipath paper? I just ran into this the
other night and put it on my reading list. If so, is it any good or
relavent to this?

> So, if a client is connected to the server and the server changes its IP
> address, the client will keep trying to use the old IP address forever.

No. If the server sends a packet to the client using the same UDP
src/dst, then it will make it to the client, and the client will learn
the new server IP.

> You would need to destroy the wireguard interface on the client and
> recreate it, so that `wg` configures the kernel module with the new IP
> address associated with the hostname.

No. And even in the worst possible case, no destruction of the wg
interface would be necessary. wg(8) can reconfigure all attributes on
the fly.

> You're right, in your case, you would need to setup port forwarding on
> your client, so that wireguard on your client device can be reached from
> any IP address.

No. In the vast majority of cases I've seen, both stateful firewalls
and NAT do not do the mapping based on the remote IP.

Re: Multiple Endpoints

[Jason A. Donenfeld]

On Sun, Jan 8, 2017 at 3:39 PM, J=C3=B6rg Thalheim <joerg@higgsboson.tk> wr=
ote:
>> I am also interested in multiple endpoints support, and I am preparing a
>> proposal that I will send soon.
>>
>
> Does your proposal also includes support for different address family? (s=
witch between ipv4/ipv6 and back)

So presumably what's needed is someway to specify multiple endpoints
per peer, and have the handshake retry mechanism cycle through the
list of endpoints, reordering them based on success. This doesn't seem
very difficult or complicated. I'll read some literature to see if
there have been some nice ideas on this in the past (multipath mosh
etc) and also examine whatever it is Baptiste has been cooking up.

Re: Multiple Endpoints

[Baptiste Jonglez]

[-- Attachment #1: Type: text/plain, Size: 1457 bytes --]

Hi Jason,

On Sun, Jan 08, 2017 at 11:18:01PM +0100, Jason A. Donenfeld wrote:
> > So, if a client is connected to the server and the server changes its IP
> > address, the client will keep trying to use the old IP address forever.
> 
> No. If the server sends a packet to the client using the same UDP
> src/dst, then it will make it to the client, and the client will learn
> the new server IP.
> 
> > You would need to destroy the wireguard interface on the client and
> > recreate it, so that `wg` configures the kernel module with the new IP
> > address associated with the hostname.
> 
> No. And even in the worst possible case, no destruction of the wg
> interface would be necessary. wg(8) can reconfigure all attributes on
> the fly.
> 
> > You're right, in your case, you would need to setup port forwarding on
> > your client, so that wireguard on your client device can be reached from
> > any IP address.
> 
> No. In the vast majority of cases I've seen, both stateful firewalls
> and NAT do not do the mapping based on the remote IP.

Please read Emmanuel's email more carefuly before being so assertive.
His use-case was a client behind a stateful firewall, so if the server
changes its IP address, roaming will not work.

I merely pointed out that a stateful firewall is similar to a symmetric
NAT, that is, both would cause issue with peer roaming.  But a full-cone
NAT would be fine, as you also mentioned.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Multiple Endpoints

[Jason A. Donenfeld]

On Sun, Jan 8, 2017 at 11:57 PM, Baptiste Jonglez
<baptiste@bitsofnetworks.org> wrote:
> Please read Emmanuel's email more carefuly before being so assertive.
> His use-case was a client behind a stateful firewall, so if the server
> changes its IP address, roaming will not work.

I understand that use-case. And as I said before, most UDP stateful
firewalls allow changing remote IPs, so long as the dport/sport remain
the same.

> I merely pointed out that a stateful firewall is similar to a symmetric
> NAT, that is, both would cause issue with peer roaming.

Are you sure about this for UDP? I did a bunch of tests several months
ago, and was able to punch holes in a variety of stateful firewalls
with changing remote IPs.

Varying source address and stateful firewalls (Was: Multiple Endpoints)

[Baptiste Jonglez]

[-- Attachment #1: Type: text/plain, Size: 2185 bytes --]

On Mon, Jan 09, 2017 at 12:00:17AM +0100, Jason A. Donenfeld wrote:
> > I merely pointed out that a stateful firewall is similar to a symmetric
> > NAT, that is, both would cause issue with peer roaming.
> 
> Are you sure about this for UDP? I did a bunch of tests several months
> ago, and was able to punch holes in a variety of stateful firewalls
> with changing remote IPs.

I must admit I had never tested :)

I just did, though, and yes, the stateful firewall from Linux does block
UDP traffic from unrelated source IP addresses.  So I guess your hole
punching was based on some other property.

Here is the setup with 3 computers A, B, C.  There is a stateful firewall
on A, and A opens a UDP connection towards B.  C then tries to pretend to
be B and contacts A with the same src/dest port.

    A# iptables -F INPUT
    A# iptables -A INPUT -p udp -m conntrack --ctstate ESTABLISHED -j LOG --log-prefix="established: "
    A# iptables -A INPUT -p udp -m conntrack --ctstate RELATED -j LOG --log-prefix="related: "
    A# iptables -A INPUT -p udp -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
    A# iptables -A INPUT -p udp -j LOG --log-prefix="drop: "
    A# iptables -A INPUT -p udp -j DROP

A and B communicate normally:

    B# nc -l -u -p 5001
    A# nc -u -p 60000 $IP_B 5001
    A# #type something
    B# #type something else

At this point, there is an entry in the conntrack table of A:

    A# conntrack -L | grep $IP_B
    udp      17 22 src=$IP_A dst=$IP_B sport=60000 dport=5001 src=$IP_B dst=$IP_A sport=5001 dport=60000 mark=0 use=1

Also, the packet from B to A has been logged by our firewall rules:

    kernel: established: IN=wlan0 OUT= SRC=$IP_B DST=$IP_A LEN=33 TOS=0x00 PREC=0x00 TTL=62 ID=43432 DF PROTO=UDP SPT=5001 DPT=60000 LEN=13

Now C tries to chime in, contacting A and pretending to be B:

    C# nc -u -p 5001 $IP_A 60000

The result:

    kernel: drop: IN=wlan0 OUT= SRC=$IP_C DST=$IP_A LEN=34 TOS=0x00 PREC=0x00 TTL=64 ID=43124 DF PROTO=UDP SPT=5001 DPT=60000 LEN=14

So, the packet from C is dropped, even though it has the same source port
and destination port as the ones from B.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Varying source address and stateful firewalls (Was: Multiple Endpoints)

[Jason A. Donenfeld]

Hey,

Thanks for the nice analysis. At first I was incredulous about the
results, but then I sat down and drew some pictures, and figured out
where the disconnect is. With hole punching, you have each peer
discovering the remote endpoint tuple, and sending an outgoing packet,
which then adjusts the stateful firewall. With em's example, there
isn't this luxury. So, I'll circle back the original thread, and
backtrack on my assertions in order to get back on track. Thanks for
investigating and showing where I erred.

Jason

Re: Multiple Endpoints

[Jason A. Donenfeld]

Hey Em,

Circling back to this thread, indeed Baptiste was right. While the
existing situation will likely work fine with NAT, it won't work as
nicely with a strict stateful firewall, when the server IP is dynamic
and changing. So, there's some headway to be made with Baptiste's
multi-endpoint idea. But in the interim, it seems likely the best
mechanism to use would be some userspace solution to check
periodically if the dynamic IP has changed, and to update the
wireguard device accordingly. Hopefully I'll implement some kind of
userspace notification logic so this can be more fine grained than a
cronjob. I'll let you know how this progresses.

Jason