Early Feedback on Container Networking, Resilience, Json Config and AcceptedIPs

Early Feedback on Container Networking, Resilience, Json Config and AcceptedIPs

[raul]

[-- Attachment #1: Type: text/plain, Size: 2900 bytes --]

Hi Jason,

First, this is a stunning piece of work.

I am working on a container networking project with vxlan and peervpn at L2
and quagga at L3 with a smattering of nearly all Linux networking tunnels
and testing Wireguard has been a revelation.

Vxlan is not useful without multicast. BGP may be overkill for simple route
management, most other tunnels require intensive management at scale.
Peervpn is nice and has autodiscovery of nodes but like most current
encrypted tunnels has a performance penalty. Ipsec doesn't but adds config
complexity. Wireguard brings something fresh to the table.

My early Wireguard Iperf tests show near line speed, 942Mbps without and
870-890Mbps with Wireguard. Around 120Mbps for Peervpn. CPU usage is at
around 100% on one core for Peervpn and around 50-60% across 4 cores for
Wireguard for sending and 35-40% receiving . These tests are on low power
1.5Ghz quad core Apollo lake Celeron J3455 clusters on Ubuntu Xenial.

You have been able to hide a tremendous amount of complexity to expose a
simple, flexible but powerful front end and I am certain this must have
taken extraordinary talent, experience, time and engineering.

I have been running a number of scenarios mainly around container
networking and service discovery across multiple clouds and systems and am
glad to report its is much more robust than I expected and just works.

There are 4 things I wanted to bring up.

1. Do you anticipate a situation where differing Wireguard versions across
distributions and kernels do not inter-operate? This introduces a huge
amount of complexity to managing clusters. I faced this with Gluster and it
is a significant pain and blocker.

2. Would you consider supporting a json configuration file? The current
Wireguard ini format has duplicate entires for 'Peers' and the python ini
parser for instance does not support duplicate section heads.

3. While the allowed IPs construct is powerful it can sometimes feel a bit
unintuitive. The way I understand it the 'AllowedIP config on the Wireguard
server is for shared networks from the client and on the client for
accepted networks. This is so the server knows where to route (and thus
there cannot be duplicate IPs, subnets or any subnet overlapping in the
server config) and the client can only receive traffic from acceptedips
recorded on the client side config.

My mindset may be a bit too container orientated with hosts sharing
internal container networks across systems at the moment, and Wireguard has
a much broader use case but the share/accept construct seems slightly more
intuitive though I could be wrong.

4. The Wireguard server is a single point of failure in a star topology. If
the server host goes down your network goes down with it. How can we add
more resilience in a simple way? A backup server in L2 with identical keys
and a floating internal IP?

Thanks for Wireguard!

Raul

[-- Attachment #2: Type: text/html, Size: 3261 bytes --]

Re: Early Feedback on Container Networking, Resilience, Json Config and AcceptedIPs

[Jonathon Fernyhough]

[-- Attachment #1.1: Type: text/plain, Size: 620 bytes --]

On 08/07/17 03:26, raul wrote:
> 4. The Wireguard server is a single point of failure in a star topology.
> If the server host goes down your network goes down with it. How can we
> add more resilience in a simple way? A backup server in L2 with
> identical keys and a floating internal IP?

I have deployed WireGuard (albeit to a limited number of servers) as
essentially a mesh - each server knows each of the others' endpoint
address and allowed IP.

Taking the most simplistic approach, I'd assume you could do something
similar for a server "cluster", e.g. deploy clients with several known
servers.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Early Feedback on Container Networking, Resilience, Json Config and AcceptedIPs

[Jason A. Donenfeld]

Hi Raul,


On Sat, Jul 08, 2017 at 07:56:28AM +0530, raul wrote:
> First, this is a stunning piece of work.
> You have been able to hide a tremendous amount of complexity to expose a
> simple, flexible but powerful front end and I am certain this must have
> taken extraordinary talent, experience, time and engineering.

Glad to hear you like it!

> 1. Do you anticipate a situation where differing Wireguard versions across
> distributions and kernels do not inter-operate? This introduces a huge
> amount of complexity to managing clusters. I faced this with Gluster and it
> is a significant pain and blocker.

Once WireGuard hits 1.0, the protocol will be stable and won't change.
Until then, it might change, but such changes have been (will be?)
extremely rare. It's certainly something I'm trying to avoid, and I
don't have any planned changes, so it very well could be that what we
have now is what we have at 1.0 time. But I don't want to make promises
until we're at 1.0, just in case.

So: things won't be too big of a pain, and at some point, there won't be
any possibility of pains.

> 2. Would you consider supporting a json configuration file? The current
> Wireguard ini format has duplicate entires for 'Peers' and the python ini
> parser for instance does not support duplicate section heads.

I wrote this toy example code:
https://git.zx2c4.com/WireGuard/tree/contrib/examples/json

The WireGuard config format is actually much simpler than "full INI",
and you should probably be able to generate the format with just a
simple printf loop.

What precisely are you doing that you think might be easier with JSON?

> 3. While the allowed IPs construct is powerful it can sometimes feel a bit
> unintuitive. 

When a packet hits the WireGuard interface for outgoing, AllowedIPs
determines for which peer the packet will be encrypted and sent by
examining the destination address. In this sense, on outgoing, it's sort
of like a routing table.

When a packet arrives and is authenticated and decrypted, the decrypted
inner-packet's source address is examined, and the packet is only
allowed if this source address matches an AllowedIP for the
authenticated peer. In this sense, on incoming, it's sort of like an
IP access control list.

> 4. The Wireguard server is a single point of failure in a star topology. If
> the server host goes down your network goes down with it. How can we add
> more resilience in a simple way? A backup server in L2 with identical keys
> and a floating internal IP?

You don't have to run WireGuard in a star topology. You can do full mesh
if you want, or whatever other topology. One interface can have multiple
peers, so you can connect things together any which way you like.

Hope this clarifies things!

Jason

Re: Early Feedback on Container Networking, Resilience, Json Config and AcceptedIPs

[raul]

[-- Attachment #1: Type: text/plain, Size: 1600 bytes --]

Hi Jason,


So: things won't be too big of a pain, and at some point, there won't be
> any possibility of pains.
>


Great to hear that!


What precisely are you doing that you think might be easier with JSON?



I did look at wg-json. I was wondering more about /etc/wireguard/*.conf and
the possibility of using json config there. I am trying to parse wgX.conf
so that we can quickly add and remove peers and subnets. Currently I am
dumping it into a python dict keyed with the pubkeys


In this sense, on outgoing, it's sort of like a routing table. on incoming,
> it's sort of like an IP access control list.



 That's a pretty succinct way of putting it. It does sounds simple put that
way.


You don't have to run WireGuard in a star topology. You can do full mesh
> if you want, or whatever other topology. One interface can have multiple
> peers, so you can connect things together any which way you like.
>


As Jonathan mentioned he is running a mesh. And it does open up
possibilities in terms of access control that I haven't fully considered.
But how do we scale a mesh? For a number of hosts lets say 20+ with 20
container subnets or more to share, one would imagine managing a peer to
peer configuration as the network scales up and down can become a chore.

A client server with let's say a /16 shared may be more feasible, as then
all the client's acceptedIPs can be the single /16 subnet, while the
individual client subnets are added to the server for routing as they are
added. I will think about this some more.

Once again this is really impressive and valuable.

Thanks!
Raul

[-- Attachment #2: Type: text/html, Size: 2596 bytes --]