Development discussion of WireGuard
 help / color / mirror / Atom feed
* Wireguard wg-quick defaults conflict with Kubernetes firewall
@ 2020-09-27 16:22 Sebastian Rose
  0 siblings, 0 replies; only message in thread
From: Sebastian Rose @ 2020-09-27 16:22 UTC (permalink / raw)
  To: wireguard

Hello,

wg-quick in its default configuration causes routing conflicts when the same host is also running a kubernetes
master node. The issue seems to be how wg-quick marks the traffic to route to the Wireguard peer: 
https://www.wireguard.com/netns/#routing-all-your-traffic
This leads to a loss of network connectivity when Kubernetes and Wireguard are running simultaneously on the
same host.

wg-quick (when not instructed otherwise) generates an wg0 interface like this:

> interface: wg0
>   public key: [REMOVED]
>   private key: (hidden)
>   listening port: 37827
>   fwmark: 0xca6c
> 
> peer: [REMOVED]
>   endpoint: [IPV4]:51820
>   allowed ips: 0.0.0.0/0, ::/0
>   latest handshake: 1 minute, 21 seconds ago
>   transfer: 161.54 KiB received, 185.11 KiB sent

Additionally, the following routing rule gets created
> rule 32765:	not from all fwmark 0xca6c lookup 51820

> # ip route show table 51820
> default dev wg0 scope link

This got created with the following configuration:

> # wg showconf wg0
> [Interface]
> ListenPort = 36768
> FwMark = 0xca6c
> PrivateKey = [REMOVED]
> 
> [Peer]
> PublicKey = [REMOVED]
> AllowedIPs = 0.0.0.0/0, ::/0
> Endpoint = [IPV4]:51820

As part of a kubernetes installation, the following firewall rules get added:

> # iptables -L KUBE-FIREWALL
> Chain KUBE-FIREWALL (2 references)
> target     prot opt source               destination         
> DROP       all  -- !127.0.0.0/8          127.0.0.0/8          /* block incoming localnet connections */ ! ctstate RELATED,ESTABLISHED,DNAT
> DROP       all  --  anywhere             anywhere             /* kubernetes firewall for dropping marked packets */ mark match 0x8000/0x8000

As far as I can tell, Wireguard marks its traffic with 0xca6c and Kubernetes wants to drop traffic that's
marked 0x8000. Since 0xca6c ∧ 0x8000 = 0x8000 all Wireguard traffic gets dropped and there is no network
connectivity.

Proposed solution:

Change FwMark in the [Interface] section of the wg0.conf to something like

> FwMark = 0x4a6c

This will cause the kubernetes firewall not to conflict anymore with the defaults of wg-quick.

Best,
Sebastian


^ permalink raw reply	[flat|nested] only message in thread

only message in thread, other threads:[~2020-10-01 11:15 UTC | newest]

Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-09-27 16:22 Wireguard wg-quick defaults conflict with Kubernetes firewall Sebastian Rose

Development discussion of WireGuard

This inbox may be cloned and mirrored by anyone:

	git clone --mirror http://inbox.vuxu.org/wireguard

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V1 wireguard wireguard/ http://inbox.vuxu.org/wireguard \
		wireguard@lists.zx2c4.com
	public-inbox-index wireguard

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.vuxu.org/vuxu.archive.wireguard


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git