Development discussion of WireGuard
 help / color / mirror / Atom feed
From: "Toke Høiland-Jørgensen" <toke@toke.dk>
To: Alex <wireguard@centromere.net>, wireguard@lists.zx2c4.com
Subject: Re: eBPF + IPv6 + WireGuard
Date: Thu, 23 Dec 2021 16:25:20 +0100	[thread overview]
Message-ID: <877dbvbaxr.fsf@toke.dk> (raw)
In-Reply-To: <20211217190659.251f006a@poseidon.quill.lan>

Alex <wireguard@centromere.net> writes:

> Hi all,
>
> I am championing WireGuard at work, and I have been granted permission
> to use it for establishing remote access to a private IPv6 VLAN for all
> employees. I have experimented with different approaches and ran in to
> an issue.
>
> With the help of a machine dedicated fully to the job of remote access
> (Ubuntu 20.04 / Linux 5.4), I've managed to establish end-to-end
> connectivity between my work laptop and the servers. The VPN gateway
> has a wg0 interface for employees and a "private0" interface for
> private VLAN connectivity. The goal is to link the two.
>
> I noticed an odd quirk: It only works when
> "net.ipv6.conf.all.forwarding" is 1. If this value is 0,
> "net.ipv6.conf.[wg0 | private0].forwarding" has no effect. In other
> words, I cannot seem to enable forwarding for *only* the interfaces
> that need it. It only works when forwarding is enabled for *all*
> interfaces. This is a problem because when the "all" value is set to 1,
> the machine will start to behave as a router on VLANs for which it is
> most definitely not a router.
>
> For the servers, I am using the officially defined[0] subnet-router
> anycast address as the default IPv6 gateway, and it works well.
> However, when I flipped the switch on "net.ipv6.conf.all.forwarding",
> the VPN gateway started using NDP to announce that it was a router
> across *all* VLANs! Specifically, it was claiming ownership over our
> global subnet anycast address 2001:aaaa:bbbb:cccc::. This is neither
> true nor desired. The VPN gateway started to receive outbound non-VPN
> Internet traffic which broke Internet connectivity for all servers. 
>
> This leads me to my first question: Why does
> "net.ipv6.conf.wg0.forwarding" have no effect?

That's how forwarding works in the kernel for IPv6:

https://elixir.bootlin.com/linux/latest/source/Documentation/networking/ip-sysctl.rst#L1981

I.e., there *is* no per-interface forwarding setting, you'll need to
setup netfilter rules or some other mechanism to control which
interfaces packets will be forwarded to/from. The per-interface
"forwarding" attribute only controls things like whether the kernel will
accept router advertisements on that interface. Yes, this is a bit
confusing (and also different from IPv4).

The isRouter flag in neighbour advertisements should be controllable via
the per-interface 'forwarding' setting; the default changes when you
enable the global setting, but I think it should be possible to re-set
it? See:

https://elixir.bootlin.com/linux/latest/source/Documentation/networking/ip-sysctl.rst#L2148

> In researching a solution, I decided to give eBPF a try. I attached an
> XDP program to "private0" (a layer 2 device, naturally) and
> successfully redirected packets to "wg0" (a layer 3 device) with
> bpf_redirect[1]. If the packets are unmodified, WireGuard will happily
> pass them to the remote hosts. This is an issue because the remote
> hosts are expecting to receive IP(v6) packets, not Ethernet frames. If
> I use bpf_xdp_adjust_head[1] to strip off the Ethernet frame, WireGuard
> will drop the packet[2] before it can be sent to the remote host.
>
> My theory is that bpf_xdp_adjust_head is modifying the data pointer
> only and not any underlying structures that may be associated with the
> packet (sk_buff perhaps).
>
> This leads me to my second question: Why can't I redirect traffic
> received on an L2 interface to an L3 interface, *even after stripping
> off the Ethernet frame?*

I assume you were using generic XDP here? Anyway, XDP is layer2-only by
definition, so there's not really any way to achieve what you're trying
to do with XDP.

> Finally, during this whole process I was using WireShark to inspect
> traffic received by the remote host (i.e. my work laptop). With the
> help of the extract-handshakes.sh script, I was able to decrypt traffic.
> I did discover a bug though. I believe "index_hashtable_insert" should
> actually be "wg_index_hashtable_insert"[3].
>
> Does anyone have any insights as to what a proper solution would look
> like? Is there a way to achieve my goal without introducing eBPF? Is
> XDP completely unsuited for this particular purpose? Do I actually need
> to operate on the sk_buff level, as opposed to the xdp_buff level?

You could maybe do something with TC-BPF, but why can't you just enable
net.ipv6.conf.all.forwarding and just use netfilter to control which
interfaces packets will be forwarded across? That seems like it would be
the simplest solution...

-Toke

      parent reply	other threads:[~2021-12-23 15:25 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-12-18  0:06 Alex
2021-12-20 21:49 ` Mike O'Connor
2021-12-23 15:25 ` Toke Høiland-Jørgensen [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=877dbvbaxr.fsf@toke.dk \
    --to=toke@toke.dk \
    --cc=wireguard@centromere.net \
    --cc=wireguard@lists.zx2c4.com \
    --subject='Re: eBPF + IPv6 + WireGuard' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).