Re: passing-through TOS/DSCP marking

[Daniel Golle <daniel@makrotopia.org>]

On Fri, Jun 18, 2021 at 02:24:29PM +0200, Jason A. Donenfeld wrote:
> Hey Toke,
> 
> On Fri, Jun 18, 2021 at 1:05 AM Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> > > I think you can achieve something similar using BPF filters, by relying
> > > on wireguard passing through the skb->hash value when encrypting.
> > >
> > > Simply attach a TC-BPF filter to the wireguard netdev, pull out the DSCP
> > > value and store it in a map keyed on skb->hash. Then, run a second BPF
> > > filter on the physical interface that shares that same map, lookup the
> > > DSCP value based on the skb->hash value, and rewrite the outer IP
> > > header.
> > >
> > > The read-side filter will need to use bpf_get_hash_recalc() to make sure
> > > the hash is calculated before the packet gets handed to wireguard, and
> > > it'll be subject to hash collisions, but I think it should generally
> > > work fairly well (for anything that's flow-based of course). And it can
> > > be done without patching wireguard itself :)
> >
> > Just for fun I implemented such a pair of eBPF filters, and tested that
> > it does indeed work for preserving DSCP marks on a Wireguard tunnel. The
> > PoC is here:
> >
> > https://github.com/xdp-project/bpf-examples/tree/master/preserve-dscp
> >
> > To try it out (you'll need a recent-ish kernel and clang version) run:
> >
> > git clone --recurse-submodules https://github.com/xdp-project/bpf-examples
> > cd bpf-examples/preserve-dscp
> > make
> > ./preserve-dscp wg0 eth0
> >
> > (assuming wg0 and eth0 are the wireguard and physical interfaces in
> > question, respectively).
> >
> > To actually deploy this it would probably need a few tweaks; in
> > particular the second filter that rewrites packets should probably check
> > that the packets are actually part of the Wireguard tunnel in question
> > (by parsing the UDP header and checking the source port) before writing
> > anything to the packet.
> >
> > -Toke
> 
> That is a super cool approach. Thanks for writing that! Sounds like a
> good approach, and one pretty easy to deploy, without the need to
> patch kernels and such.
> 
> Also, nice usage of BPF_MAP_TYPE_LRU_HASH for this.
> 
> Daniel -- can you let the list know if this works for your use case?

Turns out not exactly easy to deploy (on OpenWrt), as it depends on an
extremely recent environment. I will try pushing to that direction, but
it doesn't look like it's going to be ready very soon.

In terms of toolchain: LLVM/Clang is a very bulky beast, I gave up on
that and started working on integrating GCC-10's BPF target in our build
system...

In terms of kernel support: recent kernels don't build yet because of
gelf_getsymshndx, so we got to update libelf first for that. Recent
libelf doesn't seem to be an option yet on many of the build hosts we
currently support (Darwin and such).

In terms of library support: our build of libbpf comes from Linux
release tarballs. There isn't yet a release supporting bpf_tc_attach,
the easiest would be to wait for Linux 5.13 to be released.

I (of course ;) also tried and spend almost a day looking for a
quick-and-dirty path for temporary deployment, so I could at least give
feedback -- bpf-examples also isn't exactly made to be cross-compiled
manually, so I have failed with that as well so far.


In general I still believe this is a great approach and I would really
like to use it as soon as possible, even just to give you feedback.


Cheers


Daniel