[RFC] Replace WireGuard AllowedIPs with IP route attribute

[RFC] Replace WireGuard AllowedIPs with IP route attribute

[Daniel Gröber]

Hi wireguard, birds, and babelers,

tl;dr I want to add a new Linux route attribute (think "via $wgpeer") to
supplement wireguard's internal AllowedIPs logic for both routing and
source address filtering.

I've been pondering how to better integrate wireguard into dynamic routing
daemons, particularly BIRD and babeld. Essentially we want to be able to
dynamically add/remove AllowedIPs depending on current reachability and/or
link quality stats.

Looking at the wg netlink API I see two major efficiency/scalability
problems: 1) there is no way to be notified of changes in AllowedIPs made
by other processes meaning we have to do periodic scans and 2) a peer's
AllowedIPs set can only be replaced wholesale, not modified
incrementally. This is problematic as "someone" might, in the worst case,
want to install an entire internet routing table's worth of AllowedIPs and
the set will likely change frequently. FYI: The IPv4 table has ~1M entries
at present, yikes.

Assuming external AllowedIPs changes are infrequent occationally dumping
them all to keep a consistent view of the state shouldn't be too much of an
issue as long as the netlink interface is performant enoug, so I'm going to
concentrate on the add/remove API for now.

Instead of doing the obvious thing and adding a more efficient incremental
AllowedIPs netlink interface I figure why not just add a route attribute to
select a target wg peer on a device. That way we could not only save memory
(no separate AllowedIPs trie) but also simplify routing daemon
implementation considerably.

This would mirror how on ethernet you can have `dev eth0 via $router_ip`.
I'm still reviewing the net/ code to find the best way to do this, but I'm
thinking either a new RTA_WGPEER, like: `default dev wg0 via-wgpeer
$peer_pubkey` or perhaps re-using RTA_VIA and keying off a statically
configured AllowedIP addresses.

To start I'd make this an opt-in replacement for our usual AllowedIPs
logic, making sure to only activate it if any via* RTAs are active on a
particular device, but if it proves to work well I don't see why we
couldn't adapt the netlink code to maintain AllowedIPs using this RTA (but
invisible to userspace) to re-use the same code and get rid of allowedips.c
altogether. That's assuming this ends up being less code overall or perhaps
more performant.

Happy to hear your thoughts,
--Daniel

Re: [RFC] Replace WireGuard AllowedIPs with IP route attribute

[Daniel Gröber]

Hi Bernd,

On Sat, Aug 19, 2023 at 07:50:38PM +0200, Bernd Naumann wrote:
> Chances are high I do miss something, but I've just set AllowedIPs to
> 0.0.0.0/0 and ::/0 and just used the routing protocol of my choice and
> filters to select which routes got exported and imported... :shrug:

Right, let me expand a bit. You are absolutely right, right now if you want
to use wg with dynamic routing daemons you essentially have to have one wg
tunnel per remote node with AllowedIPs=::/0 and that works just fine at
small scales.

The idea here is that we would like to go back to having just one tunnel
for all nodes involved in this particular network instead, due to general
operations scalability, mine is a mesh network so the number of tunnels
gets rather large quickly :)

Lots of tunnels suck for various reasons, monitoring if they're all up and
configured properly is one example but my understanding from previous
discussions is the performance is probably not ideal either.

--Daniel

Re: [Babel-users] [RFC] Replace WireGuard AllowedIPs with IP route attribute

[Steffen Vogel]

Hi Daniel,

Interesting ideas! I am wondering if this complexity is really necessary?
How many routes do you have per peer? In my personal setup I have maximum of 1-100 routes per peer which I can handle with the current API quite comfortably.

My biggest concern about the introduction of a route attribute is that this adds complexity for users.
WireGuard's simplicity (and portability) have been important factors for its success.

A route attribute would introduce another source for the crypto-routing peer selection process.
What happens if the two mechanisms select different peers? Which one would have precedence?
Similarly also for incoming packets. WireGuard's current principle is really easy to understand. If the source address in in the peers AllowedIP list, we will accept the packet. If not its discarded. This is a central part of WireGuard's crypto-key routing feature which would become more complex.

Also implementation wise I would have doubts: Should WireGuard itself perform route lookups to determine which packets will be accepted? Or does WireGuard needs to synchronize the kernel routing table with its internal data structures itself?

A second concern I have with the use of route attributes is limited portability. Not all platforms support them.
How do we handle WireGuard userspace implementations?

I've tackled this problem in a userspace daemon. The synchronisation of a kernel routing table with a WireGuard AllowedIPs settings can be done by cunicu's route synchronization feature: https://cunicu.li/docs/features/rtsync

The route synchronization feature keeps the kernel routing table in sync with WireGuard's AllowedIPs setting.
This synchronization is bi-directional:

- Networks which are found in a Peers AllowedIP list will be installed as a kernel route
- Kernel routes with the peers unique IP address as next-hop will be added to the Peers AllowedIPs list.

This rather simple feature allows user to pair cunicu with a software routing daemon like Bird2 while using a single WireGuard interface with multiple peer-to-peer links.

I am assigning each WireGuard interface a link-local address which is derived from the peers public key.
I am using the peers link-local address as the next-hop in my routing daemon to differentiate to which Peer the AllowedIP entry must be added.

I am keeping track of the kernel's routing table and AllowedIPs by regularly polling the kernel.
As the route synchronisation is just one of cunicu's features [1], I have a central "watcher" routine in cunicu which observes any modification the the WireGuard interfaces and dispatches events which the individual features then can hook into. These observations are not limited to the AllowedIPs but basically any state of the WireGuard interface. E.g. last handshake time or per/peer traffic counters.

In my setup a periodic synchronization worked fine. But I agree that it would be nice if we could have a Netlink multicast group for subscribing to changes like we also have for other parts of the Linux network stack like routing tables, or link states. This feature was already discussed on the WireGuard mailing list [7]. But unfortunately the patch was never accepted. Maybe we can revisit this patch?

I would also be a big supported of extending the netlink API for supporting incremental updates the AllowedIP lists. The netlink APIs are already different for each platform. So extending it for one platform wouldn't hurt here.

Unfortunately, I have far too many ideas for cunicu and limited time to realize them all. So I've recently moved the whole cunicu project into its own organization at GitHub/Codeberg [6] in attempt to find more contributors.

Best regards,
Steffen (stv0g)

[1] Others planned features are:
- Endpoint discovery via ICE/STUN/TURN
- Peer discovery
- IP-autoconfiguration by deriving link-local addresses from peers public keys

I have a lot more ideas here like integrating my
- Go babel routing implementation [2]
- or Rosenpass PQC key-exchange [3]
- or performing proper path-MTU discovery using DPLPMTUD [4]
- or using hardware tokens, TPMs, secure enclaves to rotate pre-shared keys backed by a hardware source-of-trust [5]

[2] https://github.com/cunicu/go-babel
[3] https://github.com/cunicu/go-rosenpass
[4] https://github.com/cunicu/go-pmtud
[5] https://github.com/cunicu/go-skes
[6] https://codeberg.org/cunicu
[7] https://lists.zx2c4.com/pipermail/wireguard/2021-January/006318.html

On Saturday, August 19, 2023 16:02 CEST, Daniel Gröber <dxld@darkboxed.org> wrote:

> Hi wireguard, birds, and babelers,
> 
> tl;dr I want to add a new Linux route attribute (think "via $wgpeer") to
> supplement wireguard's internal AllowedIPs logic for both routing and
> source address filtering.
> 
> I've been pondering how to better integrate wireguard into dynamic routing
> daemons, particularly BIRD and babeld. Essentially we want to be able to
> dynamically add/remove AllowedIPs depending on current reachability and/or
> link quality stats.
> 
> Looking at the wg netlink API I see two major efficiency/scalability
> problems: 1) there is no way to be notified of changes in AllowedIPs made
> by other processes meaning we have to do periodic scans and 2) a peer's
> AllowedIPs set can only be replaced wholesale, not modified
> incrementally. This is problematic as "someone" might, in the worst case,
> want to install an entire internet routing table's worth of AllowedIPs and
> the set will likely change frequently. FYI: The IPv4 table has ~1M entries
> at present, yikes.
> 
> Assuming external AllowedIPs changes are infrequent occationally dumping
> them all to keep a consistent view of the state shouldn't be too much of an
> issue as long as the netlink interface is performant enoug, so I'm going to
> concentrate on the add/remove API for now.
> 
> Instead of doing the obvious thing and adding a more efficient incremental
> AllowedIPs netlink interface I figure why not just add a route attribute to
> select a target wg peer on a device. That way we could not only save memory
> (no separate AllowedIPs trie) but also simplify routing daemon
> implementation considerably.
> 
> This would mirror how on ethernet you can have `dev eth0 via $router_ip`.
> I'm still reviewing the net/ code to find the best way to do this, but I'm
> thinking either a new RTA_WGPEER, like: `default dev wg0 via-wgpeer
> $peer_pubkey` or perhaps re-using RTA_VIA and keying off a statically
> configured AllowedIP addresses.
> 
> To start I'd make this an opt-in replacement for our usual AllowedIPs
> logic, making sure to only activate it if any via* RTAs are active on a
> particular device, but if it proves to work well I don't see why we
> couldn't adapt the netlink code to maintain AllowedIPs using this RTA (but
> invisible to userspace) to re-use the same code and get rid of allowedips.c
> altogether. That's assuming this ends up being less code overall or perhaps
> more performant.
> 
> Happy to hear your thoughts,
> --Daniel
> 
> _______________________________________________
> Babel-users mailing list
> Babel-users@alioth-lists.debian.net
> https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/babel-users

Re: [RFC] Replace WireGuard AllowedIPs with IP route attribute

[Kyle Rose]

Daniel,

I attempted several times to send the following message to the mailing
list nearly two months ago, but they all landed in moderation for some
reason and were silently dropped after a few days. You are not the
only one who wants this functionality for mesh VPNs. I note the email
you subsequently replied to (by Bernd) also didn't show up on the
list, suggesting I am not the only one whose messages are being
inexplicably jailed.

Kyle

====
I really like the straightforward configurability of Wireguard
out-of-the-box. It was astonishingly easy to configure a mesh to
replace my previous hub-and-spoke OpenVPN setup. Thank you for making
this easy.

That said, I'd like the ability to use Linux's policy routing engine
to allow for more complex packet flows across the VPN that are
currently incompatible with Wireguard's internal packet handling. For
example, let's say I have 4 nodes and want to be able to use each of
the nodes as the default gateway for different types of flows.
Modifying the sender side to respect a route's gateway is
straightforward:

--- a/drivers/net/wireguard/allowe
dips.c
+++ b/drivers/net/wireguard/allowe
dips.c
@@ -6,6 +6,8 @@
 #include "allowedips.h"
 #include "peer.h"

+#include <net/route.h>
+
 enum { MAX_ALLOWEDIPS_BITS = 128 };

 static struct kmem_cache *node_cache;
@@ -356,10 +358,18 @@ int wg_allowedips_read_node(struct
allowedips_node *node, u8 ip[16], u8 *cidr)
 struct wg_peer *wg_allowedips_lookup_dst(stru
ct allowedips *table,
                                         struct sk_buff *skb)
 {
-       if (skb->protocol == htons(ETH_P_IP))
-               return lookup(table->root4, 32, &ip_hdr(skb)->daddr);
-       else if (skb->protocol == htons(ETH_P_IPV6))
-               return lookup(table->root6, 128, &ipv6_hdr(skb)->daddr);
+       struct rtable *rt = skb_rtable(skb);
+       if (rt->rt_uses_gateway) {
+               if (rt->rt_gw_family == AF_INET)
+                       return lookup(table->root4, 32, &rt->rt_gw4);
+               else if (rt->rt_gw_family == AF_INET6)
+                       return lookup(table->root6, 128, &rt->rt_gw6);
+       } else {
+               if (skb->protocol == htons(ETH_P_IP))
+                       return lookup(table->root4, 32, &ip_hdr(skb)->daddr);
+               else if (skb->protocol == htons(ETH_P_IPV6))
+                       return lookup(table->root6, 128, &ipv6_hdr(skb)->daddr);
+       }
        return NULL;
 }

The problem is that reply packets will be dropped via the source
address check for all but the peer with the default route listed in
AllowedIPs. The way the trie code works means a highly-invasive change
would be needed to allow for multiple peers to be associated with a
given prefix: I suspect any further complication (not to mention
possible additional data structure bloat) is undesirable, and anyway I
am looking to bypass most of the complexity created by Wireguard's
parallel packet routing infrastructure and instead leverage the far
more flexible Linux policy routing engine.

At this point, what I'd like to do is be able to skip the source
address check by configuration, instead relying on rp_filter and
firewall rules to reject bogus or unwanted packets. With such a config
knob, AllowedIPs would be used only for selecting the destination peer
based on the packet daddr or the route's gateway. For a mesh like I
described above, I would configure only a gateway IP for each peer in
AllowedIPs and use the policy routing engine for all other packet
routing behavior.

I appreciate that Wireguard works the way it does most likely because
routing is configured differently across the wide variety of devices
it supports (and in some cases may be unavailable to users), so I
don't think the AllowedIPs source address check should be removed by
default; but it would be nice if I could turn it off and rely on other
mechanisms in the kernel that would allow for more flexibility.


On Sat, Aug 19, 2023 at 10:05 AM Daniel Gröber <dxld@darkboxed.org> wrote:
>
> Hi wireguard, birds, and babelers,
>
> tl;dr I want to add a new Linux route attribute (think "via $wgpeer") to
> supplement wireguard's internal AllowedIPs logic for both routing and
> source address filtering.
>
> I've been pondering how to better integrate wireguard into dynamic routing
> daemons, particularly BIRD and babeld. Essentially we want to be able to
> dynamically add/remove AllowedIPs depending on current reachability and/or
> link quality stats.
>
> Looking at the wg netlink API I see two major efficiency/scalability
> problems: 1) there is no way to be notified of changes in AllowedIPs made
> by other processes meaning we have to do periodic scans and 2) a peer's
> AllowedIPs set can only be replaced wholesale, not modified
> incrementally. This is problematic as "someone" might, in the worst case,
> want to install an entire internet routing table's worth of AllowedIPs and
> the set will likely change frequently. FYI: The IPv4 table has ~1M entries
> at present, yikes.
>
> Assuming external AllowedIPs changes are infrequent occationally dumping
> them all to keep a consistent view of the state shouldn't be too much of an
> issue as long as the netlink interface is performant enoug, so I'm going to
> concentrate on the add/remove API for now.
>
> Instead of doing the obvious thing and adding a more efficient incremental
> AllowedIPs netlink interface I figure why not just add a route attribute to
> select a target wg peer on a device. That way we could not only save memory
> (no separate AllowedIPs trie) but also simplify routing daemon
> implementation considerably.
>
> This would mirror how on ethernet you can have `dev eth0 via $router_ip`.
> I'm still reviewing the net/ code to find the best way to do this, but I'm
> thinking either a new RTA_WGPEER, like: `default dev wg0 via-wgpeer
> $peer_pubkey` or perhaps re-using RTA_VIA and keying off a statically
> configured AllowedIP addresses.
>
> To start I'd make this an opt-in replacement for our usual AllowedIPs
> logic, making sure to only activate it if any via* RTAs are active on a
> particular device, but if it proves to work well I don't see why we
> couldn't adapt the netlink code to maintain AllowedIPs using this RTA (but
> invisible to userspace) to re-use the same code and get rid of allowedips.c
> altogether. That's assuming this ends up being less code overall or perhaps
> more performant.
>
> Happy to hear your thoughts,
> --Daniel

Re: [Babel-users] [RFC] Replace WireGuard AllowedIPs with IP route attribute

[Daniel Gröber]

Hi Steffen,

from the questions you ask I fear you've misunderstood my intention to
"replace" AllowedIPs. I'm strictly talking of the _implementation_
(allowedips.c) in Linux particular. The netlink interface would naturally
stay unchanged.

On Sat, Aug 19, 2023 at 10:00:17PM +0200, Steffen Vogel wrote:
> Interesting ideas! I am wondering if this complexity is really necessary?

I think so, right now wg's behaviour just prevents a number of advanced
use-cases which is a pitty.

> My biggest concern about the introduction of a route attribute is that
> this adds complexity for users.  WireGuard's simplicity (and portability)
> have been important factors for its success.

Nothing would change for users that don't use this feature.

> A route attribute would introduce another source for the crypto-routing
> peer selection process.  What happens if the two mechanisms select
> different peers? Which one would have precedence?

If you read my previous mail carefully you'll find I specified how this
would shake out. The rt approach will necessarily have to override
AllowedIPs to be useful.

> Similarly also for incoming packets. WireGuard's current principle is
> really easy to understand. If the source address in in the peers
> AllowedIP list, we will accept the packet. If not its discarded. This is
> a central part of WireGuard's crypto-key routing feature which would
> become more complex.

Having read Kyle's use-case I'm thinking my original plan to extend the wg
internal source-address filtering to use a rt lookup with our new attribute
would not be maximally useful so now my thinking is we should just have a
boolean toggle to disable it explicitly per device.

Then users can do whatever fancy rt (reverse-path) lookups they want with
nft. I'm particularly happy that nft rt lookups will actually do the right
thing with multipath/ECMP routes (any of the involved devices satisfies the
lookup) so this should actually work out fine in my case at least. Mmore
prototyping is required here though.

> A second concern I have with the use of route attributes is limited
> portability. Not all platforms support them.  How do we handle WireGuard
> userspace implementations?

No need. The Linux's rt table is Linux specific I have no plans to
introduce this on other platforms, that's for other intersted souls to
tackle. Again "replace" was referring to implementation concerns.

> - Networks which are found in a Peers AllowedIP list will be installed as a kernel route

That's configurable. I always turn this off when dealing with rt daemons.

> - Kernel routes with the peers unique IP address as next-hop will be added to the Peers AllowedIPs list.
> 
> This rather simple feature allows user to pair cunicu with a software
> routing daemon like Bird2 while using a single WireGuard interface with
> multiple peer-to-peer links.

Sounds like you do what I want to do at kernel level in userspace, then at
least we can agree this is a useful thing :)

> In my setup a periodic synchronization worked fine. But I agree that it
> would be nice if we could have a Netlink multicast group for subscribing
> to changes like we also have for other parts of the Linux network stack
> like routing tables, or link states. This feature was already discussed
> on the WireGuard mailing list [7]. But unfortunately the patch was never
> accepted. Maybe we can revisit this patch?

Sounds like a plan, I'll have a look at it.

> [1] Others planned features are:
> - IP-autoconfiguration by deriving link-local addresses from peers public keys

That's been discussed so many times before on the ML and someone always
realises Jason is right and there's no point to this in the end. Key
distribution is the crux of the problem.

--Daniel

Re: [Babel-users] [RFC] Replace WireGuard AllowedIPs with IP route attribute

[Kyle Rose]

On Sat, Aug 19, 2023 at 5:25 PM Daniel Gröber <dxld@darkboxed.org> wrote:
> Having read Kyle's use-case I'm thinking my original plan to extend the wg
> internal source-address filtering to use a rt lookup with our new attribute
> would not be maximally useful so now my thinking is we should just have a
> boolean toggle to disable it explicitly per device.

If there is interest among the maintainers in eventually merging a
change with a per-interface knob to turn off the source IP check, I
will go through the trouble of putting together an initial pass at
this. I don't want to spend the time if there is firm opposition to
the idea.

Thanks,
Kyle

Re: [Babel-users] [RFC] Replace WireGuard AllowedIPs with IP route attribute

[Daniel Gröber]

Hi Kyle,

On Mon, Aug 28, 2023 at 11:40:48AM -0400, Kyle Rose wrote:
> On Sat, Aug 19, 2023 at 5:25 PM Daniel Gröber <dxld@darkboxed.org> wrote:
> > Having read Kyle's use-case I'm thinking my original plan to extend the wg
> > internal source-address filtering to use a rt lookup with our new attribute
> > would not be maximally useful so now my thinking is we should just have a
> > boolean toggle to disable it explicitly per device.
> 
> If there is interest among the maintainers in eventually merging a
> change with a per-interface knob to turn off the source IP check, I
> will go through the trouble of putting together an initial pass at
> this. I don't want to spend the time if there is firm opposition to
> the idea.

I think just a patch to turn off the wg source IP check is not very useful
at the moment. It would encourage bad source IP filtering practices when
multiple peers are involved as no mechanism for identifying the sending
peer is available at the policy routing or netfilter level currently.

I think such a patch would have to get merged after some kind of mechanism
to identify and filter based on the sending wg peer is available.

So if you want to move this along I would suggest working on this
first. Since I'm also interested in having this feature I'm happy
collaborate.

It's just hard to find the motivation for writing more wg patches when my
pending ones have (mostly) been lying around for a year without a response,
but if you're also keen on this feature I'm sure it's easier to stay
motivated together :)

If my kernel patches go ignored for too long too I'll probably just resort
to getting a forked DKMS wireguard module into Debian with this
work. Perhaps that approach (or a package in a different distro) would work
for your use-case too?

--Daniel

Re: [Babel-users] [RFC] Replace WireGuard AllowedIPs with IP route attribute

[Kyle Rose]

On Mon, Aug 28, 2023 at 1:41 PM Juliusz Chroboczek <jch@irif.fr> wrote:
> I've read the whole discussion, and I'm still not clear what advantages
> the proposed route attribute has over having one interface per peer.  Is
> it because interfaces are expensive in the Linux kernel?  Or is there some
> other reason why it is better to run all WG tunnels over a single interface?

Why manage n^2 tunnels and allocate n^2 /30 CIDRs when you can just
have one tunnel and a single subnet for a full mesh? IMO, the latter
should be a feature differentiating Wireguard from other solutions to
creating a mesh VPN. That is, in fact, the whole reason I dropped
OpenVPN for it.

Kyle

Re: [RFC] Replace WireGuard AllowedIPs with IP route attribute

[Daniel Gröber]

[-- Attachment #1: Type: text/plain, Size: 1727 bytes --]

Hi Juliusz,

On Mon, Aug 28, 2023 at 07:40:51PM +0200, Juliusz Chroboczek wrote:
> I've read the whole discussion, and I'm still not clear what advantages
> the proposed route attribute has over having one interface per peer.  Is
> it because interfaces are expensive in the Linux kernel?  Or is there some
> other reason why it is better to run all WG tunnels over a single interface?

Off the top of my head UDP port exhaustion is a scalability concern here,
just as an example, not that I'd actually ever need that many peers in my
network :)

One wg-device per-peer means we need one UDP port per-peer and since
currently binding to a specific IP is also not supported by wg (I have a
patch pending for this though) there's no good way to work around this.

Frankly having tons of interfaces is just an operational PITA in all sorts
of ways. Apart from the port exhaustion having more than one wg device also
means I have to _allocate_ a new port for each node in my managment system
somehow instead of just using a static port for the entire network. This
gets dicy fast as I want to move in the direction of dynamic peering as in
tinc.

Other than that my `ip -br a` output is getting unmanagably long and having
more than one device means I have to keep ACL lists in sync all over the
system. This is a problem for daemons that don't support automatic reload
(babeld for example :P). I also have to sync the set of interface to
nftables which is easy to get wrong as it's still manual in my setup.

All of that could be solved, but I would also like to get my wg+babel VPN
setup deployed more widely at some point and all that friction isn't going
to help with that so I'd rather have this supported properly.

--Daniel

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [RFC] Replace WireGuard AllowedIPs with IP route attribute

[Ivan Labáth]

Hi,

IMO, a good tunnel solution may be if what is now called AllowedIPs,
were functionally split into:
 - AcceptIPS (to be different from AllowedIPs)
 - RouteIPs
Perhaps with a default shorthand of, say, IPs, setting both, as
AllowedIPs historically caused confusion wrt. it's meaning.

Wireguard API is a bit clunky, but I think one could dynamically manage
routes in reasonably efficient ways without extra interfaces and layers.

Not sure if it would fullfill all reasonably achievable goals.

Don't really have the time to implement anything, and I'm sure it would
not be easy, so just a possible tip to consider.

Regards,
Ivan Labáth

On Tue, Aug 29, 2023 at 12:13:12AM +0200, Daniel Gröber wrote:
> Hi Juliusz,
> 
> On Mon, Aug 28, 2023 at 07:40:51PM +0200, Juliusz Chroboczek wrote:
> > I've read the whole discussion, and I'm still not clear what advantages
> > the proposed route attribute has over having one interface per peer.  Is
> > it because interfaces are expensive in the Linux kernel?  Or is there some
> > other reason why it is better to run all WG tunnels over a single interface?
> 
> Off the top of my head UDP port exhaustion is a scalability concern here,
> just as an example, not that I'd actually ever need that many peers in my
> network :)
> 
> One wg-device per-peer means we need one UDP port per-peer and since
> currently binding to a specific IP is also not supported by wg (I have a
> patch pending for this though) there's no good way to work around this.
> 
> Frankly having tons of interfaces is just an operational PITA in all sorts
> of ways. Apart from the port exhaustion having more than one wg device also
> means I have to _allocate_ a new port for each node in my managment system
> somehow instead of just using a static port for the entire network. This
> gets dicy fast as I want to move in the direction of dynamic peering as in
> tinc.
> 
> Other than that my `ip -br a` output is getting unmanagably long and having
> more than one device means I have to keep ACL lists in sync all over the
> system. This is a problem for daemons that don't support automatic reload
> (babeld for example :P). I also have to sync the set of interface to
> nftables which is easy to get wrong as it's still manual in my setup.
> 
> All of that could be solved, but I would also like to get my wg+babel VPN
> setup deployed more widely at some point and all that friction isn't going
> to help with that so I'd rather have this supported properly.
> 
> --Daniel

Re: [RFC] Replace WireGuard AllowedIPs with IP route attribute

[Daniel Gröber]

Hi Ivan,

> IMO, a good tunnel solution may be if what is now called AllowedIPs,
> were functionally split into:
>  - AcceptIPS (to be different from AllowedIPs)
>  - RouteIPs
> Perhaps with a default shorthand of, say, IPs, setting both, as
> AllowedIPs historically caused confusion wrt. it's meaning.

That would be one way to paint the shed, yes.

This alone doesn't really address the crux of the problem though:
scalability.

> Wireguard API is a bit clunky, but I think one could dynamically manage
> routes in reasonably efficient ways without extra interfaces and layers.

The entire idea with the new route attribute is to put this functionality
into the right (pre-existing) layer and not invent a new way of expressing
this. We even get scalability for free. Win-Win.

--Daniel

PS: Your mail didn't reach my inbox for some reason, I randomly found it
while looking at the wg list archives. Consider configuring your mail
client to To/CC people you're replying to in order to better handle flaky
list servers.