Thoughts on wg-dynamic

Thoughts on wg-dynamic

[Reid Rankin]

I've been thinking about wg-dynamic -- or something quite like it -- a
lot in recent months, and I've had some ideas I think are worth
sharing even though I haven't figured out an ideal final form for them
yet.

Just to let you know where I'm coming from (and to add to the
community gestalt of potential usecases), I've been working on a
stateless WG-based mesh networking system where each node has only
read-only storage. In my application the network is in fact statically
configured -- but none of the nodes knows this configuration on boot.
Instead, the configuration is distributed dynamically and
signature-checked. For this purpose, I've been working on a
configuration protocol that seems more and more like I'm duplicating
effort with wg-dynamic, so I want to share my thoughts so far and
hopefully end up making the upstream project better. (I do realize
that wg-dynamic doesn't have signature support as a design goal, but
good protocols are extensible and I only need 32 bytes of space in
some sort of "vendor option".)

On to the fun stuff. First, I came up with the idea to run the
configuration protocol using IPv6 link-local addresses. I see that
wg-dynamic is doing the same thing, and independent engineers coming
up with the same design is usually a good sign that it's the right
fit. However, I've taken it one step further, by using
cryptographically-generated addresses; each peer automatically gets
fe80:(truncated hash of pubkey)/128 stuck in its allowed IP list. (I'm
considering harmonizing this address generation algorithm with RFC3972
in the future.) This means that initiating the protocol requires no
configuration other than the public key of the peer you'd like to
contact.

The current use of fe80::/128 as a magic address for the wg-dynamic
server brings up an important distinction between my usecase and the
current goals of wg-dynamic, and I think it might be a
significant-enough distinction to warrant changing its direction
slightly. Specifically, wg-dynamic is currently designed as a
centralized system -- nodes are not peers, because one must be in
charge of assigning IPs. I believe that my application, where each
peer knows the correct (signed) IP assignment for each other peer but
none are responsible for issuing new assignments, demonstrates that
the concerns of assigning configurations and distributing that
configuration information are fundamenally separate.

I beleive that the CGA-based approach would be more appropriate for
wg-dynamic as a whole, because it would fit better with WG's own
decentralized design, enable interesting new use cases, and simplify
configuration. Obviously it's important to specify which peers you to
trust to assign addresses -- I suggest that centralized, non-PKI-based
systems retain the "magic" address for an implicitly trusted central
server, which would use both its link-local CGA and fe80::/128. Still,
this would eliminate the chicken-and-egg problem, where some mechanism
would be needed to assign link-local IPs to each client before they
could communicate with the server to get their "real" IP assignment.
(I also suspect that if the current system were deployed as-is, many
operators would just roll their own hash-based IP allocation system
instead of maintaining some sort of database of assignments, and fewer
knobs is better when there's a sensible default and nothing to be
gained but incompatibility by tweaking it.)

Secondly, wg-dynamic needs a reliable transport to use at layer 4, but
I don't think it should be TCP. Wireguard itself is a tiny,
lightweight protocol -- one implementable on a microcontroller, if you
so desire -- but TCP is only available because it's built into kernels
everywhere, and while those implementations are battle-tested at this
point there's been a rich history of bugs in TCP stacks before the
kinks got worked out. I believe that it's important that whatever the
final form of wg-dynamic it should not preclude a completely
self-contained implementation -- say, in userspace, or on a
microcontroller. There's been plenty of work courtesy of the IDS and
DPI crowd towards reassembling TCP streams in userland, but there
aren't many full userland TCP stacks -- and the ones that are there I
wouldn't trust.

Therefore, I suggest TFTP as a transport. (No, seriously.) TFTP has
many things going for it in this application: it has simple,
well-understood state machine which can be implemented securely in a
mostly-stateless timer-driven fashion (sound familiar?), it can return
short blocks of data in only a single RTT (which TCP can't), and it's
got existing tooling that would be useful for
prototyping/experimentation/whatever. Essentially, it's got everything
you need and nothing you don't, and running it inside a WG tunnel gets
you all the security features the protocol lacks intrinsically.

TFTP is also symmetric -- there's not a huge distinction between a
server and a client, and data can be pulled or pushed as needed. A
traditional DHCP-style configuration request or renewal could be
modelled by a client requesting a magically-named "file"; a server
could send an updated configuration by pushing the same file instead,
reducing the need to rely on short lease times in networks whose
configurations (read: routes) change frequently.

TFTP is also extensible. A number of useful options already exist --
for example, one to set the block size of a transfer to take advantage
of the full MTU available, and another that provides windowed
transmission and acknowledgement and can provide throughput comparable
to TCP in many cases. And while TFTP is inherently a one-way protocol,
it's easy to add support for a request/response paradigm. (I've got a
prototype of this; the client sends a request and includes an option
requesting a response, which the server fills in with a magical file
name the client can request to retrieve said response. All simple
state-machine-based stuff.)

What I haven't solved yet -- and the reason I've delayed until now to
write this up and share it -- is all the stuff above layer 4. In my
prototype system, I use a userland daemon with access to the WG
private key to intercept incoming WG handshake requests, decrypt the
public key of the initiator, and add each new initiator as a peer with
one of these link-local CGAs before delivering the handshake packet.
(This increases my risk of DoS somewhat, but that's acceptable in my
application.) I then can use TFTP to transfer data between hosts, but
I think I've gotten too caught up in the specifics of my own
application (verifying signatures, creation and distribution of signed
config blobs, etc).

I'm hoping that by taking a step back and treating the problem as one
of developing a generic, extensible in-band configuration scheme the
natural form of a solution to that problem will emerge, and I can
build all my other fancy crap on top of it. I think link-local CGAs
and TFTP are elegant solutions, but I really need a higher-level
protocol to build my fancy stuff on top of -- which means I need
wg-dynamic. And I don't just need it to be an idea, I need it to be a
tiny, elegant, documented protocol.

So I want to get involved. Let's talk about architecture, get some
consensus, do some documentation. Let's build an MVP for Linux, but
also for Go and Rust. I'm game for all of it -- somebody hit me up and
let me know how I can help.

--Reid

Re: Thoughts on wg-dynamic

[Arti Zirk]

On P, 2020-04-05 at 19:43 -0400, Reid Rankin wrote:
> However, I've taken it one step further, by using
> cryptographically-generated addresses; each peer automatically gets
> fe80:(truncated hash of pubkey)/128 stuck in its allowed IP list.
> (I'm considering harmonizing this address generation algorithm with
> RFC3972 in the future.) This means that initiating the protocol
> requires no configuration other than the public key of the peer you'd
> like to contact.

While back there was a tool posted to this mailing list[0] that
generated WireGuard IP aadresses from the public key called wg-ip[1].

It would simplify things if that or some other link-local IP genration
algorithm would get integrated into wg-quick toolset.

I have also written a Python version of wg-ip generation algorithm that
might me slightly easier to read[2].

[0] https://lists.zx2c4.com/pipermail/wireguard/2018-April/002593.html
[1] https://github.com/chmduquesne/wg-ip
[2] https://gist.github.com/artizirk/c91e4f8c237dec07e3ad1b286f1855a7

Re: Thoughts on wg-dynamic

[Reid Rankin]

That's a neat tool, and I'd probably use it in my own projects -- I'm
a big fan of the shell-script-only approach for userland tools -- but
the algorithm it uses has a few rough edges I'd prefer didn't get
standardized:

- The hash is taken over a (newline-terminated!) Base64-encoded key
and not the raw bytes; there's nothing else in the protocol that would
require implementations to have a Base64 encoder. (This might matter
in a microcontroller implementation.) It wouldn't be hard to pipe to
`base64 -d` before hashing, which would fix this.
- I'm a fan of SHA-256 in general, but again, there's nothing else in
the protocol that would require implementers to have a SHA-256 engine.
(This would definitely matter in a microcontroller implementation.)
OTOH, `sha256sum` is a commonly available utility and reasonable to
use in a bash script, while `b2sum` is not so much.

My prototype also uses the `(subnet & mask) | (hash & ~mask)`
construction, though I use the full fe80::/10 space, instead of
fe80::/64. There's a decent argument to be made for just using the
/64, though -- it's the "interface identifier" portion of the address,
which is why that's what RFC 3972 does as well. That said, I'm not
sure there's quite the same need for subnetting in a link-local
address space. (I also feel like the subnet id/interface id dichotomy
is going the way of IPv4 class-based routing, and was probably only
conceived of as a clever ploy to keep ISPs from screwing everyone over
by only handing out out /120s.)

I've taken another look at RFC 3972, and the only way it would work
here would be by using well-known modifier and Sec parameters of zero
-- and it would force reliance on SHA-1. So I'm now thinking we don't
bother.

In any case, that makes at least two independent engineering efforts
that both use `(subnet & mask) | (hash & ~mask)` rather than using
subnet as a prefix for the hash.

--Reid

Re: Thoughts on wg-dynamic

[Reid Rankin]

I looked at something similar, but I really don't want to pay the
extra overhead -- and it just feels wrong to use
IP-in-Ethernet-in-GRE-in-UDP-in-IP(-in-WG-in-UDP-in-IP) if you don't
have to for routing purposes. (Version 2.0 of the product I'm working
on might use something similar for dynamic routing, but for now I'm
content with the assumption that all peers can reach each other.)

--Reid

On Sun, Apr 5, 2020 at 8:02 PM brian mullan <bmullan.mail@gmail.com> wrote:
>
> Reid
>
> I've been using
>
> m13253/VxWireguard-Generator: Utility to generate VXLAN over Wireguard mesh SD-WAN configuration
>
> https://github.com/m13253/VxWireguard-Generator
>
> I added BGP with BGP VRF.  It all works great in my multi cloud, multi node use-case.
>
> Brian