Hey everyone, After many months of work, Simon and I are pleased to announce the WireGuardNT project, a native port of WireGuard to the Windows kernel. This has been a monumental undertaking, and if you've noticed that I haven't read emails in about two months, now you know why. WireGuardNT, lower-cased as "wireguard-nt" like the other repos, began as a port of the Linux codebase, so that we could benefit from the analysis and scrutiny that that code has already received. After the initial porting efforts there succeeded, the NT codebase quickly diverged to fit well with native NTisms and NDIS (Windows networking stack) APIs. The end result is a deeply integrated and highly performant implementation of WireGuard for the NT kernel, that makes use of the full gamut of NT kernel and NDIS capabilities. You can read about the project and look at its source code here: https://git.zx2c4.com/wireguard-nt/about/ For the Windows platform, this project is a big deal to me, as it marks the graduation of WireGuard to being a serious operating system component, meant for more serious usage. It's also a rather significant open source release, as there generally isn't so much (though there is some) open source crypto-NIC driver code already out there that does this kind of thing while pulling together various kernel capabilities in the process. To frame what WireGuardNT is, a bit of background for how WireGuard on Windows _currently_ works, prior to this, might be in store. We currently have a cross-platform Go codebase, called wireguard-go, which uses a generic TUN driver we developed called Wintun (see wintun.net for info). The implementation lives in userspace, and shepherds packets to and from the Wintun interface. WireGuardNT will (eventually) replace that, placing all of the WireGuard protocol implementation directly into the networking stack for deeper integration, in the same way that it's done currently on Linux, OpenBSD, and FreeBSD. With the old wireguard-go/Wintun implementation, the fact of being in userspace means that for each RX UDP packet that arrives in the kernel from the NIC and gets put in a UDP socket buffer, there's a context switch to userspace to receive it, and then a trip through the Go scheduler to decrypt it, and then it's written to Wintun's ring buffer, where it is then processed upon the next context switch. For TX, things happen in reverse: userspace sends a packet, and there's a context switch to the kernel to hand it off to Wintun, which places it into a ring buffer, and then there's another context switch to userspace, and a trip through the Go scheduler to encrypt it, and then it's sent through a socket, which involves another context switch to send it. All of the ring buffers -- Wintun's rings and Winsock's RIO rings -- amortize context switches as much as possible and make this decently fast, but all and all it still constitutes overhead and latency. WireGuardNT gets rid of all of that. While performance is quite good right now (~7.5Gbps TX on my small test box), not a lot of effort has yet been spent on optimizing it, and there's still a lot more performance to eek out of it, I suspect, especially as we learn more about NT's scheduler and threading model particulars. Yet, by simply being in the kernel, we significantly reduce latency and do away with the context switch problems of wireguard-go/Wintun. Most Windows users, however, don't really care what happens beyond 1Gbps, and this is where things get interesting. Windows users with an Ethernet connection generally haven't had much trouble getting close to 1Gbps or so with the old slow wireguard-go/Wintun, but over WiFi, those same users would commonly see massive slowdowns. With the significantly decreased latency of WireGuardNT, it appears that these slowdowns are no more. Jonathan Tooker reported to me that, on his system with an Intel AC9560 WiFi card, he gets ~600Mbps without WireGuard, ~600Mbps with wireguard-go/Wintun over Ethernet, ~95Mbps with wireguard-go/Wintun over WiFi, and ~600Mbps with WireGuardNT over WiFi. In other words, the WiFi performance hit from wireguard-go/Wintun has evaporated when using WireGuardNT. Power consumption, and hence battery usage, should be lower too. And of course, on the multigig throughput side of things, Windows Server users will no doubt benefit. The project is still at its early stages, and for now (August 2021; if you're reading this in the future this might not apply) this should be considered "experimental". There's a decent amount of new code on which I'd like to spend a bit more time scrutinizing and analyzing. And hopefully by putting the code online in an "earlier" stage of development, others might be interested in studying the source and reporting bugs in it. Nonetheless, experimental or not, we still need people to test this and help shake out issues. To that end, WireGuardNT is now available in the ordinary WireGuard for Windows client -- https://www.wireguard.com/install/ -- with the 0.4.z series, in addition to having full support of the venerable wg(8) utility, but currently (August 2021; if you're reading this in the future this might not apply) it is behind a manually set registry knob. There will be three phases of the 0.4.z series: Phase 1) WireGuardNT is hidden behind the "ExperimentalKernelDriver" registry knob. If you don't manually tinker around to enable it, the client will continue to use wireguard-go/Wintun like before. Phase 2) WireGuardNT is enabled by default and is no longer hidden. However, in case there are late-stage problems that cause downtime for existing infrastructure, there'll be a new hidden knob called "UseUserspaceImplementation" that goes back to using wireguard-go/Wintun like before. Phase 3) WireGuardNT is enabled, and wireguard-go/Wintun is removed from the client. [Do note: as projects and codebases, both Wintun and wireguard-go will continue to be maintained, as they have applications and uses outside of our WireGuard client, and Wintun has uses outside of WireGuard in general.] The leap between each phase is rather large, and I'll update this thread when each one happens. Moving from 1 to 2 will happen when things seem okay for general consumption and from 2 to 3 when we're reasonably sure there's the same level of stability. Since we don't include any telemetry in the client, a lot of this assessment will be a matter of you, mailing list readers, sending bug reports or not sending bug reports. And of course, having testers during the unstable phase 1 will be a great boon. Instructions on enabling these knobs can be found in the usual place: https://git.zx2c4.com/wireguard-windows/about/docs/adminregistry.md [ If you're reading this email in the future and that page either does not exist or does not contain mention of "ExperimentalKernelDriver" or "UseUserspaceImplementation", then we have already moved to phase 3, as above, and none of this applies any more. ] So, please do give it a whirl, check out the documentation and code, and let me know what you think. I'm looking forward to hearing your thoughts and receiving bug reports, experience reports, and overall feedback. Enjoy! Jason
Hi everyone,
On Mon, Aug 2, 2021 at 7:28 PM Jason A. Donenfeld <Jason@zx2c4.com> wrote:
> Nonetheless, experimental or not, we still need people to test this and help
> shake out issues. To that end, WireGuardNT is now available in the ordinary
> WireGuard for Windows client -- https://www.wireguard.com/install/ -- with the
> 0.4.z series, in addition to having full support of the venerable wg(8)
> utility, but currently (August 2021; if you're reading this in the future this
> might not apply) it is behind a manually set registry knob. There will be
> three phases of the 0.4.z series:
>
> Phase 1) WireGuardNT is hidden behind the "ExperimentalKernelDriver"
> registry knob. If you don't manually tinker around to enable it,
> the client will continue to use wireguard-go/Wintun like before.
After the first few days, bug reports for WireGuardNT have died down
considerably. It's hard to tell whether that's a sign that the
software is stable or that nobody is using it. So, we're still in
phase 1. However, with version 0.4.5 of the client, I've added a
checkbox below the "activate" button in order to make the experimental
testing more discoverable. I dislike such knobs, but I'd rather have
more testing than less, before turning it on by default. Hopefully
this will make it easier for alpha testers to give it a spin and
report findings. If you have been testing the driver, and it's been
working well for you, please write in to say so, simply as a signal
that it is being tested. If things stay as quiet as they are now on
the bug reporting front, we'll likely move into phase 2 -- enabling
WireGuardNT by default, with a registry key to revert to the legacy
implementation -- in 2 to 3 weeks.
Jason
Hello Jason,
thanks for your work on Wireguard.
I have just updated and activated the experimental driver. The fact that
I do not need to activate it in the registry was decisive for me ;-)
I cannot report any issue so far.
Best regards,
Hendrik
------ Originalnachricht ------
Von: "Jason A. Donenfeld" <Jason@zx2c4.com>
An: "WireGuard mailing list" <wireguard@lists.zx2c4.com>
Gesendet: 13.08.2021 13:52:39
Betreff: Re: [ANNOUNCE] WireGuardNT, a high-performance WireGuard
implementation for the Windows kernel
>Hi everyone,
>
>On Mon, Aug 2, 2021 at 7:28 PM Jason A. Donenfeld <Jason@zx2c4.com> wrote:
>> Nonetheless, experimental or not, we still need people to test this and help
>> shake out issues. To that end, WireGuardNT is now available in the ordinary
>> WireGuard for Windows client -- https://www.wireguard.com/install/ -- with the
>> 0.4.z series, in addition to having full support of the venerable wg(8)
>> utility, but currently (August 2021; if you're reading this in the future this
>> might not apply) it is behind a manually set registry knob. There will be
>> three phases of the 0.4.z series:
>>
>> Phase 1) WireGuardNT is hidden behind the "ExperimentalKernelDriver"
>> registry knob. If you don't manually tinker around to enable it,
>> the client will continue to use wireguard-go/Wintun like before.
>
>After the first few days, bug reports for WireGuardNT have died down
>considerably. It's hard to tell whether that's a sign that the
>software is stable or that nobody is using it. So, we're still in
>phase 1. However, with version 0.4.5 of the client, I've added a
>checkbox below the "activate" button in order to make the experimental
>testing more discoverable. I dislike such knobs, but I'd rather have
>more testing than less, before turning it on by default. Hopefully
>this will make it easier for alpha testers to give it a spin and
>report findings. If you have been testing the driver, and it's been
>working well for you, please write in to say so, simply as a signal
>that it is being tested. If things stay as quiet as they are now on
>the bug reporting front, we'll likely move into phase 2 -- enabling
>WireGuardNT by default, with a registry key to revert to the legacy
>implementation -- in 2 to 3 weeks.
>
>Jason
Hi
I've been using it daily since 0.4.0 on a laptop and two servers in a
mixed Linux/WNT/FreeBSD (pfsense) environment.
Since 0.4.1 it has worked perfectly for me.
Thanks for the effort. The tunnel is subjectively snappier than the pure
Go implementation.
Best wishes
Peter Whisker
On 13/08/2021 12:52, Jason A. Donenfeld wrote:
> Hi everyone,
>
> On Mon, Aug 2, 2021 at 7:28 PM Jason A. Donenfeld <Jason@zx2c4.com> wrote:
>> Nonetheless, experimental or not, we still need people to test this and help
>> shake out issues. To that end, WireGuardNT is now available in the ordinary
>> WireGuard for Windows client -- https://www.wireguard.com/install/ -- with the
>> 0.4.z series, in addition to having full support of the venerable wg(8)
>> utility, but currently (August 2021; if you're reading this in the future this
>> might not apply) it is behind a manually set registry knob. There will be
>> three phases of the 0.4.z series:
>>
>> Phase 1) WireGuardNT is hidden behind the "ExperimentalKernelDriver"
>> registry knob. If you don't manually tinker around to enable it,
>> the client will continue to use wireguard-go/Wintun like before.
> After the first few days, bug reports for WireGuardNT have died down
> considerably. It's hard to tell whether that's a sign that the
> software is stable or that nobody is using it. So, we're still in
> phase 1. However, with version 0.4.5 of the client, I've added a
> checkbox below the "activate" button in order to make the experimental
> testing more discoverable. I dislike such knobs, but I'd rather have
> more testing than less, before turning it on by default. Hopefully
> this will make it easier for alpha testers to give it a spin and
> report findings. If you have been testing the driver, and it's been
> working well for you, please write in to say so, simply as a signal
> that it is being tested. If things stay as quiet as they are now on
> the bug reporting front, we'll likely move into phase 2 -- enabling
> WireGuardNT by default, with a registry key to revert to the legacy
> implementation -- in 2 to 3 weeks.
>
> Jason
I've been testing the experimental driver since the first announcement but no issues to date. On Fri, 13 Aug 2021 at 13:57, Jason A. Donenfeld <Jason@zx2c4.com> wrote: > > Hi everyone, > > On Mon, Aug 2, 2021 at 7:28 PM Jason A. Donenfeld <Jason@zx2c4.com> wrote: > > Nonetheless, experimental or not, we still need people to test this and help > > shake out issues. To that end, WireGuardNT is now available in the ordinary > > WireGuard for Windows client -- https://www.wireguard.com/install/ -- with the > > 0.4.z series, in addition to having full support of the venerable wg(8) > > utility, but currently (August 2021; if you're reading this in the future this > > might not apply) it is behind a manually set registry knob. There will be > > three phases of the 0.4.z series: > > > > Phase 1) WireGuardNT is hidden behind the "ExperimentalKernelDriver" > > registry knob. If you don't manually tinker around to enable it, > > the client will continue to use wireguard-go/Wintun like before. > > After the first few days, bug reports for WireGuardNT have died down > considerably. It's hard to tell whether that's a sign that the > software is stable or that nobody is using it. So, we're still in > phase 1. However, with version 0.4.5 of the client, I've added a > checkbox below the "activate" button in order to make the experimental > testing more discoverable. I dislike such knobs, but I'd rather have > more testing than less, before turning it on by default. Hopefully > this will make it easier for alpha testers to give it a spin and > report findings. If you have been testing the driver, and it's been > working well for you, please write in to say so, simply as a signal > that it is being tested. If things stay as quiet as they are now on > the bug reporting front, we'll likely move into phase 2 -- enabling > WireGuardNT by default, with a registry key to revert to the legacy > implementation -- in 2 to 3 weeks. > > Jason -- Phillip McMahon https://flowcrypt.com/me/phillipmcmahon OpenPGP: EA04 83D4 C864 AA7C 1099 4BE6 A11E 70AD FDA6 0CF9
Den 13-08-2021 kl. 13:52 skrev Jason A. Donenfeld:
I have used it since first announcement. Not experienced any problems.
--
Morten Christensen
> Hi everyone,
>
> On Mon, Aug 2, 2021 at 7:28 PM Jason A. Donenfeld <Jason@zx2c4.com> wrote:
>> Nonetheless, experimental or not, we still need people to test this and help
>> shake out issues. To that end, WireGuardNT is now available in the ordinary
>> WireGuard for Windows client -- https://www.wireguard.com/install/ -- with the
>> 0.4.z series, in addition to having full support of the venerable wg(8)
>> utility, but currently (August 2021; if you're reading this in the future this
>> might not apply) it is behind a manually set registry knob. There will be
>> three phases of the 0.4.z series:
>>
>> Phase 1) WireGuardNT is hidden behind the "ExperimentalKernelDriver"
>> registry knob. If you don't manually tinker around to enable it,
>> the client will continue to use wireguard-go/Wintun like before.
> After the first few days, bug reports for WireGuardNT have died down
> considerably. It's hard to tell whether that's a sign that the
> software is stable or that nobody is using it. So, we're still in
> phase 1. However, with version 0.4.5 of the client, I've added a
> checkbox below the "activate" button in order to make the experimental
> testing more discoverable. I dislike such knobs, but I'd rather have
> more testing than less, before turning it on by default. Hopefully
> this will make it easier for alpha testers to give it a spin and
> report findings. If you have been testing the driver, and it's been
> working well for you, please write in to say so, simply as a signal
> that it is being tested. If things stay as quiet as they are now on
> the bug reporting front, we'll likely move into phase 2 -- enabling
> WireGuardNT by default, with a registry key to revert to the legacy
> implementation -- in 2 to 3 weeks.
>
> Jason
On Fri Aug 13, 2021 at 01:52:39PM +0200, Jason A. Donenfeld wrote:
>this will make it easier for alpha testers to give it a spin and
>report findings. If you have been testing the driver, and it's been
>working well for you, please write in to say so, simply as a signal
>that it is being tested.
No issues on Windows 8.1 so far, using the NT kernel version from
0.4.5.
--
Mark Lawrence
Hi again,
On Fri, Aug 13, 2021 at 1:52 PM Jason A. Donenfeld <Jason@zx2c4.com> wrote:
>
> Hi everyone,
>
> On Mon, Aug 2, 2021 at 7:28 PM Jason A. Donenfeld <Jason@zx2c4.com> wrote:
> > Nonetheless, experimental or not, we still need people to test this and help
> > shake out issues. To that end, WireGuardNT is now available in the ordinary
> > WireGuard for Windows client -- https://www.wireguard.com/install/ -- with the
> > 0.4.z series, in addition to having full support of the venerable wg(8)
> > utility, but currently (August 2021; if you're reading this in the future this
> > might not apply) it is behind a manually set registry knob. There will be
> > three phases of the 0.4.z series:
> >
> > Phase 1) WireGuardNT is hidden behind the "ExperimentalKernelDriver"
> > registry knob. If you don't manually tinker around to enable it,
> > the client will continue to use wireguard-go/Wintun like before.
>
> After the first few days, bug reports for WireGuardNT have died down
> considerably. It's hard to tell whether that's a sign that the
> software is stable or that nobody is using it. So, we're still in
> phase 1. However, with version 0.4.5 of the client, I've added a
> checkbox below the "activate" button in order to make the experimental
> testing more discoverable. I dislike such knobs, but I'd rather have
> more testing than less, before turning it on by default. Hopefully
> this will make it easier for alpha testers to give it a spin and
> report findings. If you have been testing the driver, and it's been
> working well for you, please write in to say so, simply as a signal
> that it is being tested. If things stay as quiet as they are now on
> the bug reporting front, we'll likely move into phase 2 -- enabling
> WireGuardNT by default, with a registry key to revert to the legacy
> implementation -- in 2 to 3 weeks.
We're still technically in phase 1, but with the version that should
be coming out later today, users who have enrolled in the Windows
Insider Program have "ExperimentalKernelDriver" set to always on,
unless the "IgnoreInsiderProgram" admin knob is set. In other words,
we're gradually ramping up testing coverage. I still hope to move into
phase 2 soon, but this seemed like a cautious intermediate step.
Regards,
Jason
Hi everyone, As described in the original announcement: > There will be three phases of the 0.4.z series: > > Phase 1) WireGuardNT is hidden behind the "ExperimentalKernelDriver" > registry knob. If you don't manually tinker around to enable it, > the client will continue to use wireguard-go/Wintun like before. > > Phase 2) WireGuardNT is enabled by default and is no longer hidden. > However, in case there are late-stage problems that cause > downtime for existing infrastructure, there'll be a new hidden > knob called "UseUserspaceImplementation" that goes back to > using wireguard-go/Wintun like before. > > Phase 3) WireGuardNT is enabled, and wireguard-go/Wintun is removed from > the client. After quite a bit of testing, the bug reports have dried up. We've gone through several subphases of phase 1, from being masked behind a registry knob, to introducing a checkbox, to opting in users part of the Windows Insider program. There's been a pretty heavy amount of testing, from what I can gather, and I think things have shaped up nicely. Finally, tomorrow with the release of v0.4.8, we will move onto phase 2. That means that WireGuardNT will be on by default. If something goes wrong, you will still be able to disable it and revert to the older wireguard-go/Wintun using the "UseUserspaceImplementation" switch described on this page (as of September 2021; will be removed in the future): https://git.zx2c4.com/wireguard-windows/about/docs/adminregistry.md But note that if you /do/ revert to using wireguard-go/Wintun, you *must* absolutely email team@wireguard.com with details as to why you've done so, so that if there is a bug, we can fix it. If you do not do this, it is possible that your bug will never be fixed, and in a month or so, we'll move onto phase 3, and you will lose the ability to revert to using wireguard-go/Wintun. Do not rely on other people to report your bug for you; Windows is weird and diverse and there are no guarantees that somebody else will run into your same problem. So please: if you enable "UseUserspaceImplementation", send an email saying why. Depending on how Monday morning goes, this should be released in around 12 hours from now. Enjoy! Regards, Jason
Hi Jason,
On Sun, Sep 12, 2021 at 5:11 PM Jason A. Donenfeld <Jason@zx2c4.com> wrote:
> ...
> Finally, tomorrow with the release of v0.4.8, we will move onto phase
> 2. That means that WireGuardNT will be on by default. If something
> goes wrong, you will still be able to disable it and revert to the
> older wireguard-go/Wintun using the "UseUserspaceImplementation"
> switch described on this page (as of September 2021; will be removed
> in the future):
>
> https://git.zx2c4.com/wireguard-windows/about/docs/adminregistry.md
>
> But note that if you /do/ revert to using wireguard-go/Wintun, you
> *must* absolutely email team@wireguard.com with details as to why
> you've done so, so that if there is a bug, we can fix it. If you do
> not do this, it is possible that your bug will never be fixed, and in
> a month or so, we'll move onto phase 3, and you will lose the ability
> to revert to using wireguard-go/Wintun. Do not rely on other people to
> report your bug for you; Windows is weird and diverse and there are no
> guarantees that somebody else will run into your same problem. So
> please: if you enable "UseUserspaceImplementation", send an email
> saying why.
One month to move into the next phase may be a bit tight for some
folks. 30 days is probably fine for a developer or standalone
installation, but some organizations cannot move that fast.
I've worked in US Financial and US Federal, and some changes take
longer to approve. Some organizations have processes in place that
require approvals from management. It may take months to get a Change
Control Request approved.
When I worked at Treasury a trivial change could take two or three
months and it required management signoffs and complete testing before
being released to the production network. Nearly everyone dreaded a
Change Control Request.
It may be noteworthy... on Windows OSes, the trend is to move stuff
out of the kernel and into userspace to reduce risk. For example,
Microsoft moved parts of the GDI out of the kernel and into userspace.
So some folks may actually want the userland architecture to reduce
risk.
Jeff
Hello Jeff, On Sun, Sep 12, 2021 at 3:55 PM Jeffrey Walton <noloader@gmail.com> wrote: > One month to move into the next phase may be a bit tight for some > folks. 30 days is probably fine for a developer or standalone > installation, but some organizations cannot move that fast. > > I've worked in US Financial and US Federal, and some changes take > longer to approve. Some organizations have processes in place that > require approvals from management. It may take months to get a Change > Control Request approved. > > When I worked at Treasury a trivial change could take two or three > months and it required management signoffs and complete testing before > being released to the production network. Nearly everyone dreaded a > Change Control Request. I'm not sure I really follow this line of reasoning here. You've divided the into two categories: A) "a developer or standalone installation" that is presumably frequently upgrading in order to have the latest in greatest. B) "US Financial and US Federal" and similar organizations for whom changes require extensive paperwork and reliability testing. For group (A), the gradual three phase rollout outlined in the initial email works very well, as it means each component gets an appropriate amount of testing depending on the phase, with related rollback controls. It's group (B) that you've raised a concern about. But, if each change for (B) requires paperwork and reliability testing, wouldn't _that_ procedure also catch issues? Obviously a downside of not updating like (A) does is that you miss timely security updates, but I assume that (B) argues that their slow processes of sign-offs and testing and paperwork increases reliability and accountability, which may well be more important for (B). And it would seem that those processes basically accomplish the same thing as the three phase plan that's available to group (A) does. So it doesn't seem like anybody is left out. Unless, of course, all those sign-offs and testing aren't _actually_ testing anything meaningful, and they're just a Brazilesque paperwork nightmare, in which case, with regard for neither security nor reliability, what does it matter either way? Anyway, there's no exact timeline of precisely 30 days. It will be a good time after nobody has reported a bug. While products have "go to market" deadlines, projects have the luxury of being on the "when it's ready" schedule, which means we don't need to rush. At the same time, I very much look forward to the reduced maintenance burden of no longer maintaining duplicate code paths in the wireguard-windows repo. The jd/remove-wintun branch has a commit that removes about a thousand lines, which isn't _that_ much, but those are lines that are a bit stressful in the sense that abstraction layers and duplicated code paths are common sources of bugs, so they require extra vigilance from me (and auditors). > It may be noteworthy... on Windows OSes, the trend is to move stuff > out of the kernel and into userspace to reduce risk. For example, > Microsoft moved parts of the GDI out of the kernel and into userspace. > So some folks may actually want the userland architecture to reduce > risk. I don't think it actually makes a practical difference in this case: the userspace tunnel service retained SeLoadDriverPrivilege so that it could *unload* Wintun when quitting. Besides, WireGuard was made to be implemented in operating system kernels; figuring out how to do that in a minimally scary way was an original thrust of the project. Linux, OpenBSD, FreeBSD, and now NT. Jason
Hi folks,
As described in August:
On Mon, Aug 2, 2021 at 11:28 AM Jason A. Donenfeld <Jason@zx2c4.com> wrote:
> Phase 3) WireGuardNT is enabled, and wireguard-go/Wintun is removed from
> the client. [Do note: as projects and codebases, both Wintun and
> wireguard-go will continue to be maintained, as they have
> applications and uses outside of our WireGuard client, and Wintun
> has uses outside of WireGuard in general.]
After a considerable amount of testing and 11 releases, we are now in
phase 3, and the transition is complete. WireGuard for Windows v0.5 is
being released as we speak, and contains WireGuardNT 0.10.1, which is
now fully WHQL Certified and will run on all flavors of Windows
Server.
Thank you very much to everyone who helped test and stabilize this code.
Regards,
Jason