From: "Jason A. Donenfeld" <Jason@zx2c4.com>
Subject: The Future of RHEL7,8 Support in WireGuard
Date: Tue, 28 Jun 2022 13:44:20 +0200 [thread overview]
Message-ID: <YrrplG8DswM0zxQO@zx2c4.com> (raw)
I'm confronted with a few hard facts, the first of which is good and
exciting, and the rest are kind of disappointing:
1) RHEL9 has WireGuard baked into it by default, so there's no
required compatibility backporting for it. If you use RHEL9, you
can use WireGuard easily.
2) The RHEL7 and RHEL8 kernels have build errors due to trivially dumb
mismerges, if you try to build them using nearly anything other
than the RHEL-supplied .config files. I've reported these bugs to
Red Hat and I've sent series fixing these bugs (see this git repo,
for example: https://git.zx2c4.com/rhel7-kernel-misery/log/ ), but
Red Hat's perspective is that these are non-bugs that they won't
fix, since the only purpose of the kernel for them is their own
3) Those build errors make it difficult to run WireGuard's CI on them
without having to apply lots of extra patches. While RHEL7 is
mostly set in stone now, the C8S kernel changes frequently.
Recently I dropped RHEL from the CI, as C8S hadn't been building
for some time. So now I have no way of ensuring that it continues
4) RHEL7 and RHEL8 is a behind-closed-door process, which makes
tracking down new breakage and dealing with anything produced by it
an exercise in pain.
5) No companies who are using RHEL7 or RHEL8 with WireGuard have
reached out to me about trying to fund this work. (And in general,
now that WireGuard isn't a hip new marketing thing for VPN
providers and whomever, funding has significantly dropped off.)
Given all this, I wonder whether the RHEL7 and RHEL8 backports have a
future, and if they do, what that future looks like.
Note, this isn't just a matter of "backporting is hard work that nobody
likes to do, but you do it anyway because that's what you gotta do."
Because in contrast, I'm quite content doing backports for the
kernel.org stable kernels that Greg and Sasha release. The process is
open and transparent, Greg and Sasha are nice to work with, the goals
are clear, the outcomes are dependable. It is still always work, but
developing for kernel.org stable kernels is reasonable engineering work,
as opposed to RHEL backporting, which is fighting uphill against
byzantine bureaucratic processes that aren't very welcoming of my
And to be clear, it's not as though Red Hat is somehow evil here. Their
goals simply aren't aligned. The kernel.org stable releases are meant to
be general kernels that you can do things with, developed as part of an
open source project. The RHEL kernel, by contrast, is part of a
commercial product with a very specific single goal in mind. That's
fine, and there's nothing inherently wrong with that.
So when Red Hat tells me, "sorry, why should we care about this? RHEL7
and 8 don't support WireGuard. Tell people to use RHEL9 for that. We're
not going to make your life easier in RHEL7 and 8. It's not supported,"
I sigh exasperated, because as frustrating as their position is, it's
also an entirely reasonable position for them to have, based on what
their goals are and what they're doing.
That all is a very long way of saying that continuing to backport
WireGuard to RHEL7 and 8 kernels in the normal way with WireGuard's CI
as it currently exists is an uphill battle, just due to the nature of
the thing. It's constant struggle to put the square peg in the round
So what are our options here moving forward?
A) I rip out RHEL7 and RHEL8 support completely, and expect people to
just use RHEL9.
B) Somebody extremely motivated and associated with Alma Linux or
Rocky Linux or Oracle sets up a dedicated CI server using the
official kernel builds and kernel headers, and runs `make test` on
every commit to either the kernels or to wireguard-linux-compat,
and then either:
i) has the CI automatically send me an email notification on
breakage so I can fix it like usual; or
ii) fixes it and sends a proper patch to the mailing list in the
correct style and format.
Either one of those would work. (ii) would be nice, but there
aren't tons of people who like both CI devops and kernel
development, so I'd be fine with (i) too, which is more or less the
status quo but with functional CI.
Note, though, that this option (B) would have to be done well and
reliably. Were it organizationally flaky, I'd prefer to do (A), as
half-baked approaches tend to be worse than nothing.
If this interests any readers with the backing of large
RHEL-derivative projects, please get in touch.
C) Several companies who are using WireGuard with RHEL7 and RHEL8
provide significant project funding, and then I take care of
setting up dedicated and reliable RHEL CI for the project, using
official kernel builds and kernel headers. I could do this quite
well, but it doesn't seem right to do that without real funding for
If this interests any corporate readers, please get in touch.
D) Other ideas the mailing list might have.
So that's about where my thinking is at the moment. As usual, I'm not
going to rush into any decision, and I'll do my best to keep RHEL
support as afloat as I can until a careful decision in one of these
directions has been made. None of this email is written with any real
sense of urgency. But still, I'd like to at some point choose a
direction and commit to it.
Let me know what you think.
reply other threads:[~2022-06-28 11:44 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).