Re: Problems with Ubuntu 16.04 kernels and wireguard 1.0.20200611

["Jason A. Donenfeld" <Jason@zx2c4.com>]

Hi Gregory,

On Wed, Jun 17, 2020 at 2:01 AM Gregory ORIOL <fortin81@gmail.com> wrote:
> Since commit e24c9a9265af40781fa27b5de11dd5b78925c5be to
> wireguard-linux-compat with a fix for some Ubuntu versions, we are
> experiencing a problem with some older, but still LTS, versions of
> Ubuntu 16.04: older kernels 4.4.0-148, 4.4.0-166 fail to build with
> wireguard 1.0.20200611
>
> So, any system running an (or still having an installed) "older" kernel
> and doing an apt upgrade to install wireguard 1.0.20200611 would fail
> during the wireguard dkms step, while trying to build wireguard for all
> the kernels available.
>
> The problem gets more problematic when a newer kernel 4.4.0-184 gets
> installed with the same apt upgrade: then, trying to downgrade wireguard
> also fails; none of the 1.0.20200611 or 1.0.20200520 versions work
> anymore with this combination of old/new kernels...
>
> To recap :
> # wireguard 1.0.20200520
>  - ok with kernels 4.4.0-148, 4.4.0-166
>  - fails with kernel 4.4.0-184
> # wireguard 1.0.20200611
>  - fails with kernels 4.4.0-148, 4.4.0-166
>  - ok with kernel 4.4.0-184
>
> (nb: we see it now with -184 but it could have started with an earlier
> version)
>
> We could partially fix this by manually getting each deb/src and doing
> dkms install:
> dkms install wireguard/1.0.20200520 -k 4.4.0-148-generic
> dkms install wireguard/1.0.20200611 -k 4.4.0-184-generic
> ...
> But apt upgrade is still broken.
>
> While we could boot onto the newer kernel and remove the older ones to
> get rid of the problem, this situation would prevent from having a
> "previous working" kernel on the system, which is not very safe.
>
> Could there be a fix for this made to wireguard-linux-compat for those
> versions?

Unfortunately, I don't have a super good solution for you right now.
The wireguard-linux-compat repo is developed against the latest Ubuntu
kernels that they put out once every three weeks. You can see them
being tested at the bottom of
<https://www.wireguard.com/build-status/>. The backport against
upstream mainline kernels is z-granular (for an x.y.z versioning
scheme), but Ubuntu's release cycle and versioning scheme is a bit too
chaotic to make it reasonable to try to manage all the differences
between their kernels every three weeks. So for distro kernels --
Ubuntu, RHEL, Debian, etc -- we typically just develop against the
latest one, and try to make sure that we release it at the right time
so users aren't caught with no working version. This means,
unfortunately, that when there are badly breaking changes, like in
this last cycle, you have to uninstall the old kernels or mask them
from dkms, in order to get dkms to avoid building for them and only
building for the new kernel. There might be other more complicated
solutions that closely track version dependencies or do compile time
feature probing, but that comes with a maintenance burden far too
arduous for a distro frankenkernel.

But there is hope!

Canonical is adding WireGuard to 18.04 and 16.04, and this is coming
in two steps:

Step 1) The wireguard-dkms and wireguard-tools packages will be added
to the package archives, so that you won't have to use the PPA. This
means that Canonical's kernel team will include wireguard-dkms in
their development tests, so that they won't accidentally ship kernels
with build breakage, like what you experienced last week.

Step 2) The wireguard-dkms package will get built by Canonical,
signed, and shipped alongside the other modules, so that you won't
have to install wireguard-dkms, and it will just come out of the box
with the normal kernel updates. This is already the case with 20.04
and 19.10. They're working on it now for 18.04, and I really really
hope to see that happen by the next cycle. And maybe if we ask apw
(CC'd) nicely, he'll even do it for 16.04 too.

Regards,
Jason