wireguard ci hooked up to quite a few kernel trees

wireguard ci hooked up to quite a few kernel trees

[Jason A. Donenfeld]

Hi all,

With the merging of wireguard, I've hooked the project's CI up to
quite a few trees. We now have:

- net-next
- net
- linux-next
- linux (Linus' tree)
- wireguard-linux (my tree)
- wireguard-linux-compat (backports to kernels 3.10 - 5.5)

When the various pushes and pulls click a few more cranks through the
machinery, I'll probably add crypto and cryptodev, and eventually
Greg's stable trees. If anybody has suggestions on other relevant
trees that might help catch bugs as early as possible, I'm all ears.

Right now builds are kicked off for every single commit made to each
one of these trees, on x86_64, i686, aarch64, aarch64_be, arm, armeb,
mips64, mips64el, mips, mipsel, powerpc64le, powerpc, and m68k. For
each of these, a fresh kernel and miniature userland containing the
test suite is built from source, and then booted in qemu.

Even though the CI at the moment is focused on the wireguard test
suite, it has a habit of finding lots of bugs and regressions in other
weird places. For example, linux-next is failing at the moment on a
few archs.

I run this locally every day all day while developing kernel things
too. It's one command to test a full kernel for whatever thing I'm
working on, and this winds up saving a lot of time in development and
lets me debug things with printk in the dumbest ways possible while
still being productive and efficient.

You can view the current build status here:
https://www.wireguard.com/build-status/

This sort of CI is another take on the kernel CI problem; I know a few
organizations are doing similar things. I'd be happy to eventually
expand this into something more general, should there be sufficient
interest -- probably initially on networking stuff -- or it might turn
out that this simply inspires something else that is more general and
robust, which is fine too. Either way, here's my contribution to the
modicum of kernel CI things happening.

Regards,
Jason
_______________________________________________
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: wireguard ci hooked up to quite a few kernel trees

[Bhaskar Chowdhury]

[-- Attachment #1.1: Type: text/plain, Size: 2087 bytes --]

Looks bloody good Jason! thanks, man!

~Bhaskar

On 13:15 Wed 29 Jan 2020, Jason A. Donenfeld wrote:
>Hi all,
>
>With the merging of wireguard, I've hooked the project's CI up to
>quite a few trees. We now have:
>
>- net-next
>- net
>- linux-next
>- linux (Linus' tree)
>- wireguard-linux (my tree)
>- wireguard-linux-compat (backports to kernels 3.10 - 5.5)
>
>When the various pushes and pulls click a few more cranks through the
>machinery, I'll probably add crypto and cryptodev, and eventually
>Greg's stable trees. If anybody has suggestions on other relevant
>trees that might help catch bugs as early as possible, I'm all ears.
>
>Right now builds are kicked off for every single commit made to each
>one of these trees, on x86_64, i686, aarch64, aarch64_be, arm, armeb,
>mips64, mips64el, mips, mipsel, powerpc64le, powerpc, and m68k. For
>each of these, a fresh kernel and miniature userland containing the
>test suite is built from source, and then booted in qemu.
>
>Even though the CI at the moment is focused on the wireguard test
>suite, it has a habit of finding lots of bugs and regressions in other
>weird places. For example, linux-next is failing at the moment on a
>few archs.
>
>I run this locally every day all day while developing kernel things
>too. It's one command to test a full kernel for whatever thing I'm
>working on, and this winds up saving a lot of time in development and
>lets me debug things with printk in the dumbest ways possible while
>still being productive and efficient.
>
>You can view the current build status here:
>https://www.wireguard.com/build-status/
>
>This sort of CI is another take on the kernel CI problem; I know a few
>organizations are doing similar things. I'd be happy to eventually
>expand this into something more general, should there be sufficient
>interest -- probably initially on networking stuff -- or it might turn
>out that this simply inspires something else that is more general and
>robust, which is fine too. Either way, here's my contribution to the
>modicum of kernel CI things happening.
>
>Regards,
>Jason

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 148 bytes --]

_______________________________________________
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: wireguard ci hooked up to quite a few kernel trees

[Michael B. Williams]

[-- Attachment #1.1: Type: text/plain, Size: 2590 bytes --]

What’s with all the “Built Invalid Date” below each commit hash.

On Wed, Jan 29, 2020 at 23:37 Bhaskar Chowdhury <unixbhaskar@gmail.com>
wrote:

> Looks bloody good Jason! thanks, man!
>
> ~Bhaskar
>
> On 13:15 Wed 29 Jan 2020, Jason A. Donenfeld wrote:
> >Hi all,
> >
> >With the merging of wireguard, I've hooked the project's CI up to
> >quite a few trees. We now have:
> >
> >- net-next
> >- net
> >- linux-next
> >- linux (Linus' tree)
> >- wireguard-linux (my tree)
> >- wireguard-linux-compat (backports to kernels 3.10 - 5.5)
> >
> >When the various pushes and pulls click a few more cranks through the
> >machinery, I'll probably add crypto and cryptodev, and eventually
> >Greg's stable trees. If anybody has suggestions on other relevant
> >trees that might help catch bugs as early as possible, I'm all ears.
> >
> >Right now builds are kicked off for every single commit made to each
> >one of these trees, on x86_64, i686, aarch64, aarch64_be, arm, armeb,
> >mips64, mips64el, mips, mipsel, powerpc64le, powerpc, and m68k. For
> >each of these, a fresh kernel and miniature userland containing the
> >test suite is built from source, and then booted in qemu.
> >
> >Even though the CI at the moment is focused on the wireguard test
> >suite, it has a habit of finding lots of bugs and regressions in other
> >weird places. For example, linux-next is failing at the moment on a
> >few archs.
> >
> >I run this locally every day all day while developing kernel things
> >too. It's one command to test a full kernel for whatever thing I'm
> >working on, and this winds up saving a lot of time in development and
> >lets me debug things with printk in the dumbest ways possible while
> >still being productive and efficient.
> >
> >You can view the current build status here:
> >https://www.wireguard.com/build-status/
> >
> >This sort of CI is another take on the kernel CI problem; I know a few
> >organizations are doing similar things. I'd be happy to eventually
> >expand this into something more general, should there be sufficient
> >interest -- probably initially on networking stuff -- or it might turn
> >out that this simply inspires something else that is more general and
> >robust, which is fine too. Either way, here's my contribution to the
> >modicum of kernel CI things happening.
> >
> >Regards,
> >Jason
> _______________________________________________
> WireGuard mailing list
> WireGuard@lists.zx2c4.com
> https://lists.zx2c4.com/mailman/listinfo/wireguard
>
-- 
Sent from Gmail Mobile

[-- Attachment #1.2: Type: text/html, Size: 3488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 148 bytes --]

_______________________________________________
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: wireguard ci hooked up to quite a few kernel trees

[dvn]

[-- Attachment #1.1: Type: text/plain, Size: 2696 bytes --]

Hi Jason,

Impressed by how comprehensive this is already. Would you be up for
sharing some of the details of how this system works. Looks like it's
largely custom. Obviously based on QEMU/KVM, as you say - are you
emulating most of the architectures outside of x86_64?

Good to see a project taking continuous building/testing against
multiple trees and architectures so seriously!

Cheers,
Devan

Jason A. Donenfeld transcribed 2.0K bytes:
> Hi all,
> 
> With the merging of wireguard, I've hooked the project's CI up to
> quite a few trees. We now have:
> 
> - net-next
> - net
> - linux-next
> - linux (Linus' tree)
> - wireguard-linux (my tree)
> - wireguard-linux-compat (backports to kernels 3.10 - 5.5)
> 
> When the various pushes and pulls click a few more cranks through the
> machinery, I'll probably add crypto and cryptodev, and eventually
> Greg's stable trees. If anybody has suggestions on other relevant
> trees that might help catch bugs as early as possible, I'm all ears.
> 
> Right now builds are kicked off for every single commit made to each
> one of these trees, on x86_64, i686, aarch64, aarch64_be, arm, armeb,
> mips64, mips64el, mips, mipsel, powerpc64le, powerpc, and m68k. For
> each of these, a fresh kernel and miniature userland containing the
> test suite is built from source, and then booted in qemu.
> 
> Even though the CI at the moment is focused on the wireguard test
> suite, it has a habit of finding lots of bugs and regressions in other
> weird places. For example, linux-next is failing at the moment on a
> few archs.
> 
> I run this locally every day all day while developing kernel things
> too. It's one command to test a full kernel for whatever thing I'm
> working on, and this winds up saving a lot of time in development and
> lets me debug things with printk in the dumbest ways possible while
> still being productive and efficient.
> 
> You can view the current build status here:
> https://www.wireguard.com/build-status/
> 
> This sort of CI is another take on the kernel CI problem; I know a few
> organizations are doing similar things. I'd be happy to eventually
> expand this into something more general, should there be sufficient
> interest -- probably initially on networking stuff -- or it might turn
> out that this simply inspires something else that is more general and
> robust, which is fine too. Either way, here's my contribution to the
> modicum of kernel CI things happening.
> 
> Regards,
> Jason
> _______________________________________________
> WireGuard mailing list
> WireGuard@lists.zx2c4.com
> https://lists.zx2c4.com/mailman/listinfo/wireguard

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 148 bytes --]

_______________________________________________
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: wireguard ci hooked up to quite a few kernel trees

[Jason A. Donenfeld]

On Wed, Jan 29, 2020 at 1:40 PM dvn <mail@dvn.me> wrote:
> Impressed by how comprehensive this is already. Would you be up for
> sharing some of the details of how this system works. Looks like it's
> largely custom. Obviously based on QEMU/KVM, as you say - are you
> emulating most of the architectures outside of x86_64?

It uses kvm on x86_64 and i686 and TCG elsewhere. I guess it's
"custom", but it's really just a makefile that assembles some
arguments for qemu and sets some compilation flags.
_______________________________________________
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

MacOS Mojave - Tunnel activation failure

[David J Chapper]

[-- Attachment #1.1: Type: text/plain, Size: 1381 bytes --]

Hello, hoping someone else has experienced this and resolved the issue.

The Wireguard app has been running without issue for many months until today when I attempted to activate the tunnel and received the following error:

“Activation failure” Unable to retrieve tunnel information from the saved configuration.

If I try and edit the config the app crashes. Some details from the problem/crash report:

Process:               WireGuard [48844]
Path:                  /Applications/WireGuard.app/Contents/MacOS/WireGuard
Identifier:            com.wireguard.macos
Version:               0.0.20191105 (16)
App Item ID:           1451685025
App External ID:       833476010
Code Type:             X86-64 (Native)
Parent Process:        ??? [1]
Responsible:           WireGuard [48844]
User ID:               502

Date/Time:             2020-01-30 19:46:52.417 -0600
OS Version:            Mac OS X 10.14.6 (18G2022)
Report Version:        12
System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_INSTRUCTION (SIGILL)
Exception Codes:       0x0000000000000001, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Illegal instruction: 4
Termination Reason:    Namespace SIGNAL, Code 0x4
Terminating Process:   exc handler [48844]



[-- Attachment #1.2: Type: text/html, Size: 9673 bytes --]

[-- Attachment #2: Type: text/plain, Size: 148 bytes --]

_______________________________________________
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

Re: wireguard ci hooked up to quite a few kernel trees

[Joe Doss]

[-- Attachment #1.1.1: Type: text/plain, Size: 409 bytes --]

On 1/29/20 6:40 AM, dvn wrote:
> Impressed by how comprehensive this is already. Would you be up for
> sharing some of the details of how this system works. Looks like it's
> largely custom. Obviously based on QEMU/KVM, as you say - are you
> emulating most of the architectures outside of x86_64?

Pay no attention to the Bash Wizard behind the CI curtain... :)

-- 
Joe Doss
joe@solidadmin.com


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 148 bytes --]

_______________________________________________
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard