From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: henry.snow@nielsen.com Received: from krantz.zx2c4.com (localhost [127.0.0.1]) by krantz.zx2c4.com (ZX2C4 Mail Server) with ESMTP id 9fb025ad for ; Thu, 10 May 2018 21:39:12 +0000 (UTC) Received: from mail-pg0-x233.google.com (mail-pg0-x233.google.com [IPv6:2607:f8b0:400e:c05::233]) by krantz.zx2c4.com (ZX2C4 Mail Server) with ESMTP id a878a170 for ; Thu, 10 May 2018 21:39:12 +0000 (UTC) Received: by mail-pg0-x233.google.com with SMTP id z4-v6so1505022pgu.13 for ; Thu, 10 May 2018 14:41:59 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Henry Snow Date: Thu, 10 May 2018 21:41:47 +0000 Message-ID: Subject: Re: wireguard-dkms and CentOS 7.5 To: "Jason A. Donenfeld" Content-Type: multipart/alternative; boundary="0000000000008bc448056be0e35b" Cc: "wireguard@lists.zx2c4.com" List-Id: Development discussion of WireGuard List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , --0000000000008bc448056be0e35b Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hmm...yeah, I've followed these steps: Fresh CentOS 7.5 install yum install git libmnl-devel elfutils-libelf-devel kernel-devel @development-tools git clone https://git.zx2c4.com/WireGuard cd WireGuard/src make make install deploy /etc/wireguard/wg0.conf systemctl start wg-quick@wg0 kernel panic. Any additional debugging steps I can provide? This trace looks a little different, so also including it: [ 1571.621740] ------------[ cut here ]------------ [ 1571.627195] WARNING: CPU: 1 PID: 264 at arch/x86/kernel/smp.c:127 native_smp_send_reschedule+0x65/0x70 [ 1571.637324] Modules linked in: wireguard(OE) ip6_udp_tunnel udp_tunnel bonding skx_edac intel_powerclamp coretempd [ 1571.715278] CPU: 1 PID: 264 Comm: kworker/1:2 Tainted: G D OE ------------ 3.10.0-862.2.3.el7.x86_64 #1 [ 1571.726478] Hardware name: Dell Inc. PowerEdge R640/0W23H8, BIOS 1.3.7 02/08/2018 [ 1571.734820] Workqueue: wg-crypt-wg0 packet_rx_worker [wireguard] [ 1571.741700] Call Trace: [ 1571.744990] [] dump_stack+0x19/0x1b [ 1571.751604] [] __warn+0xd8/0x100 [ 1571.757319] [] warn_slowpath_null+0x1d/0x20 [ 1571.763975] [] native_smp_send_reschedule+0x65/0x70 [ 1571.771325] [] trigger_load_balance+0x191/0x280 [ 1571.778320] [] ? sched_clock_cpu+0x85/0xc0 [ 1571.784883] [] scheduler_tick+0x10a/0x150 [ 1571.791355] [] ? tick_sched_do_timer+0x50/0x50 [ 1571.798254] [] update_process_times+0x65/0x80 [ 1571.805063] [] tick_sched_handle+0x30/0x70 [ 1571.811608] [] tick_sched_timer+0x39/0x80 [ 1571.818060] [] __hrtimer_run_queues+0xd6/0x260 [ 1571.824937] [] hrtimer_interrupt+0xaf/0x1d0 [ 1571.831549] [] local_apic_timer_interrupt+0x3b/0x60 [ 1571.838863] [] smp_apic_timer_interrupt+0x43/0x60 [ 1571.846003] [] apic_timer_interrupt+0x162/0x170 [ 1571.852973] [] ? panic+0x1d8/0x21f [ 1571.859455] [] oops_end+0xd3/0xe0 [ 1571.865223] [] die+0x4b/0x70 [ 1571.870555] [] do_trap+0x60/0x170 [ 1571.876314] [] do_invalid_op+0xb4/0x130 [ 1571.882591] [] ? kfree+0x13c/0x140 [ 1571.888435] [] ? __enqueue_entity+0x78/0x80 [ 1571.895059] [] ? enqueue_entity+0x26c/0xb60 [ 1571.901676] [] ? prandom_u32+0x18/0x20 [ 1571.907858] [] ? skb_release_data+0xf5/0x140 [ 1571.914561] [] invalid_op+0x1e/0x30 [ 1571.920479] [] ? skb_release_data+0xf5/0x140 [ 1571.927164] [] ? kfree+0x13c/0x140 [ 1571.932962] [] skb_release_data+0xf5/0x140 [ 1571.939446] [] skb_release_all+0x24/0x30 [ 1571.945743] [] consume_skb+0x2c/0x90 [ 1571.951700] [] packet_rx_worker+0x2d5/0x7f0 [wireguard] [ 1571.959302] [] process_one_work+0x17f/0x440 [ 1571.965847] [] worker_thread+0x126/0x3c0 [ 1571.972117] [] ? manage_workers.isra.24+0x2a0/0x2a0 [ 1571.979327] [] kthread+0xd1/0xe0 [ 1571.984873] [] ? insert_kthread_work+0x40/0x40 [ 1571.991611] [] ret_from_fork_nospec_begin+0x7/0x21 [ 1571.998687] [] ? insert_kthread_work+0x40/0x40 [ 1572.005389] ---[ end trace 5ef54fbe37d13ebd ]--- On Thu, May 10, 2018 at 4:25 PM Jason A. Donenfeld wrote: > Hi Henry, > > Here's a transcript of things working fine with the latest master > branch: https://=D7=90.cc/GEDdGn4N > > I'm still playing to see if I can reproduce your crash. > > Jason > --=20 Henry Snow | Director, Site Reliability Engineering eXelate, a Nielsen company E: henry.snow@nielsen.com | M: 347.366.2700 675 6th Ave, 3rd Floor, New York, NY 10010 --0000000000008bc448056be0e35b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hmm...yeah, I've followed these steps:

<= div>Fresh CentOS 7.5 install
yum install git=C2=A0libmnl-devel el= futils-libelf-devel kernel-devel @development-tools
cd WireGuard/src
make
make install
deploy /etc/wireguard/wg0.conf
systemctl start wg-quick@w= g0
kernel panic.=C2=A0

Any additional de= bugging steps I can provide? This trace looks a little different, so also i= ncluding it:

[ 1571.621740] ------------[ cut= here ]------------
[ 1571.627195] WARNING: CPU: 1 PID: 264 at ar= ch/x86/kernel/smp.c:127 native_smp_send_reschedule+0x65/0x70
[ 15= 71.637324] Modules linked in: wireguard(OE) ip6_udp_tunnel udp_tunnel bondi= ng skx_edac intel_powerclamp coretempd
[ 1571.715278] CPU: 1 PID:= 264 Comm: kworker/1:2 Tainted: G=C2=A0 =C2=A0 =C2=A0 D=C2=A0 =C2=A0 OE=C2= =A0 ------------=C2=A0 =C2=A03.10.0-862.2.3.el7.x86_64 #1
[ 1571.= 726478] Hardware name: Dell Inc. PowerEdge R640/0W23H8, BIOS 1.3.7 02/08/20= 18
[ 1571.734820] Workqueue: wg-crypt-wg0 packet_rx_worker [wireg= uard]
[ 1571.741700] Call Trace:
[ 1571.744990]=C2=A0 &= lt;IRQ>=C2=A0 [<ffffffffa7d0d78e>] dump_stack+0x19/0x1b
= [ 1571.751604]=C2=A0 [<ffffffffa76916d8>] __warn+0xd8/0x100
[ 1571.757319]=C2=A0 [<ffffffffa769181d>] warn_slowpath_null+0x1d/0x= 20
[ 1571.763975]=C2=A0 [<ffffffffa7654245>] native_smp_sen= d_reschedule+0x65/0x70
[ 1571.771325]=C2=A0 [<ffffffffa76dd791= >] trigger_load_balance+0x191/0x280
[ 1571.778320]=C2=A0 [<= ffffffffa76d1f75>] ? sched_clock_cpu+0x85/0xc0
[ 1571.784883]= =C2=A0 [<ffffffffa76cd41a>] scheduler_tick+0x10a/0x150
[ 15= 71.791355]=C2=A0 [<ffffffffa7701420>] ? tick_sched_do_timer+0x50/0x50=
[ 1571.798254]=C2=A0 [<ffffffffa76a4825>] update_process_t= imes+0x65/0x80
[ 1571.805063]=C2=A0 [<ffffffffa7701220>] ti= ck_sched_handle+0x30/0x70
[ 1571.811608]=C2=A0 [<ffffffffa7701= 459>] tick_sched_timer+0x39/0x80
[ 1571.818060]=C2=A0 [<fff= fffffa76beff6>] __hrtimer_run_queues+0xd6/0x260
[ 1571.824937]= =C2=A0 [<ffffffffa76bf58f>] hrtimer_interrupt+0xaf/0x1d0
[ = 1571.831549]=C2=A0 [<ffffffffa76577fb>] local_apic_timer_interrupt+0x= 3b/0x60
[ 1571.838863]=C2=A0 [<ffffffffa7d24063>] smp_apic_= timer_interrupt+0x43/0x60
[ 1571.846003]=C2=A0 [<ffffffffa7d20= 7b2>] apic_timer_interrupt+0x162/0x170
[ 1571.852973]=C2=A0 &l= t;EOI>=C2=A0 [<ffffffffa7d07b80>] ? panic+0x1d8/0x21f
[ = 1571.859455]=C2=A0 [<ffffffffa7d17763>] oops_end+0xd3/0xe0
= [ 1571.865223]=C2=A0 [<ffffffffa762e96b>] die+0x4b/0x70
[ 1= 571.870555]=C2=A0 [<ffffffffa7d16eb0>] do_trap+0x60/0x170
[= 1571.876314]=C2=A0 [<ffffffffa762b284>] do_invalid_op+0xb4/0x130
[ 1571.882591]=C2=A0 [<ffffffffa77f6bbc>] ? kfree+0x13c/0x140<= /div>
[ 1571.888435]=C2=A0 [<ffffffffa76d3838>] ? __enqueue_entit= y+0x78/0x80
[ 1571.895059]=C2=A0 [<ffffffffa76da2fc>] ? enq= ueue_entity+0x26c/0xb60
[ 1571.901676]=C2=A0 [<ffffffffa795bb7= 8>] ? prandom_u32+0x18/0x20
[ 1571.907858]=C2=A0 [<ffffffff= a7bd60c5>] ? skb_release_data+0xf5/0x140
[ 1571.914561]=C2=A0 = [<ffffffffa7d22aee>] invalid_op+0x1e/0x30
[ 1571.920479]=C2= =A0 [<ffffffffa7bd60c5>] ? skb_release_data+0xf5/0x140
[ 15= 71.927164]=C2=A0 [<ffffffffa77f6bbc>] ? kfree+0x13c/0x140
[= 1571.932962]=C2=A0 [<ffffffffa7bd60c5>] skb_release_data+0xf5/0x140<= /div>
[ 1571.939446]=C2=A0 [<ffffffffa7bd6134>] skb_release_all+0= x24/0x30
[ 1571.945743]=C2=A0 [<ffffffffa7bd659c>] consume_= skb+0x2c/0x90
[ 1571.951700]=C2=A0 [<ffffffffc08eff35>] pac= ket_rx_worker+0x2d5/0x7f0 [wireguard]
[ 1571.959302]=C2=A0 [<f= fffffffa76b2dff>] process_one_work+0x17f/0x440
[ 1571.965847]= =C2=A0 [<ffffffffa76b3ac6>] worker_thread+0x126/0x3c0
[ 157= 1.972117]=C2=A0 [<ffffffffa76b39a0>] ? manage_workers.isra.24+0x2a0/0= x2a0
[ 1571.979327]=C2=A0 [<ffffffffa76bae31>] kthread+0xd1= /0xe0
[ 1571.984873]=C2=A0 [<ffffffffa76bad60>] ? insert_kt= hread_work+0x40/0x40
[ 1571.991611]=C2=A0 [<ffffffffa7d1f5dd&g= t;] ret_from_fork_nospec_begin+0x7/0x21
[ 1571.998687]=C2=A0 [<= ;ffffffffa76bad60>] ? insert_kthread_work+0x40/0x40
[ 1572.005= 389] ---[ end trace 5ef54fbe37d13ebd ]---


On Thu, May 10, 2018 at 4= :25 PM Jason A. Donenfeld <Jason@zx2c= 4.com> wrote:
Hi Henry,

Here's a transcript of things working fine with the latest master
branch: https://=D7=90.cc/GEDdGn4N

I'm still playing to see if I can reproduce your crash.

Jason
--
Henry Snow | Dire= ctor, Site Reliability Engineering
eXelate, a Nielsen company
E:=C2=A0henry.snow@nielsen.com=C2=A0|=C2=A0M: 347.366.2= 700
675 6th Ave, 3rd Floor, New York, NY 10010=C2=A0
--0000000000008bc448056be0e35b--