From: Rumen Telbizov <rumen.telbizov@menlosecurity.com>
To: Roman Mamedov <rm@romanrm.net>
Cc: wireguard@lists.zx2c4.com
Subject: Re: WireGuard IRQ distribution
Date: Mon, 26 Jun 2023 09:18:17 -0700 [thread overview]
Message-ID: <CA+FoirAsRV2LuEGygz3i4mAWBOAgqWDa6fqcjA8OeDNCZG+a+g@mail.gmail.com> (raw)
In-Reply-To: <20230626195311.03e86a19@nvm>
Hi Roman,
I ran UDP-based iperf and it made no difference.
Thanks for your input
On Mon, Jun 26, 2023 at 7:53 AM Roman Mamedov <rm@romanrm.net> wrote:
>
> On Tue, 9 May 2023 15:17:00 -0700
> Rumen Telbizov <rumen.telbizov@menlosecurity.com> wrote:
>
> > Baseline iperf3 performance over plain VLAN:
> > * Stable 24Gbit/s and 2Mpps
> >
> > bmon:
> > Gb (RX Bits/second)
> > 24.54 .........|.||..|.||.||.||||||..||.||.......................
> > 20.45 .........|||||||||||||||||||||||||||||.....................
> > 16.36 ........||||||||||||||||||||||||||||||.....................
> > 12.27 ........||||||||||||||||||||||||||||||.....................
> > 8.18 ........|||||||||||||||||||||||||||||||.....................
> > 4.09 ::::::::|||||||||||||||||||||||||||||||:::::::::::::::::::::
> > 1 5 10 15 20 25 30 35 40 45 50 55 60
> > M (RX Packets/second)
> > 2.03 .........|.||..|.||.||.||||||..||.||........................
> > 1.69 .........|||||||||||||||||||||||||||||......................
> > 1.35 ........||||||||||||||||||||||||||||||......................
> > 1.01 ........||||||||||||||||||||||||||||||......................
> > 0.68 ........|||||||||||||||||||||||||||||||.....................
> > 0.34 ::::::::|||||||||||||||||||||||||||||||:::::::::::::::::::::
> > 1 5 10 15 20 25 30 35 40 45 50 55 60
> >
> > top:
> > %Cpu0 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
> > %Cpu1 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
> > %Cpu2 : 1.0 us, 1.0 sy, 0.0 ni, 98.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
> > %Cpu3 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
> > %Cpu4 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
> > %Cpu5 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
> > %Cpu6 : 1.0 us, 0.0 sy, 0.0 ni, 99.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
> > %Cpu7 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
> > %Cpu8 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
> > %Cpu9 : 1.0 us, 1.0 sy, 0.0 ni, 98.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
> > %Cpu10 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
> > %Cpu11 : 0.0 us, 0.9 sy, 0.0 ni, 16.8 id, 0.0 wa, 0.0 hi, 82.2 si, 0.0 st
> > %Cpu12 : 0.0 us, 32.3 sy, 0.0 ni, 65.6 id, 0.0 wa, 0.0 hi, 2.1 si, 0.0 st
> > %Cpu13 : 1.0 us, 36.3 sy, 0.0 ni, 59.8 id, 0.0 wa, 0.0 hi, 2.9 si, 0.0 st
> > %Cpu14 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
> > %Cpu15 : 0.0 us, 1.0 sy, 0.0 ni, 99.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
> >
> > The IRQs do pile up behind CPU 11 because iperf3 is single-threaded.
>
> I'm not sure if they pile up because of that, or because of the same reason
> you point in WG's case, the 5-tuple being the same for the single TCP
> connection of iperf3.
>
> Out of interest, maybe you could try iperf3's UDP mode, and apply the same
> port randomization trick as you used for WG, and see if it also makes it see
> the better IRQ distribution, and the speed drop?
>
> --
> With respect,
> Roman
prev parent reply other threads:[~2023-06-26 16:21 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-05-09 22:17 Rumen Telbizov
2023-06-26 14:53 ` Roman Mamedov
2023-06-26 16:18 ` Rumen Telbizov [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CA+FoirAsRV2LuEGygz3i4mAWBOAgqWDa6fqcjA8OeDNCZG+a+g@mail.gmail.com \
--to=rumen.telbizov@menlosecurity.com \
--cc=rm@romanrm.net \
--cc=wireguard@lists.zx2c4.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).