Development discussion of WireGuard
 help / color / mirror / Atom feed
From: Xiaozhou Liu <liuxiaozhou@bytedance.com>
To: "Jason A. Donenfeld" <Jason@zx2c4.com>
Cc: duanxiongchun@bytedance.com, wangdongdong.6@bytedance.com,
	zhangyongsu@bytedance.com, wireguard@lists.zx2c4.com,
	wangjian@bytedance.com
Subject: Single CPU core bottleneck caused by high site-to-site traffic
Date: Sat, 2 Mar 2019 12:24:21 +0800	[thread overview]
Message-ID: <20190302042419.gv3ldcooxzbf4veq@bytedancedeMacBook-Air.local> (raw)
In-Reply-To: <CAHmME9rQ-cKSV8o-sg3fJU4Uu6r5daC=Hx4mC-4OKGA8Z8hg7w@mail.gmail.com>

Hi Jason and the list,

Here at our corporate network we run some inner site-to-site VPNs using
WireGuard. Thanks for giving out such a beautiful software to the world.

Recently we encountered some noticeable network latency during peak traffic
time. Although the traffic is pretty huge, the WireGuard box is far from
running out of any of its resources: CPU, memory, network bandwidth, etc.

It turns out that the bottleneck is caused by the single UDP connection
between the sites, which cannot be routed to different CPU cores by RSS
on receiving. The total CPU usage is not high, but one of the cores can
reach 100%.

Maybe we can improve this by:

  embedding more endpoints in one peer so that the VPN tunnel can run
  multiple UDP flows instead of one. Hence, the single huge UDP flow is
  effectively broken down to some smaller ones which can be received by
  multiple queues of the NIC and then later processed by more CPU cores.
  It will not break current users because the single UDP connection is
  still provided as the default configuration.

It is also possible to set up multiple wg interfaces and more connections
explicitly. But it would make the network administration much more complex.

We are planning to make a working demo of this idea but we would like to
hear from you first.

Any idea or comment is appreciated.


Thanks,
Xiaozhou
_______________________________________________
WireGuard mailing list
WireGuard@lists.zx2c4.com
https://lists.zx2c4.com/mailman/listinfo/wireguard

  reply	other threads:[~2019-03-08 15:26 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-02-28 19:05 [PATCH 1/2] build/tools: Add support for Haiku Alexander von Gluck IV
2019-02-28 19:05 ` [PATCH 2/2] genkey: v2. Be more aggressive in the search for entropy Alexander von Gluck IV
2019-03-01 19:54 ` [PATCH 1/2] build/tools: Add support for Haiku Jason A. Donenfeld
2019-03-02  4:24   ` Xiaozhou Liu [this message]
2019-03-09  0:16     ` Single CPU core bottleneck caused by high site-to-site traffic Samuel Holland

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190302042419.gv3ldcooxzbf4veq@bytedancedeMacBook-Air.local \
    --to=liuxiaozhou@bytedance.com \
    --cc=Jason@zx2c4.com \
    --cc=duanxiongchun@bytedance.com \
    --cc=wangdongdong.6@bytedance.com \
    --cc=wangjian@bytedance.com \
    --cc=wireguard@lists.zx2c4.com \
    --cc=zhangyongsu@bytedance.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).