How to improve Wireguard speed?

How to improve Wireguard speed?

[Houman]

Hello,

I'm compiling the latest stable Wireguard on Debian 11.

git clone https://git.zx2c4.com/wireguard-tools
make -C wireguard-tools/src -j$(nproc)
sudo make -C wireguard-tools/src install

However the speed is not that impressive.  I have a VPS with 4 vCores
and 8 GB RAM on 1 Gbps network.  The VPS is located in London and so
am I.

With wireguard enabled I get 117 Mbps
Without wireguard I get 506 Mbps

That's a drop of 75%. What are the factors to improve the download rate?

Thank you,

Re: How to improve Wireguard speed?

[Janne Johansson]

Den ons 1 juni 2022 kl 09:49 skrev Houman <houmie@gmail.com>:
> Hello,
> I'm compiling the latest stable Wireguard on Debian 11.
> git clone https://git.zx2c4.com/wireguard-tools
> make -C wireguard-tools/src -j$(nproc)
> sudo make -C wireguard-tools/src install
>
> However the speed is not that impressive.  I have a VPS with 4 vCores
> and 8 GB RAM on 1 Gbps network.  The VPS is located in London and so
> am I.
>
> With wireguard enabled I get 117 Mbps
> Without wireguard I get 506 Mbps
>
> That's a drop of 75%. What are the factors to improve the download rate?

That seems like very poor values, I would make two tests, one local to
the VPS and one test local to the other end (which you left out of the
description) and see which one is the limiting box. People (including
me) have easily gotten speeds at 1-2 gigabit on 10GE networked
computers and VPSes and several gigabits on loopbacks, so the fault is
not simply using wireguard.

If you check the results of the nightly builds for x86_64 at
https://www.wireguard.com/build-status/ you find several iperf3 runs
that end up over 2-3 Gbit/s when a box is talking "to itself" so it's
not the crypto or the tunnel that prevents you from filling up the
gigabit link. Perhaps the VPS isn't giving you a lot of cpu? Perhaps
the test reacts poorly to the lowered MTU for using (any) tunnel in
between?

-- 
May the most significant bit of your life be positive.

Re: How to improve Wireguard speed?

[Houman]

Thanks for your reply.

I have run htop to observe the CPU consumption during this test, and
the CPU(s) go up only to 10-20%. So the CPU doesn't seem to be the
bottleneck.

I didn't change the MTU settings, but I have a suspicion about MTU. I
found this article here that makes some interesting suggestions to set
MTU to 1280: https://keremerkan.net/posts/wireguard-mtu-fixes/

And beyond that iptables -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j
TCPMSS --clamp-mss-to-pmtu





On Wed, 1 Jun 2022 at 09:51, Janne Johansson <icepic.dz@gmail.com> wrote:
>
> Den ons 1 juni 2022 kl 09:49 skrev Houman <houmie@gmail.com>:
> > Hello,
> > I'm compiling the latest stable Wireguard on Debian 11.
> > git clone https://git.zx2c4.com/wireguard-tools
> > make -C wireguard-tools/src -j$(nproc)
> > sudo make -C wireguard-tools/src install
> >
> > However the speed is not that impressive.  I have a VPS with 4 vCores
> > and 8 GB RAM on 1 Gbps network.  The VPS is located in London and so
> > am I.
> >
> > With wireguard enabled I get 117 Mbps
> > Without wireguard I get 506 Mbps
> >
> > That's a drop of 75%. What are the factors to improve the download rate?
>
> That seems like very poor values, I would make two tests, one local to
> the VPS and one test local to the other end (which you left out of the
> description) and see which one is the limiting box. People (including
> me) have easily gotten speeds at 1-2 gigabit on 10GE networked
> computers and VPSes and several gigabits on loopbacks, so the fault is
> not simply using wireguard.
>
> If you check the results of the nightly builds for x86_64 at
> https://www.wireguard.com/build-status/ you find several iperf3 runs
> that end up over 2-3 Gbit/s when a box is talking "to itself" so it's
> not the crypto or the tunnel that prevents you from filling up the
> gigabit link. Perhaps the VPS isn't giving you a lot of cpu? Perhaps
> the test reacts poorly to the lowered MTU for using (any) tunnel in
> between?
>
> --
> May the most significant bit of your life be positive.

Re: How to improve Wireguard speed?

[Roman Mamedov]

On Wed, 1 Jun 2022 10:07:31 +0100
Houman <houmie@gmail.com> wrote:

> I didn't change the MTU settings, but I have a suspicion about MTU. I
> found this article here that makes some interesting suggestions to set
> MTU to 1280: https://keremerkan.net/posts/wireguard-mtu-fixes/
> 
> And beyond that iptables -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j
> TCPMSS --clamp-mss-to-pmtu

So did you apply both of that, and what was the effect?

What are the other point that you test against, is it another VPS (better if
you could try with that), or your home connection?

It could be your home provider has different speed limits (shaping) in place
for UDP. Should be possible to test this with:

  iperf3 -s                        # on VPS

  iperf3 -u -b 500M -c <VPS IP> -R # on the other side

And then see how many "Lost/Total Datagrams" (xx %) you get. A high percentage
would indicate that the actual top speed for UDP is less than 500Mbit by this
value.

-- 
With respect,
Roman

Re: How to improve Wireguard speed?

[Houman]

Thanks Roman.

> So did you apply both of that, and what was the effect?

I will create a new environment this afternoon and test the MTU
changes mentioned earlier and investigate the outcome.

> What are the other point that you test against, is it another VPS (better if
> you could try with that), or your home connection?

The iPhone is connected via Wifi to the home network, which is 500 Mbps / fibre.
I have a code snippet on the iPhone that downloads a 1 GB test file
from my AWS bucket (London) for 10 seconds. Then measures
totalBytesWritten / time elapsed / (1024.0 * 1024.0) * 8.0.
Which is the formula for Mbps as far as I am aware.

Client (iPhone) --> server (VPS) --> S3 (AWS) = 117 Mbps
Client (iPhone) --> S3 (AWS) = 506 Mbps

I run this once the Wireguard connection is established and I get 117
Mbps. Then I disconnect the VPN and run the same code again to fetch
the test file without VPN that comes down to 506 Mbps. Client
(iPhone), server (VPS) and S3 (AWS) are all in located London, UK.

I have run your iperf test.  On the VPS the Lost/Total Datagrams is
0%. On the client (Mac) the Lost/Total Datagrams is 0.13%.  This test
proves that the ISP isn't messing around with UDP.

[  5] local 192.168.1.101 port 62103 connected to xxxxx port 5201
[ ID] Interval           Transfer     Bitrate         Jitter
Lost/Total Datagrams
[  5]   0.00-1.00   sec  59.5 MBytes   499 Mbits/sec  0.034 ms  0/44538 (0%)
[  5]   1.00-2.00   sec  59.7 MBytes   500 Mbits/sec  0.012 ms  0/44677 (0%)
[  5]   2.00-3.00   sec  59.3 MBytes   497 Mbits/sec  0.021 ms
15/44400 (0.034%)
[  5]   3.00-4.00   sec  60.0 MBytes   503 Mbits/sec  0.015 ms  0/44913 (0%)
[  5]   4.00-5.00   sec  59.5 MBytes   499 Mbits/sec  0.020 ms  0/44588 (0%)
[  5]   5.00-6.00   sec  59.3 MBytes   498 Mbits/sec  0.018 ms
219/44662 (0.49%)
[  5]   6.00-7.00   sec  59.6 MBytes   500 Mbits/sec  0.065 ms  0/44633 (0%)
[  5]   7.00-8.00   sec  59.6 MBytes   500 Mbits/sec  0.037 ms  0/44614 (0%)
[  5]   8.00-9.00   sec  59.6 MBytes   500 Mbits/sec  0.024 ms  0/44633 (0%)
[  5]   9.00-10.00  sec  59.2 MBytes   497 Mbits/sec  0.024 ms
339/44686 (0.76%)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Jitter
Lost/Total Datagrams
[  5]   0.00-10.01  sec   596 MBytes   500 Mbits/sec  0.000 ms
0/446756 (0%)  sender
[SUM]  0.0-10.0 sec  657 datagrams received out-of-order
[  5]   0.00-10.00  sec   595 MBytes   499 Mbits/sec  0.024 ms
573/446344 (0.13%)  receiver


For now I'm out of ideas. I will try to play around with MTUs this
afternoon and see what happens.
Thanks,





> It could be your home provider has different speed limits (shaping) in place
> for UDP. Should be possible to test this with:
>
>   iperf3 -s                        # on VPS
>
>   iperf3 -u -b 500M -c <VPS IP> -R # on the other side
>
> And then see how many "Lost/Total Datagrams" (xx %) you get. A high percentage
> would indicate that the actual top speed for UDP is less than 500Mbit by this
> value.
>
> --
> With respect,
> Roman

Re: How to improve Wireguard speed?

[Frank Carmickle]

It's almost certainly the iPhone that's the slow part. The wireguard implementation on iOS is running in userspace.

--FC

> On Jun 1, 2022, at 7:40 AM, Houman <houmie@gmail.com> wrote:
> 
> Thanks Roman.
> 
>> So did you apply both of that, and what was the effect?
> 
> I will create a new environment this afternoon and test the MTU
> changes mentioned earlier and investigate the outcome.
> 
>> What are the other point that you test against, is it another VPS (better if
>> you could try with that), or your home connection?
> 
> The iPhone is connected via Wifi to the home network, which is 500 Mbps / fibre.
> I have a code snippet on the iPhone that downloads a 1 GB test file
> from my AWS bucket (London) for 10 seconds. Then measures
> totalBytesWritten / time elapsed / (1024.0 * 1024.0) * 8.0.
> Which is the formula for Mbps as far as I am aware.
> 
> Client (iPhone) --> server (VPS) --> S3 (AWS) = 117 Mbps
> Client (iPhone) --> S3 (AWS) = 506 Mbps
> 
> I run this once the Wireguard connection is established and I get 117
> Mbps. Then I disconnect the VPN and run the same code again to fetch
> the test file without VPN that comes down to 506 Mbps. Client
> (iPhone), server (VPS) and S3 (AWS) are all in located London, UK.
> 
> I have run your iperf test.  On the VPS the Lost/Total Datagrams is
> 0%. On the client (Mac) the Lost/Total Datagrams is 0.13%.  This test
> proves that the ISP isn't messing around with UDP.
> 
> [  5] local 192.168.1.101 port 62103 connected to xxxxx port 5201
> [ ID] Interval           Transfer     Bitrate         Jitter
> Lost/Total Datagrams
> [  5]   0.00-1.00   sec  59.5 MBytes   499 Mbits/sec  0.034 ms  0/44538 (0%)
> [  5]   1.00-2.00   sec  59.7 MBytes   500 Mbits/sec  0.012 ms  0/44677 (0%)
> [  5]   2.00-3.00   sec  59.3 MBytes   497 Mbits/sec  0.021 ms
> 15/44400 (0.034%)
> [  5]   3.00-4.00   sec  60.0 MBytes   503 Mbits/sec  0.015 ms  0/44913 (0%)
> [  5]   4.00-5.00   sec  59.5 MBytes   499 Mbits/sec  0.020 ms  0/44588 (0%)
> [  5]   5.00-6.00   sec  59.3 MBytes   498 Mbits/sec  0.018 ms
> 219/44662 (0.49%)
> [  5]   6.00-7.00   sec  59.6 MBytes   500 Mbits/sec  0.065 ms  0/44633 (0%)
> [  5]   7.00-8.00   sec  59.6 MBytes   500 Mbits/sec  0.037 ms  0/44614 (0%)
> [  5]   8.00-9.00   sec  59.6 MBytes   500 Mbits/sec  0.024 ms  0/44633 (0%)
> [  5]   9.00-10.00  sec  59.2 MBytes   497 Mbits/sec  0.024 ms
> 339/44686 (0.76%)
> - - - - - - - - - - - - - - - - - - - - - - - - -
> [ ID] Interval           Transfer     Bitrate         Jitter
> Lost/Total Datagrams
> [  5]   0.00-10.01  sec   596 MBytes   500 Mbits/sec  0.000 ms
> 0/446756 (0%)  sender
> [SUM]  0.0-10.0 sec  657 datagrams received out-of-order
> [  5]   0.00-10.00  sec   595 MBytes   499 Mbits/sec  0.024 ms
> 573/446344 (0.13%)  receiver
> 
> 
> For now I'm out of ideas. I will try to play around with MTUs this
> afternoon and see what happens.
> Thanks,
> 
> 
> 
> 
> 
>> It could be your home provider has different speed limits (shaping) in place
>> for UDP. Should be possible to test this with:
>> 
>>  iperf3 -s                        # on VPS
>> 
>>  iperf3 -u -b 500M -c <VPS IP> -R # on the other side
>> 
>> And then see how many "Lost/Total Datagrams" (xx %) you get. A high percentage
>> would indicate that the actual top speed for UDP is less than 500Mbit by this
>> value.
>> 
>> --
>> With respect,
>> Roman