From: Lonnie Abelbeck <lists@lonnie.abelbeck.com>
To: Mo Balaa <buddybalaa@gmail.com>
Cc: WireGuard mailing list <wireguard@lists.zx2c4.com>
Subject: Re: T-Mobile 4G/5G CGNAT vs WireGuard tunnel jitter
Date: Sat, 10 Apr 2021 11:03:38 -0500 [thread overview]
Message-ID: <DBC8E469-C5EC-43C2-823B-09CC876CC3A5@lonnie.abelbeck.com> (raw)
In-Reply-To: <378472A1-D6C5-428C-96FE-AC05897826C8@gmail.com>
Hi Mo,
I have the latest hardware (gray cylinder)
--
Typically 3 of 5 bars
4G Band: B2
5G Band: n71
--
Note the huge jitter only occurs inbound, which is difficult to measure without using WireGuard.
The same UDP iperf3 tests outbound (gw-lan->linode) range from 3 to 8 ms jitter, regardless of bitrate.
--
Speed ( less than 2 seconds) 10/15 Mbps (down/up)
slowly ramps up to ...
Speed ( after 20 seconds) up to 200/35 Mbps (down/up)
--
Lonnie
> On Apr 10, 2021, at 10:43 AM, Mo Balaa <buddybalaa@gmail.com> wrote:
>
> Thanks for sharing, I have also been running WG tunnels over T-Mobile home internet and haven’t seen any of the jitter you are reporting.
>
> Did you try the same tests (outbound) without running them via WG?
> Which modem do you have? How many signal bars are you getting? Also, what does an non-tunneled speed test report?
>
> Cheers
>
>
>> On Apr 10, 2021, at 10:31, Lonnie Abelbeck <lists@lonnie.abelbeck.com> wrote:
>>
>> Greetings,
>>
>> I have been testing the T-Mobile Home Internet (4G/5G fixed wireless) service to a Linode VM via WireGuard.
>>
>> The TMHI service uses CGNAT plus an additional NAT in their modem/gateway with a MTU of 1420, so WireGuard is configured with a 1340 MTU.
>>
>> Everything works, but I thought I would share some jitter results that readers here might find interesting.
>>
>> [gw-lan WGIP:10.4.1.1] -- [TMHI modem/gateway] -- 4G/5G/CGNAT -- [linode WGIP:10.4.1.10]
>>
>> gw-lan ~ # mtr -wn -c 30 -s 1340 10.4.1.10
>> ...
>> HOST: gw-lan Loss% Snt Last Avg Best Wrst StDev
>> 1.|-- 10.4.1.10 0.0% 30 88.7 88.9 77.2 99.2 5.4
>>
>> Looks to be as expected, in the direction of the CGNAT, now the other direction, against the grain of the CGNAT ...
>>
>> linode ~ # mtr -wn -c 30 -s 1340 10.4.1.1
>> ...
>> HOST: linode Loss% Snt Last Avg Best Wrst StDev
>> 1.|-- 10.4.1.1 0.0% 30 206.1 243.5 73.8 393.9 97.9
>>
>> Huge jitter, and is very reproducible. But no packet loss.
>>
>> Further investigation shows for low traffic rates (linode->gw-lan) the jitter over WireGuard is huge, here are some UDP iperf3 tests showing how the jitter goes down as the traffic rate is increased.
>>
>> linode ~ # iperf3 -c 10.4.1.1 -u -b 5k -t 30
>> ...
>> [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
>> [ 5] 0.00-30.25 sec 18.9 KBytes 5.11 Kbits/sec 68.428 ms 0/15 (0%) receiver
>>
>> linode ~ # iperf3 -c 10.4.1.1 -u -b 10k -t 30
>> ...
>> [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
>> [ 5] 0.00-30.30 sec 37.7 KBytes 10.2 Kbits/sec 82.411 ms 0/30 (0%) receiver
>>
>> linode ~ # iperf3 -c 10.4.1.1 -u -b 50k -t 30
>> ...
>> [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
>> [ 5] 0.00-30.14 sec 184 KBytes 49.9 Kbits/sec 7.532 ms 0/146 (0%) receiver
>>
>> linode ~ # iperf3 -c 10.4.1.1 -u -b 100k -t 30
>> ...
>> [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
>> [ 5] 0.00-30.10 sec 367 KBytes 100 Kbits/sec 4.182 ms 0/292 (0%) receiver
>>
>> linode ~ # iperf3 -c 10.4.1.1 -u -b 500k -t 30
>> ...
>> [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
>> [ 5] 0.00-30.11 sec 1.79 MBytes 498 Kbits/sec 1.308 ms 0/1456 (0%) receiver
>>
>>
>> So using VoIP a higher bitrate CODEC is actually better w.r.t jitter.
>>
>> Hope others find this interesting.
>>
>> Lonnie
>>
>
>
next prev parent reply other threads:[~2021-04-10 16:06 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-10 15:27 Lonnie Abelbeck
2021-04-10 15:43 ` Mo Balaa
2021-04-10 16:03 ` Lonnie Abelbeck [this message]
2021-04-10 15:59 ` Roman Mamedov
2021-04-10 16:12 ` Lonnie Abelbeck
2021-04-16 13:56 ` Lonnie Abelbeck
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=DBC8E469-C5EC-43C2-823B-09CC876CC3A5@lonnie.abelbeck.com \
--to=lists@lonnie.abelbeck.com \
--cc=buddybalaa@gmail.com \
--cc=wireguard@lists.zx2c4.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).