Development discussion of WireGuard
 help / color / mirror / Atom feed
* WireGuard is broken on iOS 15 beta
@ 2021-06-10 11:44 Andrej Mihajlov
  2021-06-10 11:51 ` Jason A. Donenfeld
  0 siblings, 1 reply; 11+ messages in thread
From: Andrej Mihajlov @ 2021-06-10 11:44 UTC (permalink / raw)
  To: WireGuard mailing list

Hi,

Just though to give you heads up. iOS 15 beta is out and it looks like the old way of obtaining the tunnel device descriptor is no longer working as the internal structure of NEPacketFlow has changed. It’s been a ticking bomb form the start as with any private API that is subject to change without a warning. 

In the short term, researching what Apple changed and looking for a quick fix for that. Longer term I believe we should have a better integration between wireguard-go and Network extension on iOS.

Cheers,
Andrej Mihajlov

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: WireGuard is broken on iOS 15 beta
  2021-06-10 11:44 WireGuard is broken on iOS 15 beta Andrej Mihajlov
@ 2021-06-10 11:51 ` Jason A. Donenfeld
  2021-06-10 15:39   ` Jason A. Donenfeld
  2021-06-16 13:25   ` Andrej Mihajlov
  0 siblings, 2 replies; 11+ messages in thread
From: Jason A. Donenfeld @ 2021-06-10 11:51 UTC (permalink / raw)
  To: Andrej Mihajlov; +Cc: WireGuard mailing list

Patches with a proper performance comparison of the two techniques in
the description would be appreciated.

Thanks,
Jason

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: WireGuard is broken on iOS 15 beta
  2021-06-10 11:51 ` Jason A. Donenfeld
@ 2021-06-10 15:39   ` Jason A. Donenfeld
  2021-06-10 22:33     ` David Crawshaw
  2021-06-16 13:25   ` Andrej Mihajlov
  1 sibling, 1 reply; 11+ messages in thread
From: Jason A. Donenfeld @ 2021-06-10 15:39 UTC (permalink / raw)
  To: Andrej Mihajlov; +Cc: WireGuard mailing list

I won't have an updated OS to test this out in until next week at the
earliest, but perhaps this hack will work?
https://git.zx2c4.com/wireguard-apple/commit/?h=jd/fd-search-hack
Let me know if that is successful, or if it blows up.

Jason

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: WireGuard is broken on iOS 15 beta
  2021-06-10 15:39   ` Jason A. Donenfeld
@ 2021-06-10 22:33     ` David Crawshaw
  2021-06-11  7:16       ` Andrej Mihajlov
  0 siblings, 1 reply; 11+ messages in thread
From: David Crawshaw @ 2021-06-10 22:33 UTC (permalink / raw)
  To: Jason A. Donenfeld; +Cc: WireGuard mailing list

I finally managed to get a device running iPadOS 15 and an Xcode 13
beta. This technique works. Some modified code with logging:

func tunnelFD(provider: NEPacketTunnelProvider) -> Int32 {
    os_log("tunnelFD searching")
    var buf = [CChar](repeating: 0, count: Int(IFNAMSIZ))
    for fd: Int32 in 4...64 {
        var len = socklen_t(buf.count)
        if getsockopt(fd, 2 /* SYSPROTO_CONTROL */, 2, &buf, &len) == 0 {
            let str = String(cString: buf)
            os_log("%{public}@", "tunnelFD \(fd): \(str)")
            if str.starts(with: "utun") {
                os_log("tunnelFD found likely fd")
                return fd
            }
        }
    }
    os_log("tunnelFD found nothing")
    return -1
}

produces the logs:

    tunnelFD searching
    tunnelFD 7: utun3
    tunnelFD found likely fd

Note that the current technique of calling
provider.packetFlow.value(forKeyPath: "socket.fileDescriptor") causes
the NetworkExtension to crash under iOS 15. The logs say the kernel
sandbox killed it:

    Sandbox: process(525) deny(2) file-test-existence /private/etc/.mdns_debug

On Thu, Jun 10, 2021 at 8:44 AM Jason A. Donenfeld <Jason@zx2c4.com> wrote:
>
> I won't have an updated OS to test this out in until next week at the
> earliest, but perhaps this hack will work?
> https://git.zx2c4.com/wireguard-apple/commit/?h=jd/fd-search-hack
> Let me know if that is successful, or if it blows up.
>
> Jason

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: WireGuard is broken on iOS 15 beta
  2021-06-10 22:33     ` David Crawshaw
@ 2021-06-11  7:16       ` Andrej Mihajlov
  2021-06-11  7:53         ` Jason A. Donenfeld
  0 siblings, 1 reply; 11+ messages in thread
From: Andrej Mihajlov @ 2021-06-11  7:16 UTC (permalink / raw)
  To: David Crawshaw; +Cc: Jason A. Donenfeld, WireGuard mailing list

IIRC one thing to consider with that lookup: iOS enables users to run Personal VPN and Custom VPN (aka WireGuard) side-by-side so there is a chance you may pick the wrong utun.

> On 11 Jun 2021, at 00:33, David Crawshaw <crawshaw@tailscale.com> wrote:
> 
> I finally managed to get a device running iPadOS 15 and an Xcode 13
> beta. This technique works. Some modified code with logging:
> 
> func tunnelFD(provider: NEPacketTunnelProvider) -> Int32 {
>    os_log("tunnelFD searching")
>    var buf = [CChar](repeating: 0, count: Int(IFNAMSIZ))
>    for fd: Int32 in 4...64 {
>        var len = socklen_t(buf.count)
>        if getsockopt(fd, 2 /* SYSPROTO_CONTROL */, 2, &buf, &len) == 0 {
>            let str = String(cString: buf)
>            os_log("%{public}@", "tunnelFD \(fd): \(str)")
>            if str.starts(with: "utun") {
>                os_log("tunnelFD found likely fd")
>                return fd
>            }
>        }
>    }
>    os_log("tunnelFD found nothing")
>    return -1
> }
> 
> produces the logs:
> 
>    tunnelFD searching
>    tunnelFD 7: utun3
>    tunnelFD found likely fd
> 
> Note that the current technique of calling
> provider.packetFlow.value(forKeyPath: "socket.fileDescriptor") causes
> the NetworkExtension to crash under iOS 15. The logs say the kernel
> sandbox killed it:
> 
>    Sandbox: process(525) deny(2) file-test-existence /private/etc/.mdns_debug
> 
> On Thu, Jun 10, 2021 at 8:44 AM Jason A. Donenfeld <Jason@zx2c4.com> wrote:
>> 
>> I won't have an updated OS to test this out in until next week at the
>> earliest, but perhaps this hack will work?
>> https://git.zx2c4.com/wireguard-apple/commit/?h=jd/fd-search-hack
>> Let me know if that is successful, or if it blows up.
>> 
>> Jason


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: WireGuard is broken on iOS 15 beta
  2021-06-11  7:16       ` Andrej Mihajlov
@ 2021-06-11  7:53         ` Jason A. Donenfeld
  2021-06-11  8:42           ` Andrej Mihajlov
  0 siblings, 1 reply; 11+ messages in thread
From: Jason A. Donenfeld @ 2021-06-11  7:53 UTC (permalink / raw)
  To: Andrej Mihajlov; +Cc: David Crawshaw, WireGuard mailing list

On 6/11/21, Andrej Mihajlov <and@mullvad.net> wrote:
> IIRC one thing to consider with that lookup: iOS enables users to run
> Personal VPN and Custom VPN (aka WireGuard) side-by-side so there is a
> chance you may pick the wrong utun.

That doesn't make any sense. File descriptors are not OS-global;
they're process-local. That's how Unix FDs have worked since forever.
Unless you're suggesting "personal VPN" is somehow resident in the
same network extension process as WireGuard's "custom VPN"?

By the way, did the experiment in your branch work? I'd prefer a
direct route to brute forcing FDs, if possible. But if not, seems like
my kludge might ultimately do the trick.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: WireGuard is broken on iOS 15 beta
  2021-06-11  7:53         ` Jason A. Donenfeld
@ 2021-06-11  8:42           ` Andrej Mihajlov
  2021-06-11  8:50             ` Jason A. Donenfeld
  0 siblings, 1 reply; 11+ messages in thread
From: Andrej Mihajlov @ 2021-06-11  8:42 UTC (permalink / raw)
  To: Jason A. Donenfeld; +Cc: David Crawshaw, WireGuard mailing list

Jason,

My bad, you’re right regarding the Personal VPN.

I just ran your patch and it works great. As David pointed out, the call to valueForKeyPath should be guarded because it throws exception if the given key path does not exist. I use the availability check to bruteforce utun on iOS 15, macOS 12 and onwards while keeping the Key value coding approach on older iOS and macOS as we know that it works great on iOS < 15 and macOS < 12. Just ran the app on macOS 11.4 and it’s still working and using the old code path.

I took a liberty to refactor the proposed solution (see: https://git.zx2c4.com/wireguard-apple/commit/?id=a7ccc8e3031f3502ea4b53a914d37589186e40f8)

Cheers,
Andrej

> On 11 Jun 2021, at 09:53, Jason A. Donenfeld <Jason@zx2c4.com> wrote:
> 
> On 6/11/21, Andrej Mihajlov <and@mullvad.net> wrote:
>> IIRC one thing to consider with that lookup: iOS enables users to run
>> Personal VPN and Custom VPN (aka WireGuard) side-by-side so there is a
>> chance you may pick the wrong utun.
> 
> That doesn't make any sense. File descriptors are not OS-global;
> they're process-local. That's how Unix FDs have worked since forever.
> Unless you're suggesting "personal VPN" is somehow resident in the
> same network extension process as WireGuard's "custom VPN"?
> 
> By the way, did the experiment in your branch work? I'd prefer a
> direct route to brute forcing FDs, if possible. But if not, seems like
> my kludge might ultimately do the trick.


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: WireGuard is broken on iOS 15 beta
  2021-06-11  8:42           ` Andrej Mihajlov
@ 2021-06-11  8:50             ` Jason A. Donenfeld
  2021-06-11  9:02               ` Andrej Mihajlov
  0 siblings, 1 reply; 11+ messages in thread
From: Jason A. Donenfeld @ 2021-06-11  8:50 UTC (permalink / raw)
  To: Andrej Mihajlov; +Cc: David Crawshaw, WireGuard mailing list

Can I infer from that that your technique didn't work? Some
confirmation one way or another there would be useful, so I know
whether to dedicate some RE time next week to it or not. If I can
avoid the fd iteration, I'd prefer that.

Jason

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: WireGuard is broken on iOS 15 beta
  2021-06-11  8:50             ` Jason A. Donenfeld
@ 2021-06-11  9:02               ` Andrej Mihajlov
  2021-06-11  9:09                 ` Jason A. Donenfeld
  0 siblings, 1 reply; 11+ messages in thread
From: Andrej Mihajlov @ 2021-06-11  9:02 UTC (permalink / raw)
  To: Jason A. Donenfeld; +Cc: David Crawshaw, WireGuard mailing list

My approach worked too. However it relies on the private interfaces of NetworkExtension framework, which we’d probably want to avoid.

In short it can be expressed in one line as:

let fd = packetFlow.perform(Selector("interface")).toOpaque().advanced(by: 0x128).assumingMemoryBound(to: Int32.self).pointee

But I bet that walking over fds is much safer.

Cheers,
Andrej

> On 11 Jun 2021, at 10:50, Jason A. Donenfeld <Jason@zx2c4.com> wrote:
> 
> Can I infer from that that your technique didn't work? Some
> confirmation one way or another there would be useful, so I know
> whether to dedicate some RE time next week to it or not. If I can
> avoid the fd iteration, I'd prefer that.
> 
> Jason


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: WireGuard is broken on iOS 15 beta
  2021-06-11  9:02               ` Andrej Mihajlov
@ 2021-06-11  9:09                 ` Jason A. Donenfeld
  0 siblings, 0 replies; 11+ messages in thread
From: Jason A. Donenfeld @ 2021-06-11  9:09 UTC (permalink / raw)
  To: Andrej Mihajlov; +Cc: David Crawshaw, WireGuard mailing list

On 6/11/21, Andrej Mihajlov <and@mullvad.net> wrote:
> let fd = packetFlow.perform(Selector("interface")).toOpaque().advanced(by:
> 0x128).assumingMemoryBound(to: Int32.self).pointee

Ahh, so that's some sort of 296 byte struct offset, rather than a
symbolic name that fails gracefully. Ooof. Okay I'll poke around for
other methods, I suppose, as that indeed seems potentially dangerous.
Thanks nonetheless for the trick - helpful for getting started.

Jason

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: WireGuard is broken on iOS 15 beta
  2021-06-10 11:51 ` Jason A. Donenfeld
  2021-06-10 15:39   ` Jason A. Donenfeld
@ 2021-06-16 13:25   ` Andrej Mihajlov
  1 sibling, 0 replies; 11+ messages in thread
From: Andrej Mihajlov @ 2021-06-16 13:25 UTC (permalink / raw)
  To: Jason A. Donenfeld; +Cc: WireGuard mailing list

It was less about performance but more about security. Just wanted to share some good news!

A little bit of background story:

There was an article a little while ago that highlighted a traffic leak, when connections established before the VPN would continue going through the primary interface outside of the tunnel: 
https://protonvpn.com/blog/apple-ios-vulnerability-disclosure/

Then at some point around iOS 14.2, Apple introduced a kill-switch to address that issue (via VPN configuration flag). 

However it never really worked with WireGuard as the tunnel would always fail to establish the connection with peers. There is a thread on Apple forums with some of Apple engineers trying to figure out what’s going on:
https://developer.apple.com/forums/thread/653116


Now it’s too early to say, but I just ran WireGuard with kill switch enabled on iOS 15 beta and the tunnel is magically up and running, which was still broken on iOS 14.3. I am going to test it on iOS 14.4 today too. So it looks like Apple did something to fix their SDK to make the kill switch work.

Potential low effort improvement for WG, in practice it looks as such:

let protocolConfig = NETunnelProviderProtocol()
if #available(iOS 15.0, *) {
    protocolConfig.includeAllNetworks = true // kill-switch
    protocolConfig.excludeLocalNetworks = true // enable LAN networking
}

Cheers,
Andrej

> On 10 Jun 2021, at 13:51, Jason A. Donenfeld <Jason@zx2c4.com> wrote:
> 
> Patches with a proper performance comparison of the two techniques in
> the description would be appreciated.
> 
> Thanks,
> Jason


^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2021-06-18 15:24 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-10 11:44 WireGuard is broken on iOS 15 beta Andrej Mihajlov
2021-06-10 11:51 ` Jason A. Donenfeld
2021-06-10 15:39   ` Jason A. Donenfeld
2021-06-10 22:33     ` David Crawshaw
2021-06-11  7:16       ` Andrej Mihajlov
2021-06-11  7:53         ` Jason A. Donenfeld
2021-06-11  8:42           ` Andrej Mihajlov
2021-06-11  8:50             ` Jason A. Donenfeld
2021-06-11  9:02               ` Andrej Mihajlov
2021-06-11  9:09                 ` Jason A. Donenfeld
2021-06-16 13:25   ` Andrej Mihajlov

Development discussion of WireGuard

This inbox may be cloned and mirrored by anyone:

	git clone --mirror http://inbox.vuxu.org/wireguard/0 wireguard/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 wireguard wireguard/ http://inbox.vuxu.org/wireguard \
		wireguard@lists.zx2c4.com
	public-inbox-index wireguard

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.vuxu.org/vuxu.archive.wireguard


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git