[WireGuard] Using wireguard link as a proxy?

[WireGuard] Using wireguard link as a proxy?

[Bruno Wolff III]

I am trying to test using a wireguard link as a proxy and I am having trouble. 
On the source machine I am trying to route packets through wg0 using a 
routing entry that has the remote end point tunnel address on a via command. 
The idea is to have the packets sent through the tunnel and then to do 
source nat on the remote side of the tunnel. The packets go to the interface, 
but do not come out the other end. Ping reports:
>From 192.168.7.2 icmp_seq=69 Destination Host Unreachable
ping: sendmsg: Required key not available

So I think the destination address in the packet is being used to find the 
tunnel key to use rather than the immediate next destination for the packet.

Has anyone else played with something like this an gotten it to work? 

Some info on what I was trying:
ip route
default via 129.89.248.129 dev em1  proto static  metric 100 
98.103.208.26 via 192.168.7.1 dev wg0 
129.89.248.128/27 dev em1  proto kernel  scope link  src 129.89.248.147  metric 100 
192.168.7.1 dev wg0  proto kernel  scope link  src 192.168.7.2 

wg source machine
  endpoint: 98.103.208.27:992
  allowed ips: 192.168.7.1/32

wg proxy machine
  endpoint: 129.89.248.147:992
  allowed ips: 192.168.7.2/32

tcpdump output from wg0
15:54:34.129798 IP 192.168.7.2 > 98.103.208.26: ICMP echo request, id 25371, seq 1, length 64

Re: [WireGuard] Using wireguard link as a proxy?

[Baptiste Jonglez]

[-- Attachment #1: Type: text/plain, Size: 2273 bytes --]

Hi,

On Thu, Jul 21, 2016 at 03:57:42PM -0500, Bruno Wolff III wrote:
> I am trying to test using a wireguard link as a proxy and I am having
> trouble. On the source machine I am trying to route packets through wg0
> using a routing entry that has the remote end point tunnel address on a via
> command. The idea is to have the packets sent through the tunnel and then to
> do source nat on the remote side of the tunnel. The packets go to the
> interface, but do not come out the other end. Ping reports:
> From 192.168.7.2 icmp_seq=69 Destination Host Unreachable
> ping: sendmsg: Required key not available
> 
> So I think the destination address in the packet is being used to find the
> tunnel key to use rather than the immediate next destination for the packet.

Yes, the notion of "immediate next destinaton" does not make sense for
Wireguard.  It encapsulates plain IP, not Ethernet.

> Has anyone else played with something like this an gotten it to work?
> 
> Some info on what I was trying:
> ip route
> default via 129.89.248.129 dev em1  proto static  metric 100
> 98.103.208.26 via 192.168.7.1 dev wg0

The "via 192.168.7.1" doesn't really make sense, because a wireguard
interface is a layer-3 device, not an Ethernet device.  You can include it
but the kernel will just ignore that part, so the route above is equivalent to:

  98.103.208.26 dev wg0

> 129.89.248.128/27 dev em1  proto kernel  scope link src 129.89.248.147 metric 100
> 192.168.7.1 dev wg0  proto kernel  scope link src 192.168.7.2
> 
> wg source machine
>  endpoint: 98.103.208.27:992
>  allowed ips: 192.168.7.1/32

You need "allowed ips 0.0.0.0/0" here.  Your situation is just a regular
client/server tunneling setup, there's nothing special about "proxying",
whatever that means.

Basically, you want your client to be able to reach the whole Internet via
the wireguard interface, so any destination IP address should be routed
through the server (and conversely, any source IP address should be
allowed from the server).

> wg proxy machine
>  endpoint: 129.89.248.147:992
>  allowed ips: 192.168.7.2/32
> 
> tcpdump output from wg0
> 15:54:34.129798 IP 192.168.7.2 > 98.103.208.26: ICMP echo request, id 25371, seq 1, length 64

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [WireGuard] Using wireguard link as a proxy?

[Bruno Wolff III]

On Fri, Jul 22, 2016 at 10:18:21 +0200,
  Baptiste Jonglez <baptiste@bitsofnetworks.org> wrote:
>
>Yes, the notion of "immediate next destinaton" does not make sense for
>Wireguard.  It encapsulates plain IP, not Ethernet.

I thought that the next IP address might have been available for wireguard 
to see as the information seems to be available for routing. But as you 
mention below and I realized, that doesn't help with the return packets 
since they can have (almost) any source address.

>You need "allowed ips 0.0.0.0/0" here.  Your situation is just a regular
>client/server tunneling setup, there's nothing special about "proxying",
>whatever that means.

Yeah I realized that when thinking about this some more. "Proxy" in this 
case means source nat will be used on the outgoing packets.

Thanks.

Re: [WireGuard] Using wireguard link as a proxy?

[Baptiste Jonglez]

[-- Attachment #1: Type: text/plain, Size: 1304 bytes --]

On Fri, Jul 22, 2016 at 04:09:13AM -0500, Bruno Wolff III wrote:
> On Fri, Jul 22, 2016 at 10:18:21 +0200,
>  Baptiste Jonglez <baptiste@bitsofnetworks.org> wrote:
> >
> >Yes, the notion of "immediate next destinaton" does not make sense for
> >Wireguard.  It encapsulates plain IP, not Ethernet.
> 
> I thought that the next IP address might have been available for wireguard
> to see as the information seems to be available for routing. But as you
> mention below and I realized, that doesn't help with the return packets
> since they can have (almost) any source address.
> 
> >You need "allowed ips 0.0.0.0/0" here.  Your situation is just a regular
> >client/server tunneling setup, there's nothing special about "proxying",
> >whatever that means.
> 
> Yeah I realized that when thinking about this some more. "Proxy" in this
> case means source nat will be used on the outgoing packets.

Ok, excellent!  Wireguard really doesn't care or even know about the
source NAT you may apply on the server (well, at least when thinking about
it at a high level).

If you had used a public IP addresses on the client side (instead of
192.168.7.2), and simply forwarded packets on the server without applying
any NAT, it would be exactly the same from the perspective of Wireguard.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [WireGuard] Using wireguard link as a proxy?

[Bruno Wolff III]

On Fri, Jul 22, 2016 at 11:32:11 +0200,
  Baptiste Jonglez <baptiste@bitsofnetworks.org> wrote:
>
>Ok, excellent!  Wireguard really doesn't care or even know about the
>source NAT you may apply on the server (well, at least when thinking about
>it at a high level).

I got this working now. I need to work a bit on setting up the routing 
so it comes up correctly at boot without manually issuing ip commands.

What I am trying to experiment with is increase privacy on an upcoming trip 
where I will be using a laptop on hotel wifi and I want to route all traffic 
through my home server via a wireguard tunnel. Depending on how they do their 
wifi I may need to do some tweaks to be able to authenticate.

Later I may have use for a similar, but more restricted, set up to help me 
work from home. I need to be careful not to open up unintended holes when 
I do that.

Re: [WireGuard] Using wireguard link as a proxy?

[Jason A. Donenfeld]

I usually do something like:

wg set wg0 peer ABCD allowed-ips 0.0.0.0/0
ip route add 0/1 dev wg0
ip route add 128/1 dev wg0
./tungate.sh proxyserver.wireguard.io

The tungate.sh script just ensures that proxyserver.wireguard.io is
reachable with the original route, and takes into account ifupdown
changes:

http://0bin.net/paste/oobWhr-vTxhwyo+V#eT1QmdqFr455L0uEHYlduNSEDeUAkfV+v9KMWordUCQ

Re: [WireGuard] Using wireguard link as a proxy?

[Bruno Wolff III]

On Fri, Jul 22, 2016 at 14:42:52 +0200,
  "Jason A. Donenfeld" <Jason@zx2c4.com> wrote:
>I usually do something like:
>
>wg set wg0 peer ABCD allowed-ips 0.0.0.0/0
>ip route add 0/1 dev wg0
>ip route add 128/1 dev wg0

The idea I am working on now is to use policy writing and rules, so that 
packets sent to the proxy server are routed using the normal routing 
table which includes the default route supplied by dhcp, but eveything 
else (well not 127/8) uses an alternate routing table that has a default 
route through wg0. This way I can use a fixed set up and not have to 
parse the routing info setup after a wireless connection is made.

I need to do some reading to get the details for creating routing tables 
and rules for picking them, but expect this will work.

Re: [WireGuard] Using wireguard link as a proxy?

[Jason A. Donenfeld]

On Fri, Jul 22, 2016 at 5:14 PM, Bruno Wolff III <bruno@wolff.to> wrote:
> On Fri, Jul 22, 2016 at 14:42:52 +0200,
> The idea I am working on now is to use policy writing and rules, so that
> packets sent to the proxy server are routed using the normal routing table
> which includes the default route supplied by dhcp, but eveything else (well
> not 127/8) uses an alternate routing table that has a default route through
> wg0. This way I can use a fixed set up and not have to parse the routing
> info setup after a wireless connection is made.
>
> I need to do some reading to get the details for creating routing tables and
> rules for picking them, but expect this will work.

1.2.3.4 is proxyserver.wireguard.io:

$ ip rule add to 1.2.3.4 lookup main pref 30
$ ip rule add to all lookup 80 pref 40
$ ip route add default dev wg0 table 80


This thread might interest you:
http://marc.info/?l=linux-netdev&m=145452157719655&w=2

Re: [WireGuard] Using wireguard link as a proxy?

[Quan Zhou]

[-- Attachment #1: Type: text/plain, Size: 1668 bytes --]

I happen to have a similar problem, using `AllowedIPs = 0.0.0.0/0` on both
sides of a wireguard link works, but when I try to add more peers, all but
one gets `(none)`. I know this is by design, but it would be sweet if I can
manually set gateways within a wireguarded network.

On Fri, Jul 22, 2016 at 11:14 PM, Bruno Wolff III <bruno@wolff.to> wrote:

> On Fri, Jul 22, 2016 at 14:42:52 +0200,
>  "Jason A. Donenfeld" <Jason@zx2c4.com> wrote:
>
>> I usually do something like:
>>
>> wg set wg0 peer ABCD allowed-ips 0.0.0.0/0
>> ip route add 0/1 dev wg0
>> ip route add 128/1 dev wg0
>>
>
> The idea I am working on now is to use policy writing and rules, so that
> packets sent to the proxy server are routed using the normal routing table
> which includes the default route supplied by dhcp, but eveything else (well
> not 127/8) uses an alternate routing table that has a default route through
> wg0. This way I can use a fixed set up and not have to parse the routing
> info setup after a wireless connection is made.
>
> I need to do some reading to get the details for creating routing tables
> and rules for picking them, but expect this will work.
>
> _______________________________________________
> WireGuard mailing list
> WireGuard@lists.zx2c4.com
> http://lists.zx2c4.com/mailman/listinfo/wireguard
>



-- 
Regards,

Quan Zhou
+------------------------+
|pub [expires 2019-05-04]|
|2C0C 4D88 E631 4C73 4C44|
|CDE0 C0E 5470 1D2D 3F3EE|
+------------------------+
|pub [revoked 2016-04-16]|
|44D2 0307 1643 E80F 2E31|
|F081 FAFA 6643 7F9F D46F|
+------------------------+
|quanzhou822@gmail.com   |
|https://keybase.io/qzhou|
+------------------------+

[-- Attachment #2: Type: text/html, Size: 3236 bytes --]

Re: [WireGuard] Using wireguard link as a proxy?

[Bruno Wolff III]

On Fri, Jul 22, 2016 at 23:35:02 +0800,
  Quan Zhou <quanzhou822@gmail.com> wrote:
>I happen to have a similar problem, using `AllowedIPs = 0.0.0.0/0` on both
>sides of a wireguard link works, but when I try to add more peers, all but
>one gets `(none)`. I know this is by design, but it would be sweet if I can
>manually set gateways within a wireguarded network.

You only want 0.0.0.0/0 on one side. The proxy server should expect a 
single source IP address the clients can get packets from any address 
through the server.

If the clients are also proxing some local traffic, then you'll probably 
need to do nat on each client so the source addresses don't conflict.

Re: [WireGuard] Using wireguard link as a proxy?

[Bruno Wolff III]

On Fri, Jul 22, 2016 at 17:30:37 +0200,
  "Jason A. Donenfeld" <Jason@zx2c4.com> wrote:
>
>$ ip rule add to 1.2.3.4 lookup main pref 30
>$ ip rule add to all lookup 80 pref 40
>$ ip route add default dev wg0 table 80

OK. That is more or less what I am doing.

I was trying to name the routing table because I saw that people were doing 
that, but doing that in an idempotent way seems like it will require a hack. 
I'm going to just try using a table number like you do to see if that avoids 
the problem. 

The other issue I am seeing is that the default route generated for the wlan 
connection specifies a gateway in addition to the interface, and it looks 
like routing is checked for the gateway rather than assuming it is at the 
speficied interface because it looks like the routing doesn't work if 
I don't add the wlan route to the wg routing table. I'm going to do some 
more testing around that.

Re: [WireGuard] Using wireguard link as a proxy?

[Bruno Wolff III]

On Fri, Jul 22, 2016 at 17:30:37 +0200,
  "Jason A. Donenfeld" <Jason@zx2c4.com> wrote:
>
>$ ip rule add to 1.2.3.4 lookup main pref 30
>$ ip rule add to all lookup 80 pref 40
>$ ip route add default dev wg0 table 80
>
>
>This thread might interest you:
>http://marc.info/?l=linux-netdev&m=145452157719655&w=2

While this doesn't cover the general case, what I found usable was 
after checking for packets to the proxy, suppressing the default route when 
searching the main table and letting stuff that matches a narrower route go 
through directly. And then left over stuff goes to the tunnel via a 
default route in the extra routing table.

So for a real example that appears to be working, my systemd service file 
is below. 98.103.208.27 and 192.168.1.2 are bith the same machine. I need 
to include both because replies from 98.103.208.27 on my local network 
sometimes have a source address of 192.168.1.2. Most people won't have 
that issue.
There is extra cleanup commands so the service can be restarted without 
accumalating extra rules or routing table entries.
It seems to work now, but I had been fooled previously by cached router 
information.
The main thing of interest at this point in the thread is:
ip rule add suppress_prefixlength 0 lookup main priority 102

[Unit]
Description=WireGuard Server

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=-/usr/sbin/ip link del dev wg0
ExecStart=/usr/sbin/ip rule flush
ExecStart=/usr/sbin/ip rule add from all lookup default priority 32767
ExecStart=/usr/sbin/ip rule add from all lookup main priority 32766
ExecStart=/usr/sbin/ip route flush table 200
ExecStart=/usr/sbin/ip link add dev wg0 type wireguard
ExecStart=/usr/sbin/ip address add dev wg0 192.168.7.3 peer 192.168.7.1/32
ExecStart=/usr/bin/wg setconf wg0 /etc/wireguard/config
ExecStart=/usr/sbin/ip link set up dev wg0
ExecStart=/usr/sbin/ip route add default dev wg0 table 200
ExecStart=/usr/sbin/ip rule add to 98.103.208.27/32 lookup main priority 100
ExecStart=/usr/sbin/ip rule add to 192.168.1.2/32 lookup main priority 101
ExecStart=/usr/sbin/ip rule add suppress_prefixlength 0 lookup main priority 102
ExecStart=/usr/sbin/ip rule add from all lookup 200 priority 103
ExecStopPost=/usr/sbin/ip link del dev wg0
ExecStopPost=/usr/sbin/ip rule flush
ExecStopPost=/usr/sbin/ip rule add from all lookup default priority 32767
ExecStopPost=/usr/sbin/ip rule add from all lookup main priority 32766
ExecStopPost=/usr/sbin/ip route flush table 200

[Install]
WantedBy=multi-user.target

Re: [WireGuard] Using wireguard link as a proxy?

[Bruno Wolff III]

[-- Attachment #1: Type: text/plain, Size: 1435 bytes --]

On Fri, Jul 22, 2016 at 13:05:27 -0500,
  Bruno Wolff III <bruno@wolff.to> wrote:
>
>So for a real example that appears to be working, my systemd service 

I had another issue and that is the proxy server was used for some 
other services and I didn't want to connect to those from outside the 
tunnel. So I wanted some traffic to the proxy server to go direct and 
some to go through the tunnel. This involves marking packets. But the 
guessed source addresses don't use the marks, so you need to rewrite 
(SNAT) the source address for some of the outgoing packets. If you 
want static rules to do this you need to have the incorrect guesses 
be to use normal routing and then rewrite the source address for 
packets going over the tunnel. As the tunnel address is fixed, but 
the normal gateway address will change when moving between networks or 
possibly when dhcp leases expire.
The explanations for marking and policy routing aren't explicit about 
how you need to handle the source address issue and why it happens, though 
there are lots of mentions that there are problems related to the source 
address.
Another gotcha is that ip rule can't negate a test for fwmark and testing 
for fwmark equal to zero is a flag not to test it at all. So you need to 
do more complicated packet marking.
I'm attaching the real systemd service file (with the routing policy 
commands and other wireguard setup) and the iptables information.

[-- Attachment #2: wireguard.service --]
[-- Type: text/plain, Size: 1169 bytes --]

[Unit]
Description=WireGuard Server

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=-/usr/sbin/ip link del dev wg0
ExecStart=/usr/sbin/ip rule flush
ExecStart=/usr/sbin/ip rule add from all lookup default priority 32767
ExecStart=/usr/sbin/ip rule add from all lookup main priority 32766
ExecStart=/usr/sbin/ip route flush table 200
ExecStart=/usr/sbin/ip link add dev wg0 type wireguard
ExecStart=/usr/sbin/ip address add dev wg0 192.168.7.3 peer 192.168.7.1/32
ExecStart=/usr/bin/wg setconf wg0 /etc/wireguard/config
ExecStart=/usr/sbin/ip link set up dev wg0
ExecStart=/usr/sbin/ip route add default dev wg0 table 200
ExecStart=/usr/sbin/ip rule add suppress_prefixlength 0 lookup main priority 101
ExecStart=/usr/sbin/ip rule add fwmark 2 lookup 200 priority 102
ExecStart=/usr/sbin/ip route flush cache
ExecStopPost=/usr/sbin/ip link del dev wg0
ExecStopPost=/usr/sbin/ip rule flush
ExecStopPost=/usr/sbin/ip rule add from all lookup default priority 32767
ExecStopPost=/usr/sbin/ip rule add from all lookup main priority 32766
ExecStopPost=/usr/sbin/ip route flush table 200
ExecStopPost=/usr/sbin/ip route flush cache

[Install]
WantedBy=multi-user.target

[-- Attachment #3: iptables --]
[-- Type: text/plain, Size: 1041 bytes --]

*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
-A POSTROUTING ! -s 192.168.7.3/32 -o wg0 -j SNAT --to-source 192.168.7.3
COMMIT
*mangle
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
-A OUTPUT -j MARK --set-xmark 0x2/0xffffffff
-A OUTPUT -d 98.103.208.27 -p udp -m udp --dport 992 -j MARK --set-xmark 0x1/0xffffffff
-A OUTPUT -d 192.168.1.2 -p udp -m udp --dport 992 -j MARK --set-xmark 0x1/0xffffffff
COMMIT
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -i lo -j ACCEPT
-A INPUT -i wg0 -j ACCEPT
-A INPUT -p icmp -m icmp --icmp-type any -j ACCEPT
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A INPUT -s 98.103.208.24/29 -p tcp -m conntrack --ctstate NEW -m tcp --dport 22 -j ACCEPT
-A INPUT -p udp -m conntrack --ctstate NEW -m udp --dport 992 -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
COMMIT

Re: [WireGuard] Using wireguard link as a proxy?

[Bruno Wolff III]

On Sat, Jul 23, 2016 at 11:36:37 -0500,
  Bruno Wolff III <bruno@wolff.to> wrote:
>The explanations for marking and policy routing aren't explicit about 
>how you need to handle the source address issue and why it happens, 
>though there are lots of mentions that there are problems related to 
>the source address.

You also need to turn off rp_filter on interfaces because packets coming 
from the tunnel can look like they should have come from the normal 
gateway and will get dropped locally.

Re: [WireGuard] Using wireguard link as a proxy?

[Jason A. Donenfeld]

Hi Bruno,

Jeeze louise. Seems woefully complex. Inspired by your attempts here,
I thought of another method involving network namespaces that you
might consider instead. Voila: https://www.wireguard.io/netns/

Jason

Re: [WireGuard] Using wireguard link as a proxy?

[Bruno Wolff III]

On Mon, Jul 25, 2016 at 14:57:56 +0200,
  "Jason A. Donenfeld" <Jason@zx2c4.com> wrote:
>Hi Bruno,
>
>Jeeze louise. Seems woefully complex. Inspired by your attempts here,
>I thought of another method involving network namespaces that you
>might consider instead. Voila: https://www.wireguard.io/netns/

So far I have just skimmed it. It looks very promising. I am about to leave 
for my trip and won't finish reading it until I'm there. I am bringing a 
spare laptop so might risk messing with one while I'm there.

I don't know about network name spaces, so I should learn a lot from the 
write up. Thanks.