Hi guys. I'm experiencing a pretty weird wireguard, or perhaps kernel/OS stack bits behavior. I have three nodes which all can ping each other on wg0's IPs but when I add a secondary IP: -> $ ip addr add 10.0.0.226/24 dev wg0 it gets weird, namely, say when that sec IP is on A -> B ping returns; C ping waits, no errors, no return B -> both C & A pings return C -> neither A nor B ping returns I'm on CentOS with 4.18.0-301.1.el8.x86_64. All three nodes are virtually identical kvm VMs. any suggestions as to what is not working here or how to troubleshoot are vey appreciated. many thanks, L.
On Sat, 8 May 2021 17:31:58 +0100
lejeczek <peljasz@yahoo.co.uk> wrote:
> I'm experiencing a pretty weird wireguard, or perhaps
> kernel/OS stack bits behavior.
>
> I have three nodes which all can ping each other on wg0's
> IPs but when I add a secondary IP:
>
> -> $ ip addr add 10.0.0.226/24 dev wg0
>
> it gets weird, namely, say when that sec IP is on
> A -> B ping returns; C ping waits, no errors, no return
> B -> both C & A pings return
> C -> neither A nor B ping returns
>
> I'm on CentOS with 4.18.0-301.1.el8.x86_64.
> All three nodes are virtually identical kvm VMs.
>
> any suggestions as to what is not working here or how to
> troubleshoot are vey appreciated.
> many thanks, L.
Did you add the new IP to AllowedIPs of that node on all the other nodes?
Also remember that sets of AllowedIPs should be unique within the network,
i.e. can't have the same AllowedIPs or ranges listed for multiple nodes at the
same time. Setting it to the same /24 on all nodes will not work.
If still not clear, better post your complete config (without keys).
--
With respect,
Roman
On 08/05/2021 17:50, Roman Mamedov wrote:
> On Sat, 8 May 2021 17:31:58 +0100
> lejeczek <peljasz@yahoo.co.uk> wrote:
>
>> I'm experiencing a pretty weird wireguard, or perhaps
>> kernel/OS stack bits behavior.
>>
>> I have three nodes which all can ping each other on wg0's
>> IPs but when I add a secondary IP:
>>
>> -> $ ip addr add 10.0.0.226/24 dev wg0
>>
>> it gets weird, namely, say when that sec IP is on
>> A -> B ping returns; C ping waits, no errors, no return
>> B -> both C & A pings return
>> C -> neither A nor B ping returns
>>
>> I'm on CentOS with 4.18.0-301.1.el8.x86_64.
>> All three nodes are virtually identical kvm VMs.
>>
>> any suggestions as to what is not working here or how to
>> troubleshoot are vey appreciated.
>> many thanks, L.
> Did you add the new IP to AllowedIPs of that node on all the other nodes?
>
> Also remember that sets of AllowedIPs should be unique within the network,
> i.e. can't have the same AllowedIPs or ranges listed for multiple nodes at the
> same time. Setting it to the same /24 on all nodes will not work.
>
> If still not clear, better post your complete config (without keys).
>
It's the same single subnet 10.0.0.0/24 and to reiterate -
wg0's "primary" IPs can all ping each other.
All nodes have, respectively:
eg. node-B
[peer]
...
AllowedIPs = 10.0.0.1/32, 10.0.0.226/32
Endpoint = 10.1.1.223:51851
[peer]
...
AllowedIPs = 10.0.0.3/32, 10.0.0.226/32
Endpoint = 10.1.1.225:51853
On 08/05/2021 17:31, lejeczek wrote:
> Hi guys.
>
> I'm experiencing a pretty weird wireguard, or perhaps
> kernel/OS stack bits behavior.
>
> I have three nodes which all can ping each other on wg0's
> IPs but when I add a secondary IP:
>
> -> $ ip addr add 10.0.0.226/24 dev wg0
>
> it gets weird, namely, say when that sec IP is on
> A -> B ping returns; C ping waits, no errors, no return
> B -> both C & A pings return
> C -> neither A nor B ping returns
>
> I'm on CentOS with 4.18.0-301.1.el8.x86_64.
> All three nodes are virtually identical kvm VMs.
>
> any suggestions as to what is not working here or how to
> troubleshoot are vey appreciated.
> many thanks, L.
>
>
>
>
What I've just noticed for the first time is, config eg.:
..
[Peer]
..
AllowedIPs = 10.0.0.2/32, 10.0.0.226/32
Endpoint = 10.1.1.224:51852
[Peer]
..
AllowedIPs = 10.0.0.3/32, 10.0.0.226/32
Endpoint = 10.1.1.225:51853
> $ wg
interface: wg0
public key: c+gJArxYd8+=
private key: (hidden)
listening port: 51851
peer: K/=
preshared key: (hidden)
endpoint: 10.1.1.225:51853
allowed ips: 10.0.0.3/32, 10.0.0.226/32
latest handshake: 16 seconds ago
transfer: 124 B received, 2.14 KiB sent
peer: /KidNfhqgP/+c3A=
preshared key: (hidden)
endpoint: 10.1.1.224:51852
allowed ips: 10.0.0.2/32 # !! no
10.0.0.226/32 ?
latest handshake: 3 minutes, 15 seconds ago
transfer: 180 B received, 92 B sent
That is probably why only 10.0.0.3 with secondary IP is
"reachable". Right?
If that is by design and expected - why is that and how to
make a "floating" IP work if that is by design?
thanks, L.
On Sat, 8 May 2021 19:49:06 +0100
lejeczek <peljasz@yahoo.co.uk> wrote:
> > Also remember that sets of AllowedIPs should be unique within the network,
> > i.e. can't have the same AllowedIPs or ranges listed for multiple nodes at the
> > same time. Setting it to the same /24 on all nodes will not work.
> >
> > If still not clear, better post your complete config (without keys).
> >
> It's the same single subnet 10.0.0.0/24 and to reiterate -
> wg0's "primary" IPs can all ping each other.
> All nodes have, respectively:
> eg. node-B
> [peer]
> ...
> AllowedIPs = 10.0.0.1/32, 10.0.0.226/32
> Endpoint = 10.1.1.223:51851
>
> [peer]
> ...
> AllowedIPs = 10.0.0.3/32, 10.0.0.226/32
> Endpoint = 10.1.1.225:51853
See above for "Also remember...", you cannot have 10.0.0.226/32 added to
multiple peers as AllowedIPs at the same time.
--
With respect,
Roman