Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

VRRPv3 IPv6: persistent FAULT state when updating VMAC with a non-existent VMAC base interface #2280

Open
louis-6wind opened this issue Apr 4, 2023 · 2 comments
Open

VRRPv3 IPv6: persistent FAULT state when updating VMAC with a non-existent VMAC base interface #2280

louis-6wind opened this issue Apr 4, 2023 · 2 comments

Comments

@louis-6wind
Copy link
Contributor

louis-6wind commented Apr 4, 2023 •
edited

Describe the bug
VRRPv3 IPv6 remains in a persistent FAULT state after setting an non-existent VMAC base interface. When the base interface appears, the problem is not solved. Base interface is never updated on the vmac.

To Reproduce

On a standalone VM machine

# Apply the following commands
killall keepalived
sleep 1
ip link set iface5 down
ip link set iface5 name ens5
ip link set ens5 down

ip -6 addr add  fd00:100::3/64 dev ens4
ip addr add 10.0.0.1/24 dev ens4
ip link set dev ens4 up
sleep 2
cat>/etc/keepalived/keepalived.conf <<\EOF
global_defs
{
router_id router
dynamic_interfaces
}

vrrp_sync_group group15 {
group {
vrrp
}
}

vrrp_instance vrrp {
version 3
state BACKUP
interface ens4

use_vmac vrrp


garp_master_delay 5

virtual_router_id 15

priority 200
advert_int 1.0

virtual_ipaddress {
fd00:100::1/64
}

preempt_delay 0
}
EOF
keepalived -D
sleep 5
journalctl _PID=$(pgrep keepalived | tail -n1) | cat
sed -e 's|ens4|iface5|g' -i /etc/keepalived/keepalived.conf
killall -s SIGHUP keepalived
sleep 4
ip -br l
journalctl _PID=$(pgrep keepalived | tail -n1) | cat
ip link set ens5 name iface5 up
sleep 4
journalctl _PID=$(pgrep keepalived | tail -n1) | cat
ip -br l

Expected behavior

Instance in MASTER state. VMAC updated with the correct base interface

Keepalived version

Keepalived v2.2.7 (04/02,2023), git commit v2.2.7-148-g58be65ee

Copyright(C) 2001-2023 Alexandre Cassen, <acassen@gmail.com>

Built with kernel headers for Linux 4.15.18
Running on Linux 5.4.0-135-generic #152~18.04.2-Ubuntu SMP Tue Nov 29 08:23:49 UTC 2022
Distro: Ubuntu 18.04.1 LTS

configure options: CFLAGS=-g -O0 --prefix=/usr --sysconfdir=/etc --with-extra-cflags=-I/usr/include/libnl3 --with-extra-ldflags= --disable-lvs --with-init=systemd --host=x86_64-linux-gnu host_alias=x86_64-linux-gnu

Config options:  NFTABLES VRRP VRRP_AUTH VRRP_VMAC OLD_CHKSUM_COMPAT INIT=systemd SYSTEMD_NOTIFY

System options:  VSYSLOG MEMFD_CREATE IPV4_DEVCONF RTA_ENCAP RTA_EXPIRES RTA_NEWDST RTA_PREF FRA_SUPPRESS_PREFIXLEN FRA_SUPPRESS_IFGROUP FRA_TUN_ID RTAX_CC_ALGO RTAX_QUICKACK RTEXT_FILTER_SKIP_STATS FRA_L3MDEV FRA_UID_RANGE RTAX_FASTOPEN_NO_COOKIE RTA_VIA RTA_TTL_PROPAGATE IFA_FLAGS LWTUNNEL_ENCAP_MPLS LWTUNNEL_ENCAP_ILA IPTABLES NET_LINUX_IF_H_COLLISION VRRP_IPVLAN IFLA_LINK_NETNSID GLOB_BRACE GLOB_ALTDIRFUNC INET6_ADDR_GEN_MODE VRF SO_MARK

Distro (please complete the following information):

  • Name: Ubuntu
  • Version: 18.04-hwe
  • Architecture: x86_64

Details of any containerisation or hosted service (e.g. AWS)

NA

Configuration file:
See above
Notify and track scripts
NA

System Log entries

Apr 04 18:43:01 ubuntu1804hwe Keepalived_vrrp[21742]: Registering Kernel netlink reflector
Apr 04 18:43:01 ubuntu1804hwe Keepalived_vrrp[21742]: Registering Kernel netlink command channel
Apr 04 18:43:01 ubuntu1804hwe Keepalived_vrrp[21742]: Sync group group15 has only 1 virtual router(s) - this probably isn't what you want
Apr 04 18:43:01 ubuntu1804hwe Keepalived_vrrp[21742]: (vrrp) the first IPv6 VIP address should be link local
Apr 04 18:43:01 ubuntu1804hwe Keepalived_vrrp[21742]: use_vmac or no_accept/strict specified, but no firewall configured - using nftables
Apr 04 18:43:01 ubuntu1804hwe Keepalived_vrrp[21742]: Assigned address 10.0.0.1 for interface ens4
Apr 04 18:43:01 ubuntu1804hwe Keepalived_vrrp[21742]: Assigned address fe80::dced:1ff:fe70:f533 for interface ens4
Apr 04 18:43:01 ubuntu1804hwe Keepalived_vrrp[21742]: (vrrp): Success creating VMAC interface vrrp
Apr 04 18:43:01 ubuntu1804hwe Keepalived_vrrp[21742]: Assigned address fe80::dced:1ff:fe70:f533 for interface vrrp
Apr 04 18:43:01 ubuntu1804hwe Keepalived_vrrp[21742]: Registering gratuitous NDISC shared channel
Apr 04 18:43:01 ubuntu1804hwe Keepalived_vrrp[21742]: (vrrp) Entering BACKUP STATE (init)
Apr 04 18:43:01 ubuntu1804hwe Keepalived_vrrp[21742]: VRRP sockpool: [ifindex(164), family(IPv6), proto(112), fd(13,14) multicast, address(ff02::12)]
Apr 04 18:43:04 ubuntu1804hwe Keepalived_vrrp[21742]: (vrrp) Receive advertisement timeout
Apr 04 18:43:04 ubuntu1804hwe Keepalived_vrrp[21742]: (vrrp) Entering MASTER STATE
Apr 04 18:43:04 ubuntu1804hwe Keepalived_vrrp[21742]: (vrrp) setting VIPs.
Apr 04 18:43:04 ubuntu1804hwe Keepalived_vrrp[21742]: (vrrp) Sending/queueing Unsolicited Neighbour Adverts on vrrp for fd00:100::1
Apr 04 18:43:04 ubuntu1804hwe Keepalived_vrrp[21742]: Sending unsolicited Neighbour Advert on vrrp for fd00:100::1
Apr 04 18:43:04 ubuntu1804hwe Keepalived_vrrp[21742]: Sending unsolicited Neighbour Advert on vrrp for fd00:100::1
Apr 04 18:43:04 ubuntu1804hwe Keepalived_vrrp[21742]: Sending unsolicited Neighbour Advert on vrrp for fd00:100::1
Apr 04 18:43:04 ubuntu1804hwe Keepalived_vrrp[21742]: Sending unsolicited Neighbour Advert on vrrp for fd00:100::1
Apr 04 18:43:04 ubuntu1804hwe Keepalived_vrrp[21742]: Sending unsolicited Neighbour Advert on vrrp for fd00:100::1
Apr 04 18:43:04 ubuntu1804hwe Keepalived_vrrp[21742]: VRRP_Group(group15) Syncing instances to MASTER state
Apr 04 18:43:06 ubuntu1804hwe Keepalived_vrrp[21742]: Reloading
Apr 04 18:43:06 ubuntu1804hwe Keepalived_vrrp[21742]: Configuration specifies interface iface5 which doesn't currently exist - will use if created
Apr 04 18:43:06 ubuntu1804hwe Keepalived_vrrp[21742]: Sync group group15 has only 1 virtual router(s) - this probably isn't what you want
Apr 04 18:43:06 ubuntu1804hwe Keepalived_vrrp[21742]: (vrrp) the first IPv6 VIP address should be link local
Apr 04 18:43:06 ubuntu1804hwe Keepalived_vrrp[21742]: use_vmac or no_accept/strict specified, but no firewall configured - using nftables
Apr 04 18:43:06 ubuntu1804hwe Keepalived_vrrp[21742]: (vrrp) VMAC vrrp already exists but is incompatible. It will be deleted/updated
Apr 04 18:43:06 ubuntu1804hwe Keepalived_vrrp[21742]: Assigned address fe80::dced:1ff:fe70:f533 for interface vrrp
Apr 04 18:43:06 ubuntu1804hwe Keepalived_vrrp[21742]: (vrrp): entering FAULT state (interface iface5 down)
Apr 04 18:43:06 ubuntu1804hwe Keepalived_vrrp[21742]: (vrrp) entering FAULT state (no IPv6 address for interface)
Apr 04 18:43:06 ubuntu1804hwe Keepalived_vrrp[21742]: VRRP_Group(group15): Syncing vrrp to FAULT state
Apr 04 18:43:06 ubuntu1804hwe Keepalived_vrrp[21742]: (vrrp) entering FAULT state
Apr 04 18:43:06 ubuntu1804hwe Keepalived_vrrp[21742]: (vrrp) sent 0 priority
Apr 04 18:43:06 ubuntu1804hwe Keepalived_vrrp[21742]: (vrrp) removing VIPs.
Apr 04 18:43:06 ubuntu1804hwe Keepalived_vrrp[21742]: VRRP sockpool: [ifindex(164), family(IPv6), proto(112), fd(13,14) multicast, address(ff02::12)]
Apr 04 18:43:10 ubuntu1804hwe Keepalived_vrrp[21742]: Interface name has changed from ens5 to iface5
Apr 04 18:43:10 ubuntu1804hwe Keepalived_vrrp[21742]: Interface iface5 added
Apr 04 18:43:10 ubuntu1804hwe Keepalived_vrrp[21742]: (vrrp) interface iface5 is down
Apr 04 18:43:10 ubuntu1804hwe Keepalived_vrrp[21742]: Netlink reports iface5 up
Apr 04 18:43:12 ubuntu1804hwe Keepalived_vrrp[21742]: Assigned address fe80::dced:1ff:fe70:f534 for interface iface5
Apr 04 18:43:12 ubuntu1804hwe Keepalived_vrrp[21742]: Assigned address fe80::dced:1ff:fe70:f534 for interface vrrp
+ ip -br l
lo               UNKNOWN        00:00:00:00:00:00 <LOOPBACK,UP,LOWER_UP> 
mgmt0            UP             de:ad:de:01:02:03 <BROADCAST,MULTICAST,UP,LOWER_UP> 
ens4             UP             de:ed:01:70:f5:33 <BROADCAST,MULTICAST,UP,LOWER_UP> 
iface5           UP             de:ed:01:70:f5:34 <BROADCAST,MULTICAST,UP,LOWER_UP> 
ens6             DOWN           de:ed:01:70:f5:35 <BROADCAST,MULTICAST> 
ens7             DOWN           de:ed:01:70:f5:35 <BROADCAST,MULTICAST> 
vrrp@ens4        UP             00:00:5e:00:02:0f <BROADCAST,MULTICAST,UP,LOWER_UP>

Did keepalived coredump?
No
@louis-6wind
Copy link
Contributor Author

To fix the issue, patches #2277 and #2281 are needed.

@louis-6wind
Copy link
Contributor Author

A more complicated scenario works with the patches

# Apply the following commands
killall keepalived
sleep 1
ip link set iface5 down
ip link set iface5 name ens5
ip link set ens5 down

ip -6 addr add  fd00:100::3/64 dev ens4
ip addr add 10.0.0.1/24 dev ens4
ip link set dev ens4 up
sleep 2
cat>/etc/keepalived/keepalived.conf <<\EOF
global_defs
{
router_id router
dynamic_interfaces
}

vrrp_sync_group group15 {
group {
vrrp
}
}

vrrp_instance vrrp {
version 3
state BACKUP
interface ens4

use_vmac vrrp


garp_master_delay 5

virtual_router_id 15

priority 200
advert_int 1.0

virtual_ipaddress {
fd00:100::1/64
}

preempt_delay 0
}
EOF
keepalived -D
sleep 5
journalctl _PID=$(pgrep keepalived | tail -n1) | cat
sed -e 's|ens4|iface6|g' -i /etc/keepalived/keepalived.conf
killall -s SIGHUP keepalived
sleep 4
ip -br l
journalctl _PID=$(pgrep keepalived | tail -n1) | cat
sed -e 's|iface6|iface5|g' -i /etc/keepalived/keepalived.conf
killall -s SIGHUP keepalived
sleep 4
ip -br l

ip link set ens5 name iface5 up
sleep 4
journalctl _PID=$(pgrep keepalived | tail -n1) | cat
ip -br l

@louis-6wind louis-6wind changed the title VRRPv3 IPv6: persistent FAULT state when setting inexistent VMAC base interface VRRPv3 IPv6: persistent FAULT state when setting non-existent VMAC base interface Apr 4, 2023
@louis-6wind louis-6wind changed the title VRRPv3 IPv6: persistent FAULT state when setting non-existent VMAC base interface VRRPv3 IPv6: persistent FAULT state when updating VMAC with a non-existent VMAC base interface Apr 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@louis-6wind