IPv6 on vrrp interface #757
Comments
|
Can you please describe the environment that keepalived is being run in, e.g. what kernel version, is keepalived being run in a container or a VM? I take it by FC27 you mean Fedora 27. Please also provide a copy of your configuration and the output of How are you disabling ipv6 on vrrp.6 and vrrp.7? If keepalived can't update net.ipv4.conf.all.rp_filter I don't understand why I don't understand what you mean by: It appears that the disabling of ipv6 is commented out. Or is this some configuration file or is it the output of some command? I have run keepalived in a Fedora 27 VM and I am not experiencing these problems. |
|
Yes, It is Fedora 27 with all the latest updates, all standard rpm packages. Keepalived (1.3.9) works on top of OpenvSwitch (2.8.1) interfaces in KVM guest. This is my custom sysctl configuration : [root@mx1 ~]# cat /etc/sysctl.conf Previously I had no *.disable_ipv6 settings in sysctl.conf. Now keepalived on this system works only when IPv6 is disabled on vrrp intefaces. It is wierd, that I have some systems where IPv6 is properly disabled by keepalived and few systems where IPv6 is still enabled running keepalived. This is my Keepalived configuration. It differs in state (BACKUP) and priority (51) from backup node. [root@mx1 ~]# cat /etc/keepalived/keepalived.conf vrrp_sync_group mx { vrrp_instance mx.vlan103 { vrrp_instance mx.vlan212 { |
|
Looking at the code, it checks whether the read of /proc/sys/net/ipv4/conf/all/rp_filter is successful or not, and we can see that it is failing. Unfortunately the code does not check (at the moment) whether the write to /proc/sys/net/ipv6/conf/vrrp.6/disable_ipv6 etc. is successful or not, and hence no error message is reported. It appears that the /proc filesystem is not accessible from within the KVM. Can you check whether you are able to modify /proc/sys/net/ipv4/conf/all/rp_filter from within the KVM, since I think that is where your problem lies. |
|
Thanks for the hint! It is/was SELinux and this is confirmation: [root@mx1 ~]# ifconfig vrrp.6 [root@mx1 ~]# setenforce 0 [root@mx1 ~]# systemctl restart keepalived.service [root@mx1 ~]# ifconfig vrrp.6 [root@mx1 ~]# setenforce 1 [root@mx1 ~]# systemctl restart keepalived.service [root@mx1 ~]# ifconfig vrrp.6 |
|
Since I could not find any good documentation about sysctl settings for keepalived (per linux distribution), can you please enlight us which sysctl setting should be changed in sysctl.conf that keepalived do not set? Is there actually need for custom sysctl changes? Now I have only net.ipv4.conf.all.rp_filter = 0 set in sysctl.conf: [root@mx1 ~]# cat /etc/sysctl.conf #rp_filter settings needed for keepalived VRRP #conntrackd/keepalived |
|
With SELinux enabled I can modify /proc/sys/* settings in terminal. So why keepalived can't, running [root@mx1 ~]# sestatus [root@mx1 ~]# cat /proc/sys/net/ipv6/conf/vrrp.6/disable_ipv6 [root@mx1 ~]# echo 1 > /proc/sys/net/ipv6/conf/vrrp.6/disable_ipv6 [root@mx1 ~]# cat /proc/sys/net/ipv6/conf/vrrp.6/disable_ipv6 Maybe this is connected with latest kernel security patches?! |
|
If I'm readings things properly, mx1 is your Fedora 27 system, and mx2 is the KVM system. If the above is correct, the problem is with keepalived running in the KVM (mx2), but when you have executed It seems to me that SELinux in mx1 is preventing the KVM (mx2) from accessing the /proc filesystem in mx1; this though makes mx2 sound more like a container. |
|
Sorry, I didn't answer your question about which sysctl settings need to be set for keepalived. The simple answer is none; keepalived sets what it needs to. See keepalived/vrrp/vrrp_if_config.c for the details of what needs to be set and why, and if you follow the code you will see that keepalived is setting them, as you experience when you say As I wrote above, the problem appears to be that mx2 cannot access the /proc filesystem, and that SELinux in the host system mx1 is stopping that access. |
|
Actually mx1 and mx2, they are both KVM guests. [root@mx2 ~]# restorecon -r / With SELinux disabled and restored default SELinux security contexts I still get: Keepalived_vrrp[1595]: Unable to read sysctl net.ipv4.conf.all.rp_filter. Jan 21 21:32:44 mx2 Keepalived[1567]: Stopped Keepalived v1.3.9 (10/21,2017) [root@mx2 ~]# uname -a [root@mx2 ~]# sestatus [root@mx2 ~]# cat /proc/sys/net/ipv4/conf/all/rp_filter [root@mx2 ~]# cat /proc/sys/net/ipv4/conf/all/rp_filter [root@mx2 ~]# cat /proc/sys/net/ipv4/conf/all/rp_filter |
|
Case solved. I had to re-enabled SELinux to repair security contexts during (re)boot. Jan 23 13:09:30 mx2 Keepalived[1580]: Starting Keepalived v1.3.9 (10/21,2017) |
|
Somehow there should be Keepalived SELinux rules to allow reading and writeing /proc/sys/* settings with SELinux enabled. Jan 23 13:09:30 mx2 Keepalived_vrrp[1587]: Opening file '/etc/keepalived/keepalived.conf'. |
|
OK, that's good from the keepalived perspective, since this is an SELinux issue on mx2. Am I correct in understanding that you have the problems on both mx1 and mx2, or am I getting muddled (I had previously understood that mx1 was OK but mx2 was not). It appears that some of the testing has been done on mx1 and some on mx2, which is certainly causing me some confusion. I know that Fedora/RedHat have had to update their SELinux policy to allow keepalived to access various files. What distro's SELinux policy are you using in your VMs? |
|
Closing due to problem solved and no update for over 3 weeks. |
I've noticed that IPv6 is configured on vrrp interface getting split brain on first failover
vrrp.6: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::200:5eff:fe00:106 prefixlen 64 scopeid 0x20
ether 00:00:5e:00:01:06 txqueuelen 1000 (Ethernet)
RX packets 247 bytes 11978 (11.6 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 12 bytes 952 (952.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vrrp.7: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::200:5eff:fe00:107 prefixlen 64 scopeid 0x20
ether 00:00:5e:00:01:07 txqueuelen 1000 (Ethernet)
RX packets 247 bytes 11978 (11.6 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 12 bytes 952 (952.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
If I enable disable_ipv6 for vrrp interfaces everything works normal:
net.ipv4.conf.all.rp_filter = 0
net.ipv4.conf.lo.rp_filter = 0
net.ipv4.conf.default.rp_filter = 0
#net.ipv6.conf.vrrp/6.disable_ipv6 = 1
#net.ipv6.conf.vrrp/7.disable_ipv6 = 1
net.ipv4.ip_nonlocal_bind = 1
Also I noticed that net.ipv4.conf.all.rp_filter is ureadable on keepalived startup:
Jan 21 19:11:44 mx2 Keepalived_vrrp[1582]: vmac: Success creating VMAC interface vrrp.6 for vrrp_instance mx.vlan103
Jan 21 19:11:44 mx2 Keepalived_vrrp[1582]: Unable to read sysctl net.ipv4.conf.all.rp_filter
Jan 21 19:11:44 mx2 Keepalived_vrrp[1582]: vmac: Success creating VMAC interface vrrp.7 for vrrp_instance mx.vlan212
Jan 21 19:11:44 mx2 Keepalived_vrrp[1582]: Unable to read sysctl net.ipv4.conf.all.rp_filter
net.ipv6.conf.vrrp/7.disable_ipv6 = 1
So why vrrp->family for disable_ipv6 is calculated diffrently on other systems with the same OS (FC27) and keepalived (1.3.9) and same (copied) system setup where IPv6 is properly disabled on vrrp interfaces:
vrrp.21: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 00:00:5e:00:01:15 txqueuelen 1000 (Ethernet)
RX packets 1490723 bytes 74286032 (70.8 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vrrp.22: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 00:00:5e:00:01:16 txqueuelen 1000 (Ethernet)
RX packets 1489359 bytes 74162544 (70.7 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
The text was updated successfully, but these errors were encountered: