prevent default fallback route lookup from user-defined VRF table to local table(default vrf). #1557

sumanbrcm · 2020-12-17T12:52:14Z

What I did
By default fallback to local table if l3mdev table lookup fails is
enabled in kernel.
This enables packet to move to default-vrf if route lookup in
non-default-vrf fails.
To disable this fallback feature below IPv4 & IPv6 rules are added to
FIB Routing Policy Data Base.

ip ru add pref 1001 l3mdev unreachable
ip -6 ru add pref 1001 l3mdev unreachable

Why I did it
This fix is needed as it prevents default fallback route lookup from user-defined VRF table to local table(default vrf).

How I verified it
Please refer to below output after fix :
admin@sonic: ip -4 rule ls
1000: from all lookup [l3mdev-table]
1003: from 10.59.133.11 lookup mgmt
1004: from all to 10.0.0.0/8 lookup mgmt
32765: from all lookup local
32766: from all lookup main
32767: from all lookup default
admin@sonic:

admin@sonic: ip -6 rule ls
1000: from all lookup [l3mdev-table]
1003: from 2100::2 lookup mgmt
32765: from all lookup local
32766: from all lookup main
admin@sonic:

…to local table(default vrf).

preetham-singh · 2020-12-23T09:36:00Z

Adding few details for adding l3mdev unreachable rule:

If l3mdev unreachable rule is not present, route lookup by default falls back to local table(default vrf routing table which has connected prefixes). If connected prefix is present in local table for which lookup is performed in non-default vrf, result will lead to nexthop in local table causing packet leak from non-default vrf to default vrf.
Below are some linux output for route lookup before and after adding l3mdev unreachable rule.

Before Fix:

root@sonic:/home/admin# show vrf
VRF Interfaces

Vrf-red Ethernet56

root@sonic:/home/admin# ip -4 ru ls
1000: from all lookup [l3mdev-table]
1001: from all lookup local
32766: from all lookup main
32767: from all lookup default

root@sonic:/home/admin# ip -4 route get 24.0.0.100 vrf Vrf-red
24.0.0.100 dev Ethernet56 table 1001 src 24.0.0.2 uid 0
cache

root@sonic:/home/admin# config interface ip remove Ethernet56 24.0.0.2/24

root@sonic:/home/admin# ip -4 route get 24.0.0.100 vrf Vrf-red
24.0.0.100 dev Ethernet24 src 24.0.0.2 uid 0
cache

root@sonic:/home/admin# ip -4 route get 24.0.0.100
24.0.0.100 dev Ethernet24 src 24.0.0.2 uid 0
cache

root@sonic:/home/admin# ip -s addr show Ethernet24
36: Ethernet24: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9100 qdisc pfifo_fast state UP group default qlen 1000
link/ether 3c:2c:99:2d:84:35 brd ff:ff:ff:ff:ff:ff
inet 24.0.0.2/24 brd 24.0.0.255 scope global Ethernet24
valid_lft forever preferred_lft forever
inet6 fe80::3e2c:99ff:fe2d:8435/64 scope link
valid_lft forever preferred_lft forever
RX: bytes packets errors dropped overrun mcast
14396 62 0 2 0 0
TX: bytes packets errors dropped carrier collsns
14572 63 0 0 0 0

root@sonic:/home/admin# ip -s addr show Ethernet56
44: Ethernet56: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9100 qdisc pfifo_fast master Vrf-red state UP group default qlen 1000
link/ether 3c:2c:99:2d:84:35 brd ff:ff:ff:ff:ff:ff
inet 56.0.0.2/24 brd 56.0.0.255 scope global Ethernet56
valid_lft forever preferred_lft forever
inet6 fe80::3e2c:99ff:fe2d:8435/64 scope link
valid_lft forever preferred_lft forever
RX: bytes packets errors dropped overrun mcast
218601 1633 0 5 0 0
TX: bytes packets errors dropped carrier collsns
16973 98 0 0 0 0
root@sonic:/home/admin#

After fix:

root@sonic:~# ip -4 ru ls
1000: from all lookup [l3mdev-table]
1001: from all lookup [l3mdev-table] unreachable
1002: from all lookup local
32766: from all lookup main
32767: from all lookup default

root@sonic:~# show vrf
VRF Interfaces

Vrf-red Ethernet48

root@sonic:~# ip -s addr show Ethernet48
27: Ethernet48: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9100 qdisc pfifo_fast master Vrf-red state UP group default qlen 1000
link/ether 3c:2c:99:2e:d8:75 brd ff:ff:ff:ff:ff:ff
inet 24.0.0.1/24 brd 24.0.0.255 scope global Ethernet48
valid_lft forever preferred_lft forever
inet 48.0.0.1/24 brd 48.0.0.255 scope global Ethernet48
valid_lft forever preferred_lft forever
inet6 fe80::3e2c:99ff:fe2e:d875/64 scope link
valid_lft forever preferred_lft forever
RX: bytes packets errors dropped overrun mcast
16018 69 0 0 0 0
TX: bytes packets errors dropped carrier collsns
16657 73 0 0 0 0

root@sonic:~# ip -s addr show Ethernet24
25: Ethernet24: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9100 qdisc pfifo_fast state UP group default qlen 1000
link/ether 3c:2c:99:2e:d8:75 brd ff:ff:ff:ff:ff:ff
inet 24.0.0.1/24 brd 24.0.0.255 scope global Ethernet24
valid_lft forever preferred_lft forever
inet6 fe80::3e2c:99ff:fe2e:d875/64 scope link
valid_lft forever preferred_lft forever
RX: bytes packets errors dropped overrun mcast
18322 105 0 0 0 0
TX: bytes packets errors dropped carrier collsns
15719 67 0 0 0 0

root@sonic:~# ip -4 route get 24.0.0.100 vrf Vrf-red
24.0.0.100 dev Ethernet48 table 1001 src 24.0.0.1 uid 0
cache

root@sonic:~# config interface ip remove Ethernet48 24.0.0.1/24

root@sonic:~# ip -s addr show Ethernet48
27: Ethernet48: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9100 qdisc pfifo_fast master Vrf-red state UP group default qlen 1000
link/ether 3c:2c:99:2e:d8:75 brd ff:ff:ff:ff:ff:ff
inet 48.0.0.1/24 brd 48.0.0.255 scope global Ethernet48
valid_lft forever preferred_lft forever
inet6 fe80::3e2c:99ff:fe2e:d875/64 scope link
valid_lft forever preferred_lft forever
RX: bytes packets errors dropped overrun mcast
16259 70 0 0 0 0
TX: bytes packets errors dropped carrier collsns
16899 74 0 0 0 0

root@sonic:~# ip -4 route get 24.0.0.100 vrf Vrf-red
RTNETLINK answers: Network is unreachable

root@sonic:~#

ben-gale · 2020-12-31T20:05:53Z

@lguohan - can we get some review on this please?

lguohan · 2021-01-06T22:54:40Z

cfgmgr/vrfmgr.cpp

-            << IP_CMD << " -6 rule add pref " << TABLE_LOCAL_PREF << " table local && " << IP_CMD << " -6 rule del pref 0";
+            << IP_CMD << " -6 rule add pref " << TABLE_LOCAL_PREF << " table local && " << IP_CMD << " -6 rule del pref 0 && "
+                << IP_CMD << " rule add pref " << VRF_FALLBACK_DISABLE_PREF << " l3mdev unreachable && "
+                << IP_CMD << " -6 rule add pref " << VRF_FALLBACK_DISABLE_PREF << " l3mdev unreachable";


do we have vstest for this feature?

Thanks @lguohan for the review . I will soon update the test run results .

preventing default fallback route lookup from user-defined VRF table …

d559413

…to local table(default vrf).

sumanbrcm mentioned this pull request Dec 17, 2020

preventing default fallback route lookup from user-defined VRF table … sonic-net/sonic-buildimage#6236

Closed

ben-gale added the brcm-upstream label Dec 17, 2020

lguohan reviewed Jan 6, 2021

View reviewed changes

prsunny self-requested a review as a code owner September 2, 2022 23:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

prevent default fallback route lookup from user-defined VRF table to local table(default vrf). #1557

prevent default fallback route lookup from user-defined VRF table to local table(default vrf). #1557

sumanbrcm commented Dec 17, 2020 •

edited

Loading

preetham-singh commented Dec 23, 2020

ben-gale commented Dec 31, 2020

lguohan Jan 6, 2021

sumanbrcm Jan 7, 2021

prevent default fallback route lookup from user-defined VRF table to local table(default vrf). #1557

Are you sure you want to change the base?

prevent default fallback route lookup from user-defined VRF table to local table(default vrf). #1557

Conversation

sumanbrcm commented Dec 17, 2020 • edited Loading

preetham-singh commented Dec 23, 2020

ben-gale commented Dec 31, 2020

lguohan Jan 6, 2021

Choose a reason for hiding this comment

sumanbrcm Jan 7, 2021

Choose a reason for hiding this comment

sumanbrcm commented Dec 17, 2020 •

edited

Loading