Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Got BGP Rejected at Routing Table (Show IPv6 Route) - SRv6 L3VPN #10896

Closed
iqraaswad opened this issue Mar 27, 2022 · 18 comments
Closed

Got BGP Rejected at Routing Table (Show IPv6 Route) - SRv6 L3VPN #10896

iqraaswad opened this issue Mar 27, 2022 · 18 comments
Labels
triage Needs further investigation

Comments

@iqraaswad
Copy link

Describe the bug

[ X] Did you check if this is a duplicate issue?
[ X] Did you test it on the latest FRRouting/frr master branch?

To Reproduce

Router 1 :

Router Configuration :

Current configuration:
!
frr version 8.1
frr defaults traditional
hostname pi3
no ip forwarding
service integrated-vtysh-config
!
ipv6 route 2001:d30:8000:8000::5/128 2001:d30:8000:243::15
!
vrf vrf1
 vni 4001
exit-vrf
!
interface lo
 ipv6 address 2001:d30:8000:8000::3/128
 ipv6 router isis SRv6
 isis circuit-type level-1
 isis passive
exit
!
interface eth0.241
 ipv6 router isis SRv6
 isis bfd
 isis circuit-type level-1
 isis fast-reroute ti-lfa level-1 node-protection link-fallback
 isis fast-reroute ti-lfa level-2
 isis network point-to-point
exit
!
interface eth0.243
 ipv6 router isis SRv6
 isis bfd
 isis circuit-type level-1
 isis fast-reroute ti-lfa level-1 node-protection link-fallback
 isis fast-reroute ti-lfa level-2
 isis network point-to-point
exit
!
interface eth0.244
 ipv6 router isis SRv6
 isis bfd
 isis circuit-type level-1
 isis fast-reroute ti-lfa level-1 node-protection link-fallback
 isis fast-reroute ti-lfa level-2
 isis network point-to-point
exit
!
router bgp 30
 bgp router-id 3.3.3.3
 bgp log-neighbor-changes
 no bgp ebgp-requires-policy
 no bgp default ipv4-unicast
 bgp bestpath as-path multipath-relax
 neighbor iPartner peer-group
 neighbor iPartner remote-as 30
 neighbor iPartner tcp-mss 1200
 neighbor iPartner update-source lo
 neighbor iPartner capability extended-nexthop
 neighbor 2001:d30:8000:8000::5 peer-group iPartner
 neighbor 2001:d30:8000:8000::5 tcp-mss 1200
 !
 segment-routing srv6
  locator SRv6_Loc
 exit
 !
 address-family ipv6 unicast
  maximum-paths 2
 exit-address-family
 !
 address-family ipv6 vpn
  neighbor iPartner activate
  neighbor iPartner route-reflector-client
 exit-address-family
exit
!
router bgp 30 vrf vrf1
 bgp router-id 3.3.3.3
 no bgp ebgp-requires-policy
 no bgp default ipv4-unicast
 !
 address-family ipv6 unicast
  network 2001:d30:8000:241::/64
  redistribute connected
  sid vpn export auto
  rd vpn export 30:131
  rt vpn both 30:13
  export vpn
  import vpn
 exit-address-family
exit
!
router isis SRv6
 is-type level-1
 net 49.0001.0001.0001.0003.00
 spf prefix-priority critical ACL_CRITICAL
 topology ipv6-unicast
 fast-reroute priority-limit critical level-1
 segment-routing on
 segment-routing node-msd 8
exit
!
ipv6 access-list ACL_CRITICAL seq 5 permit 2001:d30:8000:243::/64
ipv6 access-list ACL_CRITICAL seq 10 permit 2001:d30::/48
!
segment-routing
 srv6
  locators
   locator SRv6_Loc
    prefix 2001:d30:8000:3::/64
   exit
   !
  exit
  !
 exit
 !
 traffic-eng
 exit
exit
!
end

Router 2 :

Current configuration:
!
frr version 8.1
frr defaults traditional
hostname pi5
no ip forwarding
service integrated-vtysh-config
!
ipv6 route 2001:d30:8000:8000::3/128 2001:d30:8000:243::13
!
vrf vrf1
 vni 4001
exit-vrf
!
interface lo
 ipv6 address 2001:d30:8000:8000::5/128
 ipv6 router isis SRv6
 isis circuit-type level-1
 isis passive
exit
!
interface eth0.243
 ipv6 router isis SRv6
 isis bfd
 isis circuit-type level-1
 isis fast-reroute ti-lfa level-1 node-protection link-fallback
 isis fast-reroute ti-lfa level-2
 isis network point-to-point
exit
!
router bgp 30
 bgp router-id 5.5.5.5
 bgp log-neighbor-changes
 no bgp ebgp-requires-policy
 no bgp default ipv4-unicast
 bgp bestpath as-path multipath-relax
 neighbor iPartner peer-group
 neighbor iPartner remote-as 30
 neighbor iPartner update-source lo
 neighbor iPartner capability extended-nexthop
 neighbor 2001:d30:8000:8000::3 peer-group iPartner
 !
 segment-routing srv6
  locator SRv6_Loc
 exit
 !
 address-family ipv6 vpn
  neighbor iPartner activate
 exit-address-family
exit
!
router bgp 30 vrf vrf1
 bgp router-id 5.5.5.5
 no bgp ebgp-requires-policy
 no bgp default ipv4-unicast
 !
 address-family ipv6 unicast
  redistribute connected
  sid vpn export auto
  rd vpn export 30:151
  rt vpn both 30:15
  export vpn
  import vpn
 exit-address-family
exit
!
router isis SRv6
 is-type level-1
 net 49.0001.0001.0001.0005.00
 lsp-mtu 1300
 spf prefix-priority critical ACL_CRITICAL
 topology ipv6-unicast
 fast-reroute priority-limit critical level-1
 segment-routing on
 segment-routing node-msd 8
exit
!
ipv6 access-list ACL_CRITICAL seq 5 permit 2001:d30:8000:243::/64
ipv6 access-list ACL_CRITICAL seq 10 permit 2001:d30::/48
!
segment-routing
 srv6
  locators
   locator SRv6_Loc
    prefix 2001:d30:8000:5::/64
   exit
   !
  exit
  !
 exit
 !
 traffic-eng
 exit
exit
!
end

Expected behavior

B> 2001:d30:8000:3:100::/128 [20/0] is directly connected, vrf1, seg6local End.DT6 table 10, seg6 ::, weight 1, 00:04:31

Screenshots

image

I got B>r 2001:d30:8000:3:100::/128 [20/0] is directly connected, vrf1, seg6local End.DT6 table 10, seg6 ::, weight 1, 00:04:31 **Versions**
  • OS Version: Debian GNU/Linux 11 (bullseye)
  • Kernel: Linux 5.10.63-v8+
  • FRR Version: 8.1 stable version

Additional context

I got rejected status at routing table when using show ipv6 route command
@iqraaswad iqraaswad added the triage Needs further investigation label Mar 27, 2022
@iqraaswad iqraaswad changed the title Got BGP Rejected at Routing Table (Show IPv6 Route) Got BGP Rejected at Routing Table (Show IPv6 Route) - SRv6 L3VPN Mar 27, 2022
@donaldsharp
Copy link
Member

what does the log say about the rejection?

@iqraaswad
Copy link
Author

I don't know how to produce a log. I have checked file /var/log/frr/frr.log but found nothing 🥲. I just got daemon status from that file

@iqraaswad
Copy link
Author

what does the log say about the rejection?

I don't know exactly how to do that.. but this is my interface brief

Interface       Status  VRF             Addresses
---------       ------  ---             ---------
eth0            up      default         169.254.214.48/16
eth0.231        up      default         202.249.25.143/27
                                        + 2001:d30:101:2:b7bc:71c7:3ba1:badb/64
eth0.243        down    default         
eth0.244        up      default         192.168.244.13/24
                                        + 2001:d30:8000:244::13/64
lo              up      default         2001:d30:8000:8000::3/128
pimreg          up      default         
wlan0           down    default         

Interface       Status  VRF             Addresses
---------       ------  ---             ---------
eth0.241        up      vrf1            192.168.241.13/24
                                                  + 2001:d30:8000:241::13/64
pimreg10        up      vrf1            
vrf1                 up      vrf1            169.254.68.80/16

i create manually vrf1 using this command

 ip link add vrf1 type vrf table 10 
ip link set dev eth0.241 master vrf1

and when trying to ping eth0.241 from same router (FRRouting cli) ... i got

pi3# ping 2001:d30:8000:241::13
PING 2001:d30:8000:241::13(2001:d30:8000:241::13) 56 data bytes
From 2001:d30:8000:241::13 icmp_seq=1 Destination unreachable: Address unreachable
From 2001:d30:8000:241::13 icmp_seq=2 Destination unreachable: Address unreachable
From 2001:d30:8000:241::13 icmp_seq=3 Destination unreachable: Address unreachable
^C
--- 2001:d30:8000:241::13 ping statistics ---
5 packets transmitted, 0 received, +3 errors, 100% packet loss, time 4078ms

any solutions please.. 😢

@donaldsharp
Copy link
Member

log file /tmp/zebra.log
debug zebra kernel
debug zebra rib detail

then cause bgp to reinstall the route

@iqraaswad
Copy link
Author

after do that... i got some error from zebra.log

2022/04/02 16:41:32 BGP: [KTTWK-0CPJ7][EC 100663303] sockopt_tcp_signature: setsockopt(24): Protocol not available
2022/04/02 16:41:32 BGP: [S3XWN-YG5FD][EC 33554494] Unable to set TCP MD5 option on socket for peer 2001:d30:8000:8000::4 (sock=24): Protocol not available
2022/04/02 16:41:32 ZEBRA: [S77YB-SX0AW] assign_srv6_locator_chunk: locator SRv6_Loc was not found
2022/04/02 16:41:32 ZEBRA: [XTSCQ-WA14G] Unable to assign locator chunk to bgp instance 0
2022/04/02 16:41:32 ZEBRA: [T1KCW-RSXEY] zebra_redistribute: client bgp 2001:d30:8000:240::/64(71) checking: selected=8, type=2, distance=0, metric=0 zebra_check_addr=1
2022/04/02 16:41:32 ZEBRA: [T1KCW-RSXEY] zebra_redistribute: client bgp fe80::/64(71) checking: selected=8, type=2, distance=0, metric=0 zebra_check_addr=0
2022/04/02 16:41:32 ZEBRA: [T1KCW-RSXEY] zebra_redistribute: client bgp fe80::/64(71) checking: selected=0, type=2, distance=0, metric=0 zebra_check_addr=0
2022/04/02 16:41:32 BGP: [QKHW2-PWEQT] ensure_vrf_tovpn_sid: not allocated new sid for vrf VRF vrf1: afi IPv6
2022/04/02 16:41:32 BGP: [QKHW2-PWEQT] ensure_vrf_tovpn_sid: not allocated new sid for vrf VRF vrf1: afi IPv6
2022/04/02 16:41:32 BGP: [QKHW2-PWEQT] ensure_vrf_tovpn_sid: not allocated new sid for vrf VRF vrf1: afi IPv6
2022/04/02 16:41:32 BGP: [QKHW2-PWEQT] ensure_vrf_tovpn_sid: not allocated new sid for vrf VRF vrf1: afi IPv6
2022/04/02 16:41:32 ZEBRA: [V98V0-MTWPF] client 125 says hello and bids fair to announce only isis routes vrf=0
2022/04/02 16:41:32 ZEBRA: [TAFZ7-2C7QV] client protocol instance 1
2022/04/02 16:41:32 ZEBRA: [PNYQ5-RZEVA] rib_delete: (0:254):2001:d30:8000:10::/64: type 8 doesn't exist in rib
2022/04/02 16:41:32 ZEBRA: [PNYQ5-RZEVA] rib_delete: (0:254):2001:d30:8000:242::/64: type 8 doesn't exist in rib
2022/04/02 16:41:32 ZEBRA: [PNYQ5-RZEVA] rib_delete: (0:254):2001:d30:8000:8000::1/128: type 8 doesn't exist in rib
2022/04/02 16:41:32 ZEBRA: [MFYWV-KH3MC] rib_add_multipath_nhe: (0:?):2001:d30:8000:1:100::/128: Inserting route rn 0x559537ee80, re 0x5594c08520 (bgp) existing 0x0, same_count 0
2022/04/02 16:41:32 ZEBRA: [Q4T2G-E2SQF] rib_add_multipath_nhe: dumping RE entry 0x5594c08520 for 2001:d30:8000:1:100::/128 vrf default(0)
2022/04/02 16:41:32 ZEBRA: [M5M58-9PD2R] 2001:d30:8000:1:100::/128: uptime == 8831241, type == 9, instance == 0, table == 254
2022/04/02 16:41:32 ZEBRA: [RVZMM-N7DME] 2001:d30:8000:1:100::/128: metric == 0, mtu == 0, distance == 20, flags == Recursion status == None 
2022/04/02 16:41:32 ZEBRA: [Q1NW5-NWY7P] 2001:d30:8000:1:100::/128: nexthop_num == 1, nexthop_active_num == 0
2022/04/02 16:41:32 ZEBRA: [TFHQ8-TC30H] 2001:d30:8000:1:100::/128: NH vrf1[71] vrf default(0) wgt 1, with flags 
2022/04/02 16:41:32 ZEBRA: [SCETK-GQ9E4] 2001:d30:8000:1:100::/128: dump complete
2022/04/02 16:41:32 ZEBRA: [GCGMT-SQR82] rib_link: (0:?):2001:d30:8000:1:100::/128: rn 0x559537ee80 adding dest
2022/04/02 16:41:32 ZEBRA: [QEVVE-G3FQQ] rib_meta_queue_add: (0:254):2001:d30:8000:1:100::/128: queued rn 0x559537ee80 into sub-queue 6
2022/04/02 16:41:32 ZEBRA: [NZNZ4-7P54Y] default(0:254):2001:d30:8000:1:100::/128: Processing rn 0x559537ee80
2022/04/02 16:41:32 ZEBRA: [ZJVZ4-XEGPF] default(0:254):2001:d30:8000:1:100::/128: Examine re 0x5594c08520 (bgp) status: Changed flags: Recursion dist 20 metric 0
2022/04/02 16:41:32 ZEBRA: [JPJF4-TGCY5] default(0:254):2001:d30:8000:1:100::/128: After processing: old_selected 0x0 new_selected 0x5594c08520 old_fib 0x0 new_fib 0x5594c08520
2022/04/02 16:41:32 ZEBRA: [ZFRN1-EZNJZ] default(0:254):2001:d30:8000:1:100::/128: Adding route rn 0x559537ee80, re 0x5594c08520 (bgp)
2022/04/02 16:41:32 ZEBRA: [QZ1V6-CRT8D] default(0:254):2001:d30:8000:1:100::/128 rn 0x559537ee80 dequeued from sub-queue 6
2022/04/02 16:41:32 ZEBRA: [YXCJP-0WZWV] netlink_nexthop_msg_encode: ID (42): directly connected, vrf1(71) vrf default(0)  
2022/04/02 16:41:32 ZEBRA: [R43C6-KYHWT] netlink_nexthop_msg_encode: RTM_NEWNEXTHOP, id=42
2022/04/02 16:41:32 ZEBRA: [YXPF5-B2CE0] netlink_route_multipath_msg_encode: RTM_NEWROUTE 2001:d30:8000:1:100::/128 vrf 0(254)
2022/04/02 16:41:32 ZEBRA: [J87BH-XW5PP] netlink_route_multipath_msg_encode: 2001:d30:8000:1:100::/128 nhg_id is 42
2022/04/02 16:41:32 ZEBRA: [HYEHE-CQZ9G] nl_batch_send: netlink-dp (NS 0), batch size=132, msg cnt=2
2022/04/02 16:41:32 ZEBRA: [HSYZM-HV7HF] Extended Error: lwt encapsulation type not supported
2022/04/02 16:41:32 ZEBRA: [WVJCK-PPMGD][EC 4043309093] netlink-dp (NS 0) error: Operation not supported, type=RTM_NEWNEXTHOP(104), seq=39, pid=2619130526
2022/04/02 16:41:32 ZEBRA: [QTT8V-3ZQ34] nl_batch_read_resp: netlink error message seq=39 
2022/04/02 16:41:32 ZEBRA: [HSYZM-HV7HF] Extended Error: Nexthop id does not exist
2022/04/02 16:41:32 ZEBRA: [WVJCK-PPMGD][EC 4043309093] netlink-dp (NS 0) error: Invalid argument, type=RTM_NEWROUTE(24), seq=40, pid=2619130526
2022/04/02 16:41:32 ZEBRA: [QTT8V-3ZQ34] nl_batch_read_resp: netlink error message seq=40 
2022/04/02 16:41:32 ZEBRA: [P2XBZ-RAFQ5][EC 4043309074] Failed to install Nexthop ID (42) into the kernel
2022/04/02 16:41:32 ZEBRA: [TX7F6-F3TSG] default(0:254):2001:d30:8000:1:100::/128: Route install failed
2022/04/02 16:41:32 ZEBRA: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:01
2022/04/02 16:41:32 RIP: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:01
2022/04/02 16:41:32 RIPNG: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:01
2022/04/02 16:41:32 OSPF: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:01
2022/04/02 16:41:32 OSPF6: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:01

@donaldsharp
Copy link
Member

2022/04/02 16:41:32 ZEBRA: [HSYZM-HV7HF] Extended Error: lwt encapsulation type not supported looks like your kernel does not support this encapsulation.

@donaldsharp
Copy link
Member

@slankdev do you know what version of the linux kernel @iqraaswad should be using?

@iqraaswad
Copy link
Author

thanks for mentioning kernel.. i've recompiling kernel.. and everything works now...

@ale0524
Copy link

ale0524 commented May 5, 2023

你知道应该使用什么版本的 Linux 内核吗？

Hello, I also encountered the same problem on my end;

Operating system: Centos 7.6

Kernel version: 5.4.241-1. el7.elrepo.x86_ sixty-four

May I ask which kernel version should I use??
ipv6 route

@ale0524
Copy link

ale0524 commented May 5, 2023

thanks for mentioning kernel.. i've recompiling kernel.. and everything works now...
Hello, I also encountered the same problem on my end;

Operating system: Centos 7.6

Kernel version: 5.4.241-1. el7.elrepo.x86_ sixty-four

May I ask which kernel version should I use??
ipv6 route

@ale0524
Copy link

ale0524 commented May 5, 2023

May 5 07:04:58 R1 zebra[5409]: [WVJCK-PPMGD][EC 4043309093] netlink-dp (NS 0) error: Invalid argument, type=RTM_NEWROUTE(24), seq=6491, pid=4093804124
May 5 07:04:58 R1 zebra[5409]: [X5XE1-RS0SW][EC 4043309074] Failed to install Nexthop (392[fe80::4bfa:b4f7:bb81:6b1d if 3]) into the kernel
May 5 07:04:58 R1 zebra[5409]: [VYKYC-709DP] evpn1(11:10):2001:db8:b::/64: Route install failed
May 5 07:05:00 R1 pim6d[5440]: [PWY4E-BRD9V] zclient_read_nexthop: found non-ifindex nexthop type=6 for address *(default)

@ale0524
Copy link

ale0524 commented May 5, 2023

May 5 06:23:52 R1 zebra[5409]: [SWQK6-6JY63][EC 4043309074] 0:254:2001:db8:2::/128: Failed to enqueue dataplane install

@ale0524
Copy link

ale0524 commented May 6, 2023

1

@ale0524
Copy link

ale0524 commented May 6, 2023

May 6 04:00:35 R1 pim6d[4083]: [PWY4E-BRD9V] zclient_read_nexthop: found non-ifindex nexthop type=6 for address *(default)
May 6 04:01:31 R2 pim6d[4064]: [PWY4E-BRD9V] zclient_read_nexthop: found non-ifindex nexthop type=6 for address *(default)

@liuyuefengcn
Copy link

liuyuefengcn commented Oct 16, 2023 •
edited
Loading

@iqraaswad
Hello, I am also having the same issue, can you tell me what version of linux kernel I should use? Thank you so much!
CentOS Linux release 8.5.2111
Linux version 4.18.0-348.el8.x86_64
image

@liuyuefengcn
Copy link

I've upgraded the linux kernel version to 6.5.7-1.el8.elrepo.x86_64, but the problem persists, can anyone help figure it out?

@ale0524
Copy link

ale0524 commented Oct 16, 2023

@liuyuefengcn
Currently, only these use cases are supported:
IPv6 L3VPN (with SRv6 End.DT6) -> kernel >=4.14 -> config example here
IPv4 L3VPN (with SRv6 End.DT4) -> kernel >= 5.11 -> config example here
IPv4/IPv6 L3VPN (with SRv6 End.DT46) -> kernel >= 5.14 -> config example here

@liuyuefengcn
Copy link

liuyuefengcn commented Nov 1, 2023 •
edited
Loading

@ale0524 ok, thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
triage Needs further investigation
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@donaldsharp @iqraaswad @ale0524 @liuyuefengcn