Hi Andrew,
OK so it sounds like the next-hop in vrf:CustA is known because of the VPNv4 address family advertising it from PE2 to PE1 via a Path Attribute, but why does the BGP table accept it as a valid route if the next-hop is in a different VRF? I thought that VRFs were totally isolated routing tables that only leaked over with manual intervention (aka import an RT or something). Am I just thinking about this the wrong way?
The way that I learned to understand basic IP routing back in the day was as follows:
- Look at PC1’s routing table. Find the egress interface and next-hop.
- Look at PC1’s L2 packet header towards the next-hop IP (aka review the ARP entry of the next-hop IP).
- Look at R1’s routing table and find the next-hop towards the destination.
- Look recursively for R1’s egress interface toward the next-hop
- Look at R1’s L2 packet rewrite towards the next-hop IP
- Repeat all the way to destination.
Seemed simple in old-school CCNA/CCNP networking. Find next-hop, find egress interface, build L2, remove L2, find next-hop, find egress interface rebuild L2, etc.
I expected to be able to rebuild this lab and a similar one over at PacketLife on MPLS L3 VPNs and be able to review L3 forwarding, find the egress interface, and rebuild the L2 portion of the packet. It seems that either my above method kinda falls apart and I have to assume that there is magic working within MPLS and VPNv4 that allows it to extend beyond the boundaries of a VRF that I had grown to know from VRF-Lite, or I’m dumb and am just not thinking about it the right way.
The following walkthrough of my madness was taken from Rene’s MPLS L3 VPN Configuration article.
!!! CE1 shows 5.5.5.5 with a next-hop of 192.168.12.2. That’s fine. Same VRF !!!
CE1#show ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route
Gateway of last resort is not set
C 192.168.12.0/24 is directly connected, FastEthernet0/0
1.0.0.0/32 is subnetted, 1 subnets
C 1.1.1.1 is directly connected, Loopback0
5.0.0.0/32 is subnetted, 1 subnets
B 5.5.5.5 [20/0] via 192.168.12.2, 00:00:40
!!!! CE1 builds an L2 header as follows !!!!
CE1#show ip arp 192.168.12.2
Protocol Address Age (min) Hardware Addr Type Interface
Internet 192.168.12.2 3 c203.1cb4.0000 ARPA FastEthernet0/0
!!! PE1 receives the packet in on F0/0 (vrf:CUSTOMER) and looks for the destination. It finds the destination with a next-hop of 4.4.4.4. !!!
PE1#show ip route vrf CUSTOMER
Routing Table: CUSTOMER
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route
Gateway of last resort is not set
C 192.168.12.0/24 is directly connected, FastEthernet0/0
1.0.0.0/32 is subnetted, 1 subnets
B 1.1.1.1 [20/0] via 192.168.12.1, 00:03:21
5.0.0.0/32 is subnetted, 1 subnets
B 5.5.5.5 [200/0] via 4.4.4.4, 00:01:53
!!! My problem is that I don’t understand how BGP can add it to the routing table if the next-hop isn’t reachable in the CUSTOMER VRF. !!!
PE1#show ip route vrf CUSTOMER 4.4.4.4
% Network not in table
!!! The 4.4.4.4 network exists only in the provider’s backbone VRF (vrf:Default) !!!
PE1#show ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route
Gateway of last resort is not set
2.0.0.0/32 is subnetted, 1 subnets
C 2.2.2.2 is directly connected, Loopback0
3.0.0.0/32 is subnetted, 1 subnets
O 3.3.3.3 [110/11] via 192.168.23.3, 00:05:05, FastEthernet0/1
4.0.0.0/32 is subnetted, 1 subnets
O 4.4.4.4 [110/21] via 192.168.23.3, 00:04:18, FastEthernet0/1
C 192.168.23.0/24 is directly connected, FastEthernet0/1
O 192.168.34.0/24 [110/20] via 192.168.23.3, 00:04:28, FastEthernet0/1