MPLS Layer 3 VPN Explained

(Justin D) #48

Rene,

If the provider backbone exists in a different VRF from Customer A (vrf:CustA vs. vrf:default), how does the BGP next-hop on PE1 (vrf:CustA) towards Customer A’s 192.168.2.0/24 prefix reflect a next-hop from a different VRF (i.e. next-hop = router P’s IP from vrf:default)? I am trying to trace the decision making process from one end to the other and this part has me stumped.

(Andrew P) #49

The BGP VPNV4 address family solves this issue. The PE routers, based on RT Exports and RT Imports, knows which other PE endpoint the VRF traffic needs to travel to, and this is encapsulated as part of VPNV4 using a VPN Label (this will be the bottom label). The PE then adds the second LDP label (called the transit label) on top of this, so that the data will traverse the provider’s core MPLS.

Doing it this way allows the core to have no knowledge of the details of the VRFs.

If I am not understanding your question, please try asking again :slight_smile:

(Justin D) #50

Hi Andrew,

OK so it sounds like the next-hop in vrf:CustA is known because of the VPNv4 address family advertising it from PE2 to PE1 via a Path Attribute, but why does the BGP table accept it as a valid route if the next-hop is in a different VRF? I thought that VRFs were totally isolated routing tables that only leaked over with manual intervention (aka import an RT or something). Am I just thinking about this the wrong way?

The way that I learned to understand basic IP routing back in the day was as follows:

  1. Look at PC1’s routing table. Find the egress interface and next-hop.
  2. Look at PC1’s L2 packet header towards the next-hop IP (aka review the ARP entry of the next-hop IP).
  3. Look at R1’s routing table and find the next-hop towards the destination.
  4. Look recursively for R1’s egress interface toward the next-hop
  5. Look at R1’s L2 packet rewrite towards the next-hop IP
  6. Repeat all the way to destination.

Seemed simple in old-school CCNA/CCNP networking. Find next-hop, find egress interface, build L2, remove L2, find next-hop, find egress interface rebuild L2, etc.

I expected to be able to rebuild this lab and a similar one over at PacketLife on MPLS L3 VPNs and be able to review L3 forwarding, find the egress interface, and rebuild the L2 portion of the packet. It seems that either my above method kinda falls apart and I have to assume that there is magic working within MPLS and VPNv4 that allows it to extend beyond the boundaries of a VRF that I had grown to know from VRF-Lite, or I’m dumb and am just not thinking about it the right way.

The following walkthrough of my madness was taken from Rene’s MPLS L3 VPN Configuration article.

!!! CE1 shows 5.5.5.5 with a next-hop of 192.168.12.2. That’s fine. Same VRF !!!

CE1#show ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

C    192.168.12.0/24 is directly connected, FastEthernet0/0
     1.0.0.0/32 is subnetted, 1 subnets
C       1.1.1.1 is directly connected, Loopback0
     5.0.0.0/32 is subnetted, 1 subnets
B       5.5.5.5 [20/0] via 192.168.12.2, 00:00:40
!!!! CE1 builds an L2 header as follows !!!!
CE1#show ip arp 192.168.12.2
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  192.168.12.2            3   c203.1cb4.0000  ARPA   FastEthernet0/0

!!! PE1 receives the packet in on F0/0 (vrf:CUSTOMER) and looks for the destination. It finds the destination with a next-hop of 4.4.4.4. !!!

PE1#show ip route vrf CUSTOMER

Routing Table: CUSTOMER
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

C    192.168.12.0/24 is directly connected, FastEthernet0/0
     1.0.0.0/32 is subnetted, 1 subnets
B       1.1.1.1 [20/0] via 192.168.12.1, 00:03:21
     5.0.0.0/32 is subnetted, 1 subnets
B       5.5.5.5 [200/0] via 4.4.4.4, 00:01:53

!!! My problem is that I don’t understand how BGP can add it to the routing table if the next-hop isn’t reachable in the CUSTOMER VRF. !!!

PE1#show ip route vrf CUSTOMER 4.4.4.4
% Network not in table

!!! The 4.4.4.4 network exists only in the provider’s backbone VRF (vrf:Default) !!!

PE1#show ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

     2.0.0.0/32 is subnetted, 1 subnets
C       2.2.2.2 is directly connected, Loopback0
     3.0.0.0/32 is subnetted, 1 subnets
O       3.3.3.3 [110/11] via 192.168.23.3, 00:05:05, FastEthernet0/1
     4.0.0.0/32 is subnetted, 1 subnets
O       4.4.4.4 [110/21] via 192.168.23.3, 00:04:18, FastEthernet0/1
C    192.168.23.0/24 is directly connected, FastEthernet0/1
O    192.168.34.0/24 [110/20] via 192.168.23.3, 00:04:28, FastEthernet0/1
(ROHITENDU M) #51

Hi Rene,

One doubt, when PE2 router receives VPNV4 routes from PE1 ,PE2 will import the routes to correct vrf, but you mentioned that PE2 will export the routes to vrf.
My understanding is that PE1 will export the route and PE2 will import the routes. pls explain

(Andrew P) #52

Rohitendu,
You are both right. The problem is one of language. Whether R2 imports or exports routes to the VRF is a matter of perspective–are you looking at it from the point of view of the router or the VRF? How about instead of the word “export” you just use the word “place.” So, R2 receives the VPNV4 route from PE1, and places these routes into the correct VRF based on the route-target.

(Marek O) #53

Hi Rene, Andrew
I am afraid I still don’t understand one thing- why do we need vpn label if we have both RD and RT’s ?
It was said the router wouldn’t know what VRF the route belongs to… well:
When PE1 advertises the route to PE2 , this route is unique for BGP because of RD and PE2 also knows in what VRF to install it thanks to Route Target value.
So the MPLS VPN label seems to be redundant as the BGP can figure the VRF out based solely on the Route Targets …
What am I missing in this puzzle :slight_smile:
Thank you
Edit- ok, I think I mix up the control and data plane again , found nice explanation here :

2 Likes
(Samir S) #54

Rene,
this is really very Excellent article. Simple and Concise

(Shantel - Networklessons.com) split this topic #55

19 posts were merged into an existing topic: MPLS Layer 3 VPN Explained

(Harmeet S) #56

Hi, I too have the same doubt. But I heard somewhere that , nexthop will be 4.4.4.4 but interface will be some “magic interface”. And transmit function of that “magic interface” will give it to the chip (MPLS L3VPN aware chip) (after adding transport and VPN label , I don’t know) and then chip will forward it. Means after PE1 (VRF 1), packet will travel in data plane and it will not come to the Kernel for forwarding. So after PE1 (VRF 1), we will not refer to Kernel routing table till PE2 (VRF 1).

Hope my explanation is clear. Rene to confirm the same ?

(Harmeet S) #57

Hi Rene, please tell some use cases to explain, when to go with MPLS L2 VPN and when to go with MPLS L3 VPN.

(Lazaros Agapides) #58

Hello Harmeet

In essence, MPLS L2 and L3 VPNs will be used depending on the needs of the application. The needs in this case don’t depend so much on the actual protocols, but on whether you require a separate subnet for each remote site or if you require two or more sites to be on the same subnet.

It’s always a good idea to separate subnets and not to span them over WAN links, however, if the needs of the applications you are running require this, it can be done. It all depends on your requirements.

I hope this has been helpful!

Laz

(Kumara N) #59

Thanks Rene for the great article.

So VPN label only comes into picture when we use VRFs in PE routers? Please clarify.

By using RTs, it will place customer routes into respective RIB of VRFs, and then LIB will be derived from it. Now if we don’t have VPN label attached to the packet then it will not be part of MPLS forwarding table of that particular VRF and it doesn’t get the out going interface details so it doesn’t know where to send this packet!. Is my understanding correct?

Thanks,
Kumara

(jonrandall) #60

Hi @kumaracp10,

Many thanks for your excellent question. If you are referring to MPLS labels, this is primarily used as a method to quickly switch IP packets within the MPLS core. This is the most basic feature of MPLS so it is used in all MPLS networks even if there is no VPN overlay. The 1st MPLS tag exists only to enable MPLS forwarding plane operations.

**If we decide to operate a VPN over MPLS, a second MPLS tag is added** to allow PEs to know how to efficiently forward incoming packets.

In MPLS there are two basic rules that help us unpick the architecture:

  1. Separate the control plane from the forwarding plane:
    Control plane: routes being shared between CE<==any routing protocol==>PE and PE<==BGP==>PE)
    Forwarding plane: MPLS labels applied to real customer packets full of customer data, to forward them efficiently.

  2. Always remember, packets with routes go one way, packet with useful customer data go the other way. “We send routes in order to receive data”.

I hope that helps a little to clarify. Kind regards,
Jon.

(Manami B) #61

Hi Rene,

I have couple of doubts.

  1. Why we need RD? We have for example VRF Red or VRF Blue which is separate from each other so why we are using RD to make the prefix unique.
  2. After using RD, PE Router will come to know which prefix will be import to which CE Router but again we are using RT here for import and export though we have RD.

Thanks,
Manami

(Lazaros Agapides) #62

Hello Manami

It is true that there is a VRF for each customer at each PE router. In the example in the lesson, Customer A sends a routing update to PE1. PE1 knows that this route belongs to customer A because it has come in on the interface connected to customer A. It is then placed in the appropriate routing table.

However, when PE1 redistributes this route to PE2, there is no way to distinguish which customer it belongs to. Both Customer A and B routing updates will go to PE2 via the same interface. So the RD is required so the PE router will know to which customer the routes belong.

Once the PE receives these routes, it knows to which customer they belong, however, it does not know what to do with it. The router will NOT automatically export each route to the correct customer VRF. This is why the RT is required.

Essentially, the RD makes the route unique (even if the same IP ranges are used) and the RT, by being added to all routes for the Customer A VRF, allow the routers to know which VRF to import and export to.

I hope this has been helpful!

Laz

(Manami B) #63

Hi Laz,

Thank you. Awesome explanation.

I have one more doubt.

Why we are saying MPLS SP network to BGP Free Core? We are running iBGP in MPLS Network?

Can you please help me to understand.

Thanks,
Manami

(Rene Molenaar) #64

Hi Manami,

We run iBGP but only on the PE (Provider Edge) routers. The P routers form the core, and those don’t run BGP.

Rene

(Deep) #65

Awesome simplified lessons here! I did not know forum has such in depth explanations too! Any plans for half duplex VRF explanation?

(Rene Molenaar) #66

Glad to hear you like it :smile:

I might add something on half duplex VRF. I still have MPLS VPN hub and spoke on the list, adding half duplex to it shouldn’t be much work.

(MAODO T) #67

The Conclusion says “In the next lesson I will show you the configuration of everything that I explained above …”. The next lesson I see is “DMVPN”.