MPLS Layer 3 VPN Explained

Hi,

Thanks for the nice explanation. I am still trying to understand the RD and RT concept correctly. I have three questions for you.

  1. What will the receiving PE (in our case lets assume PE2) do with RD ? I know it will use RT to download the route to corresponding VRF. But what is the use of RD here ?

  2. Why can’t we use RD itself(maybe as label) in the transport instead having an extra VPN label ?

Looking forward to your answer.

Thanks Rene for good lesson. Do you have any lesson for L2VPN…?

Thanks,
Durga

Durga,
NetworkLessons.com has one on Any Transport over MPLS

Aswin,
This is a very good question, and I thought I understood this topic well until you made me really think about what each component does. Part of what is confusing about RD vs RT is that their format can be so similar, but they do very different things.

The key thing to understand about RT vs RD is the relationship they have with BGP’s VPNV4 (or V6). Route Targets are BGP extended community attributes, so these are just extra information that is carried along with a BGP update. Route Distinguishers, however, modify the actual address being advertised by being pre-pended to it. For example, with the RD of 15:15 and address space of 172.16.15.0/24, these would be combined as 15:15:172.16.15.0/24.

Why does this matter? It matters once you understand BGP does not use the extended community attributes to determine uniqueness–instead it is the combination of the advertised address prefix and subnet mask. Here’s an example of why the RD is needed:

Imagine you have two PE routers, and each has connections two different customers–say Customer A using VRF A and Customer B using VRF B. The two customers are sharing the same address space - say 10.10.10.0/24. PE1 sends an update to PE2 about Customer A. Of course, included in the update is the RT Export corresponding to VRF A. PE2 would process the update and place it into its VRF A thanks to the RT included. Now, suppose PE1 sends another update, but this time about VRF B. When PE2 receives it, it believes PE1 is talking about the SAME network. Remember, this is because BGP looks only at the prefix + mask to determine uniqueness. In this case, PE2 would actually remove 10.10.10.0/24 from VRF A, and put it in VRF B.

This is why we need the RD. It tells BGP that even though the same prefix + mask happens to be used across multiple VRFs, they are actually different objects and should be treated differently.

1 Like

Great Article. Many thx.

Hi Rene,

what is VPN Label, is it a RD + IPV4 ?

Regards
Pradeep

Hi Pradeep,

The label only has the label value, EXP, TTL and bottom of stack bit. You can find the explanation here:

MPLS Labels

Rene

Rene,

If the provider backbone exists in a different VRF from Customer A (vrf:CustA vs. vrf:default), how does the BGP next-hop on PE1 (vrf:CustA) towards Customer A’s 192.168.2.0/24 prefix reflect a next-hop from a different VRF (i.e. next-hop = router P’s IP from vrf:default)? I am trying to trace the decision making process from one end to the other and this part has me stumped.

The BGP VPNV4 address family solves this issue. The PE routers, based on RT Exports and RT Imports, knows which other PE endpoint the VRF traffic needs to travel to, and this is encapsulated as part of VPNV4 using a VPN Label (this will be the bottom label). The PE then adds the second LDP label (called the transit label) on top of this, so that the data will traverse the provider’s core MPLS.

Doing it this way allows the core to have no knowledge of the details of the VRFs.

If I am not understanding your question, please try asking again :slight_smile:

Hi Andrew,

OK so it sounds like the next-hop in vrf:CustA is known because of the VPNv4 address family advertising it from PE2 to PE1 via a Path Attribute, but why does the BGP table accept it as a valid route if the next-hop is in a different VRF? I thought that VRFs were totally isolated routing tables that only leaked over with manual intervention (aka import an RT or something). Am I just thinking about this the wrong way?

The way that I learned to understand basic IP routing back in the day was as follows:

  1. Look at PC1’s routing table. Find the egress interface and next-hop.
  2. Look at PC1’s L2 packet header towards the next-hop IP (aka review the ARP entry of the next-hop IP).
  3. Look at R1’s routing table and find the next-hop towards the destination.
  4. Look recursively for R1’s egress interface toward the next-hop
  5. Look at R1’s L2 packet rewrite towards the next-hop IP
  6. Repeat all the way to destination.

Seemed simple in old-school CCNA/CCNP networking. Find next-hop, find egress interface, build L2, remove L2, find next-hop, find egress interface rebuild L2, etc.

I expected to be able to rebuild this lab and a similar one over at PacketLife on MPLS L3 VPNs and be able to review L3 forwarding, find the egress interface, and rebuild the L2 portion of the packet. It seems that either my above method kinda falls apart and I have to assume that there is magic working within MPLS and VPNv4 that allows it to extend beyond the boundaries of a VRF that I had grown to know from VRF-Lite, or I’m dumb and am just not thinking about it the right way.

The following walkthrough of my madness was taken from Rene’s MPLS L3 VPN Configuration article.

!!! CE1 shows 5.5.5.5 with a next-hop of 192.168.12.2. That’s fine. Same VRF !!!

CE1#show ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

C    192.168.12.0/24 is directly connected, FastEthernet0/0
     1.0.0.0/32 is subnetted, 1 subnets
C       1.1.1.1 is directly connected, Loopback0
     5.0.0.0/32 is subnetted, 1 subnets
B       5.5.5.5 [20/0] via 192.168.12.2, 00:00:40
!!!! CE1 builds an L2 header as follows !!!!
CE1#show ip arp 192.168.12.2
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  192.168.12.2            3   c203.1cb4.0000  ARPA   FastEthernet0/0

!!! PE1 receives the packet in on F0/0 (vrf:CUSTOMER) and looks for the destination. It finds the destination with a next-hop of 4.4.4.4. !!!

PE1#show ip route vrf CUSTOMER

Routing Table: CUSTOMER
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

C    192.168.12.0/24 is directly connected, FastEthernet0/0
     1.0.0.0/32 is subnetted, 1 subnets
B       1.1.1.1 [20/0] via 192.168.12.1, 00:03:21
     5.0.0.0/32 is subnetted, 1 subnets
B       5.5.5.5 [200/0] via 4.4.4.4, 00:01:53

!!! My problem is that I don’t understand how BGP can add it to the routing table if the next-hop isn’t reachable in the CUSTOMER VRF. !!!

PE1#show ip route vrf CUSTOMER 4.4.4.4
% Network not in table

!!! The 4.4.4.4 network exists only in the provider’s backbone VRF (vrf:Default) !!!

PE1#show ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

     2.0.0.0/32 is subnetted, 1 subnets
C       2.2.2.2 is directly connected, Loopback0
     3.0.0.0/32 is subnetted, 1 subnets
O       3.3.3.3 [110/11] via 192.168.23.3, 00:05:05, FastEthernet0/1
     4.0.0.0/32 is subnetted, 1 subnets
O       4.4.4.4 [110/21] via 192.168.23.3, 00:04:18, FastEthernet0/1
C    192.168.23.0/24 is directly connected, FastEthernet0/1
O    192.168.34.0/24 [110/20] via 192.168.23.3, 00:04:28, FastEthernet0/1

Hi Rene,

One doubt, when PE2 router receives VPNV4 routes from PE1 ,PE2 will import the routes to correct vrf, but you mentioned that PE2 will export the routes to vrf.
My understanding is that PE1 will export the route and PE2 will import the routes. pls explain

Rohitendu,
You are both right. The problem is one of language. Whether R2 imports or exports routes to the VRF is a matter of perspective–are you looking at it from the point of view of the router or the VRF? How about instead of the word “export” you just use the word “place.” So, R2 receives the VPNV4 route from PE1, and places these routes into the correct VRF based on the route-target.

Hi Rene, Andrew
I am afraid I still don’t understand one thing- why do we need vpn label if we have both RD and RT’s ?
It was said the router wouldn’t know what VRF the route belongs to… well:
When PE1 advertises the route to PE2 , this route is unique for BGP because of RD and PE2 also knows in what VRF to install it thanks to Route Target value.
So the MPLS VPN label seems to be redundant as the BGP can figure the VRF out based solely on the Route Targets …
What am I missing in this puzzle :slight_smile:
Thank you
Edit- ok, I think I mix up the control and data plane again , found nice explanation here :

3 Likes

Rene,
this is really very Excellent article. Simple and Concise

19 posts were merged into an existing topic: MPLS Layer 3 VPN Explained

Hi, I too have the same doubt. But I heard somewhere that , nexthop will be 4.4.4.4 but interface will be some “magic interface”. And transmit function of that “magic interface” will give it to the chip (MPLS L3VPN aware chip) (after adding transport and VPN label , I don’t know) and then chip will forward it. Means after PE1 (VRF 1), packet will travel in data plane and it will not come to the Kernel for forwarding. So after PE1 (VRF 1), we will not refer to Kernel routing table till PE2 (VRF 1).

Hope my explanation is clear. Rene to confirm the same ?

Hi Rene, please tell some use cases to explain, when to go with MPLS L2 VPN and when to go with MPLS L3 VPN.

Hello Harmeet

In essence, MPLS L2 and L3 VPNs will be used depending on the needs of the application. The needs in this case don’t depend so much on the actual protocols, but on whether you require a separate subnet for each remote site or if you require two or more sites to be on the same subnet.

It’s always a good idea to separate subnets and not to span them over WAN links, however, if the needs of the applications you are running require this, it can be done. It all depends on your requirements.

I hope this has been helpful!

Laz

Thanks Rene for the great article.

So VPN label only comes into picture when we use VRFs in PE routers? Please clarify.

By using RTs, it will place customer routes into respective RIB of VRFs, and then LIB will be derived from it. Now if we don’t have VPN label attached to the packet then it will not be part of MPLS forwarding table of that particular VRF and it doesn’t get the out going interface details so it doesn’t know where to send this packet!. Is my understanding correct?

Thanks,
Kumara

Hi @kumaracp10,

Many thanks for your excellent question. If you are referring to MPLS labels, this is primarily used as a method to quickly switch IP packets within the MPLS core. This is the most basic feature of MPLS so it is used in all MPLS networks even if there is no VPN overlay. The 1st MPLS tag exists only to enable MPLS forwarding plane operations.

**If we decide to operate a VPN over MPLS, a second MPLS tag is added** to allow PEs to know how to efficiently forward incoming packets.

In MPLS there are two basic rules that help us unpick the architecture:

  1. Separate the control plane from the forwarding plane:
    Control plane: routes being shared between CE<==any routing protocol==>PE and PE<==BGP==>PE)
    Forwarding plane: MPLS labels applied to real customer packets full of customer data, to forward them efficiently.

  2. Always remember, packets with routes go one way, packet with useful customer data go the other way. “We send routes in order to receive data”.

I hope that helps a little to clarify. Kind regards,
Jon.

1 Like