Service Provider- Redundant links between PE-to PE routers and PE routers redundancy setups

Hi Lazarus,

I have another query on the MPLS setups. Please make me to understand on these setups how it looks in real scenario.

  1. Can we interconnect the PE routers which are at the edges from one location to another location using redundant links or it is not a good idea?
  2. How are the redundancy made on PE routers at a single site/location, and which protocol is used to have redundant routers?
  3. Which protocol is used for redundancy of PE routers- VRRP, HSRP or
    MC-LAG.

Please refer my topology attached for your reference.

Hello Maheshwaran

You can achieve redundancy by placing two PE devices at one customer. How you would achieve your redundancy depends on several factors.

PE devices are typically routers, and provide you with a layer 3 operation with your on site equipment. In other words, the PEs will act as a default gateway for your customer networks. If this is the case, then the most common solution is to use BGP at the customer edge between your CEs and PEs and perform multi homing. That way, your redundancy is achieved via the BGP routing protocol. Alternatively, you can use an IGP like OSPF or EIGRP to load balance all of your traffic towards the service provider across the PEs. So in such a scenario, you are configuring redundancy using Layer 3 (routing).

Now in the diagrams you shared, you are suggesting using multi chassis LAG, that is, a portchannel (EtherChannel) to connect to PE routers. This is not ideal, because of several reasons:

  1. PE devices are typically routers (Layer 3) and they are provided by the ISP. Although some router platforms do support it, Etherchannel is primarily a technology deployed on switches.
  1. Secondly, even if you have PE routers that do support it, you would have to arrange with the ISP to have them set up their end appropriately, and it must be within the parameters that they are willing to deliver for you.
  2. If you were to create a port channel as shown in your first diagram, you wouldn’t have an active and backup PE, nor would you run iBGP between them or any type of FHRP such as HSRP or VRRP. You would have to use some high availability option for routers such as VSS or vPC (for nexus switches only). These are not good solutions for a PE device that’s part of an ISP.

Now the portchannel option in your second diagram is more feasible because you are eliminating the need for the ISP’s PE router to support MC-LAG, but you are asking the PE to deliver a Layer 2 connection to your premises. That has to be done in coordination with your ISP, but it’s doable.

Keep in mind that EtherChannel, MC-LAG, FHRPs such as VRRP and HSRP etc are all technologies that are typically used within an enterprise network, and often within datacenters. They are not often deployed at the network edge. The preferrable solutions for the redundancy you’re looking for is Layer 3 routing enabling redundancy via multiple paths. That way the PE routers can remain independent and less coordination is necessary with the ISP.

Ultimately, I believe that the most often used solution for such redundancy would be layer 3 redundancy

I hope this has been helpful!

Laz

1 Like

Dear Lazarus,

Greetings for the day…

I have a query on one of the setups which I always gets confused, which will be the recommended option for the redundancy links for a specific customer using L3 MPLS connection acquired from an ISP.

Based on my screenshots attached for your reference,
Case 1:
Here, the topology states the Bangalore District has only one PE1 router to serve the customers connected to it. If Cust-A requires multi-home setup like redundancy links, how service provider would connect them?

Is the case 1 screenshot being a good option to connect Cust-A with redundant link from single router? and this is a single point of failure as well.

Case 2:

Here, the topology states the Chennai District has only one PE3 router to serve the customers connected to it.

If this district is nearer to Bangalore district and can service provider lay or connects alternate link from this PE3 router to Customer Cust-A who resides at Bangalore District?

Can this be a good design if all districts have only one PE routers which is serving?

Will there be more expensive on this cable connectivity for multi homed required designs for the customers? coming from two different districts which is nearby area.

Which of these two cases you would suggest being the best design for
Multi-home setups for the customers and how they should be connected.

Case 3:

There is another district called Karaikudi and has two PE routers acting as primary and backup devices which serves customers with redundant link facility. Does this design fit the real scenario?

Could you please suggest us on this design connectivity links like how real scenario works.

Do you suggest Case 3 topology for Karaikudi district works better as redundant links for Cust-F ?

Which of these districts is the recommended design as per yours?

Case1-Topology
Case2-Topology

Thanks in Advance Laz. Appreciate your help on this.

Hello Maheshwaran

Among the three cases that you shared, it is the third one that I would consider best practice. It provides redundancy at the PEs as well as at the CEs, with a Layer 3 routing protocol providing the redundancy through multiple routing paths. Having said that, let me say a few words about the other cases:

If you only have the option of having a single PE router, then yes, this is a sufficient setup. You achieve redundancy at the CE with customer A with two links, but the PE does remain a single point of failure.

This depends on how that link from the PE3 router connects to the R2 router at customer A. If that link is expensive as you suggest, or if it traverses the same backbone network, it may not be the best choice. You have to weigh the cost/benefit of the deployment. Technically speaking, it does provide a higher level of redundancy than Case 1.

Ultimately, you have to do a real-world cost-benefit analysis. You can throw money at the problem and get ridiculous levels of redundancy anywhere you want, but the question is, is the redundancy worth the cost? Technically speaking, case 3 is ideal, but there are situations where you can argue for case 2 or even case 1, depending on the details of the deployment.

I hope this has been helpful!

Laz