Very good information, appreciate it!
1 Like
Hi Guys,
I have 2 x ISPs which connected to my edge router R3 with BGP. My public subnet 209.24.12.0/24 is advertised to both ISPs. Iām currently receiving default route from both ISPs and partial internet routes. I have BGP neighbor setup and configured and Iām able to go to the internet from inside of my LAN.
I only have access to my R3 for modification and donāt have access to ISPsā routers (attachment).
Here are my questions:
1. From inside network, for every outbound traffic to 72.12.0.0/16, I'd like it to go thru ISP #2 and return the same path. How do I set that up?
2. Right now, most of my internet traffic is going thru ISP #2 as well, very little go thru ISP #1, how do I setup my internet traffic to go thru ISP #1 except 72.12.0.0/16?
3. If when one of my 2 ISPs was down, I'd like to have all my outbound traffic (72.12.0.0/16 and other internet traffic) to go to the active ISP. How do I make sure that would happen automatically?
Thanks.
Hello Tom
Remember that routing that occurs in each direction is an independent operation. This means that if you want a particular route to be taken by your traffic, you must adjust the routing parameters for each direction. Keep in mind that you have complete control for the BGP routing of all outgoing traffic, but, although you can influence it, you do not have ultimate control over incoming traffic. This control belongs to the ISPs. To cause traffic to 72.12.0.0/16 to go through ISP#2, you simply need to use one of the BGP attributes to do this. The easiest way is to use the weight attribute. For more info about this attribute, take a look at the following lesson:
For incoming traffic, take a look at the following post that will answer your question.
The answer here is similar. To direct all the rest of your outbound traffic out of the ISP you want, you can simply use the weight attribute once again.
Since you are already receiving default routes from both ISPs, if one of the ISPs goes down, traffic should automatically use the other ISP. However, the issue here is that BGP may take a while to converge (on the order of dozens of seconds, to several minutes). In order to speed up convergence, there are several features that can be used including BGP Next Hop Address Tracking, and Additional Paths. The first monitors next hop address changes in the routing table to speed up convergence, while the second allows the advertising of multiple paths for the same prefix. Note the second is not suitable for your topology, since it only works with iBGP.
Itās best to talk to your ISPs beforehand. If you attempt to influence their routing, they may see this as a āhostileā or at the very least a ārudeā action on your part, and may be annoyed with you. If you approach them and let them know what you want to achieve, they should be willing to help you out.
I hope this has been helpful!
Laz
Hello,
In the Dual ISP scenario, I assume that the customer would always need to use his own public ASN to advertise his public IP range to both ISPs?
Or would it be possible for the customer to use a private ASN?
Thanks,
LP
Hello Luis
If a customer has their own public ASN (which in general has strict requirements, and is generally difficult and costly to obtain) then yes, they can use that ASN to advertise their public IP addresses. However, it can also be done using a private ASN.
Private ASN numbers for advertised prefixes can be removed and replaced with the ASN of the ISP when advertised into the Internet. More about this can be found at the following lesson:
I hope this has been helpful!
Laz
1 Like
Hi Rene and Laz,
Could you explain this along with Network Diagram and Configuration?
Hello Pradyumna
This lesson describes various scenarios for single and multihomed connections to ISPs. In addition, you can find some lessons that involve such a configuration below:
I hope this has been helpful.
Laz
Thanks Lagapides, I was able to follow your suggestion and setup in my lab. Weight and AS-path prepend work fine. Sorry, It took me few days to setup the lab and get back to you.
I just realized that I forgot to mention earlier. If we have a second edge router and iBGP is configured between these edge router R3 and R4. The R4 is also connected to ISP #2 and ISP #1. With this case, I canāt really use weight?!! Please see attachment.
Questions:
From inside network, for every outbound traffic to 72.12.0.0/16 that hits R3, I still want it to go thru ISP #2 and return the same path. The rest of outbound internet traffic that hits R3, Iād like it to go to ISP #1. How do we accomplish this? Local pref, MED and as-path prepend?
Since we have R4 now, Iād like to set it as a backup of R3. When those 2 x eBGP links of R3 failed, the traffic would flow to R4 and the R4 would act the same that every outbound traffic to 72.12.0.0/16 would go to ISP #2. And the rest of out bound internet traffic that will go to ISP #1. How do I do it? Thanks.
Thanks Laz for sharing.
1 Like
Rene - please help me understand thisā¦
From the picture - Bottom left - is Customer with ASN 55. I have a multihomed scenario with Router (CST5551_WANRTR_1) 1 (bottom) ebgp to iSP 1 and same with router 2 āthe one right above_CST5552_WANRTR2ā. Router 2 is the active router for HSRP therefore receives the 1st packet coming from server 1 behind a nexus 7 āCST5555N7Kā. Both routers are internetconnected thru interface Gi0/1 ip 10.55.11.0/30 and iBGP with the same IP schema. Although - router 2 is the active i have that router to its ISP Peer āATT_13ā local pref as 150 and router 1 to isp ISP3 rotuer as local pref 200.
The issue im encountering is that traffic hits router 2 ābeing HSRP Activeā and sends the traffic out to its ISP peer instead of sending it to Router 1 āibgp with local pref of 200āā¦ when i do the output of sho ip bgp 44.44.44.0 ". there are two possible outbound gateways with routerās 2 ISP being best "local pref of 150. and for ibgp i get a message that says (inaccessable) local pref 200.
Why is that? Why
Hello Diego
Based on your output, for the route to 44.44.44.0/24, a next hop of 10.64.13.3 is found in the BGP table, but, this is inaccessible, so the next hop of 10.67.113.3 is used instead.
Why is this inaccessible? Well, BGP checks the interface the BGP next-hop was learned from. This is the Gi0/1 interface on R2. If 10.64.13.3 was learned via a different interface than that used to peer with 10.55.11.1, the the route will be marked as inaccessible. Examine your topology to see via which interface the next hop IP was learned from.
Depending on your configuration, one solution to this issue is to use a loopback interface to create the BGP peering, and have it advertised via the IGP, so that the peering routers will learn the looback subnet via the same interface the next-hop was learned from.
I hope this sheds some light into your troubleshooting procedures. Let us know how you get on.
I hope this has been helpful!
Laz
Hello Tom
Weight is an attribute that is local to each individual router. There is nothing that prevents you from using weight on both R3 and R4 to direct traffic to the ISP of your choice.
For your new topology now, remember, you have full control of your outbound traffic. Just like you did with your topology with a single router on the company network, you simply configure the same thing on each of the R3 and R4 routers. Set up the prefixes you want to send via ISP1 and those you want to send via ISP2.
Now in order to configure R3 as your primary router and R4 as the backup router for your traffic, you can do this in multiple ways. These include:
- Configure HSRP, VRRP, or GLBP which are gateway redundancy protocols, on the enterprise-network-facing interfaces of the routers and make R3 the primary gateway and R4 the secondary/backup.
- Configure routing (either IGP or iBGP) between the edge routers and internal routers on the enterprise network to route traffic via R3, and have backup routes go via R4.
- You can even configure load balancing/sharing using GLBP, or equal cost load balancing of a routing protocol to send traffic to both R3 and R4, taking advantage of the bandwidth available for both devices to each ISP.
A couple of notes:
- If internal traffic is sent primarily to R3 using one of the above methods, then BGP routing in R3 will take care of where to send such outbound traffic, ISP1 or ISP2, depending on the destination. The same goes for traffic to R4.
- The benefit that iBGP between R3 and R4 will provide is if the physical link between R3 and R2 fails, for example, then traffic that hits R3 that should be routed via ISP1, will be sent to R4, and then to ISP1. In case of such a failure, it may take BGP some minutes to reconverge, so you may need to set up BGP Additional Paths in order to allow for R3 to have the additional path added via R3 using iBGP.
I hope this has been helpful!
Laz
Lazaros,
Thank you for your reply. I wanted to replicate a real scenario as much as possible.
1st. I did Reneās configuration and as per your suggestion of IGP and loopback IPs and it worked also with the next-hop-self command
2nd. Went back to the original configuration and just added the next-hop-self command - and it worked as well.
3rd. Whatās a suggestion or commonly used configuration in real environments?
3.2 - Use an additional Pool IPs from the ISP āpublic IPsā to configure iBGP and use the next hop -self command?
3.3 Do most companies use an IP as part of the block that they are announcing to make a loopbacks and iBGP peer to do the configuration you are suggesting ?
Im trying to do configurations that are as close as possible to real scenarios. At work we have two wan routers all internetconnected with public IPs and down into the environment FIrewalls all with public IPsā¦ not sure how thatās configured as access to those are thru an NRV "console serverā that i dont have access to for some X reason.
Hello Diego
For 1 and 2, glad to hear that those worked out for you.
In general it is best to use a loopback for the source of the BGP peering. This is typically best practice, primarily because it eliminates the possibility of losing a peering due to a downed interface. If you are using BGP on the edge of your network with the ISP, then you must use some public IPs to configure BGP on your devices. If youāre using iBGP, then yes, the next hop self configuration is necessary. Most often, however, you will see eBGP peerings between the enterprise edge and the ISP, so the next hop self doesnāt have any meaning there.
There is no single correct scenario, so sometimes you will see several other arrangements. It depends on the specific requirements and network topologies.
I hope this has been helpful!
Laz
Hi Laz, I have a clarification in the below.
Single Dual homed- Assuming we have two CE routers ( CE1 and CE2 ) , Both connected to their respective ISP1 and ISP2. Now we have a bought a public IP for our servers and attached only to CE1 router. Question is, how well we can set failover as in if ISP1 fails it has to go through by ISP2.?
Hi,
I have a question about this topology:
- Company with office on different sites.
- Each site have his ebgp peering with different ISPs
*Exist an iBGP between each office.
*On each site exist a webfarm with server with public IPs.
is possible to configure the same vlan on access switches between sites with public IPs to permit to move a server to a different webfarm?
Thanks.
Hello Giovanni
Iāve created the following diagram to help us out:
So youāre saying that the server farms at each site are on the same subnet, correct? Well, there are indeed several ways you can achieve this. Since the two remote offices have iBGP peerings between the routers, this means that they have some sort of WAN between them, that is independent of the ISPs themselves. Without knowing more about the type of WAN, here are a few thoughts:
- Create a L2 connection between the two sites, and span the server VLAN across the link, allowing the servers to be in the same VLAN. You could then advertise this subnet out of one or the other or both R1 routers to the Internet. This would require some changes in the WAN and internal topology of the network.
- If the WAN only functions at layer 3, then use a tunneling protocol such as L2TPv3 to tunnel layer 2 over a layer 3 link. This would allow the server farm VLAN to span the two sites. You could then advertise the subnet out of the two ISPs as you see fit.
- The use of Ciscoās Overlay Transport Virtualization (OTV) technology will allow you to have the same subnet at remote locations, while still maintaining the benefit of having dual redundant ISP connections that can be leveraged by both sites. This is done by allowing R1 and R2 to use first hop redundancy protocols such as HSRP and VRRP across the WAN. You can findout more about this at the following post:
Now notice that all of the options simply speak about the necessary topology to allow for the spanning of the VLAN across the WAN. The mechanism of then advertising these prefixes to both ISPs is the same in all cases. All that is necessary is to make that VLAN/subnet accessible on some interface on both R1 and R2, so that it can then be advertised however you see fit to the ISPs using eBGP peerings.
I hope this has been helpful!
Laz
Hello Dakshinamurthy
Based on your description, weāre looking at something like this:
Now the first thing Iād say is in such a situation, you wouldnāt have the servers connected just to CE1 but you would find a way to have them connect to both CE1 and CE2 either via internal routing or using some first hop redundancy protocol like HSRP or VRRP.
However, if you want to maintain only a single physical connection from the servers to CE1, then you would at least have a connection between the two CE routers since they are on the same enterprise network. This way, both CE1 and CE2 will learn of the subnet of the servers, and can then be advertised to both ISP1 and ISP2.
Otherwise, if CE1 and CE2 are disconnected from each other, then we donāt have a multihomed network as you suggest but two separate enterprise networks each connected to a single ISP. The multihoming component of such an arrangement is nullified.
There are multiple ways to affect incoming traffic on a BGP enabled edge network with multiple ISPs. You can find out more about these possibilities at the following post:
I hope this has been helpful!
Laz
Thanks, mabye the vxlan can be another solution?
What is the difference between vxlan and OTV?
Hi Laz,
Thanks for the reply. I read the post you shared and in that it says " assuming you are running EBGP between your equipment and each ISPās equipment". Here you are saying we are running EBGP between CE1 and CE2 and also EbGP peering between CE1 -ISP1 and CE2-ISP2 right ?
And lets assume we are manipulating our outbound traffic from CE1 and CE2 with routing policies. What kind of attributes can we use to manipulate outbound traffic ? Local preference ? And you mentioned that there are 4 ways to influence incoming traffic. If we want to change either MED value or AS-Path prepending for example in this case, it has to be done from ISP side towards CE ?
Can you provide any examples of influencing inbound and outbound traffic with the topology.?