DMVPN Phase 1 BGP Routing

lagapidis · January 21, 2022, 2:13pm

Hello Raghu

According to Rene’s lesson on the Introduction to DMVPN, he states that:

With phase 1 we use NHRP so that spokes can register themselves with the hub. The hub is the only router that is using a multipoint GRE interface, all spokes will be using regular point-to-point GRE tunnel interfaces. This means that there will be no direct spoke-to-spoke communication, all traffic has to go through the hub!

What makes phase 1 do this is the way the GRE tunnel is created at the spokes. No change in routing or in IPv4 or IPv6 can change that.

What is the role of BGP? To advertise networks. It won’t change the way DMVPN operates. Even creating IPv4 and IPv6 tunnels will still make traffic go through the hub.

So what have you done in your configuration? You’ve simply overriden a loop prevention mechanism of BGP which simply states that “a BGP route will not be installed via eBGP if the local router’s AS is in the path attribute”. In other words, spoke 1 sent a route with its own AS 65023, it went to the hub, and then with eBGP it was advertised to spoke 2. But spoke 2’s AS is 65203, which is the same as the advertised route, thus the path was not installed. Communication is still possible due to the default route.

You have overridden this mechanism by using the allowas-in keyword. Thus, the routes to the other spokes appear in the local routing table. However, if you look at the next hop IP in the routing table, you will see that it still goes through the hub.

You can find out more about the allowas-in keyword in the following links:

I hope this has been helpful!

Laz

raghu_kummar · January 21, 2022, 2:31pm

Hi Mr Lazaros

Thanks for the explanation. Yes , it is overriding the BGP rule.
can we say it is similar to "no split-horizon eigrp " configured on the dmvpn Hub site tunnel interface, with eigrp routing ,to enable spoke-to-spoke communication.

lagapidis · January 24, 2022, 6:21am

Hello Raghu

When applying EIGRP in a DMVPN Phase 1 environment, we must disable split horizon, as seen in the related lesson. In a sense, overriding the eBGP rule is indeed similar.

However, in the EIGRP case, when you disable split horizon, each spoke installs a route to the network of the other spoke with the hub as the next-hop IP. In your case with BGP, the next-hop IP installed is not the IP of the other spoke, but that of the other spoke.

Remember, in the context of DMVPN, spoke-to-spoke communication means direction communication, that is, not via the hub. In both the EIGRP and BGP cases, there is no such spoke-to-spoke communication simply because the underlying topology of the GRE tunnels cannot allow it. Even though in the BGP case you see the other spoke’s IP address as the next-hop IP, it still uses NHRP to resolve that IP, and the result given is the hub IP address as the next hop.

I hope this has been helpful!

Laz

wilder7bc · March 10, 2022, 3:11pm

Thanks was about to post question but you answered it already here as I was not sure why NULL0 was added.

As to your second part the Route-Map because it has an implicit deny after the first match blocks everything out correct?

if we had added another match statement with permit any any then it would have allowed the other prefixes as well correct?

------------I tested myself and that seems to be the case-------------------------

no ip http secure-server
ip route 0.0.0.0 0.0.0.0 Null0
!
!
ip prefix-list DEFAULT_ROUTE seq 5 permit 0.0.0.0/0
!
route-map SPOKE_ROUTERS permit 10
 match ip address prefix-list DEFAULT_ROUTE
!
route-map SPOKE_ROUTERS permit 20

then here is the spoke results first only the default route and then when sequence 20 kicks in we have all of them.

Spoke1#show ip route bgp

Gateway of last resort is 172.16.123.1 to network 0.0.0.0

B*    0.0.0.0/0 [20/0] via 172.16.123.1, 01:10:00


Spoke1#show ip route bgp

Gateway of last resort is 172.16.123.1 to network 0.0.0.0

B*    0.0.0.0/0 [20/0] via 172.16.123.1, 01:10:27
      1.0.0.0/32 is subnetted, 1 subnets
B        1.1.1.1 [20/0] via 172.16.123.1, 00:00:08
      3.0.0.0/32 is subnetted, 1 subnets
B        3.3.3.3 [20/0] via 172.16.123.3, 00:00:08
Spoke1#

Its hard to get use to thinking of an implicit deny after a permit statement lol… but I guess since on seq 10 we use a match and basically saying allow this it does kind of make sense that if we having to specifically allow something that means the rest is blocked.

lagapidis · March 12, 2022, 6:42am

Hello Brian

Yes, you are correct, the reasoning behind it is indeed the implicit deny at the end. Thanks for sharing your test results as well!

Laz

icekeki0127 · April 7, 2022, 6:38am

Hello

After I was done configuring it with OSPF and BGP , I compared OSPF routing table to BGP.

For example,
OSPF
Spoke1

o 1.1.1.1[110/1001] via 172.16.123.1 tunnel0

However
BGP
Spoke1
B 1.1.1.1[20/0/ via 172.16.123.1
(no interface information but when I captured bgp packets , it was sent out through the tunnel)

Is it normal?

Many thank!!
DY

lagapidis · April 9, 2022, 4:43am

Hello Dy Han

Yes, this is normal. BGP will never show the exit interface in the routing table, it will only show the next-hop IP. It uses recursive routing to determine the exit interface for the specific next-hop IP. This is why a BGP-learned route will be installed in the local routing table only if the local router has a valid path to the next-hop IP. For more info, take a look at the following two NetworkLessons Notes:

I hope this has been helpful!

Laz

icekeki0127 · April 12, 2022, 4:48am

Hello Laz,

Many thanks for your information!!
DY

kansangeeth · December 15, 2022, 8:01am

@ReneMolenaar Why in ebgp config the next-hop is not updated to hub’s address in spokes as the next hop should be updated in ebgp?

lagapidis · December 19, 2022, 7:02am

Hello Nipun

For both eBGP scenarios, we see that the next hop IP is updated to that of the hub, for routes to 1.1.1.1/32. Maybe I do not understand the question. Can you clarify your question so we can respond more appropriately?

Thanks!

Laz

kansangeeth · December 19, 2022, 8:33am

As an example,when configured as ebgp,why the next hop ip of 3.3.3.3/32 at spoke 1 showing as 172.16.123.3 ? shouldn’t it be 172.16.123.1 ? @lagapidis

lagapidis · December 22, 2022, 5:55am

Hello Nipun

This is expected behavior. Now it is important not to confuse the operation of NHRP with the operation of the routing protocol.

In Phase 1 all spokes use point-to-point GRE tunnel interfaces, and therefore there is no direct spoke-to-spoke communication. All traffic goes through the hub. Now this does not mean that if you use a routing protocol, the next hop given in the routing table will be the hub!

What next hop will appear in the routing table depends on several factors, including the routing protocol used.

For example, if you take a look at EIGRP in a DMVPN Phase 1 topology, you will see that (once you disable split horizon) the next hop IP for the Spoke 2 network from the Spoke 1 router is the hub:

Spoke1#show ip route eigrp | include 3.3.3.3
D        3.3.3.3 [90/28288000] via 172.16.123.1, 00:00:27, Tunnel0

Why? Because, as stated in that lesson,

EIGRP changes the next hop IP address when it advertises networks.

You will find this behavior of EIGRP in DMVPN phase 2 and phase 3! So it is a routing-protocol-specific behavior.

OSPF, on the other hand, when applied to a DMVPN Phase 1 topology, will behave differently based on the network type that has been configured.

A broadcast network type and a non-broadcast network type will use the IP address of the remote spoke as the next hop IP in the routing table
A point-to-multipoint network type and point-to-multipoint non-broadcast network type will use the IP address of the hub as the next hop IP in the routing table
When a spoke is configured as a stub area, it will use the IP address of the hub as the next hop IP in the routing table

So for eBGP with a different AS on each spoke, you will see the IP address of the remote spoke as the next hop IP in the routing table simply because that’s how this particular routing protocol works. eBGP with the same AS on the spokes, as you can see from the lesson, works differently, as does iBGP with dynamic peers, as shown.

I hope this has been helpful!

Laz

kansangeeth · December 23, 2022, 8:33am

Thanks for the clear explanation @lagapidis
When a router sends a prefix to a ebgp neighbor,shouldn’t it update the next hop to self(to hub tunnel ip as it advertise to spokes) ?

lagapidis · January 2, 2023, 6:28am

Hello Nipun

Yes, you are correct. However, in this case, it is Spoke 2 that is advertising the 3.3.3.3/32 subnet. Therefore the next hop IP should be that of Spoke 2, which is indeed 172.16.123.3. It is the Hub that acts as the NHRP server, resolving the addresses to be used to reach the other spoke.

For those prefixes advertised by the hub (such as 1.1.1.1/32) you see that the next hop IP is indeed that of the hub (172.16.123.1).

I hope this has been helpful!

Laz

kansangeeth · January 7, 2023, 9:51am

@lagapidis
I think I am missing something here.Please correct me.here is my understanding.

Spoke 2 advertises 3.3.3.3 prefix to the hub setting next hop as 172.16.123.2.
Then Hub should advertise it to Spoke 1 setting next hop to it’s ip which used to form neighborship with spoke 1

lagapidis · January 12, 2023, 11:34am

Hell Nipun

So I went into the lab, and created a topology (non DMVPN) where we have three routers, each in its own AS in the following topology with the loopback interfaces shown:

(1.1.1.1/32) ---- R1 ---- R2 ---- R3 ---- (3.3.3.3/32)

Where R1 is in AS1, R2 is in AS2 and R3 is in AS3. BGP peerings were configured, and indeed, R3 had the next hop IP of R2 in both its BGP table and its routing table, as expected and as you suggest in your post.

I also labbed up a DMVPN Phase 1 topology with eBGP where we are using a different AS for each of the hub and the two spokes, where R2 is the hub, we find that this is not the case. R3 will have the next hop IP address of R1 for the 1.1.1.1/32 destination in both the BGP table and the routing table. Here is a wireshark capture of the update sent from the hub to spoke 1:

Note the next hop that the hub sends is indeed that of the spoke.

Why does it do this? Well, the only difference between the two situations is that one uses DMVPN and the other does not. I tried looking at the wirshark captures of the BGP messages sent, but I was unable to find any indication of why this difference occurs.

It definitely has to do with the use of NHRP, but I am unable to tell you more. I will ask Rene as well if he has any experience with this to see why this is happening…

I wish I could have been more helpful!

Laz

lagapidis · January 13, 2023, 6:26am

Hello again Nipun

OK, so here’s what I got from Rene as well.

eBGP next hop is not changed if the BGP next hop in the BGP table belongs to the same IP subnet as the eBGP neighbor to which the update is sent.

This is a type of next hop optimization, and it makes sense, because it ensures that the shortest path is used, and no unnecessary intermediate ASes are added to the AS_PATH. This is also what happens in the DMVPN Phase 1 BGP topology we are looking at. Because the next hop of Spoke2 is on the same subnet as the next hop of the Hub, the Hub chooses to advertise the next hop address as that of Spoke2 rather than its own, when advertising Spoke2’s routes to Spoke1. Specificlly, this is called a third party next hop.

Now in DMVPN Phase 1, spoke to spoke communication is not possible, so the next hop of Spoke2 should be unreachable from Spoke1 right? Well, that’s where NHRP comes in, and makes the spokes believe they can communicate directly with each other, but using the next hop resolution mechanism of NHRP, the actual traffic indeed travels via the hub.

This is further described in section 5.1.3 of RFC 4271.

I hope this has been helpful!

Laz

kansangeeth · March 3, 2023, 4:57pm

Thank you very much @lagapidis

davidilles · October 15, 2023, 12:47pm

Hello, everyone!

What’s generally the best routing protocol to run in the DMVPN cloud? I’ve seen that we can use EIGRP, OSPF and BGP. OSPF is not always the best solution due to its strict design hierarchy but what about EIGRP and BGP? What should we take into consideration when we’re picking between these two?

And BGP has a lot of options, but the best one should be iBGP with dynamic peers right?

Kind regards,
David

lagapidis · October 26, 2023, 6:00am

Hello David

As you can see from the CCNP ENARSI course, Rene has exhaustively covered the use of various routing protocols within the range of DMVPN topologies, so having gone over those, you should have a well-rounded view of the advantages and disadvantages of each.

Now the choice of routing protocol you will use really depends on your specific network requirements and design. Generally speaking, you are correct, OSPF is typically avoided when it comes to DMVPN due to its strict design hierarchy as you put it.

Both EIGRP and BGP are good choices for a DMVPN environment and both are often used.

EIGRP has advantages like faster convergence and simplicity in configuration, but it’s a Cisco proprietary protocol and might not be the best choice if you’re considering a multi-vendor environment. It also works well in small to medium-sized networks.

BGP, on the other hand, is more complex to configure and has slower convergence, but it’s highly scalable and works well in larger networks. It’s also vendor-neutral and offers more granular control over routing updates.

If you choose BGP, using iBGP with dynamic peers is indeed a good option for DMVPN. This setup allows for more flexibility and scalability, as it can accommodate a large number of spokes without needing to manually configure each peer relationship.

If your network is small to medium-sized and primarily Cisco, EIGRP might be the best choice. If it’s a larger, multi-vendor network, or if you need more control over your routing updates, BGP would likely be the better option.

That’s not to say that OSPF should never be used. Many organizations have used it successfully, however, with the appropriate care, planning, and design needed to ensure correct operations.

So there’s no clear-cut answer to what’s best, as it primarily depends upon the nature of the network itself and the requirements of the topology.

I hope this has been helpful!

Laz