GRE Tunnel Recursive Routing Error

Lessons Discussion
41

Hello Matteo

There are several ways in which you can resolve the recursive routing problem. As stated in the lesson, these include:

  • Don’t advertise the tunnel destination IP address on the tunnel interface. Don’t advertise it or use route filtering.
  • Make sure the administrative distance of the tunnel destination IP address through the tunnel is higher (worse) than what you have in the routing table now.
  • Make sure the metric to the tunnel destination IP address through the tunnel is worse than what you have in the routing table now.

These suggestions are based on the use of RIP as the routing protocol. But because RIP doesn’t establish neighbor adjacencies, it is somewhat different for OSPF. The first suggestion will not work with OSPF because then the adjacency won’t form over the link.

The other two however are possibilities. My suggestion would be to do the following:

In R1, create a more specific static route to the tunnel destination of R2 via the physical interface. Specifically:

R1(config)# ip route 172.16.10.2 255.255.255.255 10.0.149.1 gigabitethernet 0/0

(I’m assuming the IP address of Gi0/0 of the Internet router is 10.0.149.1)

And do the same (in reverse) for R2:

R2(config)# ip route 10.0.149.220 255.255.255.255 172.16.10.1 gigabitethernet 0/1

(I’m assuming the IP address of Gi0/1 of the Internet router is 172.16.10.1)

Now, because the static route has a lower AD than the OSPF route, the physical interface will be used for all traffic to the tunnel destination IPs in both directions. Note that this will only change the routing for that particular destination. That’s why we made the static route so specific. All other OSPF routing will remain the same, i.e. over the tunnel, which is the result we want.

Alternatively you can change the metric that OSPF gives for that particular destination as well using a route map, but that is slightly more complex.

I hope this has been helpful!

Laz

42

Hi Laz,

thanks for the support but in this case, my problem is another, now I try to explain it to you.

The goal of this lab is to demonstrate that multicast can pass through an IPSec tunnel and that it’s possible to establish an OSPF/EIGRP neighborship with the remote peer through the tunnel itself.

In this case, I use OSPF and I try to simulate the Intenet through a simple router that has configured with only two point-to-point towards R1 (10.0.149.0/24) and R2 (172.16.10.0/24).

Now I already tried to use two static routes as you have recommended but the problem is still present, the recursive error messages continue to appear on the screen and I’m not sure but probably it will be necessary to configure something like route-map or similar.

I know that maybe this is not a common enterprise scenario but I’m really involved in this lab and I want to address this issue in some way :wink:

Matteo

43

To be even clearer this is the error message that appears :

R1(config-if)#
*Mar 26 10:40:46.403: %TUN-5-RECURDOWN: Tunnel0 temporarily disabled due to recursive routing
*Mar 26 10:40:46.404: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to down
*Mar 26 10:40:46.416: %OSPF-5-ADJCHG: Process 1, Nbr 2.2.2.2 on Tunnel0 from INIT to DOWN, Neighbor Down: Interface down or detached

Now I tried to add ip ospf 1 area 0 into the Tunnel interface but nothing is changed.

44

Hello Matteo

These are the types of problems that are really interesting, and as you get deeper, and ultimately solve them, that’s when you get a really good understanding of the concepts.

It’s really strange to me that the static routes did not do the trick. Remember, the static routes should include only the IP address of the physical interface terminating the tunnel. Just to confirm, it should be:

R1(config)# ip route 172.16.10.2 255.255.255.255 10.0.149.1 gigabitethernet 0/0

R2(config)# ip route 10.0.149.220 255.255.255.255 172.16.10.1 gigabitethernet 0/1

You don’t have to include the exit interface in this case, but just make sure you are indicating a /32 address.

After you put in these routing commands, what do you get in your routing table?

Secondly, I notice that you attempted to change the bandwidth setting in the tunnel and the physical interfaces. You’ve made the tunnel a cost of 10 and the physical interface a cost of 1000. That means that traffic will prefer going over the tunnel. But so will the traffic that is attempting to create the tunnel, which is the initial problem, so changing the cost will not benefit you.

Take a look at the above and verify your settings. If you continue to have problems, let us know and we’ll see if we can duplicate it…

I hope this has been helpful!

Laz

45

We got it!

With the precise static route, the tunnel goes UP and consequently, also the neighborship was also established as you can see:

R1# *Mar 26 11:51:38.990: OSPF-1 HELLO Tu0: Send hello to 224.0.0.5 area 0 from 192.168.10.2
R2# *Mar 26 12:55:01.963: OSPF-1 HELLO Tu0: Rcv hello from 1.1.1.1 area 0 192.168.10.2

Thanks for the advice, now I have understood that another “guideline” when configuring OSPF (probably EIGRP also) over VTI IPSec tunnel we need to declare a static route specifically for the remote peer physical interface.

Thank you very much Laz, see you soon,
Matteo

46

Hello Matteo

Great to hear! Glad I could be of help, my pleasure.

Talk soon!

Laz

47

We currently work through this at work with a front door VRF setup where we logically segment the routes for the DMVPN network behind the transport layer reachability. 2 VRF’s, 1 for only transport, and the other for the rest of our enterprise routes. We source the tunnel interface from the Transport VRF, so it has outbound reachability and won’t go through the flapping process that occurs because of the recursive lookup process you went over.

I did have a thought though in regards to the metric solution proposed if you were to use an offset list to modify the metric for the overlay route, wouldn’t that never be installed into the routing table unless the feasible route (presuming an EIGRP deployment in this fictitious situation) was to go down or the metric was to recalculate and become worse than the tunnel?

Maybe I am just missing something here in the fundamentals of route selection, but that’s the manner by which route selection works as I understand it. Let’s use this fictitious example:

Correct me if I am wrong here:

Given the destination IP of 10.0.1.5

The most specific route to the network from the associated route process table, EIGRP Topology table in this case, is selected so 10.0.1.4/30 would be selected over 10.0.0.0/24, and yes I realize that in classless networking they are considered different networks.

If the 10.0.1.4/30 network was learned from two different sources, one through an OSPF process, and one through EIGRP, the EIGRP would be installed due to Administrative distance winning out.

If upon checking the EIGRP topology table, we noticed the route was learned via two different paths, which would lead to a difference in metrics (feasible distance in the case of EIGRP), then the second entry in the topology table would be installed, if the primary path (that was currently installed into the routing table) went down due to a lack of connectivity for reason XYZ.

Is anything I said above wrong? Because if it isn’t I don’t get how your metric offset situation works.

49

@lagapidis or @ReneMolenaar any insight to be provided here?

50

Hello Adam

Yes, that’s correct. That’s actually what we want to happen. The overlay route (via the tunnel) to never be installed under normal conditions. If for some reason it fails, the tunnel will fail as well, and thus the feasible successor route via the tunnel will also not be possible. Here’s some more detail in the issue:

If you choose to use EIGRP in this scenario, you can still successfully use the offset-list command. Let’s take the topology and the addresses as they appear in the lesson like so:

image
image791×343 35.3 KB

Imagine you configure EIGRP instead of RIP, and you configure an offset similar to that in the lesson. Notice what happens:

  • R1 will advertise 1.1.1.1 through the tunnel as being of a higher metric than 1.1.1.1 via R2. So R3 will choose to go via R2 to get to 1.1.1.1 because that path has a better metric. Thus, the tunnel itself remains up.
  • similarly, R2 will advertise 3.3.3.3 through the tunnel as being of a higher metric than 3.3.3.3 via R2. So R1 will choose to go via R2 to get to 3.3.3.3 because that path has a better metric.

So it is only the 1.1.1.1/32 and 3.3.3.3/32 prefixes for which the metric is being changed.

Now with EIGRP, from the point of view of R1, the successor route to 3.3.3.3/32 is via R2, while the feasible successor is indeed via the tunnel. But that doesn’t bother us, since the successor is indeed the path we want, and has been achieved.

As for the rest of your post, keep in mind that these metric offsets are applied only for the loopback prefixes and their purpose is simply to ensure that the routing involved to establish the tunnel always goes via R2, so the tunnel can remain up. All other routing, for other prefixes on your network, will not require any offsets and can function normally for any routing protocol including EIGRP.

I hope this has been helpful!

Laz

51

Great lesson Rene. So the take away point is if you end up with recursive routing error due to misconfiguration, you need to ensure you have a more preferable route to the tunnel destination which does not go via the tunnel itself.

This could be achieved in different ways, depending on the topology. e.g. if you learning about the tunnel destination via an IGP then you could use metric, or you could use AD, or a more specific static route ect.

Is that correct?

52

Hello Bhawandeep

Yes, that is correct. You simply have to ensure that the tunnel destination is not reached through the tunnel itself. As stated in the lesson, you can also do it in these ways:

  • Don’t advertise the tunnel destination IP address on the tunnel interface. You can do this by using route filtering for example.
  • Make sure the administrative distance of the tunnel destination IP address through the tunnel is higher (worse) than what you have in the routing table now.
  • Make sure the metric to the tunnel destination IP address through the tunnel is worse than what you have in the routing table now.

I hope this has been helpful!

Laz

1 Like
53

Thank you Laz for clarifying

1 Like
54

Can you guys explain offset 0 in the GRE recursive lesson in detail to people who do now know, it would be appreciated. Thanks.

55

Hello George

For more info about offset-lists for both EIGRP and RIP, take a look at this NetworkLessons note. There you will find information about the command that contains offset 0 as well.

I hope this has been helpful!

Laz

57

Hello, everyone!

What are the chances of this error occuring in the real world? If I follow up on Rene’s topology

recursive-routing-gre-topology
recursive-routing-gre-topology791×343 34.1 KB

if R1 and R3 wanted to build a GRE tunnel across the internet (assume that R2 is the ISP), they wouldn’t really enable the ISP facing links for OSPF, would they?

Are there any other scenarios where this error could potentially occur?

Thank you!

58

Hello David

If the GRE tunnel is established over the Internet, then it is unlikely that an OSPF configuration would cause a recursive routing error since you wouldn’t have OSPF running on the Internet, or on your edge devices. However, the recursive routing error is not limited to OSPF. It could occur with any routing protocol, as well as static routing.

The truth is that it is not a very common error because, in most cases, edge devices will have a default route configured sending all traffic destined for the Internet via the Internet-facing physical interface. That will typically be enough to eliminate such an error, since routing over the GRE tunnel is achieved using a more specific route. However, if, for some reason, you have a more specific route (either learned dynamically or configured statically) to the tunnel endpoint via the tunnel itself, the recursive routing error will occur.

The key here is that recursive routing errors in GRE tunnels stem from a router or routing protocol choosing the tunnel as the path to the tunnel’s own endpoint regardless of what routing protocol is used.

I hope this has been helpful!

Laz

1 Like
59

At the end of the lesson you state:

How do we solve this? You need to ensure the router doesn’t reach the tunnel destination through the tunnel itself. There are some options to do this:

  1. Don’t advertise the tunnel destination IP address on the tunnel interface. Don’t advertise it or use route filtering.
  2. Make sure the administrative distance of the tunnel destination IP address through the tunnel is higher (worse) than what you have in the routing table now.
  3. Make sure the metric to the tunnel destination IP address through the tunnel is worse than what you have in the routing table now.

Could you give a quick config example of 1 and 2? Especially, 1 as I don’t understand what you mean at all with that one.

I think for 2 I could just say:

Ip route x.x.x.x 255.255.255 y.y.y.y 130 (which is higher than rip’s 120), right?

60

Hello Scott

Indeed, the lesson covers the third solution. The first two options can also be applied, but the config for each may differ depending upon the routing protocol used. You can find solutions with configuration examples, assuming the use of RIP, at the following NetworkLessons notes:

  1. GRE - Recursive routing error
  2. GRE - Recursive routing error - AD solution
  3. GRE - Recursive routing error - filtering solution

I hope this has been helpful!

Laz

61

Yeah, the wording is confusing. You say “don’t advertise the tunnel destination on the tunnel interface” but you apply the distribute list to the physical interface. The word “on” here is ambiguous.

62

Hello Scott

For the GRE Recursive routing error filtering solution, you are correct and I have fixed the error in the related NetworkLessons note. And you are correct, terminology and wording can become confusing. Specifically in this solution, we want to avoid advertising the tunnel endpoints through the tunnel itself. In other words, the 1.1.1.1/32 and 3.3.3.3/32 should not be advertised out of the tunnel interfaces on their respective routers.

I hope this has been helpful!

Laz