GRE Tunnel Recursive Routing Error

I noticed the discussion of “what about this and that” concerning metric and AD changes.
In my opinion you should NEVER need too, nor want too, advertise the tunnel destination networks thru the tunnel interface itself.
So the only corrective action should be some type of filtering. There is no reason whatsoever to advertise that network back thru the tunnel interface as far as I can see. Makes no sense even with higher cost or AD.

Hi Rene,

  I'm not sure I quite get this ! Say in your previous lesson you have R2 as an HQ router, R3 as a Branch and R1 as an ISP Router, If we create a Tunnel between HQ and Branch as Described. Then say secure the tunnel between HQ and Branch with IPSEC so data can be secure between the 2 offices.

When you change the cost to make the Tunnel cost higher ( Say 3 if RIP  or 50 if OSPF ) the tunnel wouldn't collapse, However the better path between the sites would then appear to be the UNENCRYPTED path via R1 ( or the ISP Router ) 

 The Tunnel would stay up but wouldn't be used. In the previous lesson we set Static routes between R1 and R3 ( HQ and Branch ) and used EIGRP to add the Tunnel Routes. Static Routes  have an AD of 1 so they stay in the table.

 Static routes would be ok on a 3 router scenario but not so good if there were 20 or 30 Branch offices.

  I'm sure I'm missing something

@Daniel That is correct, normally you should never advertise the tunnel destination addresses through the tunnel itself. Filtering so they are not advertised is the best solution. However, understanding how to use the AD and/or metric is important in case you want to do the CCIE R&S lab…you’ll have to know all your options.

@John To simplify this, let’s forget about IPsec for now. It doesn’t matter if the tunnel is encrypted or not…recursive routing is a routing issue.

The only destinations that you don’t want to learn through the tunnel are the IP addresses that are used to establish the tunnel itself. In my example, that’s 192.168.12.1 on R1 (HQ) and 192.168.23.3 on R3 (branch). R3 should never learn about 192.168.12.1 through the tunnel and R1 should never learn 192.168.23.3 through the tunnel. You can achieve this by filtering (best option) or by increasing the AD/metric.

Once that is solved, the tunnel will be up and remain up…

Now you can run an IGP (RIP, OSPF or EIGRP) on the tunnel interface and advertise any networks that you have. Your routers will form a neighbor adjacency through the tunnel interface and advertise networks to each other. To reach these networks, the tunnel interface will be used.

Rene

1 Like

Makes Sense Now ! It’s only the TUNNEL Addresses that should be advertised through the tunnel, The Router IP addresses should not I assume any Loopbacks / Networks can also be advertised through the tunnel !

That’s right :slight_smile:

In other words, you can do whatever you want but you have to make sure that the IP addresses that are used to build the tunnel, are never advertised through the tunnel interfaces. In my example, that’s 192.168.12.1 and 192.168.23.3.

If they are advertised, you have to make sure you change the AD or metric so they never get installed in the routing table.

19 posts were merged into an existing topic: GRE Tunnel Recursive Routing Error

I got “recursive routing” before applying the offset command.

However ; I noted that traceroute works, before and after applying offset. It, also, displays the same result (below).

**R1#traceroute 3.3.3.3**

Type escape sequence to abort.
Tracing the route to 3.3.3.3

  1 192.168.12.2 56 msec 8 msec 12 msec
  2 192.168.23.3 20 msec 20 msec 20 msec

Hello,

I have a question about the use of GRE
what I have understood that main use of GRE is to connect site to site via the Internet. If that is the case this problem (Recursive Routing Error) won’t appear because our connection is via the ISP and the tunnel source will be the interface that is connected to ISP and we will configure default static route so it won’t be advertised by the routing protocol.

another question do we use GRE tunnel in other cases?

Hello Rehab

If you implement GRE over the Internet in the way that you described, you are correct that you wouldn’t run into the recursive routing error. However, there are many cases where a GRE tunnel would be implemented where the recursive routing error would be a problem. These involve situations where you have a large enterprise network that may span several campuses, for example, in a university.

Say you have a university network with several campuses and many buildings within each campus. All the buildings and campuses are interconnected using a private network of some sort.
Say the computer science department has it’s main offices in a particular building in campus A but has also been given two offices in campus B. The department wants to use a contiguous set of IP addresses so that communication between devices in campus A and those in the offices in campus B are on the same subnet. One way to do this is via a GRE tunnel, over the private network.

A GRE tunnel can be used any time you want to connect two particular networks over a third network, whether than third network is the Internet or some other private network.

I hope this has been helpful!

Laz

1 Like

Hello Rene,

Thanks for such a good explanation. I can understand properly what causes the recursive routing in tunnel.

Now i am trying to resolve the recursive routing issue by changing the Administrative distance but not exactly getting as which network to include in the access-list

R1(config)#interface tunnel 1
R1(config-if)#tunnel source fa0/0
R1(config-if)#ip address 192.168.13.1 255.255.255.0
R1(config-if)#tunnel destination 192.168.23.3

-

R3(config)#interface tunnel 1
R3(config-if)#tunnel source Fa0/0
R3(config-if)#ip address 192.168.13.3 255.255.255.0
R3(config-if)#tunnel destination 192.168.12.1

R1 has loopback interface which is advertising network 10.10.10.0 255.255.255.0

now iI am using OSPF in R1 and adverting as -

router ospf 2 
network 192.168.12.0 0.0.0.255 area 0 
network 192.168.13.0 0.0.0.255 area 0 
network 10.10.10.0 0.0.0.255 area 0 

R3 has loopback interface which is advertising network 20.20.20.0 255.255.255.0

Now on R3 also i am using OSPF in R3 and advertising networks as

router ospf 2 
network 192.168.23.0 0.0.0.255 area 0 
network 192.168.13.0 0.0.0.255 area 0 
network 20.20.20.0 0.0.0.255 area 0 

Now this will cause recursive routing issue, as advertising tunnel destination through the tunnel itself.

now to resolve this i changed the Administrative distance using the command router ospf 2

distance 255 0.0.0.0 255.255.255.255 tunnelprefix

This command is configured on both R1 and R3

now i created an access list

ip access-list standard tunnelprefix
permit 192.168.13.0 0.0.0.255

still getting the routing recursive error and the tunnel is flapping and ospf neighborship is breaking in tunnel.

I am not sure which networks to include in the access-list,

192.168.13.0 or 10.10.10.0 also

or network 192.168.12.0 and 192.168.23.0

I can see the administrative distance is changing using the command show ip route ospf
but routing recursive issue is not resolving.

Is it possible if you can create one example with ospf and EIGRP also as you created with RIP
so that we can understand how administrative distance can resolve this issue.

Rene, it will be very helpful for us.

Hello Tejpal

In order to achieve the correct configuration, you must specify the administrative distance of the specific destination subnet. To do this, you simply configure the following:

distance 200 192.168.13.0 0.0.0.255

This will cause the destination of 192.168.13.0/24 to have an AD of 200, and will override any other information received. The access list is not needed. It is only used it is applied when a network is being inserted into the routing table. This behavior allows you to filter networks based on the IP prefix supplying the routing information. So the access list would reject any routes for which AD has not been explicitly configured.

More info about this command can be found here:

Try it out and let us know. I hope this has been helpful!

Laz

In this topic with RIP, I don’t understand, what is the purpose of the tunnel when the metric will be worse, than through R2 to reach the network 1.1.1.1 and 2.2.2.2 through tunnel using offset list.

Hello Jan

In the specific topology, using a GRE tunnel will make very little difference in the actual routing of traffic from one loopback to the other. In both cases, the packets go via R2. There really is no reason to create a GRE tunnel for such a topology, so you are right, the purpose of the tunnel is not immediately perceivable.

However, the point of the lesson is to show that when you do have a GRE tunnel, which in most cases will be over a much larger network like the Internet for example, you have this recursive routing error that can occur. Creating a GRE tunnel over the Internet is useful and does indeed have many useful applications.

It doesn’t really matter what the actual metric is, as long as the routing between the routers over the GRE tunnel is achieved successfully. Because the recursive routing error for GRE tunnels is a real problem when used in a real world scenario, such as over the Internet, this lesson shows us how to resolve it.

I hope this has been helpful!

Laz

Hi Renè,

This is my topology:

Now, the configurations are these:

R1

crypto isakmp policy 1
 encr 3des
 authentication pre-share
 group 2
crypto isakmp key cisco123 address 0.0.0.0
crypto isakmp keepalive 10
!
!
crypto ipsec transform-set TSET esp-3des esp-sha-hmac
 mode tunnel
!
crypto ipsec profile VTI
 set transform-set TSET
!
!
interface Tunnel0
 ip address 192.168.10.2 255.255.255.0
 ip ospf cost 10
 tunnel source 10.0.149.220
 tunnel mode ipsec ipv4
 tunnel destination 172.16.10.2
 tunnel protection ipsec profile VTI
!
interface GigabitEthernet0/0
 ip address 10.0.149.220 255.255.255.0
 ip ospf cost 1000
 duplex auto
 speed auto
 media-type rj45
!
interface GigabitEthernet0/1
 ip address 192.168.100.1 255.255.255.0
 duplex auto
 speed auto
 media-type rj45
!
router ospf 1
 router-id 1.1.1.1
 network 10.0.149.0 0.0.0.255 area 0
 network 192.168.10.0 0.0.0.255 area 0
 network 192.168.100.0 0.0.0.255 area 0
!

R2
crypto isakmp policy 1
 encr 3des
 authentication pre-share
 group 2
crypto isakmp key cisco123 address 0.0.0.0
crypto isakmp keepalive 10
!
!
crypto ipsec transform-set TSET esp-3des esp-sha-hmac
 mode tunnel
!
crypto ipsec profile VTI
 set transform-set TSET
!
interface Tunnel0
 ip address 192.168.10.1 255.255.255.0
 ip ospf cost 10
 tunnel source 172.16.10.2
 tunnel mode ipsec ipv4
 tunnel destination 10.0.149.220
 tunnel protection ipsec profile VTI
!
interface GigabitEthernet0/1
 ip address 172.16.10.2 255.255.255.0
 ip ospf cost 1000
 duplex auto
 speed auto
 media-type rj45
!
interface GigabitEthernet0/2
 ip address 192.168.200.1 255.255.255.0
 duplex auto
 speed auto
 media-type rj45
!
router ospf 1
 router-id 2.2.2.2
 network 172.16.10.0 0.0.0.255 area 0
 network 192.168.10.0 0.0.0.255 area 0
 network 192.168.200.0 0.0.0.255 area 0

Router Internet 
interface GigabitEthernet0/0
 ip address 10.0.149.221 255.255.255.0
 ip ospf cost 1000
 duplex auto
 speed auto
 media-type rj45
!
interface GigabitEthernet0/1
 ip address 172.16.10.1 255.255.255.0
 ip ospf cost 1000
 duplex auto
 speed auto
 media-type rj45
!
router ospf 1
 router-id 3.3.3.3
 network 10.0.149.0 0.0.0.255 area 0
 network 172.16.10.0 0.0.0.255 area 0

Now the purpose of this lab is to test the OSPF multicast packets over an IPsec tunnel configured with VTI.
I wish that R1 and R2 become neighbors through the tunnel.

The problem now, as you already know, is the recursive routing that brings down my tunnel… how can I solve this issue?

I tried to disable the OSPF from the Internet router but the problem is still present.

Hello Matteo

There are several ways in which you can resolve the recursive routing problem. As stated in the lesson, these include:

  • Don’t advertise the tunnel destination IP address on the tunnel interface. Don’t advertise it or use route filtering.
  • Make sure the administrative distance of the tunnel destination IP address through the tunnel is higher (worse) than what you have in the routing table now.
  • Make sure the metric to the tunnel destination IP address through the tunnel is worse than what you have in the routing table now.

These suggestions are based on the use of RIP as the routing protocol. But because RIP doesn’t establish neighbor adjacencies, it is somewhat different for OSPF. The first suggestion will not work with OSPF because then the adjacency won’t form over the link.

The other two however are possibilities. My suggestion would be to do the following:

In R1, create a more specific static route to the tunnel destination of R2 via the physical interface. Specifically:

R1(config)# ip route 172.16.10.2 255.255.255.255 10.0.149.1 gigabitethernet 0/0

(I’m assuming the IP address of Gi0/0 of the Internet router is 10.0.149.1)

And do the same (in reverse) for R2:

R2(config)# ip route 10.0.149.220 255.255.255.255 172.16.10.1 gigabitethernet 0/1

(I’m assuming the IP address of Gi0/1 of the Internet router is 172.16.10.1)

Now, because the static route has a lower AD than the OSPF route, the physical interface will be used for all traffic to the tunnel destination IPs in both directions. Note that this will only change the routing for that particular destination. That’s why we made the static route so specific. All other OSPF routing will remain the same, i.e. over the tunnel, which is the result we want.

Alternatively you can change the metric that OSPF gives for that particular destination as well using a route map, but that is slightly more complex.

I hope this has been helpful!

Laz

Hi Laz,

thanks for the support but in this case, my problem is another, now I try to explain it to you.

The goal of this lab is to demonstrate that multicast can pass through an IPSec tunnel and that it’s possible to establish an OSPF/EIGRP neighborship with the remote peer through the tunnel itself.

In this case, I use OSPF and I try to simulate the Intenet through a simple router that has configured with only two point-to-point towards R1 (10.0.149.0/24) and R2 (172.16.10.0/24).

Now I already tried to use two static routes as you have recommended but the problem is still present, the recursive error messages continue to appear on the screen and I’m not sure but probably it will be necessary to configure something like route-map or similar.

I know that maybe this is not a common enterprise scenario but I’m really involved in this lab and I want to address this issue in some way :wink:

Matteo

To be even clearer this is the error message that appears :

R1(config-if)#
*Mar 26 10:40:46.403: %TUN-5-RECURDOWN: Tunnel0 temporarily disabled due to recursive routing
*Mar 26 10:40:46.404: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to down
*Mar 26 10:40:46.416: %OSPF-5-ADJCHG: Process 1, Nbr 2.2.2.2 on Tunnel0 from INIT to DOWN, Neighbor Down: Interface down or detached

Now I tried to add ip ospf 1 area 0 into the Tunnel interface but nothing is changed.

Hello Matteo

These are the types of problems that are really interesting, and as you get deeper, and ultimately solve them, that’s when you get a really good understanding of the concepts.

It’s really strange to me that the static routes did not do the trick. Remember, the static routes should include only the IP address of the physical interface terminating the tunnel. Just to confirm, it should be:

R1(config)# ip route 172.16.10.2 255.255.255.255 10.0.149.1 gigabitethernet 0/0

R2(config)# ip route 10.0.149.220 255.255.255.255 172.16.10.1 gigabitethernet 0/1

You don’t have to include the exit interface in this case, but just make sure you are indicating a /32 address.

After you put in these routing commands, what do you get in your routing table?

Secondly, I notice that you attempted to change the bandwidth setting in the tunnel and the physical interfaces. You’ve made the tunnel a cost of 10 and the physical interface a cost of 1000. That means that traffic will prefer going over the tunnel. But so will the traffic that is attempting to create the tunnel, which is the initial problem, so changing the cost will not benefit you.

Take a look at the above and verify your settings. If you continue to have problems, let us know and we’ll see if we can duplicate it…

I hope this has been helpful!

Laz

We got it!

With the precise static route, the tunnel goes UP and consequently, also the neighborship was also established as you can see:

R1# *Mar 26 11:51:38.990: OSPF-1 HELLO Tu0: Send hello to 224.0.0.5 area 0 from 192.168.10.2
R2# *Mar 26 12:55:01.963: OSPF-1 HELLO Tu0: Rcv hello from 1.1.1.1 area 0 192.168.10.2

Thanks for the advice, now I have understood that another “guideline” when configuring OSPF (probably EIGRP also) over VTI IPSec tunnel we need to declare a static route specifically for the remote peer physical interface.

Thank you very much Laz, see you soon,
Matteo

Hello Matteo

Great to hear! Glad I could be of help, my pleasure.

Talk soon!

Laz