FlexVPN Hub and Spoke

Hello can FlexVPN implemented on Cat 9500 and 9300 switches pls ?

Hello Hab

FlexVPN is a feature that is available only on Cisco routers. You can see more info about compatibility and platform support of FlexVPN in the following documentation:

So, unfortunately, the 9500 and 9300 do not support FlexVPN, because these devices are primarily designed for campus and data center switching, rather than being VPN concentrators.

I hope this has been helpful!

Laz

I followed the outline but I’m getting a few errors.

001609: *Jan  6 03:26:24.389: %ADJ-5-PARENT: Midchain parent maintenance for IP midchain out of Tunnel1 - looped chain attempting to stack
001610: *Jan  6 03:26:26.037: %TUN-5-RECURDOWN: Tunnel1 temporarily disabled due to recursive routing
001611: *Jan  6 03:26:26.037: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel1, changed state to down
001612: *Jan  6 03:26:26.289: IKEv2:(SA ID = 1):Retransmitting packet

001613: *Jan  6 03:26:26.289: IKEv2:(SA ID = 1):Sending Packet [To 98.xx.xx.xx:4500/From 192.168.1.190:4500/VRF i0:f0]
Initiator SPI : 06D974DD25B1A0DF - Responder SPI : 0FEBD183B05898BB Message id: 2
IKEv2 INFORMATIONAL Exchange REQUEST
Payload contents:
 ENCR

001614: *Jan  6 03:26:26.297: IKEv2:Detected an invalid IKE SPI

001615: *Jan  6 03:26:26.297: IKEv2:Couldn't find matching SA

001616: *Jan  6 03:26:26.297: IKEv2:(SA ID = 0):Received Packet [From 98.xx.xx.xx:4500/To 192.168.1.190:4500/VRF i0:f0]
Initiator SPI : ABCEA0BC8F907150 - Responder SPI : 037F1AFB09699BCE Message id: 0
IKEv2 INFORMATIONAL Exchange REQUEST
001617: *Jan  6 03:26:26.297: IKEv2:A supplied parameter is incorrect

001618: *Jan  6 03:26:26.297: IKEv2:
001619: *Jan  6 03:26:30.157: IKEv2:Detected an invalid IKE SPI

001620: *Jan  6 03:26:30.157: IKEv2:Couldn't find matching SA

001621: *Jan  6 03:26:30.157: IKEv2:(SA ID = 0):Received Packet [From 98.xx.xx.xx.xx:4500/To 192.168.1.190:4500/VRF i0:f0]
Initiator SPI : ABCEA0BC8F907150 - Responder SPI : 037F1AFB09699BCE Message id: 0
IKEv2 INFORMATIONAL Exchange REQUEST
001622: *Jan  6 03:26:30.157: IKEv2:A supplied parameter is incorrect

Hello Matthew

The error messages you’re encountering while configuring FlexVPN on Cisco devices indicate several issues that need to be addressed:

The first one has to do with Recursive Routing Issues. The messages %TUN-5-RECURDOWN: Tunnel1 temporarily disabled due to recursive routing and %ADJ-5-PARENT: Midchain parent maintenance... suggest a recursive routing problem. This typically happens when the route to the tunnel destination is learned through the tunnel itself, causing a loop.

Secondly, it seems that you have IKEv2 Errors. The repeated IKEv2 errors (Detected an invalid IKE SPI, Couldn't find matching SA, A supplied parameter is incorrect) point to issues with the IKEv2 Security Association (SA) negotiation. This could be due to a mismatch in the IKEv2 policy parameters (encryption, hash, authentication, etc.) between the FlexVPN peers.

Without knowing more about your configuration, I can make some suggestions and give you some guidelines as to how you can proceed in your troubleshooting:

  1. Check Routing Configuration: Ensure that the routes to the tunnel endpoints are not learned through the tunnel itself. You might need to define specific static routes or adjust your dynamic routing protocol configuration to prevent recursive routing.

  2. Verify IKEv2 Configuration:

    • Check the IKEv2 policies on both ends of the VPN to ensure they match exactly in terms of encryption algorithms, hash algorithms, and other parameters.
    • Ensure that the correct keyring and profiles are being used and that the endpoint addresses in the IKEv2 profiles are correctly configured.
    • The IKEv2:A supplied parameter is incorrect error suggests that there might be a misconfiguration in one of the IKEv2 parameters. Double-check all the parameters carefully.
  3. Review Tunnel Configuration: Verify the tunnel interface settings, including source and destination addresses, and ensure they are correctly configured.

Looking at these configuration parameters you should be able to determine the problem in your topology. Let us know how you get along and if you need any further assistance.

I hope this has been helpful!

Laz

Hi,

I used this lesson as a model to build my own FexVPN hub and spoke. I did NOT wish to use ikev2 routing and instead plan to use BGP at a future date. so my hub and spoke crypto ikev2 profile has only:

show run | se crypto ikev2 profile
crypto ikev2 profile LNEFLEXVPN_PROFILE
 match identity remote address 44.228.114.254 255.255.255.255 
 match identity remote address 0.0.0.0 
 authentication remote pre-share
 authentication local pre-share
 keyring local LNBRwestspoke_KEYRING
 aaa authorization group psk list LNE-FLEXVPN default

Currently the tunnel is showing up up on the spoke and I see a “Virtual-Access1” interface on my hub.

AWSBR02aDC-NEW#show crypto ipsec sa

interface: Virtual-Access1
    Crypto map tag: Virtual-Access1-head-0, local addr 10.149.135.44

   protected vrf: (none)
   local  ident (addr/mask/prot/port): (10.149.135.44/255.255.255.255/47/0)
   remote ident (addr/mask/prot/port): (65.243.77.214/255.255.255.255/47/0)
   current_peer 65.243.77.214 port 5761
     PERMIT, flags={origin_is_acl,}
    #pkts encaps: 5, #pkts encrypt: 5, #pkts digest: 5
    #pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
    #pkts compressed: 0, #pkts decompressed: 0
    #pkts not compressed: 0, #pkts compr. failed: 0
    #pkts not decompressed: 0, #pkts decompress failed: 0
    #send errors 0, #recv errors 0

     local crypto endpt.: 10.149.135.44, remote crypto endpt.: 65.243.77.214
     plaintext mtu 1426, path mtu 1500, ip mtu 1500, ip mtu idb GigabitEthernet2
     current outbound spi: 0x487A84DA(1215988954)
     PFS (Y/N): N, DH group: none

     inbound esp sas:
      spi: 0x29F1C09F(703709343)
        transform: esp-256-aes esp-sha512-hmac ,
        in use settings ={Transport UDP-Encaps, }
        conn id: 2001, flow_id: CSR:1, sibling_flags FFFFFFFF80000008, crypto map: Virtual-Access1-head-0, initiator : False
         sa timing: remaining key lifetime (k/sec): (4608000/2342)
        IV size: 16 bytes
        replay detection support: Y
        Status: ACTIVE(ACTIVE)

     inbound ah sas:

     inbound pcp sas:

     outbound esp sas:
      spi: 0x487A84DA(1215988954)
        transform: esp-256-aes esp-sha512-hmac ,
        in use settings ={Transport UDP-Encaps, }
        conn id: 2002, flow_id: CSR:2, sibling_flags FFFFFFFF80000008, crypto map: Virtual-Access1-head-0, initiator : False
         sa timing: remaining key lifetime (k/sec): (4607999/2342)
        IV size: 16 bytes
        replay detection support: Y
        Status: ACTIVE(ACTIVE)
          
     outbound ah sas:

     outbound pcp sas:

and on spoke:

show crypto ipsec sa                 

interface: Tunnel150
    Crypto map tag: Tunnel150-head-0, local addr 10.44.86.2

   protected vrf: (none)
   local  ident (addr/mask/prot/port): (10.44.86.2/255.255.255.255/47/0)
   remote ident (addr/mask/prot/port): (44.228.114.254/255.255.255.255/47/0)
   current_peer 44.228.114.254 port 4500
     PERMIT, flags={origin_is_acl,}
    #pkts encaps: 176, #pkts encrypt: 176, #pkts digest: 176
    #pkts decaps: 5, #pkts decrypt: 5, #pkts verify: 5
    #pkts compressed: 0, #pkts decompressed: 0
    #pkts not compressed: 0, #pkts compr. failed: 0
    #pkts not decompressed: 0, #pkts decompress failed: 0
    #send errors 0, #recv errors 0

     local crypto endpt.: 10.44.86.2, remote crypto endpt.: 44.228.114.254
     plaintext mtu 1426, path mtu 1500, ip mtu 1500, ip mtu idb Port-channel1.801
     current outbound spi: 0x29F1C09F(703709343)
     PFS (Y/N): N, DH group: none

     inbound esp sas:
      spi: 0x487A84DA(1215988954)
        transform: esp-256-aes esp-sha512-hmac ,
        in use settings ={Transport UDP-Encaps, }
        conn id: 2020, flow_id: ESG:20, sibling_flags FFFFFFFF80004008, crypto map: Tunnel150-head-0, initiator : True
         sa timing: remaining key lifetime (k/sec): (4607999/2307)
        IV size: 16 bytes
        replay detection support: Y
        Status: ACTIVE(ACTIVE)

     inbound ah sas:

     inbound pcp sas:

     outbound esp sas:
      spi: 0x29F1C09F(703709343)
        transform: esp-256-aes esp-sha512-hmac ,
        in use settings ={Transport UDP-Encaps, }
        conn id: 2019, flow_id: ESG:19, sibling_flags FFFFFFFF80004008, crypto map: Tunnel150-head-0, initiator : True
         sa timing: remaining key lifetime (k/sec): (4607985/2307)
        IV size: 16 bytes
        replay detection support: Y
        Status: ACTIVE(ACTIVE)

     outbound ah sas:

     outbound pcp sas:

I am unable to ping the hub (172.16.150.1) from the spoke (172.16.150.5).

Any ideas on how I can troubleshoot this issue further?

-Brian

Hello Brian

From the configs you have shared, there don’t seem to be any outstanding issues that immediately point to the problem. The IPsec tunnel itself is successfully established (both inbound and outbound SAs are ACTIVE), so Phase 1 and Phase 2 look fine. The main problem is that there is no end-to-end connectivity to transfer traffic inside the tunnel.

This could be due to several factors. I can share with you some thoughts that may help point you in the right direction for your troubleshooting process:

  • Since you’re not using IKEv2 routing, the first thing that comes to mind is that you need to look at your routing configuration. Verify that the hub’s routing table includes the spoke’s subnet. Spokes should have a default route pointing to the hub’s tunnel IP.
  • Ensure that the tunnel interfaces are configured correctly and are in the same subnet.
  • Verify using debug cyrpto ikev2 packet and debug crypto ipsec. See below document for more debugging information:

I hope this gives you some insight into your next steps, let us know how you get along!

I hope this has been helpful!

Laz

Hello, I found a typo in 1.1.1. IKEv2 Keyring.
Dash between pre-shared and key is missing.

Hub1(config)#crypto ikev2 keyring IKEV2_KEYRING
Hub1(config-ikev2-keyring)#peer SPOKE_ROUTERS
Hub1(config-ikev2-keyring-peer)#address 0.0.0.0 0.0.0.0
Hub1(config-ikev2-keyring-peer)#pre-shared key local CISCO
Hub1(config-ikev2-keyring-peer)#pre-shared key remote CISCO

Hello Bernd

Indeed you are correct, thanks for pointing that out! I will let @ReneMolenaar know to make the correction.

Thanks again!

Laz

In your reply, you mentioned the overlay and underlay networks. But the 0/0 route seems to be the point of confusion for me on this if I have 3 sites 1 being the hub and the two being the spokes. What happens if I need internet access on any of the sites using a 0/0 route. How does that work when the hub is configuring that on all the sites?

Thank you,
Alan

Sorry, this is a response to one of the first posts on this page:

Hello Alan

If we want to establish internet connectivity at the spokes via the local internet connection, how do we do that? With the current config, the hub pushes a default route to the spokes, so the routing over the overlay network to the internet will go via the hub. Such an arrangement would enable centralized internet access for all spokes via the hub.

The hub uses the IKEv2 authorization policy to “push” a route to the spokes during tunnel establishment. This is where the hub specifies that the spoke should install a default route via the tunnel. In the lesson, this is done like so:

Hub1(config)#crypto ikev2 authorization policy IKEV2_AUTHORIZATION
Hub1(config-ikev2-author-policy)#route set interface
Hub1(config-ikev2-author-policy)#route set access-list FLEXVPN_ROUTES

Hub1(config)#ip access-list standard FLEXVPN_ROUTES
Hub1(config-std-nacl)#permit any

It is the FLEXVPN_ROUTES ACL that defines what routes will be sent to the hub. Right now, this is everything, and that’s why we see the default route on the spokes. If you want to change that, you can change this ACL to only include the internal routes, and then set up routing at each spoke that will allow internet traffic to exit the local internet connection. This is known as a split tunnel ACL.

I hope this has been helpful!

Laz

Lazarus,

Thank you for the response. I have a lab for different builds for DMVPN and to say the least it’s been some work. Trying to do DMVPN over a VRF or tying the source interfaces to a loopback has been challenging. With having a VRF for the overlay DMVPN I could have two default routes in each context since both route tables are different, correct? So global routing table has default to internet for basic routing let’s assume to connect to service provider. Then have a VRF for the DMVPN that has its own default route. Am I piecing this together correctly? I guess my main confusion here is that I am understanding your response with the default route over the VRF then all spokes get to the internet over that hub default route. But then what happens with the default route the hub and spokes use to connect to each other for the tunnel endpoints to talk? Hopefully this makes sense.

Thank you,
Alan

Hello Alan

DMVPN can be a challenging technology, especially if you are combining it with other mechanisms as well.

You mention different VRFs. Are you referring to the overlay network routing and the underlay network routing? Or have you introduced VRFs in your topology to fulfil an additional requirement? Because in conventional DMVPN or FlexVPN, you don’t explicitly configure VRFs unless you need them for something else. I’m assuming you mean the overlay and underlay networks.

If that is the case, then we have to look at these differently and not in the context of VRFs. The underlay network for DMVPN is used to allow the Hub and Spokes to reach each other’s physical interfaces. This is a prerequisite in order for them to establish mGRE tunnels between them. This uses conventional routing. Once the mGRE tunnels are created, you then have your overlay network. From the spoke’s perspective, the underlay and overlay exist within the same routing table (unless VRFs are explicitly used), but they behave like separate planes, with routing decisions based on the next-hop and exit interface. Any routing in the spoke to the underlay or overlay network will be determined based on the next hop. Any route that uses the tunnel as the exit interface goes on the overlay network, and any route that uses the physical interface goes on the underlay network. If you want to direct some local traffic to exit directly to the Internet, then you must configure routing to exit the physical interface for such traffic.

The confusion may come from the fact that you would like to use a default route for both the underlay traffic (to the internet) and the overlay traffic (via the DMVPN topology). Since you can’t have two default routes in the same RIB without conflict, you’ll need to prioritize underlay reachability for tunnel establishment, perhaps using a static host route for the tunnel destination and then using the default route for overlay traffic learned via routing protocols. But this is not an issue of VRFs, it is an issue of employing your routing such that the correct traffic (i.e. the traffic you want) goes over the overlay and the underlay as you desire.

I hope this has been helpful!

Laz