FlexVPN Remote Access AnyConnect

This topic is to discuss the following lesson:

Hi Rene and the NWL Team,

I managed to resolve nearly all issues by using the same anyconnect version utilised in the lesson (4.8.03052)

However once I click connect anyway after the self-signed certificate warning, anyconnect requests the local AAA credentials upon entering those credentials

“The IPsec VPN connection was terminated due to an authentication failure or timeout”?

Any help would be much appreciated.

P.S I am using IOS 15.7(3)(M3)

Hello Taha

Keep in mind that the AnyConnect client depending upon its version can support various different configurations. You may have configured the ASA exactly the same way as in the lesson, but the client may or may not support certain configurations.

You may also find that depending upon your operating system or connecting device (Windows 10, Windows 7/8, Android, iOS, Mac etc…) you will have different capabilities. The following Cisco community thread describes some of these issues that have resulted in the same error as you are getting here. Hopefully this will shed some light on the problems you are facing:

Let us know how your troubleshooting is coming along…

I hope this has been helpful!

Laz

1 Like

After consistently spending time in the lab troubleshooting/reading/learning the fundamentals of IKEV2. I was able to use the false positive debug information to track the root cause of this “passed authentication and failed authorization” issue.

I have finally managed to resolve this issue and I hope this small write-up will help others in future too.

I was using the same version of anyconnect 4.8 in the article and IOS 15.7.

Authorization Failure:
The article really helped! however it left out a small issue:

From the article I learned that:

A VPN with Ikev2 requires the following:

IKEv2 proposal

IKEv2 policy

IKEv2 Authorization Policy (missing from the article and caused small confusion)

IKEv2 profile

IKEv2 keyring

IPSec:

IPSec transform-set

IPSec profile

When you are configuring the profile in IKEv2
and you are declaring the aaa authorization group anyconnect-eap list 'NAME OF YOUR AAA AUTHORIZATION NETWORK You must FOLLOW this up with the KEv2 Authorization Policy!!

This applies to you if you are using a radius or local authentication!

for example aaa authorization group anyconnect-eap list 'AAA_AUTHORIZATION_NETWORK' 'IKEV2_AUTHORIZATION_POLICY'

even if you are using a policy derived from radius you must use a “dummy” authorization policy!

A populated authorization policy example:

crypto ikev2 authorization policy IKEV2_AUTHORIZATION_POLICY
pool VPN_POOL
dns 1.1.1.1
def-domain NWL.LAB
route set remote ipv4 1.1.1.1 255.255.255.255

a “dummy” authorization policy:

crypto ikev2 authorization policy IKEV2_AUTHORIZATION_POLICY

(EMPTY)

The issue with “Smart Defaults”
The main issue with smart defaults IS even IF you don’t configure it and you let “smart defaults” predefine a “default” authorization policy, that authorization policy name will be required to be called in the profile! otherwise authentication will pass but authorization will FAIL!( my experience). If you don’t call and name a authorization policy I’m not sure how we can specify it in the profile!

Cryptographic Errors when using anyconnect 4.9+

AnyConnect 4.9.00086 disables certain cryptography encryption/hash and groups DES, 3DES, MD5, and DH groups 2,5, 14, and 24

Workaround:

SOLUTION 1: Simply specify all encryption and hash in the proposal

crypto ikev2 proposal MY_IKEV2_PROPOSAL
encryption aes-cbc-256 aes-cbc-192 aes-cbc-128 3des
integrity sha512 sha384 sha256 sha1
group 21 20 19 16 14 5 2

crypto ikev2 policy MY_IKEV2_POLICY
proposal MY_IKEV2_PROPOSAL

Solution 2:
Write separate proposals and specify them in the ikev2 Policy(not to be confused with the “authorization policy”)

crypto ikev2 proposal HIGH
encryption aes-cbc-256 aes-cbc-192 aes-cbc-128
integrity sha512 sha384 sha256
group 21 20 19
crypto ikev2 proposal MEDIUM
encryption aes-cbc-256 aes-cbc-192 aes-cbc-128
integrity sha256 sha1
group 16 14
crypto ikev2 proposal LOW
encryption aes-cbc-128 3des
integrity sha1 md5
group 5 2
crypto ikev2 policy MY_IKEV2_POLICY
proposal HIGH
proposal MEDIUM
proposal LOW

Solution 3: Use “anyconnect version pre 4.9”
NOTE: You are then resorting to utilising depreciated cryptography “encryption/hash and groups”

Remember both these issue apply if you are using radius ISE OR local database for AAA

I hope that makes sense, please leave me a comment for any further clarification.

Hello Taha

I suggest you go into the device and take a look at the syslogs. They should tell you the reason for the failure. If it is credentials, if it is a mismatch in authentication parameters, or if it is actually a timeout. Looking at the logs will give you a clearer picture so that you can more correctly approach the troubleshooting process…

I hope this has been helpful!

Laz

1 Like

Hi Lazaros

I have been using debug crypto ikev2.

This really helped, unfortunately didn’t point me in the right direction, but it made a difference. I could see the client was replying back to the server with authen credentials! It gave me the exact line that was failing. So I knew where to start

Taha

Hello Taha!

Thanks so much for your post above, it really adds to the value and relevance of the forum. Your contribution is much appreciated! I’ll let Rene know so he can consider including some of that information in the lesson for clarification.

Thanks again for your contribution!

Laz

Hi Laz,

I am extremely flattered by your compliments thank you. I am novice in the cisco world.
I think very highly of any individual who has acquired a CCIE. I see them as a role-model!

As of late networking and virtualization has become a hobby activity, subsequently this aids ones ability to learn rather quickly.

The issue came to my attention, after reading cisco article on IKEv2:

Under the proposal section it states:

“Manually configured IKEv2 proposals must be linked with an IKEv2 policy; otherwise, the proposals are not used in the negotiation”

consequently when a proposal is manually configured, this must be called/linked to a policy for example:

> Crypto IKEv2 proposal nwlesson_Proposal
> {Encryption Algorithm} e.g aes-cbc256 etc
*> {Integrity Algorithm} any hash of your choice e.g SHA etc *
> {Pseudo-Random function (PRF)} The PRF algorithm is the same as integrity algorithm, subesquently this does not to separately configured
{Diffie-Hellman(DH) Group}

I discovered yesterday in the lab, if you want this proposal to be utilised in the IKEv2 negotiation it MUST be called/linked by a policy for example:

Crypto IKEv2 Policy nwlesson_Policy
proposal nwlesson_Proposal (our proposal we configured above)

During a friday night dinner out with the family. I kept thinking, what happens if we don’t link the proposal to a policy?

Subsequently from 10pm untill 1am in the lab, I tested this theory. The configured proposal(encryption, Integrity, PRF algorithm and DH group) is not utilised in the SA_INIT exchange of the IKEv2 negotiation, if there isn’t a policy!

subsequently the question becomes, what proposal is being utilised if the one we configured isn’t being used in the exchange?

The answer: the default proposal!(remember smart defaults?)

now on this fantastic article by Rene that allowed me to learn about FlexVPN in general, I am very appreciative of this tutorial! it’s simply outstanding!

You will notice there is no policy that links the proposal, just a authorization policy which technically is completely different. Subsequently we are not using those proposal declared in the tutorial for the negotiation we are simply using default values provided by IOS:

in order to utilise the proposa(IKEV2_PROPOSAL)l that was declared in the lesson we must include the following in the tutorial guide:

Crypto IKEv2 Policy nwlesson_Policy
proposal IKEV2_PROPOSAL

many thanks again!

Taha

Hello Taha

Well, credit where credit is due. Being curious about “what happens if…” is part of being good at networking, especially when it comes to complex interactions of various features on a device.

Thanks for adding these details, it helps all of us get better at what we do.

Laz

1 Like

Hi @ReneMolenaar ,

It’s an absolute pleasure to meet you :raising_hand_man:

I apologise I opened up multiple topics regarding this issue, as I was experiencing multiple issues with the lesson. Lets use this forum as I have included details on how I came to conclusions with some of the issues above.

I would like to state the article is wonderful. Between each step you provide details on a) why we are performing the the particular step b)How we will perform this config.
With this type of break-down one is able to fully comprehend and understand the config, subsequently this allows you to manipulate the code to satisfy individualised criteria.

I was further able to troubleshoot some of the issues, which I raised with @lagapidis

There was thee major issues which caused the config to not function, or caused compatibility issues with other versions of anyconnect and IOS.

I will outline those technical details below:

1. Manually configured IKEv2 proposals must be linked with an IKEv2 policy; otherwise, the proposals are not used in the negotiation.
However in the original There was no policy declared, I learned from Cisco’s official IKEv2. That a policy must be configured derived from the manually declared IKEv2 proposal.

IF there isn’t “CRYPTO IKEv2 POLICY” that will link back to the propsal, consequently the manually configured propsal in the lesson was not being utilised during the IKEv2_INIT part of the IKEv2 negoation.

2.The Default proposal must be disabled in order to utilise the manually configured proposal
This subsequent issue we run into is a direct consequence of the above configuration.
Now that we have declared a policy that will be linked with a manually configured proposal, we must disable the default policy. Otherwise the config will still prioritise the default policy, this will result in the default proposal being utilised.
When you have configured a custom IKEv2 Proposal and Policy you can and should disable the defaults.
no crypto ikev2 proposal default
or
no crypto ikev2 policy default
Because the default policy was being utilised, this was initiating the default proposal. This resulted in IKEv2 using depreciated cryptography, integrity and Diffie-Hellman group in the IKEv2_INIT part of the negation. Consequently only anyconnect <4.8 was supported by the server.

3. IKEv2 Authorization Policy needs to be linked to the IKEv2 Profile.
When you are configuring the profile in IKEv2, You must declare the aaa authorization group anyconnect-eap list 'NAME OF YOUR AAA AUTHORIZATION NETWORK You must FOLLOW this up with the KEv2 Authorization Policy

This applies even if are using a radius server(i.e ISE) or local authentication!

for example in the profile in aaa authorization group anyconnect-eap list it must be linked with authorization policy e.g ‘AAA_AUTHORIZATION_NETWORK' 'IKEV2_AUTHORIZATION_POLICY’

During my time in the lab, I learned even if you are a using authorization policy attributes deriving from radius(e.g ISE), that you must use a “dummy” authorization policy!(strange I know)

This is not mentioned by cisco, which I found rather strange.

For example:
Here is the current attribute populated authorization policy :

crypto ikev2 authorization policy IKEV2_AUTHORIZATION_POLICY
pool VPN_POOL
dns 1.1.1.1
def-domain NWL.LAB
route set remote ipv4 1.1.1.1 255.255.255.255

Here is a “dummy” authorization policy that needs to be declared, this applies even if attributes is to derived from ISE.
crypto ikev2 authorization policy IKEV2_AUTHORIZATION_POLICY

(EMPTY)

in the tutorial we are using LOCAL derived attributes, however if both are configured(ISE and Local) LOCAL will take preference.
The IKEv2 authorization policy serves as a container of IKEv2 local AAA group authorization parameters.

The issue was the tutorial’s authorization policy was configured with attributes. However it was NOT linked to the profile. Consequently this was causing the authorization error, in the AAA part of the negotiations.

I managed to troubleshoot most of these issues using:

debug crypto IKEv2
debug aaa authorization
debug aaa authentication

Update: I noticed you managed to update the tutorials to address some of these issues I raised, that is absolutely brilliant!

Many thanks again!
Taha

1 Like

Please add a note to disable the AnyConnect profile downloader in AnyConnectLocalPolicy.xml by updating: true.

Otherwise we get the error: “The VPN client failed to establish a connection.” after authenticating.
AnyConnect-Error

MAC OS: /opt/cisco/anyconnect/AnyConnectLocalPolicy.xml
Windows: C:\ProgramData\Cisco\Cisco AnyConnect Secure Mobility Client\AnyConnectLocalPolicy.xml

I’m running IOS-XE 17.3.3. The Cisco links below reference turning this off when using anycast-eap aggregate. The second link implies this is ‘optional’ and not needed after 16.9.1, but I did not find this to be the case. After setting it to true, everything works as expected!

References:

1 Like

Hello Jason

Thanks so much for sharing this on the forum. I’ll let Rene know to take a look and make the appropriate updates to the lesson.

Thanks again!

Laz

2 Likes

Thanks for the great post: it’s the clearest tutorial on remove vpn I’ve seen so far!
My question is - what change do I need to make in order to allow the system MacOS IKEv2 client connect?
I’ve run some tests and while the IKEv2 SA comes up properly, the process fails at the EAP exchange with a very obscure error (note in the log remote-client is the client ID I’m using on macOS):

Jan 16 12:59:02.918: AAA/AUTHEN/LOGIN (00000FE8): Pick method list 'vpn-eap-auth-local'
Jan 16 12:59:02.918: IKEv2:sending remote-client [IDi] as username to AAA
Jan 16 12:59:02.918: IKEv2:(SA ID = 3):[IKEv2 -> AAA] Authentication request sent
Jan 16 12:59:02.919: AAA Failed to validate User blocking for one or more of below reasons
Jan 16 12:59:02.919: AAA Shim Block User Enable status : 1
Jan 16 12:59:02.919: AAA Global user Block status      : 0
Jan 16 12:59:02.919: AAA Username                      : remote-client
Jan 16 12:59:02.919:
AAA Local EAP : Recieved AAA authentication request with EAP Attribute
Jan 16 12:59:02.919: AAA Local EAP: Local EAP termination Halted !!
Jan 16 12:59:02.919: AAA/LOCAL/LOGIN Validating for user lock out

Jan 16 12:59:02.919: AAA/LOCAL/LOGIN configured max failed attempt = 0

Jan 16 12:59:02.919: AAA/LOCAL/LOGIN user lock out status          = 0

Jan 16 12:59:02.919: AAA/LOCAL/LOGIN: failure_reason: [2]
Jan 16 12:59:02.919: AAA/LOCAL/LOGIN(00000FE8): failover
Jan 16 12:59:02.919: AAA Failed to validate User blocking for one or more of below reasons
Jan 16 12:59:02.919: AAA Shim Block User Enable status : 1
Jan 16 12:59:02.919: AAA Global user Block status      : 0
Jan 16 12:59:02.919: AAA Username                      : remote-client
Jan 16 12:59:02.919: IKEv2-ERROR:%Unsuccessful AAA response FAIL

I feel stuck because I can’t figure out what the issue is - MacOS’ client logs are of no help at all. Any idea?

1 Like

Great Lesson! Thanks!
I am quite new to this topic.
Can I change the default TCP port number to do not conflict with web service running on the outside interface?

Cheers, Hannes

Hello Johann

When you configure any type of VPN on a device, if you take no further action, only one VPN client can connect to the VPN at any one time. Remember, the VPN functionality takes place at Layer 3. Yes, IKE negotiation uses UDP 500, but that’s just for IKE. The actual traffic is sent using regular TCP and UDP ports as determined by the applications using them.

This is a “problem” that exists whether you are attempting to connect multiple VPN clients to a single VPN headend device, or if you are trying to reserve specific ports to use to access the CLI interface of the router itself. Both of these problems can be resolved by using NAT.

There are basically three solutions to this:

  1. NAT Traversal - This method uses UDP port 500 for IKE negotiation, but then tunnels IPSec data within UDP packets using port 4500. This should free up other ports to be used for other purposes.
  2. IPSec over UDP - This method uses UDP port 500 for IKE negotiation, but tunnels all IPSec data within a predefined UDP port, which by default is port 10000.
  3. IPSec over TCP - This method tunnels both IKE negotiation and IPsec data traffic within a predefined TCP port.

FlexVPN supports NAT traversal. For more information about IPSec over UDP and TCP, take a look at this Cisco community thread:

https://community.cisco.com/t5/vpn/difference-between-ipsecover-udp-and-ipsec-over-tcp/td-p/2481653#:~:text=Whereas%20IPSec%20over%20UDP%2C%20similar,still%20use%20UDP%20port%20500.

I hope this has been helpful!

Laz

Thank you very much!
In the meantime FlexVPN via AnyConnect IPSec is up and running.

Now I play with AnyConnect via SSL VPN.
It work till username and password login.
But then AnyConnect gets disconnected.

regards, Hannes

Now I try SSL-VPN with anyconnect to Cisco 1111!

Getting following debug messages:

077275: Nov  3 09:51:14.694 MESZ: %SEC_LOGIN-5-LOGIN_SUCCESS: Login Success [user: hannes2] [Source: 213.142.97.2] [localport: 55443] at 09:51:14 MESZ Thu Nov 3 2022
C1111#
077276: Nov  3 09:51:15.214 MESZ: 
077277: Nov  3 09:51:15.214 MESZ: 
077278: Nov  3 09:51:15.214 MESZ: [CRYPTO-SSL-TUNL-EVT]:[FFFF4AFB0800] CSTP Version recd , using 1
077279: Nov  3 09:51:15.214 MESZ: [CRYPTO-SSL-TUNL-ERR]:[FFFF4AFB0800] Full Tunnel CONNECT request failed, Sending error
077280: Nov  3 09:51:15.214 MESZ: HTTP/1.1 401 Unauthorized
077281: Nov  3 09:51:15.215 MESZ: 
077282: Nov  3 09:51:15.216 MESZ: 
077283: Nov  3 09:51:15.216 MESZ: 
077284: Nov  3 09:51:15.216 MESZ: [CRYPTO-SSL-TUNL-ERR]:[FFFF4AFB0800] User hannes2 not authorized to access Full tunnel

SSL Profile:

C1111#sh cry ssl prof

SSL Profile: SSL_PROFILE
 Status: ACTIVE
 Match Criteria:
   URL: none
   Policy: SSL_POLICY
 AAA accounting List      : local
 AAA Authentication List  : ANYCONNECT-USERS
 AAA Authorization User List   : ANYCONNECT-USERS
   User : hannes2
   Cached : True
 AAA Authorization Group List  : ANYCONNECT-USERS
   Group List: hannes2
   Override: True
 Authentication Mode      : user credentials
 Interface                : SSLVPN-VIF0
   Status: DISABLE
 Max Users                : 10000

Any ideas?

Hello Johann

Doing some research, I have found that others are having similar difficulties with the 1000 series routers. I believe it may have to do with a slight misconfiguration or an issue with what the specific router supports.

Can you share your SSL profile and crypto configuration with us as well? That way we can help you further. Some additional links that may also be helpful include:

I hope this has been helpful!

Laz

1 Like

Hi!

Here is my config.
IKEV2 is working.
SSL is not working.
I don’t know how to configure the virtual-template - interface and type.
On my MacBook I can ping the following tunnel IP:

utun6: flags=80d1<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST> mtu 1406
	inet 192.168.255.63 --> 192.168.255.63 netmask 0xffffff00 
	inet6 fe80::bed0:74ff:fe1a:5cf9%utun6 prefixlen 64 scopeid 0x19 
	inet6 fe80::2642:4245:6d5c:3ce6%utun6 prefixlen 128 scopeid 0x19 
	nd6 options=201<PERFORMNUD,DAD>
C1111#sh run
Building configuration...

Current configuration : 31756 bytes
!
version 17.6
hostname C1111
!
boot-start-marker
boot system bootflash:c1100-universalk9.17.06.04.SPA.bin
boot system flash 
boot-end-marker
!
!
!
aaa new-model
!
!
aaa authentication login default local
aaa authentication login no-login none
aaa authentication login vpn-authen local
aaa authentication login FLEX local
aaa authorization exec default local 
aaa authorization exec no-login none 
aaa authorization network default local 
aaa authorization network vpn-author local 
aaa authorization network AAA_AUTHORIZATION_NETWORK local 
aaa authorization network FLEX_SSL local 
!
!
!
!
!
!
!
!
crypto ikev2 authorization policy IKEV2_AUTHORIZATION_POLICY 
 pool vpnusers
 def-domain NWL.LAB
 route set remote ipv4 10.0.0.0 255.255.255.0
!
crypto ikev2 proposal IKEV2_PROPOSAL 
 encryption aes-cbc-256
 integrity sha256
 group 19
!
crypto ikev2 policy IKEV2_POLICY 
 proposal IKEV2_PROPOSAL
!
!
!
crypto ikev2 profile IKEV2_PROFILE
 match identity remote key-id *$AnyConnectClient$*
 authentication local rsa-sig
 authentication remote anyconnect-eap aggregate
 pki trustpoint TP-self-signed-2984925144
 aaa authentication anyconnect-eap AAA_AUTHENTICATION_LOGIN
 aaa authorization group anyconnect-eap list AAA_AUTHORIZATION_NETWORK IKEV2_AUTHORIZATION_POLICY
 virtual-template 1
!
!
!
! 
crypto ssl proposal SSL_PROPOSAL 
 protection rsa-aes256-sha1
!
crypto ssl authorization policy SSL_AUTH_POL 
 homepage https://google.at
 netmask 255.255.255.0
 pool vpnusers
 dns 8.8.8.8
 banner Hallo Hannes
!
crypto ssl policy SSL_POLICY 
 ssl proposal SSL_PROPOSAL
 pki trustpoint TP-self-signed-2984925144 sign
 ip interface GigabitEthernet0/0/0 port 55443
!
crypto ssl profile SSL_PROFILE 
 match policy SSL_POLICY 
 aaa authentication user-pass list FLEX 
 aaa authorization user user-pass list FLEX_SSL SSL_AUTH_POL 
 aaa authorization group user-pass list default hannes2 
 authentication remote user-pass 
 virtual-template 10
!
!
!
!
crypto ipsec transform-set TRANSFORM_SET esp-aes 256 esp-sha256-hmac 
 mode tunnel
!
crypto ipsec profile IKEV2_PROFILE
 set transform-set TRANSFORM_SET 
 set ikev2-profile IKEV2_PROFILE
!
!
!
!
!
!
interface GigabitEthernet0/0/0
 description outside - 62.178.173.16 per dhcp
 bandwidth 150000
 ip ddns update hostname hummer.no-ip.org
 ip ddns update no-ip.org
 ip address dhcp
 ip nbar protocol-discovery ipv4
 ip nat outside
!
interface Virtual-Template1 type tunnel
 ip unnumbered Vlan1
 ip mtu 1400
 tunnel mode ipsec ipv4
 tunnel protection ipsec profile IKEV2_PROFILE
!
interface Virtual-Template2
 ip unnumbered Vlan1
!
interface Virtual-Template5
 ip unnumbered GigabitEthernet0/0/0
 ip nat inside
 peer default ip address pool NAT-ACL
 keepalive 5
 compress mppc
!
interface Virtual-Template10 type vpn 
 ip unnumbered Vlan1
 ip mtu 1400
!
interface Vlan1
 ip address 10.0.0.1 255.255.255.0
 ip pim sparse-dense-mode
 ip nat inside
 ip access-group from-vlan1 in
 ip tcp adjust-mss 1452
!
ip local pool vpnusers 192.168.255.1 192.168.255.254
ip tcp synwait-time 10
ip telnet tos 48
ip http server
ip http authentication local
ip http secure-server
ip http timeout-policy idle 60 life 86400 requests 10000 

ip access-list standard line
 10 permit 10.0.0.0 0.255.255.255
 20 permit 192.168.0.0 0.0.255.255
 30 permit any
!
!
!
!
end

Hi @hannes.1967.hiller

I would suspect this has to do with your authentication and/or authorization (lists):

crypto ssl profile SSL_PROFILE 
 match policy SSL_POLICY 
 aaa authentication user-pass list FLEX 
 aaa authorization user user-pass list FLEX_SSL SSL_AUTH_POL 
 aaa authorization group user-pass list default hannes2 
 authentication remote user-pass 
 virtual-template 10

If you use debug aaa authentication and debug aaa authorization does it tell you anything?

Rene