Cisco ASA VPN to Cisco Router “MM_WAIT_MSG3”

Posted on 27/03/2019 by PeteLong

KB ID 0001531

Problem

While migrating a VPN tunnel from an ASA 5520 firewall to a new 5516-X I got this problem. The other end was a Cisco router (2900). As soon as I swapped it over, it was stuck at MM_WAIT_MSG3, and phase 1 would not establish;

[box]

NUFC-ASA5516x(config-tunnel-ipsec)# show crypto isa

IKEv1 SAs:

   Active SA: 6
    Rekey SA: 0 (A tunnel will report 1 Active and 1 Rekey SA during rekey)
Total IKE SA: 6

1   IKE Peer: 1.1.1.1
    Type    : user            Role    : responder
    Rekey   : no              State   : MM_WAIT_MSG3

[/box]

Debugs didn’t help much either;

[box]

Mar 25 2019 18:50:49: %ASA-7-713236: IP = 1.1.1.1, IKE_DECODE RECEIVED Message (msgid=0) with payloads : HDR + SA (1) + VENDOR (13) + VENDOR (13) + VENDOR (13) + VENDOR (13) + NONE (0) total length : 164
Mar 25 2019 18:50:49: %ASA-7-715047: IP = 1.1.1.1, processing SA payload
Mar 25 2019 18:50:49: %ASA-7-713906: IP = 1.1.1.1, Oakley proposal is acceptable
Mar 25 2019 18:50:49: %ASA-7-715047: IP = 1.1.1.1, processing VID payload
Mar 25 2019 18:50:49: %ASA-7-715049: IP = 1.1.1.1, Received NAT-Traversal RFC VID
Mar 25 2019 18:50:49: %ASA-7-715047: IP = 1.1.1.1, processing VID payload
Mar 25 2019 18:50:49: %ASA-7-715047: IP = 1.1.1.1, processing VID payload
Mar 25 2019 18:50:49: %ASA-7-715049: IP = 1.1.1.1, Received NAT-Traversal ver 03 VID
Mar 25 2019 18:50:49: %ASA-7-715047: IP = 1.1.1.1, processing VID payload
Mar 25 2019 18:50:49: %ASA-7-715049: IP = 1.1.1.1, Received NAT-Traversal ver 02 VID
Mar 25 2019 18:50:49: %ASA-7-715047: IP = 1.1.1.1, processing IKE SA payload
Mar 25 2019 18:50:49: %ASA-7-715028: IP = 1.1.1.1, IKE SA Proposal # 1, Transform # 1 acceptable  Matches global IKE entry # 8
Mar 25 2019 18:50:49: %ASA-7-715046: IP = 1.1.1.1, constructing ISAKMP SA payload
Mar 25 2019 18:50:49: %ASA-7-715046: IP = 1.1.1.1, constructing NAT-Traversal VID ver RFC payload
Mar 25 2019 18:50:49: %ASA-7-715046: IP = 1.1.1.1, constructing Fragmentation VID + extended capabilities payload
Mar 25 2019 18:50:49: %ASA-7-713236: IP = 1.1.1.1, IKE_DECODE SENDING Message (msgid=0) with payloads : HDR + SA (1) + VENDOR (13) + VENDOR (13) + NONE (0) total length : 128
Mar 25 2019 18:50:57: %ASA-7-713236: IP = 1.1.1.1, IKE_DECODE RESENDING Message (msgid=0) with payloads : HDR + SA (1) + VENDOR (13) + VENDOR (13) + NONE (0) total length : 128
Mar 25 2019 18:50:59: %ASA-7-713906: IKE Receiver: Packet received on 2.2.2.2:500 from 1.1.1.1:500
Mar 25 2019 18:50:59: %ASA-5-713202: IP = 1.1.1.1, Duplicate first packet detected.  Ignoring packet.
Mar 25 2019 18:51:05: %ASA-7-713236: IP = 1.1.1.1, IKE_DECODE RESENDING Message (msgid=0) with payloads : HDR + SA (1) + VENDOR (13) + VENDOR (13) + NONE (0) total length : 128
Mar 25 2019 18:51:21: %ASA-7-715065: IP = 1.1.1.1, IKE MM Responder FSM error history (struct &0x00007f4d2d293690)  , :  MM_DONE, EV_ERROR-->MM_WAIT_MSG3, EV_TIMEOUT-->MM_WAIT_MSG3, NullEvent-->MM_SND_MSG2, EV_SND_MSG-->MM_SND_MSG2, EV_START_TMR-->MM_SND_MSG2, EV_RESEND_MSG-->MM_WAIT_MSG3, EV_TIMEOUT-->MM_WAIT_MSG3, NullEvent
Mar 25 2019 18:51:21: %ASA-7-713906: IP = 1.1.1.1, IKE SA MM:5f3d6a94 terminating:  flags 0x01000002, refcnt 0, tuncnt 0
Mar 25 2019 18:51:21: %ASA-7-713906: IP = 1.1.1.1, sending delete/delete with reason message
Mar 25 2019 18:51:21: %ASA-6-713905: IP = 1.1.1.1, Warning: Ignoring IKE SA (src) without VM bit set

[/box]

Solution

Well, as you can tell from my Troubleshooting Phase 1 Cisco Site to Site (L2L) VPN Tunnels article MM_WAIT_MSG3 usually happens if something is blocking ISAKMP (UDP 500) in-between the peers. Or there’s a ‘bug’ that needs some newer or older code. The problem was not spotted by me, I was chatting to an ex colleague about it and he knew what it was straight away (annoyingly). The Router had an ACL on its outside interface that was NOT allowing the peer in to establish a VPN.

On the router take a look a the outside interface, and look for an access-list;

[box]

Remote-Router#show run interface GigabitEthernet 0/0
Building configuration...

Current configuration : 214 bytes
!
interface GigabitEthernet0/0
 bandwidth 200000
 ip address 1.1.1.1 255.255.255.248
 ip access-group outside-in in
 ip nat outside
 ip virtual-reassembly in
 duplex auto
 speed auto
 crypto map CMAP-1
end

Remote-Router#

[/box]

So it’s called outside-in what’s it doing?

[box]

Remote-Router#show access-list outside-in
Extended IP access list outside-in
    10 permit udp any eq domain any
    20 permit icmp any any echo-reply (956 matches)
    30 permit ip 123.123.123.64 0.0.0.15 any (127341882 matches)
    40 permit ip 123.123.123.128 0.0.0.127 any (572 matches)
    50 permit ip 222.222.222.96 0.0.0.15 any (4 matches)
    60 permit ip host 123.123.123.68 any
    70 permit udp host 222.222.222.76 eq ntp any
    80 permit udp host 222.222.222.204 eq ntp any
    90 permit udp host 222.222.222.232 eq ntp any
    100 permit icmp any any (1320 matches)
    110 permit ip 223.223.233.0 0.0.0.255 any

[/box]

Add in our peer IP address;

[box]

Remote-Router#configure terminal
Enter configuration commands, one per line.  End with CNTL/Z.
Remote-Router(config)#ip access-list extended outside-in
Remote-Router(config-ext-nacl)# permit esp host 2.2.2.2 any
Remote-Router(config-ext-nacl)# permit udp host 2.2.2.2 any eq isakmp
Remote-Router(config-ext-nacl)# permit udp host 2.2.2.2 any eq non500-isakmp
Remote-Router(config-ext-nacl)#exit
Remote-Router(config)#

[/box]

Let’s just make sure it’s there;

[box]

Remote-Router#show access-list outside-in
Extended IP access list outside-in
    10 permit udp any eq domain any
    20 permit icmp any any echo-reply (956 matches)
    30 permit ip 123.123.123.64 0.0.0.15 any (127341882 matches)
    40 permit ip 123.123.123.128 0.0.0.127 any (572 matches)
    50 permit ip 222.222.222.96 0.0.0.15 any (4 matches)
    60 permit ip host 123.123.123.68 any
    70 permit udp host 222.222.222.76 eq ntp any
    80 permit udp host 222.222.222.204 eq ntp any
    90 permit udp host 222.222.222.232 eq ntp any
    100 permit icmp any any (1320 matches)
    110 permit ip 223.223.233.0 0.0.0.255 any
    120 permit esp host 2.2.2.2 any
    130 permit udp host 2.2.2.2 any eq isakmp
    140 permit udp host 2.2.2.2 any eq non500-isakmp

[/box]

Don’t forget to save the changes with a ‘write mem’ command!

Cisco ASA: Allow VPN Traffic “Through” A Cisco Firewall

Posted on 06/05/2018 by PeteLong

KB ID 0001428

Problem

I got asked to put in a VPN for a client, this week, it went from a simple site to site, to a site to site with a Fortigate firewall at one end, to a VPN from and ASA to a Fortigate ‘through’ another ASA.

It’s been a few years since I had to tunnel ‘through’ a firewall, and experience tells me, if you don’t have control of BOTH ends of a new VPN tunnel, anything that stops the tunnel coming up, MUST be my fault. So I thought I’d build it on my test bench, (admittedly without a Fortigate, but thankfully that’s not my problem, so I used another ASA instead.) This is what I built.

Note: The routers are just there, so I can ping the other site to test the tunnel.

Solution

So what I’m worried about, is how to configure the ASA in the middle (The Corporate Perimeter Firewall). This is not an exercise in setting up the VPNs, if that’s what you require, then see the link at the bottom of the page.

There are FIVE things you need to do;

Create a Static (One-To-One) NAT so that the ASA that has a private IP on its outside interface, (192.168.2.1) has a PUBLIC IP mapped to it, (I’m using 1.1.1.3).
Allow UDP 500 (ISAKMP) from the ASA (1.1.1.1) to the ASA (192.168.2.1)
Allow UDP 4500 (NAT-TRAVERSAL) from the ASA (1.1.1.1) to the ASA (192.168.2.1)
Allow UDP 500 (ISAKMP) from the ASA (192.168.2.1) to the ASA (1.1.1.1)
Allow UDP 4500 (NAT-TRAVERSAL) from the ASA (192.168.2.1) to the ASA (1.1.1.1)

That’s It! You DON’T need to allow ESP or AH, (protocols 50 and 51,) nor do you need to enable ‘ipsec-pass-through’. I’ve brought up the tunnel with both IKEv1 and IKEv2.

Config;

[box]

!
object network OBJ-ASA-SITE-1
 host 1.1.1.1
object network OBJ-ASA-SITE-2
 host 192.168.2.1
 nat (inside,outside) static 1.1.1.3
!
access-list outbound extended permit udp object OBJ-ASA-SITE-2 object OBJ-ASA-SITE-1 eq isakmp
access-list outbound extended permit udp object OBJ-ASA-SITE-2 object OBJ-ASA-SITE-1 eq 4500
!
access-list inbound extended permit udp object OBJ-ASA-SITE-1 object OBJ-ASA-SITE-2 eq isakmp
access-list inbound extended permit udp object OBJ-ASA-SITE-1 object OBJ-ASA-SITE-2 eq 4500
!
access-group inbound in interface outside
access-group outbound in interface inside
!

[/box]

WARNING: Read the following article before executing the access-group commands;

Cisco ASA – ‘access-group’ Warning

Cisco ASA Site to Site VPN’sSite to Site ISAKMP VPN (Main Mode)

Posted on 17/11/2015 by Migrated

KB ID 0000213

Problem

As with most things, before you have a hope of fixing something, you will stand a better chance if you know how it works in the first place. Below is a quick run though of what’s happening with your site to site VPN‘s and how they work.

For the entire process we will have two Cisco ASA 5500 firewalls and a site to site VPN.

Solution

What’s an Initiator and a Responder?

1. Our Laptop 192.168.1.50 wants to talk to a server on the other site at 172.16.1.50

2. To get out of the local network the Laptop goes through the ASA at its local site, The ASA knows that traffic destined for 172.16.1.50 needs to be sent down the VPN tunnel, so it needs to bring up the tunnel. IT BECOMES THE INITIATOR, contacts the ASA on the other site THAT BECOMES THE RESPONDER.

3 Once that’s complete the tunnel is up and traffic can pass.

So how does it bring up the Tunnel?

To establish an ISAKMP VPN tunnel 3 things have to happen.

1. Phase 1 has to complete.

2. Phase 2 has to complete.

3. The Traffic has to be allowed to pass.

VPN Phase 1 (ISAKMP)

This stage brings up the first secure tunnel (eventually there will be three tunnels) and for it to establish the firewalls need to agree what they are going to do to bring up the tunnel, then Secure the tunnel. This process uses SIX MESSAGES (Note: We are dealing to Main Mode here not Aggressive mode). Both firewalls need a matching Phase 1 Policy to continue. And the Policy is proposed in MESSAGE1 and accepted in MESSAGE2.

A Phase 1 policy consists of,

1. The Authentication method (either a pre shared key or an RSA signature is usual).

2. The Encryption method (DES, 3DES, AES, AES-192, or AES-256).

3. The Hashing Method (MD5 or SHA).

4. The Diffie Helman Group (1, 2 or 5 usually).

5. Lifetime (In seconds before phase 1 should be re-established – usually 86400 seconds [1 day]).

MESSAGE 1

The Initiator sends policies that it proposes to use, for phase 1 to the other ASA.

MESSAGE 2

Providing the responder has a matching policy it will accept one of those proposed by the initiator and send it back in message 2.

Now the two ends have agreed HOW they will establish phase 1, they then need to agree on a “Shared Key” both ends must use the same shared key, but the shared key cant be sent between them because the network link is not secure. To do this they use a Diffie Hellman key exchange, this uses a mathematical process called modular exponentiation, a simple example of how that works (The math’s involved in a real key exchange are much more complicated!).

How Diffie Hellman works (simply)

Problem Site A and Site B need to use the same secret key (which will be a big long number). they cant send that number to each other because if they do it will be seen.

Solution:

Both sites pick a random number, and they have a common number, this common number can be passed between sites, In our example Site A chooses 4 and Site B chooses 5

Both sites use the common number and raise it by the power of the random number they are using so Site A arrives at 16, and Site B at 32.

The sites then send the number they have arrived at, to the other site.

Each site uses the other sites total and raises it to the power of their original random number, this results in them both having the same key, with only the numbers 2, 16 and 32 being passed between them.

Back to our VPN Tunnel

The next two messages are the initiator and responder swapping their Diffie Hellman information, Each side produces a DH Public Key, and mathematically computes a long number called a “Nonce”

MESSAGE 3

The initiator generates a “Public Key” also called the DH Public Value or Xa It also generates a Nonce or Ni and sends both of them to the responder.

MESSAGE 4

The responder generates a “Public Key” also called the DH Public Value or Xb It also generates a Nonce or Nr and sends both of them to the initiator.

At this point both the initiator and the responder can calculate the DH Shared secret key, they then use the DH Secret Key, the “Shared Secret” that is manually entered onto both peers, and the Nonce from the other peer to create 3 DIGITAL KEYS, because of the nature of Diffie Hellman each end will produce the same keys.

Key 1 = SKEYID_d Used to work out any future IPsec keying Key 2 = SKEYID_a Used for data integrity and authentication (IKE) Key 3 = SKEYID_e Used to encrypt all further IKE traffic.

MESSAGE 5

The initiator now sends its ID to the responder (this is either its IP address or a hostname). It also sends a “Hash” this authenticates the initiator to the responder as its made from the SKEYID, the pre-shared key and other information only known to the two peers.

MESSAGE 6

Message 6 is basically the mirror of Message 5, the responder sends its ID (IP or Hostname) Back the the initiator with its “Hash” and authenticates itself back to the initiator.

At this point both peers recalculate the hash they have received from the other peer, and they should both come out the same, if this happens then the IKE SA’s are established and phase 1 is complete.

So what’s PFS?

Perfect Forward Secrecy is a method by which new keys are generated, each new key is mathematically linked to the key that came before it, the prior key being a “Grandfather” key. With PFS enabled this link is broken so a key can not be forward/reverse engineered to guess a previous/new key value). Every new negotiation produces a new fresh key.

VPN Phase 2

Once Phase 1 has completed the second stage of the VPN can start. Like phase 1 this state also requires messages to be sent between the peers, IPsec usually executes in “Quick mode” this means that there are only 3 MESSAGES.

Note: If PFS is configured only on one end then it will fail at this point with an “Attribute not supported” error.

MESSAGE 1

The Initiator sends another Hash to the responder, this is similar to the one used in phase 1 but also includes info within this message to guarantee integrity.

The Phase 2 proposal includes

1. Encapsulation method either ESP or AH.

2. Hashing method (Integrity checking) either SHA-HMAC or MD5-HMAC.

3. Diffie Hellman Group (1, 2, or 5).

4. The SPI – This number is the LABEL for the end of the tunnel the initiator will use for outbound traffic.

Tunnel mode (Tunnel or Transport). A timeout in seconds is specified, as is the ID (usually the subnet of both ends of the tunnel).

MESSAGE 2

The Responder replies with its own “Hash” with the accepted proposal and its own SPI for outgoing encrypted traffic from the responder, and finally its own Key Exchange Payload.

Once this is complete both peers generate new DH secret keys and combine them with the SKEYID_d key from phase 1 to create keys for IPsec encryption.

MESSAGE 3

The final Message is sent from imitator to responder, and serves to inform the responder that its previous message was received.

Once phase 2 is complete IPsec SA’s have been established and the tunnel is up.

PeteNetLive

Technology that 'Just Works'

Tag Archives: esp

Cisco ASA VPN to Cisco Router “MM_WAIT_MSG3”

Problem

Solution

Related Articles, References, Credits, or External Links

Cisco ASA: Allow VPN Traffic “Through” A Cisco Firewall

Problem

Solution

Config;

Related Articles, References, Credits, or External Links