Cisco ASA VPN to Cisco Router “MM_WAIT_MSG3”

KB ID 0001531

Problem

While migrating a VPN tunnel from an ASA 5520 firewall to a new 5516-X I got this problem. The other end was a Cisco router (2900). As soon as I swapped it over, it was stuck at MM_WAIT_MSG3, and phase 1 would not establish;

[box]

NUFC-ASA5516x(config-tunnel-ipsec)# show crypto isa

IKEv1 SAs:

   Active SA: 6
    Rekey SA: 0 (A tunnel will report 1 Active and 1 Rekey SA during rekey)
Total IKE SA: 6

1   IKE Peer: 1.1.1.1
    Type    : user            Role    : responder
    Rekey   : no              State   : MM_WAIT_MSG3

[/box]

Debugs didn’t help much either;

[box]

Mar 25 2019 18:50:49: %ASA-7-713236: IP = 1.1.1.1, IKE_DECODE RECEIVED Message (msgid=0) with payloads : HDR + SA (1) + VENDOR (13) + VENDOR (13) + VENDOR (13) + VENDOR (13) + NONE (0) total length : 164
Mar 25 2019 18:50:49: %ASA-7-715047: IP = 1.1.1.1, processing SA payload
Mar 25 2019 18:50:49: %ASA-7-713906: IP = 1.1.1.1, Oakley proposal is acceptable
Mar 25 2019 18:50:49: %ASA-7-715047: IP = 1.1.1.1, processing VID payload
Mar 25 2019 18:50:49: %ASA-7-715049: IP = 1.1.1.1, Received NAT-Traversal RFC VID
Mar 25 2019 18:50:49: %ASA-7-715047: IP = 1.1.1.1, processing VID payload
Mar 25 2019 18:50:49: %ASA-7-715047: IP = 1.1.1.1, processing VID payload
Mar 25 2019 18:50:49: %ASA-7-715049: IP = 1.1.1.1, Received NAT-Traversal ver 03 VID
Mar 25 2019 18:50:49: %ASA-7-715047: IP = 1.1.1.1, processing VID payload
Mar 25 2019 18:50:49: %ASA-7-715049: IP = 1.1.1.1, Received NAT-Traversal ver 02 VID
Mar 25 2019 18:50:49: %ASA-7-715047: IP = 1.1.1.1, processing IKE SA payload
Mar 25 2019 18:50:49: %ASA-7-715028: IP = 1.1.1.1, IKE SA Proposal # 1, Transform # 1 acceptable  Matches global IKE entry # 8
Mar 25 2019 18:50:49: %ASA-7-715046: IP = 1.1.1.1, constructing ISAKMP SA payload
Mar 25 2019 18:50:49: %ASA-7-715046: IP = 1.1.1.1, constructing NAT-Traversal VID ver RFC payload
Mar 25 2019 18:50:49: %ASA-7-715046: IP = 1.1.1.1, constructing Fragmentation VID + extended capabilities payload
Mar 25 2019 18:50:49: %ASA-7-713236: IP = 1.1.1.1, IKE_DECODE SENDING Message (msgid=0) with payloads : HDR + SA (1) + VENDOR (13) + VENDOR (13) + NONE (0) total length : 128
Mar 25 2019 18:50:57: %ASA-7-713236: IP = 1.1.1.1, IKE_DECODE RESENDING Message (msgid=0) with payloads : HDR + SA (1) + VENDOR (13) + VENDOR (13) + NONE (0) total length : 128
Mar 25 2019 18:50:59: %ASA-7-713906: IKE Receiver: Packet received on 2.2.2.2:500 from 1.1.1.1:500
Mar 25 2019 18:50:59: %ASA-5-713202: IP = 1.1.1.1, Duplicate first packet detected.  Ignoring packet.
Mar 25 2019 18:51:05: %ASA-7-713236: IP = 1.1.1.1, IKE_DECODE RESENDING Message (msgid=0) with payloads : HDR + SA (1) + VENDOR (13) + VENDOR (13) + NONE (0) total length : 128
Mar 25 2019 18:51:21: %ASA-7-715065: IP = 1.1.1.1, IKE MM Responder FSM error history (struct &0x00007f4d2d293690)  , :  MM_DONE, EV_ERROR-->MM_WAIT_MSG3, EV_TIMEOUT-->MM_WAIT_MSG3, NullEvent-->MM_SND_MSG2, EV_SND_MSG-->MM_SND_MSG2, EV_START_TMR-->MM_SND_MSG2, EV_RESEND_MSG-->MM_WAIT_MSG3, EV_TIMEOUT-->MM_WAIT_MSG3, NullEvent
Mar 25 2019 18:51:21: %ASA-7-713906: IP = 1.1.1.1, IKE SA MM:5f3d6a94 terminating:  flags 0x01000002, refcnt 0, tuncnt 0
Mar 25 2019 18:51:21: %ASA-7-713906: IP = 1.1.1.1, sending delete/delete with reason message
Mar 25 2019 18:51:21: %ASA-6-713905: IP = 1.1.1.1, Warning: Ignoring IKE SA (src) without VM bit set

[/box]

Solution

Well, as you can tell from my Troubleshooting Phase 1 Cisco Site to Site (L2L) VPN Tunnels article MM_WAIT_MSG3 usually happens if something is blocking ISAKMP (UDP 500) in-between the peers. Or there’s a ‘bug’ that needs some newer or older code. The problem was not spotted by me, I was chatting to an ex colleague about it and he knew what it was straight away (annoyingly). The Router had an ACL on its outside interface that was NOT allowing the peer in to establish a VPN.

On the router take a look a the outside interface, and look for an access-list;

[box]

Remote-Router#show run interface GigabitEthernet 0/0
Building configuration...

Current configuration : 214 bytes
!
interface GigabitEthernet0/0
 bandwidth 200000
 ip address 1.1.1.1 255.255.255.248
 ip access-group outside-in in
 ip nat outside
 ip virtual-reassembly in
 duplex auto
 speed auto
 crypto map CMAP-1
end

Remote-Router#

[/box]

So it’s called outside-in what’s it doing?

[box]

Remote-Router#show access-list outside-in
Extended IP access list outside-in
    10 permit udp any eq domain any
    20 permit icmp any any echo-reply (956 matches)
    30 permit ip 123.123.123.64 0.0.0.15 any (127341882 matches)
    40 permit ip 123.123.123.128 0.0.0.127 any (572 matches)
    50 permit ip 222.222.222.96 0.0.0.15 any (4 matches)
    60 permit ip host 123.123.123.68 any
    70 permit udp host 222.222.222.76 eq ntp any
    80 permit udp host 222.222.222.204 eq ntp any
    90 permit udp host 222.222.222.232 eq ntp any
    100 permit icmp any any (1320 matches)
    110 permit ip 223.223.233.0 0.0.0.255 any

[/box]

Add in our peer IP address;

[box]

Remote-Router#configure terminal
Enter configuration commands, one per line.  End with CNTL/Z.
Remote-Router(config)#ip access-list extended outside-in
Remote-Router(config-ext-nacl)# permit esp host 2.2.2.2 any
Remote-Router(config-ext-nacl)# permit udp host 2.2.2.2 any eq isakmp
Remote-Router(config-ext-nacl)# permit udp host 2.2.2.2 any eq non500-isakmp
Remote-Router(config-ext-nacl)#exit
Remote-Router(config)# 

[/box]

Let’s just make sure it’s there;

[box]

Remote-Router#show access-list outside-in
Extended IP access list outside-in
    10 permit udp any eq domain any
    20 permit icmp any any echo-reply (956 matches)
    30 permit ip 123.123.123.64 0.0.0.15 any (127341882 matches)
    40 permit ip 123.123.123.128 0.0.0.127 any (572 matches)
    50 permit ip 222.222.222.96 0.0.0.15 any (4 matches)
    60 permit ip host 123.123.123.68 any
    70 permit udp host 222.222.222.76 eq ntp any
    80 permit udp host 222.222.222.204 eq ntp any
    90 permit udp host 222.222.222.232 eq ntp any
    100 permit icmp any any (1320 matches)
    110 permit ip 223.223.233.0 0.0.0.255 any
    120 permit esp host 2.2.2.2 any
    130 permit udp host 2.2.2.2 any eq isakmp
    140 permit udp host 2.2.2.2 any eq non500-isakmp

[/box]

Don’t forget to save the changes with a ‘write mem’ command!

Related Articles, References, Credits, or External Links

Once again, thanks to SteveH for spotting, (in less than sixty seconds) what was wrong.

Cisco ASA: Allow VPN Traffic “Through” A Cisco Firewall

KB ID 0001428

Problem

I got asked to put in a VPN for a client, this week, it went from a simple site to site, to a site to site with a Fortigate firewall at one end, to a VPN from and ASA to a Fortigate ‘through’ another ASA. 

It’s been a few years since I had to tunnel  ‘through’ a firewall, and experience tells me, if you don’t have control of BOTH ends of a new VPN tunnel, anything that stops the tunnel coming up, MUST be my fault. So I thought I’d build it on my test bench, (admittedly without a Fortigate, but thankfully that’s not my problem, so I used another ASA instead.) This is what I built.

Note: The routers are just there, so I can ping the other site to test the tunnel.

Solution

So what I’m worried about, is how to configure the ASA in the middle (The Corporate Perimeter Firewall). This is not an exercise in setting up the VPNs, if that’s what you require, then see the link at the bottom of the page.

There are FIVE things you need to do;

  • Create a Static (One-To-One) NAT so that the ASA that has a private IP on its outside interface, (192.168.2.1) has a PUBLIC IP mapped to it, (I’m using 1.1.1.3).
  • Allow UDP 500 (ISAKMP) from the ASA (1.1.1.1) to the ASA (192.168.2.1)
  • Allow UDP 4500 (NAT-TRAVERSAL) from the ASA (1.1.1.1) to the ASA (192.168.2.1)
  • Allow UDP 500 (ISAKMP) from the ASA (192.168.2.1) to the ASA (1.1.1.1)
  • Allow UDP 4500 (NAT-TRAVERSAL) from the ASA (192.168.2.1) to the ASA (1.1.1.1)

That’s It! You DON’T need to allow ESP or AH, (protocols 50 and 51,) nor do you need to enable ‘ipsec-pass-through’. I’ve brought up the tunnel with both IKEv1 and IKEv2.

Config;

[box]

!
object network OBJ-ASA-SITE-1
 host 1.1.1.1
object network OBJ-ASA-SITE-2
 host 192.168.2.1
 nat (inside,outside) static 1.1.1.3
!
access-list outbound extended permit udp object OBJ-ASA-SITE-2 object OBJ-ASA-SITE-1 eq isakmp
access-list outbound extended permit udp object OBJ-ASA-SITE-2 object OBJ-ASA-SITE-1 eq 4500
!
access-list inbound extended permit udp object OBJ-ASA-SITE-1 object OBJ-ASA-SITE-2 eq isakmp
access-list inbound extended permit udp object OBJ-ASA-SITE-1 object OBJ-ASA-SITE-2 eq 4500
!
access-group inbound in interface outside
access-group outbound in interface inside
!

[/box]

WARNING: Read the following article before executing the access-group commands;

Cisco ASA – ‘access-group’ Warning

 

Related Articles, References, Credits, or External Links

Cisco ASA Site To Site VPN IKEv2 “Using CLI”

Cisco ASA5500 Site to Site VPN from ASDM

Add a Static (One to One) NAT Translation to a Cisco ASA 5500 Firewall