Cisco ASA Site to Site VPN’sSite to Site ISAKMP VPN (Main Mode)


KB ID 0000213 Dtd 28/03/13


As with most things, before you have a hope of fixing something, you will stand a better chance if you know how it works in the first place. Below is a quick run though of what’s happening with your site to site VPN‘s and how they work.

site to site vpn

For the entire process we will have two Cisco ASA 5500 firewalls and a site to site VPN.


What’s an Initiator and a Responder?

1. Our Laptop wants to talk to a server on the other site at

vpn initiator

2. To get out of the local network the Laptop goes through the ASA at its local site, The ASA knows that traffic destined for needs to be sent down the VPN tunnel, so it needs to bring up the tunnel. IT BECOMES THE INITIATOR, contacts the ASA on the other site THAT BECOMES THE RESPONDER.

vpn responder

3 Once that’s complete the tunnel is up and traffic can pass.

vpn up

So how does it bring up the Tunnel?

To establish an ISAKMP VPN tunnel 3 things have to happen.

1. Phase 1 has to complete.

2. Phase 2 has to complete.

3. The Traffic has to be allowed to pass.

VPN Phase 1 (ISAKMP)

This stage brings up the first secure tunnel (eventually there will be three tunnels) and for it to establish the firewalls need to agree what they are going to do to bring up the tunnel, then Secure the tunnel. This process uses SIX MESSAGES (Note: We are dealing to Main Mode here not Aggressive mode). Both firewalls need a matching Phase 1 Policy to continue. And the Policy is proposed in MESSAGE1 and accepted in MESSAGE2.

A Phase 1 policy consists of,

1. The Authentication method (either a pre shared key or an RSA signature is usual).

2. The Encryption method (DES, 3DES, AES, AES-192, or AES-256).

3. The Hashing Method (MD5 or SHA).

4. The Diffie Helman Group (1, 2 or 5 usually).

5. Lifetime (In seconds before phase 1 should be re-established – usually 86400 seconds [1 day]).


The Initiator sends policies that it proposes to use, for phase 1 to the other ASA.

ike message 1


Providing the responder has a matching policy it will accept one of those proposed by the initiator and send it back in message 2.

ike message 2


Now the two ends have agreed HOW they will establish phase 1, they then need to agree on a “Shared Key” both ends must use the same shared key, but the shared key cant be sent between them because the network link is not secure. To do this they use a Diffie Hellman key exchange, this uses a mathematical process called modular exponentiation, a simple example of how that works (The math’s involved in a real key exchange are much more complicated!).

How Diffie Hellman works (simply)

Problem Site A and Site B need to use the same secret key (which will be a big long number). they cant send that number to each other because if they do it will be seen.


How Diffie Hellman work

Both sites pick a random number, and they have a common number, this common number can be passed between sites, In our example Site A chooses 4 and Site B chooses 5

How Diffie Hellman works simply

Both sites use the common number and raise it by the power of the random number they are using so Site A arrives at 16, and Site B at 32.

Diffie Hellman explained

The sites then send the number they have arrived at, to the other site.

How Diffie Hellman work

Each site uses the other sites total and raises it to the power of their original random number, this results in them both having the same key, with only the numbers 2, 16 and 32 being passed between them.

Back to our VPN Tunnel

The next two messages are the initiator and responder swapping their Diffie Hellman information, Each side produces a DH Public Key, and mathematically computes a long number called a “Nonce”


The initiator generates a “Public Key” also called the DH Public Value or Xa It also generates a Nonce or Ni and sends both of them to the responder.

ike message 3


The responder generates a “Public Key” also called the DH Public Value or Xb It also generates a Nonce or Nr and sends both of them to the initiator.

ike message 4

At this point both the initiator and the responder can calculate the DH Shared secret key, they then use the DH Secret Key, the “Shared Secret” that is manually entered onto both peers, and the Nonce from the other peer to create 3 DIGITAL KEYS, because of the nature of Diffie Hellman each end will produce the same keys.

Key 1 = SKEYID_d Used to work out any future IPsec keying Key 2 = SKEYID_a Used for data integrity and authentication (IKE) Key 3 = SKEYID_e Used to encrypt all further IKE traffic.


The initiator now sends its ID to the responder (this is either its IP address or a hostname). It also sends a “Hash” this authenticates the initiator to the responder as its made from the SKEYID, the pre-shared key and other information only known to the two peers.

ike message 5


Message 6 is basically the mirror of Message 5, the responder sends its ID (IP or Hostname) Back the the initiator with its “Hash” and authenticates itself back to the initiator.

ike message 6

At this point both peers recalculate the hash they have received from the other peer, and they should both come out the same, if this happens then the IKE SA’s are established and phase 1 is complete.

So what’s PFS?

Perfect Forward Secrecy is a method by which new keys are generated, each new key is mathematically linked to the key that came before it, the prior key being a “Grandfather” key. With PFS enabled this link is broken so a key can not be forward/reverse engineered to guess a previous/new key value). Every new negotiation produces a new fresh key.

VPN Phase 2

Once Phase 1 has completed the second stage of the VPN can start. Like phase 1 this state also requires messages to be sent between the peers, IPsec usually executes in “Quick mode” this means that there are only 3 MESSAGES.

Note: If PFS is configured only on one end then it will fail at this point with an “Attribute not supported” error.


The Initiator sends another Hash to the responder, this is similar to the one used in phase 1 but also includes info within this message to guarantee integrity.

The Phase 2 proposal includes

1. Encapsulation method either ESP or AH.

2. Hashing method (Integrity checking) either SHA-HMAC or MD5-HMAC.

3. Diffie Hellman Group (1, 2, or 5).

4. The SPI – This number is the LABEL for the end of the tunnel the initiator will use for outbound traffic.

Tunnel mode (Tunnel or Transport). A timeout in seconds is specified, as is the ID (usually the subnet of both ends of the tunnel).

ipsec phase 1


The Responder replies with its own “Hash” with the accepted proposal and its own SPI for outgoing encrypted traffic from the responder, and finally its own Key Exchange Payload.

ipsec phase 2

Once this is complete both peers generate new DH secret keys and combine them with the SKEYID_d key from phase 1 to create keys for IPsec encryption.


The final Message is sent from imitator to responder, and serves to inform the responder that its previous message was received.

IPSEC message 3

Once phase 2 is complete IPsec SA’s have been established and the tunnel is up.


Related Articles, References, Credits, or External Links


Author: Migrated

Share This Post On