When Email was a new medium, pretty much all mail servers were open relays, and nobody really cared, after all if someone in Nigeria wanted to relay mail through a college server in Manchester then why shouldn’t they? Sadly with the explosion of Email and internet use this is no longer an option, because all those annoying Emails you get for Viagra or insider stock tips have probably been sent to you through an open relay. That’s to say someone like you with a mail server that’s happily sending out someone else’s mail.
Now you may be the sort of person who does not care, if that’s true then “be warned” Major ISP’s and Email handlers are routinely blocking “Suspect” mail servers, you might not even know you had a problem, and the first thing you know about it is,you cant send mail to a particular domain because you are on a blacklist, or worse your ISP cuts off your internet access, or you get an early knock on the door from some nice men in suits wanting to know why 400Gb of “dodgy photos” was sent from your IP address this morning.
Solution
Test your Mail Server with Telnet
1. OK – the first thing we need to do is get on a PC that isn’t logged into the domain (Exchange can be set to relay mail from a client that’s authenticated to the domain regardless) – So take that out of the equation by not being in the domain. Now open a good old fashioned command window . Click Start > Run > CMD {enter} Alert! – To put “Run” on your start menu > Right click your task bar > Properties > Start Menu Tab > Customise > Tick “Run Command” > OK > OK.
2. You now need to connect to your mail server using a telnet command, to do this you need to know either the name or the IP address of the server. the command is,
telnet <IP Address or Server Name> 25 {enter}
3. The server should respond with a “Banner” this lets you know you connects successfully NOTE some anti virus programs block this (McAfee for example) you need to go to its access protection settings and untick “Prevent mass mailing worms from sending mail”. Don’t forget to turn it back on again later 🙂
4. You are going to send an Email from command line, the first thing you need to do is say hello to the server, though being an exchange server the command is,
ehlo {enter}
What we want to see are 250 messages in our example we got,
250-server1.petenetlive.com Hello [10.254.254.60]
5. Type the following,
mail from:test@test.com{enter}
again we want to be seeing a 250 message, if you didn’t get one you made a spelling mistake start again 🙁
In our example we got,
250 2.1.0 test@test.com….Sender OK
6. Now we are going to attempt to relay mail for a different domain this will tell us if the server is an open relay or not. Type the following,
rcpt to:badperson@nastyspammer.com{enter}
Note if the Server gives you a message like,
550 5.7.1 Unable to relay for badperson@nastyspammer.com THIS MEANS YOU ARE NOT AN OPEN RELAY.
NOTE if the server responds with
250 2.1.5 badperson@nastyspammer.com
Then either you ignored me in step 1 and your in the domain – or YOU ARE AN OPEN RELAY.
Officially there are three things, that if set wrong can leave you as an open relay,
1. Your Default SMTP Virtual Server.
2. Your SMTP Connector.
3. You have ISA server installed and set incorrectly.
Step 1 – Check the SMTP Virtual Server
1. On the Exchange Server Click Start > All Programs > Microsoft Exchange > System Manager.
2. Expand Administrative Groups > First Administrative group > Servers > {your server name} > Protocols > SMTP > Right Click “Default SMTP Virtual Server” > Properties.
3. On the properties window select the Access tab > Click the “Relay” Button.
4. On the “Relay Restrictions” window Check that, “Only the list below” is selected > It’s not unusual (in fact its the default) that the window is empty, you may see the Exchange server IP addresses in here – or in some cases other hosts on your network that have been set up to relay mail – (Backup software that emails you, or SQL servers that email events for example) > And ensure the box at the bottom that says “Allow all computers that successfully authenticate to relay, regardless of the list above” IS TICKED.
Step 2 – Check the SMTP Connector
NOTE: You might not have an Exchange connector don’t panic if its not there 🙂
1. On the exchange Server Click Start > All Programs > Microsoft Exchange > System Manager.
2. Expand Administrative Groups > First Administrative group > Routing Groups > First Routing Group > Connectors. > {your connector name} NOTE – you may have many different routing groups and the path in your exchange system manger might be under Servers > {your server name} > Connectors.
3. Right click your connector and select Properties.
4 Select the “Address Space” Tab > You should see the following > Address Type = SMTP > Address = * > Cost = 1 > Connector Scope = “Entire Organisation” > “Allow Messages to be relayed to these domains” IS NOT TICKED.
Step 3 – Check ISA
ISA Server 2000 – had a problem where if you had a mail publishing rule for SMTP it set an open relay -check! Also make sure make sure 127.0.0.1 is NOT in the list of IP addresses that are allowed to relay in the the properties section of the default SMTP Virtual server.
Related Articles, References, Credits, or External Links
Site to Site VPN’s either work faultlessly straight away, or involve head scratching and a call to Cisco TAC, or someone like me to come and take a look. If I’m honest, the simplest and best answer to the problem is “Remove the Tunnel from both ends and put it back again”. Just about every VPN tunnel I’ve put in that did not work, was a result of my fat fingers putting in the wrong subnet, IP address or shared secret.
However you can’t always remove the tunnel and start again, especially if you only have control of your end of the tunnel. In that case you need to do some troubleshooting and debugging.
Solution
To Troubleshoot and debug a VPN tunnel you need to have an appreciation of how VPN Tunnels work READ THIS.
Now you have read that you are an expert on IKE VPN Tunnels 🙂
Step 1
To bring up a VPN tunnel you need to generate some “Interesting Traffic” Start by attempting to send some traffic over the VPN tunnel.
User Access Verification
Password:
Type help or '?' for a list of available commands.
PetesASA> enable
Password: ********
PetesASA# show crypto isakmp
[/box]
You may see a lot more information if you have Existing VPN tunnels, but what you are looking for is this,
[box]
IKEv1 SAs:
Active SA: 2
Rekey SA: 0 (A tunnel will report 1 Active and 1 Rekey SA during rekey)
Total IKE SA: 2
1 IKE Peer: 123.123.123.123
Type : L2L Role : initiator
Rekey : no State : MM_ACTIVE <<YOUR SIDE BROUGHT THE VPN UP
There are no IKEv2 SAs
[/box]
If you see MM_ACTIVE (This means phase 1 has completed in Main Mode, and is active) So phase 1 has completed successfully, you need to jump forward and troubleshoot Phase 2.
Note: If you see AG_{something} this means you are trying to bring the tunnel up in aggressive mode!
If there is nothing listed at all – then your side is not even trying to bring up the tunnel. Try and generate a lot of VPN traffic – Like a persistent ping {ping 192.168.1.1 -t} and issue the show crypto isakmp command a few times to be sure. if you never see anything then its not getting as far as phase 1!
If your still reading this, then your problem is with Phase 1, and you have an ISAKMPSA state error.
ISAKMP SA MESSAGE STATES (On the Initiator)
MM_WAIT_MSG2
Message 1 has been sent to the responder but there has been no reply.
Causes:
1. There is no network connectivity to the firewallsecurity device at the other end, can you ping it?
2. The IP address of the far firewall is incorrect in the tunnel-group, issue a “show run tunnel-group” command, check you have a tunnel group with the correct IP address.
3. The IP address in the “Crypto Map” is incorrect, issue a “show run crypto map” command and check the line that ends “crypto map {name} {number} set peer xxx.xxx.xxx.xxx” to make sure.
4. You do not have a matching phase 1 policy with the other end, issue a “show run crypto isakmp” command make sure the other end has a matching policy, if you cant check the other end then generate some VPN traffic, issue the following command and check for the following,
[box]
EXAMPLE PHASE 1 POLICIES DONT MATCH
Password: Type help or ‘?’ for a list of available commands. PetesASA> en Password: ******** PetesASA#debug crypto isakmp 200
<<<<<<<LOTS Of DEBUG TEXT REMOVED>>>>>>>
Apr 01 14:48:48 [IKEv1]: IP = 123.123.123.123, IKE_DECODE RECEIVED Message (msgid=ce4a3ffe) with payloads : HDR + NOTIFY (11) + NONE (0) total length : 56 Apr 01 14:48:48 [IKEv1]: IP = 123.123.123.123, Received an un-encrypted NO_PROPOSAL_CHOSEN notify message, dropping Apr 01 14:48:48 [IKEv1]: IP = 123.123.123.123, Information Exchange processing failed
<<<<<<<LOTS Of DEBUG TEXT REMOVED>>>>>>>
[/box]
MM_WAIT_MSG4
The Phase 1 Policies have been agreed with both peers, the initiator is waiting for the responder to send it its keying information. I’ve seen two things cause this.
1. Different Vendors equipment talking the the ASA, or simply the version of OS on the ASA have been different.
2. There is a comms error, check there’s no router with firewall capabilities in the link.
MM_WAIT_MSG6
If there’s a firewall ‘in-between’ make sure UDP port 4500 is open for both peers.
Check your Pre-Shared Keys match on the ASA issue a “more system:running-config” then keep pressing the space bar till you see the tunnel- group and shared key
Again if you can’t check the other end then issue the following debug and the following will tell you if there is a key mismatch.
This error can also be seen if one end has PFS set and the other end does not. In this case the error will appear and dissapear and the connection is repeatedly “torn down”
e.g
crypto map outside_map 20 set pfs [box]
EXAMPLE PHASE 1 PRE SHARED KEYS DONT MATCH
Password: Type help or ‘?’ for a list of available commands. PetesASA> en Password: ******** PetesASA#debug crypto isakmp 200
<<<<<<<LOTS Of DEBUG TEXT REMOVED>>>>>>>
Apr 01 15:11:47 [IKEv1]: IP = 123.123.123.123, IKE_DECODE RECEIVED Message (msgid=5456d64e) with payloads : HDR + NOTIFY (11) + NONE (0) total length : 56 Apr 01 15:11:47 [IKEv1]: Group = 123.123.123.123, IP = 123.123.123.123, Received an un-encrypted PAYLOAD_MALFORMED notify message, dropping Apr 01 15:11:47 [IKEv1]: Group = 123.123.123.123, IP = 123.123.123.123, Error, peer has indicated that something is wrong with our message. This could indicate a pre-shared key mismatch. Apr 01 15:11:47 [IKEv1]: Group = 123.123.123.123, IP = 123.123.123.123, Information Exchange processing failed
<<<<<<<LOTS Of DEBUG TEXT REMOVED>>>>>>>
[/box]
ISAKMP SA MESSAGE STATES (On the Responder)
MM_WAIT_MSG3
The Phase 1 Policies have been agreed with both peers, the responder is waiting for the initiator to send it its keying information. I’ve seen two things cause this.
1. Different Vendors equipment talking the the ASA, or simply the version of OS on the ASA have been different.
2. There is a comms error, check there’s no router with firewall capabilities in the link.
3. I’ve seen this on a VPN from a VMware Edge Gateway, that had PFS (perfect forward secrecy) enabled, and the ASA did not.
If there’s a firewall ‘in-between’ make sure UDP port 4500 is open for both peers.
Here’s an Example of Phase one completing message by message successfully.
MESSAGE 1 (Leaving the Initiator)
[box]
Apr 01 11:38:51 [IKEv1]: IP = 123.123.123.123, IKE Initiator: New Phase 1, Intf inside, IKE Peer 123.123.123.123 local Proxy Address 192.168.1.0, remote Proxy Address 172.16.1.0, Crypto map (outside_map) Apr 01 11:38:51 [IKEv1 DEBUG]: IP = 123.123.123.123, constructing ISAKMP SA payload Apr 01 11:38:51 [IKEv1 DEBUG]: IP = 123.123.123.123, constructing NAT-Traversal VID ver 02 payload Apr 01 11:38:51 [IKEv1 DEBUG]: IP = 123.123.123.123, constructing NAT-Traversal VID ver 03 payload Apr 01 11:38:51 [IKEv1 DEBUG]: IP = 123.123.123.123, constructing NAT-Traversal VID ver RFC payload Apr 01 11:38:51 [IKEv1 DEBUG]: IP = 123.123.123.123, constructing Fragmentation VID + extended capabilities payload Apr 01 11:38:51 [IKEv1]: IP = 123.123.123.123, IKE_DECODE SENDING Message (msgid=0) with payloads : HDR + SA (1) + VENDOR (13) + VENDOR (13) + VENDOR (13) + VENDOR (13) + NONE (0) total length : 168
[/box]
MESSAGE 2 (Arriving at the Responder)
[box]
Apr 01 11:38:52 [IKEv1]: IP = 123.123.123.123, IKE_DECODE RECEIVED Message (msgid=0) with payloads : HDR + SA (1) + VENDOR (13) + NONE (0) total length : 117
Apr 01 11:38:52 [IKEv1 DEBUG]: IP = 123.123.123.123, processing SA payload
Apr 01 11:38:52 [IKEv1 DEBUG]: IP = 123.123.123.123, Oakley proposal is acceptable
Apr 01 11:38:52 [IKEv1 DEBUG]: IP = 123.123.123.123, processing VID payload
Apr 01 11:38:52 [IKEv1 DEBUG]: IP = 123.123.123.123, constructing ke payload
Apr 01 11:38:52 [IKEv1 DEBUG]: IP = 123.123.123.123, constructing nonce payload
Apr 01 11:38:52 [IKEv1 DEBUG]: IP = 123.123.123.123, constructing Cisco Unity VID payload
Apr 01 11:38:52 [IKEv1 DEBUG]: IP = 123.123.123.123, constructing xauth V6 VID payload
Apr 01 11:38:52 [IKEv1 DEBUG]: IP = 123.123.123.123, Send IOS VID
Apr 01 11:38:52 [IKEv1 DEBUG]: IP = 123.123.123.123, Constructing ASA spoofing IOS Vendor ID payload (version: 1.0.0, capabilities: 20000001)
Apr 01 11:38:52 [IKEv1 DEBUG]: IP = 123.123.123.123, constructing VID payload
Apr 01 11:38:52 [IKEv1 DEBUG]: IP = 123.123.123.123, Send Altiga/Cisco VPN3000/Cisco ASA GW VID
[/box]
MESSAGE 3 (Leaving the Initiator)
[box]
Apr 01 11:38:52 [IKEv1]: IP = 123.123.123.123, IKE_DECODE SENDING Message (msgid=0) with payloads : HDR + KE (4) + NONCE (10) + VENDOR (13) + VENDOR (13) + VENDOR (13) + VENDOR (13) + NONE (0) total length : 256
[/box]
MESSAGE 4 (Arriving at the Initiator)
[box]
Apr 01 11:38:52 [IKEv1]: IP = 123.123.123.123, IKE_DECODE RECEIVED Message (msgid=0) with payloads : HDR + KE (4) + NONCE (10) + NONE (0) total length : 228
Apr 01 11:38:52 [IKEv1 DEBUG]: IP = 123.123.123.123, processing ke payload
Apr 01 11:38:52 [IKEv1 DEBUG]: IP = 123.123.123.123, processing ISA_KE payload
Apr 01 11:38:52 [IKEv1 DEBUG]: IP = 123.123.123.123, processing nonce payload
[/box]
MESSAGE 5 (Leaving the Initiator)
[box]
Apr 01 11:38:52 [IKEv1]: IP = 123.123.123.123, Connection landed on tunnel_group 123.123.123.123
Apr 01 11:38:52 [IKEv1 DEBUG]: Group = 123.123.123.123, IP = 123.123.123.123, Generating keys for Initiator...
Apr 01 11:38:52 [IKEv1 DEBUG]: Group = 123.123.123.123, IP = 123.123.123.123, constructing ID payload
Apr 01 11:38:52 [IKEv1 DEBUG]: Group = 123.123.123.123, IP = 123.123.123.123, constructing hash payload
Apr 01 11:38:52 [IKEv1 DEBUG]: Group = 123.123.123.123, IP = 123.123.123.123, Computing hash for ISAKMP
Apr 01 11:38:52 [IKEv1 DEBUG]: Group = 123.123.123.123, IP = 123.123.123.123, constructing dpd vid payload
Apr 01 11:38:52 [IKEv1]: IP = 123.123.123.123, IKE_DECODE SENDING Message (msgid=0) with payloads : HDR + ID (5) + HASH (8) + VENDOR (13) + NONE (0) total length : 84
[/box]
MESSAGE 6 (Arriving at the Initiator)
[box]
Apr 01 11:38:53 [IKEv1]: IP = 123.123.123.123, IKE_DECODE RECEIVED Message (msgid=0) with payloads : HDR + ID (5) + HASH (8) + NONE (0) total length : 64
Apr 01 11:38:53 [IKEv1 DEBUG]: Group = 123.123.123.123, IP = 123.123.123.123, processing ID payload
Apr 01 11:38:53 [IKEv1 DECODE]: Group = 123.123.123.123, IP = 123.123.123.123, ID_IPV4_ADDR ID received 123.123.123.123
Apr 01 11:38:53 [IKEv1 DEBUG]: Group = 123.123.123.123, IP = 123.123.123.123, processing hash payload
Apr 01 11:38:53 [IKEv1 DEBUG]: Group = 123.123.123.123, IP = 123.123.123.123, Computing hash for ISAKMP
Apr 01 11:38:53 [IKEv1]: IP = 123.123.123.123, Connection landed on tunnel_group 123.123.123.123
Apr 01 11:38:53 [IKEv1 DEBUG]: Group = 123.123.123.123, IP = 123.123.123.123, Oakley begin quick mode
Apr 01 11:38:53 [IKEv1 DECODE]: Group = 123.123.123.123, IP = 123.123.123.123, IKE Initiator starting QM: msg id = 26f952ae
Apr 01 11:38:53 [IKEv1]: Group = 123.123.123.123, IP = 123.123.123.123, PHASE 1 COMPLETED
[/box]
Note: You can debug Phase 1 traffic on a particular tunnel, with the following command.
debug crypto condition peer 123.123.123.123
or, simply;
debug crypto isakmp
Phase 1 Debug Errors
[box]
Petes-ASA((config)# debug crypto isakmp
Feb 29 11:49:08 [IKEv1]Group = 123.123.123.123, IP = 123.123.123.123, QM FSM error (P2 struct &0x00007fda92308b10, mess id 0xc02b7a5d)!
Feb 29 11:49:08 [IKEv1]Group = 123.123.123.123, IP = 123.123.123.123, Removing peer from correlator table failed, no match!
Feb 29 11:49:08 [IKEv1]Group = 123.123.123.123, IP = 123.123.123.123, Session is being torn down. Reason: crypto map policy not found
[/box]
This was due to more than one misconfiguration, firstly the source and destination network objects in the interesting traffic ACL were the wrong way round! (Don’t forget to check your static NAT statement as well). And the TRANSFORM SET didn’t match, (sometimes you can see phase one established but then it disappears).
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
[box]
Petes-ASA((config)# debug crypto ikev1
Apr 19 16:36:10 [IKEv1]IP = 123.123.123.123, No Group found by matching OU(s) from ID payload: Unknown
Apr 19 16:36:10 [IKEv1]IP = 123.123.123.123, Trying to find group via IKE ID...
Apr 19 16:36:10 [IKEv1]IP = 123.123.123.123, Trying to find group via IP ADDR...
Apr 19 16:36:10 [IKEv1]IP = 123.123.123.123, Connection landed on tunnel_group 123.123.123.123
Apr 19 16:36:10 [IKEv1 DEBUG]Group = 123.123.123.123, IP = 123.123.123.123, peer ID type 2 received (FQDN)Apr 19 16:36:10 [IKEv1]Group = 123.123.123.123, IP = 123.123.123.123, Unable to compare IKE ID against peer cert Subject Alt Name
Apr 19 16:36:10 [IKEv1 DEBUG]Group = 123.123.123.123, IP = 123.123.123.123, IKE MM Responder FSM error history (struct &0x00007ffd9e230670) <state>, <event>: MM_DONE, EV_ERROR-->MM_BLD_MSG6, EV_COMPARE_IDS-->MM_BLD_MSG6, EV_CERT_OK-->MM_BLD_MSG6, NullEvent-->MM_BLD_MSG6, EV_ACTIVATE_NEW_SA-->MM_BLD_MSG6, NullEvent-->MM_BLD_MSG6, EV_VALIDATE_CERT-->MM_BLD_MSG6, EV_UPDATE_CERT
Apr 19 16:36:10 [IKEv1 DEBUG]Group = 123.123.123.123, IP = 123.123.123.123, IKE SA MM:3a1ed893 terminating: flags 0x01018002, refcnt 0, tuncnt 0
Apr 19 16:36:10 [IKEv1 DEBUG]Group = 123.123.123.123, IP = 123.123.123.123, sending delete/delete with reason message
[/box]
The ASA did not like the certificate presented by the remote peer, (Even though is was a good cert issued by NDES). To get past this you need to make a change to the tunnel group.
Petes-ASA((config)# debug crypto ikev1
%ASA-3-717009: Certificate validation failed. Peer certificate key usage is invalid, serial number: 6B00002B3F8571E2605FA02883000100002C3E, subject name: hostname=Petes-Router-Petes-HQ.petenetlive.com.
%ASA-3-717027: Certificate chain failed validation. Certificate chain is either invalid or not authorized.
[/box]
The ASA did not like the certificate presented by the remote peer, (Even though is was a good cert issued by NDES). To get past this you need to make a change to the trustpoint on the ASA.
[box]
Petes-ASA(config)# crypto ca trustpoint PNL-Trustpoint
Petes-ASA(config-ca-trustpoint)# ignore-ipsec-keyusage
[/box]
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
[box]
Petes-ASA# debug crypto ikev1
Petes-ASA# Feb 17 12:25:17 [IKEv1]Group = 123.123.123.123, IP = 123.123.123.123, Received encrypted Oakley Main Mode packet with invalid payloads, MessID = 0
Feb 17 12:25:17 [IKEv1]Group = 212.20.251.44, IP = 123.123.123.123, ERROR, had problems decrypting packet, probably due to mismatched pre-shared key. Aborting
Feb 17 12:25:23 [IKEv1]IP = 123.123.123.123, Header invalid, missing SA payload! (next payload = 4)
\
[/box]
Amazingly this had nothing to do with a mismatched pre shared key, the other end was set to use PFS (Perfect Forward Secrecy,) and my end (the ASA) was not.
If you have got this far the next step is to troubleshoot Phase 2
Related Articles, References, Credits, or External Links