Category Archives: Routers

Posts, tutorials, and solutions covering the Cisco IOS, Routers and configuration

macOS – SSH Error ‘No Matching Exchange Method Found’

Posted on by
138

Mac SSH Error KB ID 0001245 

Problem

Certified working all the way up to macOS Ventura version 13.6
Certified working all the way up to macOS Sonoma version 14.1

I thought my RoyalTSX had broken today, I upgraded it a couple of weeks ago, and I upgraded to macOS Catalina 10.15 the other day. After this, all my SSH sessions refused to connect with this error;

 

Unable to negotiate with x.x.x.x port 22: no matching key exchange found. Their offer diffie-hellman-group1-sha1

Note: You may also see the following error;

Unable to negotiate with x.x.x.x port 22: no matching cipher found. Their offer: aes128-cbc,3des-cbc,aes192-cbc,aes256-cbc

Update: 10/04/20: With newer equipment you may see the following error;

Unable to negotiate with x.x.x.x port 22: no matching MAC found Their offer: hmac-sha2-256

Update 08/12/21: With newer Cisco equipment you may see the following error;

Unable to negotiate with x.x.x.x.1 port 22: no matching key exchange method found. Their offer: diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1

Mac SSH Error – Fix

WARNING: My usual fix for this is to edit the macs ssh_config file directly and allow the older (less secure) connection methods. It has been (correctly) pointed out, that this is the ‘least preferred‘ method, as it will allow these less sure methods for ALL SSH connections. The preferred fix, is to create a new file in your own home folder, with the connection settings for EACH SSH CONNECTION. However, I manage hundreds of firewalls. routers and switches so that method is a little unwieldy for me. But I will include it at the end of the article, for the more security conscious of you.

Not Apple’s fault to be fair, it’s OpenSSH version 7. SHA1 is weak, so support for it has been removed. Which is fine, but all my clients Cisco Firewalls, Routers, Switches are probably all using  RSA/SHA1. So until they re all updated I’m going to need to re-enable SHA1.

Open a terminal windows and execute the following;

[box]

sudo nano /etc/ssh/ssh_config
ENTER YOUR PASSWORD

[/box]

Locate the line ‘ #   MACs hmac-md5,hmac-sha1,hmac-sha2-256,umac-64@openssh.com,hmac-ripemd160′ and remove the Hash/Pound sight from the beginning, and add the extra hashing algorithm that I’ve shown above in red. 

Locate the line ‘ #   Ciphers aes128-ctr,aes192-ctr,aes256-ctr,aes128-cbc,3des-cbc’ and remove the Hash/Pound sight from the beginning.

Then paste the following on the end;

HostkeyAlgorithms ssh-dss,ssh-rsa

KexAlgorithms +diffie-hellman-group1-sha1,diffie-hellman-group14-sha1

Like so;

Mac SSH Error – Quitting Nano

To quit nano, use the Ctrl-X key combination. because you are working on has been modified since the last time you saved it, you will be prompted to save the file first. Type y to save the file.

Theres no reason to reboot, it should work straight away.

But Wait – Theres More!

It’s going to happen every time you upgrade your mac, I’ve started taking a backup of the ssh_config file, then I can just restore it back again, like so;

Backup macOS SSH Settings & Ciphers

[box]

sudo cp /etc/ssh/ssh_config /etc/ssh/ssh_config.bak

[/box]

Restore macOS SSH Settings & Ciphers

[box]

sudo rm /etc/ssh/ssh_config
sudo mv /etc/ssh/ssh_config.bak /etc/ssh/ssh_config
sudo cp /etc/ssh/ssh_config /etc/ssh/ssh_config.bak

[/box]

Mac SSH Error (The More Secure & Preferred) Fix

Using this method you will create a connection (entry) for each asset you want to connect to. This will be saved in your Home directory so if multiple people use the Mac you will need to do this for each user.

Firstly create a config file in you home directory.

[box]

sudo nano ~/ssh/config

[/box]

What this does is create a blank file that you can then create an entry for each of the assets you want to connect to. Firstly specify the hostname, then the settings for that particular asset, like so:

[box]

Host YOUR-SWITCH
User YOUR-USER
port 22
KexAlgorithms +diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
Ciphers +aes128-cbc,3des-cbc,aes192-cbc,aes256-cbc
PubkeyAcceptedAlgorithms +ssh-rsa
HostkeyAlgorithms +ssh-rsa

[/box]

Note: Simply add new host entries at a new line in the SAME config file.

Related Articles, References, Credits, or External Links

NA

Use Azure MFA With Microsoft NPS (RADIUS) Server

Posted on by
12

 

KB ID 0001759

Problem

I was in a forum last week and someone asked, “Can I enable Azure MFA, on my RADIUS server, to secure access to my switches and routers etc”. It turns out if you want to enable Azure MFA with Microsoft NPS it’s actually quite  simple.

So, I’m using RADIUS auth (above) on my NPS server, and it’s simply checking the authenticating user is a member of a domain security group. Once it has satisfied that requirement, it will authenticate against my Azure AD, which will trigger an MFA event, (in my case send a request to the Microsoft Authenticator Application on my Android Phone).

Azure MFA With Microsoft NPS Pre-Requisites

The remote user needs EITHER an Azure P1 License, or a Microsoft 365 license. 

“But I can use the Authenticator App with my Office 365 subscription?”

Well yes you can, but we are not authenticating to office 365 are we?

Below you can prove the licence is allocated in Office 365

And the same in Azure AD.

Now your user needs to have MFA enabled, (this should be pretty obvious), to use the Microsoft authenticator application the USER chooses that method of authentication, when you enable MFA for them (the first time they login). You can re-force that, from the following screen if you wish.

Azure MFA With Microsoft NPS: Deploying NPS

So I’ve pretty much covered this half a dozen times before, but for completeness I’ll quickly run though setting up NPS / NPAS. The quickest simplest method is to use PowerShell.

[box]

Install-WindowsFeature NPAS -IncludeManagmentTools

[/box]

From administrative tools open > Network Policy Server >Right click (Top Level) > Register Server in Active Directory  > OK > OK

Execute the following PowerShell command to create a registry key

[box]

New-Item 'HKLM:\SOFTWARE\Microsoft\AzureMfa' -Force | New-ItemProperty -Name REQUIRE_USER_MATCH -Value TRUE -Force | Out-Null

[/box]

Enable NPS RADIUS on Windows Firewall

Now for some reason installing NPS does not open the correct ports on the Windows Firewall? So issue the following command;

[box]

Get-NetFirewallRule -DisplayGroup "Network Policy Server" | where DisplayName -like "*RADIUS*" | Set-NetFirewallRule -Service Any

[/box]

Azure MFA With Microsoft NPS: Domain (on Premises and Azure AD)

You will need to know what your Azure Tenant ID is, keep a copy of this handy either in notepad or on the clipboard because you will need it in a minute.

Below you can see I’ve got my domain user, their remote access (Dial In Tab) is set to control access though policy, and I’ve placed them in a security group called SG-Azure-MFA.

Configure NPS for RADIUS Access

Note: You may already have this configured, if so please skip to the next section.

The first task is to define the RADIUS CLIENT, in my case it will be a Cisco firewall, yours could be any device that requires RADIUS authentication. Locate REDIUS Clients  > New > Provide a ‘Friendly Name’ (REMEMBER WHAT IT IS) > Enter its IP address > Then provide and confirm a shared secret (think of it like a password, you will need to add this to the radius clients config) > OK

Policies > Network Policies > New > Give it a sensible name > Next.

Add in a ‘Condition‘ for User Group, then add in the user group you created/used above.

Add in another ‘Condition‘ > Set the friendly name to the one you used when you created your RADIUS client.

Accepts all the defaults until you get to Configure Authentication Methods > Tick ‘Unencrypted Authentication (PAP, SPAP)’> Click yes if you want to read the warning > Next > Accept all the defaults from this point forward.

Enable Azure MFA With Microsoft NPS

Download the ‘NPS Extension For Azure MFA‘ software form Microsoft, and install it on your NPS server.

To actually enable it against your Azure AD, Execute the following PowerShell commands;

[box]

cd "c:\Program Files\Microsoft\AzureMfa\Config"
.\AzureMfaNpsExtnConfigSetup.ps1

[/box]

Eventually you will be asked to authenticate to Azure, do so with an administrative account.

You will be asked to provide your Azure Tennant ID.

When complete REBOOT THE NPS SERVER!

Testing Azure MFA With NPS

Again for Cisco ASA I’ve already blogged about this, but for completeness here’s me making sure it works;

Remember to RAISE the RADIUS timeout, by default its 10 seconds, I raised it to 30 seconds.

And on my phone I get prompted to allow

 

Authentication successful!

Troubleshooting (NPS Azure MFA Not Working)

Event ID 6274: The Request Was Discarded by a third-party extension DLL file. 

This happens when the user you are authenticating does not have the correct license in Azure (or you have just allocated the license and have not waited for a while).

Full Error

[box]

Log Name:      Security
Source:        Microsoft-Windows-Security-Auditing
Date:          15/07/2021 16:42:58
Event ID:      6274
Task Category: Network Policy Server
Level:         Information
Keywords:      Audit Failure
User:          N/A
Computer:      PKI-02.pnl.com
Description:
Network Policy Server discarded the request for a user.

Contact the Network Policy Server administrator for more information.

User:
	Security ID:			PNL\tanya.long
	Account Name:			tanya.long
	Account Domain:			PNL
	Fully Qualified Account Name:	pnl.com/PNL/Users/Tanya Long

Client Machine:
	Security ID:			NULL SID
	Account Name:			-
	Fully Qualified Account Name:	-
	Called Station Identifier:		-
	Calling Station Identifier:		-

NAS:
	NAS IPv4 Address:		192.168.254.254
	NAS IPv6 Address:		-
	NAS Identifier:			-
	NAS Port-Type:			Virtual
	NAS Port:			6

RADIUS Client:
	Client Friendly Name:		Firewall
	Client IP Address:			192.168.254.254

Authentication Details:
	Connection Request Policy Name:	Use Windows authentication for all users
	Network Policy Name:		NP-Azure-MFA
	Authentication Provider:		Windows
	Authentication Server:		PKI-02.pnl.com
	Authentication Type:		PAP
	EAP Type:			-
	Account Session Identifier:		-
	Reason Code:			9
	Reason:				The request was discarded by a third-party extension DLL file.

[/box]

Event ID 6273: An NPS extension dynamic link library (DLL) that is installed on the NPS server rejected the connection

In my case I had re-install the NPS Azure extension.

Full Error

[box]

Log Name:      Security
Source:        Microsoft-Windows-Security-Auditing
Date:          15/07/2021 17:24:39
Event ID:      6273
Task Category: Network Policy Server
Level:         Information
Keywords:      Audit Failure
User:          N/A
Computer:      PKI-02.pnl.com
Description:
Network Policy Server denied access to a user.

Contact the Network Policy Server administrator for more information.

User:
	Security ID:			NULL SID
	Account Name:			tanya.long
	Account Domain:			PNL
	Fully Qualified Account Name:	PNL\tanya.long

Client Machine:
	Security ID:			NULL SID
	Account Name:			-
	Fully Qualified Account Name:	-
	Called Station Identifier:		-
	Calling Station Identifier:		-

NAS:
	NAS IPv4 Address:		192.168.254.254
	NAS IPv6 Address:		-
	NAS Identifier:			-
	NAS Port-Type:			Virtual
	NAS Port:			10

RADIUS Client:
	Client Friendly Name:		Firewall
	Client IP Address:			192.168.254.254

Authentication Details:
	Connection Request Policy Name:	Use Windows authentication for all users
	Network Policy Name:		-
	Authentication Provider:		Windows
	Authentication Server:		PKI-02.pnl.com
	Authentication Type:		Extension
	EAP Type:			-
	Account Session Identifier:		-
	Logging Results:			Accounting information was written to the local log file.
	Reason Code:			21
	Reason:				An NPS extension dynamic link library (DLL) that is installed on the NPS server rejected the connection request.

[/box]

 

Related Articles, References, Credits, or External Links

NA

Cisco: Getting a SKU (Product ID) From a Serial Number

Posted on by
Reply

KB ID 0001674

Problem

I had a situation a couple of weeks ago where I had the serial numbers for a bunch of Cisco switches, I needed to get some extended cover for them, but what I didn’t have were the Cisco SKU (Stock Keeping Unit) codes.

Solution

You will need to have a Cisco CCO login, once you have that go here > Add devices.

Give the device a name, (it does not matter what) > Paste in the serial number > Add.

Boom, there’s your SKU (Product ID)

Repeat as required.

Related Articles, References, Credits, or External Links

NA

Configuring Cisco HSRP

Posted on by

KB ID 0000946 

Problem

Cisco HSRP: Normally your client machines have one route off the network, (their default gateway). But what if that goes down? HSRP aims to solve this problem by assigning a ‘Virtual IP address’ to your default gateway (or default route). So that IP can be shared amongst two or more possible devices (routers, or layer 3 switches).

Above, we have a client 192.168.1.10 that has two possible routes off the network, (.254 and .253). We will setup a virtual IP of .250 and both routers can use that IP, (if they are the active gateway). Below is a brief overview of how to set it up.

Deploy Cisco HSRP

Setting up Cisco HSRP

1. On the first router (Router0), add the standby IP address (192.168.1.250) the ‘1’ denotes the standby group (a number from 0 to 4096). It comes up as standby, then after it has checked (via multicast address 224.0.0.2 on UDP port 1985). It finds no other live HSRP devices using that IP address so it becomes ‘Active’.

[box]

Router0>enable
Router0#configure terminal 
Enter configuration commands, one per line. End with CNTL/Z.
Router0(config)#interface GigabitEthernet0/0
Router0(config-if)#standby 1 ip 192.168.1.250
Router0(config-if)#

%HSRP-6-STATECHANGE: GigabitEthernet0/0 Grp 1 state Speak -> Standby

%HSRP-6-STATECHANGE: GigabitEthernet0/0 Grp 1 state Standby -> Active

Router0(config-if)#

[/box]

2. Repeat this on the second Router, this one discovers the ‘Active’ router and sets itself up as ‘Standby’.

[box]

Router1>
Router1>enable
Router1#configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
Router1(config)#interface GigabitEthernet0/0
Router1(config-if)#standby 1 ip 192.168.1.250
Router1(config-if)#
%HSRP-6-STATECHANGE: GigabitEthernet0/0 Grp 1 state Speak -> Standby

Router1(config-if)#

[/box]

3. You can prove this by running show standby (or do show standby in configure terminal mode).

[box]

Router0

Router0#show standby
GigabitEthernet0/0 - Group 1 (version 2)
State is Active
8 state changes, last state change 00:02:02
Virtual IP address is 192.168.1.250
Active virtual MAC address is 0000.0C9F.F001
Local virtual MAC address is 0000.0C9F.F001 (v2 default)
Hello time 3 sec, hold time 10 sec
Next hello sent in 0.055 secs
Preemption disabled
Active router is local
Standby router is 192.168.1.253
Priority 100 (default 100)
Group name is hsrp-Gig0/0-1 (default)
Router0#

Router1

Router1#show standby
GigabitEthernet0/0 - Group 1 (version 2)
State is Standby
3 state changes, last state change 00:10:44
Virtual IP address is 192.168.1.250
Active virtual MAC address is unknown
Local virtual MAC address is 0000.0C9F.F001 (v2 default)
Hello time 3 sec, hold time 10 sec
Next hello sent in 2.125 secs
Preemption disabled
Active router is 192.168.1.254
Standby router is local
Priority 100 (default 100)
Group name is hsrp-Gig0/0-1 (default)
Router1#

[/box]

4. That is HSRP configured! However there are a few changes you might want to make, for example, what if one router had a 100MB leased line, and the other was a 2MB ADSL line, you would want the fastest one to be in use, (as long as it was up). To achieve that, give the router with the fastest connection a higher priority (you may notice above, that by default the priority is 100). Be aware, even if a router has the highest priority, it wont ‘seize’ the virtual IP, it just sits and waits until it’s available. For our 100MB and 2MB example that’s not good. We would want Router1 to seize the virtual IP as soon as it can. To do that we need to set it to preempt. (Note: This process is called ‘launching a coup’).

[box]

Router1(config)#interface GigabitEthernet0/0
Router1(config-if)#standby 1 priority 105
Router1(config-if)#standby 1 preempt

[/box]

5. At this point it’s important to say, that in our scenario we would also need to setup a virtual IP for the ‘other side’ of the routers (i.e their GigabitEthernet 0/1 interfaces), or the remote client (172.16.1.10) would not be able to return our ‘pings’ or get any traffic back to us. So lets setup a virtual HSRP address on that side as well. Notice I just use another standby group number.

Note: To work the remote host 172.16.1.10 will need its default gateway changing to the HSRP Virtual IP of 172.16.1.250.

[box]

Router0 

Router0(config)#interface GigabitEthernet0/1
Router0(config-if)#standby 2 ip 172.16.1.250
Router0(config-if)#

Router1

Router1(config)#interface GigabitEthernet0/1
Router1(config-if)#standby 2 ip 172.16.1.250
Router1(config-if)#

[/box]

6. Finally we have set Router0 with the highest priority and set it to seize the virtual IP as soon as it can. But what if another interface on Router1 goes down? e.g. If the GigabitEthernet 0/1 interface were to go down, HSRP would not do anything because it’s tracking both the GigabitEthernet 0/0 interfaces, so communications would fail.

To solve the problem we need to tell it which interfaces to ‘Track’. In our example we need to track GigabitEthernet 0/1, if that goes down we need to give the virtual IP address to the standby ‘router’. This works because once we tell it to ‘track’ the GigabitEthernet 0/1 interface, if that were to fail it will DECREMEMT the routers priority by 10. So for Router0 its priority would drop to 95, this is five less than the default value of 100 (on Router1). But Remember, at the moment that fail-over would still fail, unless you allow Router1 to preempt and launch a coup.

[box]

Router0 

Router0(config)#interface GigabitEthernet0/0
Router0(config-if)#standby 1 track GigabitEthernet0/1
Router0(config-if)#

Router1

Router1(config)#interface GigabitEthernet0/0
Router1(config-if)#standby 1 preempt
Router1(config-if)#

[/box]

7. We can see that by running a ‘show standby’ on Router0.

[box]

Router0#show standby
GigabitEthernet0/0 - Group 1 (version 2)
State is Active
7 state changes, last state change 00:00:31
Virtual IP address is 192.168.1.250
Active virtual MAC address is 0000.0C9F.F001
Local virtual MAC address is 0000.0C9F.F001 (v2 default)
Hello time 3 sec, hold time 10 sec
Next hello sent in 2.81 secs
Preemption enabled
Active router is local
Standby router is 192.168.1.253, priority 115 (expires in 7 sec)
Priority 115 (configured 115)
Track interface GigabitEthernet0/1 state Up decrement 10
Group name is hsrp-Gig0/0-1 (default)
GigabitEthernet0/1 - Group 2 (version 2)
State is Active
6 state changes, last state change 00:00:28
Virtual IP address is 172.16.1.250
Active virtual MAC address is 0000.0C9F.F002
Local virtual MAC address is 0000.0C9F.F002 (v2 default)
Hello time 3 sec, hold time 10 sec
Next hello sent in 0.81 secs
Preemption disabled
Active router is local
Standby router is 172.16.1.253
Priority 100 (default 100)
Group name is hsrp-Gig0/1-2 (default)
Router0#

[/box]

Related Articles, References, Credits, or External Links

NA

Cisco IOS: Ether-Channel Trunks

Posted on by
1

KB ID 0001533

Problem

This is a subject that every time I need to create an Ether-Channel I end up checking beforehand, so it’s about time I wrote it up. We are combining two different things, an Ether-channel, (an aggregation of links) and a Trunk (the ability to carry many VLANS). If you are NOT from a Cisco background then you might want to read though the following post first to avoid confusion about the world ‘Trunk‘.

HP and Cisco – VLANs and Trunks Confusion!

So this is what I’m going to create;

I will combine the TWO links between the switches to act at one link (Ether-Channel). An Ether-Channel can have up to eight links.

Note: I’m only concentrating on the Ether-Channel setup so VLANs/VTP and Routing are not covered.

Solution

You can use two types of Ether-Channels PAgP (Port Aggregation Protocol), but WHY when it only works on Cisco switches. LACP (Link Aggregation control Protocol) which is supported by just about everything else, so let’s stick with that! By default a ‘Trunk’ will pass ALL VLANS, you might not want that, I’ll cover filtering VLANs a bit further down.

WARNING: If you simply connect two switches with two cables you will create a LOOP, if you have STP enabled the network will recover and block one of the links, but your colleagues will shake their heads and pull a ‘frowny face’. For that reason ‘SHUT THE PORTS DOWN BEFORE YOU CABLE / CONFIGURE THEM“. 

Starting on Switch1 make sure there’s is no existing Ether-Channels configured;

[box]

SW-1#show etherchannel
                Channel-group listing:
                ----------------------

Group: 1
----------
Group state = L2
Ports: 2   Maxports = 4
Port-channels: 1 Max Port-channels = 4
Protocol:   LACP
Minimum Links: 0

[/box]

Above there is already an Ether-Channel (port-channel) on the switch (group 1) so you would have to use group 2. For arguments sake we will say I don’t have one, so I can use group 1.

Note: ‘show etherchannel summary’ is also a handy command to remember!

[box]

First shut down the uplinks, Note the syntax for the 'range interfaces', may differ from device to device, 
so use the TAB key.

SW-1#configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
SW-1(config)#interface range ethernet 0/1 - 2
SW-1(config-if-range)#shutdown

Add the ports to channel group 1, Note 'Active' denotes use LACP, (Passive also works, but one (or both) ends
should be active.)

SW-1(config-if-range)#channel-group 1 mode active
Creating a port-channel interface Port-channel 1

Create a 'trunk' with 802.1q encapsulation.

SW-1(config-if-range)#switchport trunk encapsulation dot1q
SW-1(config-if-range)#switchport mode trunk

[/box]

Then configure the other end the same, (assuming the port numbers are the same!) As mentioned above you can use LACP mode ‘passive‘ but I tend to set both ends active.

Once you have both ends configured and the cables in place, enable the interfaces with a ‘no shutdown‘ command, on both ends!

[box]

SW-1(config)#interface range ethernet 0/1 - 2
SW-1(config-if-range)#no shutdown

[/box]

Filtering VLANs on an Ether-Channel Trunk

Any further port-channel changes need to be done on the port-channel interface, so if you want to filter what’s allowed you simply use the following syntax;

[box]

SW-1#configure terminal
Enter configuration commands, one per line.  End with CNTL/Z.
SW-1(config)#interface Port-channel1
SW-1(config-if)#switchport trunk allowed vlan 1,10,100,200

[/box]

Note: When adding any future VLANS check the syntax, if you simply add a new one it will overwrite all the others, and things will break!

Do all Ether-Channels need to be Trunks? No! Not at all, they can be access ports (as long as they are all in the same VLAN), and they can also be routed uplinks, with an IP address at both ends, (specified in the port-channel interface). 

Related Articles, References, Credits, or External Links

NA

Cisco ASA VPN to Cisco Router “MM_WAIT_MSG3”

Posted on by
Reply

KB ID 0001531

Problem

While migrating a VPN tunnel from an ASA 5520 firewall to a new 5516-X I got this problem. The other end was a Cisco router (2900). As soon as I swapped it over, it was stuck at MM_WAIT_MSG3, and phase 1 would not establish;

[box]

NUFC-ASA5516x(config-tunnel-ipsec)# show crypto isa

IKEv1 SAs:

   Active SA: 6
    Rekey SA: 0 (A tunnel will report 1 Active and 1 Rekey SA during rekey)
Total IKE SA: 6

1   IKE Peer: 1.1.1.1
    Type    : user            Role    : responder
    Rekey   : no              State   : MM_WAIT_MSG3

[/box]

Debugs didn’t help much either;

[box]

Mar 25 2019 18:50:49: %ASA-7-713236: IP = 1.1.1.1, IKE_DECODE RECEIVED Message (msgid=0) with payloads : HDR + SA (1) + VENDOR (13) + VENDOR (13) + VENDOR (13) + VENDOR (13) + NONE (0) total length : 164
Mar 25 2019 18:50:49: %ASA-7-715047: IP = 1.1.1.1, processing SA payload
Mar 25 2019 18:50:49: %ASA-7-713906: IP = 1.1.1.1, Oakley proposal is acceptable
Mar 25 2019 18:50:49: %ASA-7-715047: IP = 1.1.1.1, processing VID payload
Mar 25 2019 18:50:49: %ASA-7-715049: IP = 1.1.1.1, Received NAT-Traversal RFC VID
Mar 25 2019 18:50:49: %ASA-7-715047: IP = 1.1.1.1, processing VID payload
Mar 25 2019 18:50:49: %ASA-7-715047: IP = 1.1.1.1, processing VID payload
Mar 25 2019 18:50:49: %ASA-7-715049: IP = 1.1.1.1, Received NAT-Traversal ver 03 VID
Mar 25 2019 18:50:49: %ASA-7-715047: IP = 1.1.1.1, processing VID payload
Mar 25 2019 18:50:49: %ASA-7-715049: IP = 1.1.1.1, Received NAT-Traversal ver 02 VID
Mar 25 2019 18:50:49: %ASA-7-715047: IP = 1.1.1.1, processing IKE SA payload
Mar 25 2019 18:50:49: %ASA-7-715028: IP = 1.1.1.1, IKE SA Proposal # 1, Transform # 1 acceptable  Matches global IKE entry # 8
Mar 25 2019 18:50:49: %ASA-7-715046: IP = 1.1.1.1, constructing ISAKMP SA payload
Mar 25 2019 18:50:49: %ASA-7-715046: IP = 1.1.1.1, constructing NAT-Traversal VID ver RFC payload
Mar 25 2019 18:50:49: %ASA-7-715046: IP = 1.1.1.1, constructing Fragmentation VID + extended capabilities payload
Mar 25 2019 18:50:49: %ASA-7-713236: IP = 1.1.1.1, IKE_DECODE SENDING Message (msgid=0) with payloads : HDR + SA (1) + VENDOR (13) + VENDOR (13) + NONE (0) total length : 128
Mar 25 2019 18:50:57: %ASA-7-713236: IP = 1.1.1.1, IKE_DECODE RESENDING Message (msgid=0) with payloads : HDR + SA (1) + VENDOR (13) + VENDOR (13) + NONE (0) total length : 128
Mar 25 2019 18:50:59: %ASA-7-713906: IKE Receiver: Packet received on 2.2.2.2:500 from 1.1.1.1:500
Mar 25 2019 18:50:59: %ASA-5-713202: IP = 1.1.1.1, Duplicate first packet detected.  Ignoring packet.
Mar 25 2019 18:51:05: %ASA-7-713236: IP = 1.1.1.1, IKE_DECODE RESENDING Message (msgid=0) with payloads : HDR + SA (1) + VENDOR (13) + VENDOR (13) + NONE (0) total length : 128
Mar 25 2019 18:51:21: %ASA-7-715065: IP = 1.1.1.1, IKE MM Responder FSM error history (struct &0x00007f4d2d293690)  , :  MM_DONE, EV_ERROR-->MM_WAIT_MSG3, EV_TIMEOUT-->MM_WAIT_MSG3, NullEvent-->MM_SND_MSG2, EV_SND_MSG-->MM_SND_MSG2, EV_START_TMR-->MM_SND_MSG2, EV_RESEND_MSG-->MM_WAIT_MSG3, EV_TIMEOUT-->MM_WAIT_MSG3, NullEvent
Mar 25 2019 18:51:21: %ASA-7-713906: IP = 1.1.1.1, IKE SA MM:5f3d6a94 terminating:  flags 0x01000002, refcnt 0, tuncnt 0
Mar 25 2019 18:51:21: %ASA-7-713906: IP = 1.1.1.1, sending delete/delete with reason message
Mar 25 2019 18:51:21: %ASA-6-713905: IP = 1.1.1.1, Warning: Ignoring IKE SA (src) without VM bit set

[/box]

Solution

Well, as you can tell from my Troubleshooting Phase 1 Cisco Site to Site (L2L) VPN Tunnels article MM_WAIT_MSG3 usually happens if something is blocking ISAKMP (UDP 500) in-between the peers. Or there’s a ‘bug’ that needs some newer or older code. The problem was not spotted by me, I was chatting to an ex colleague about it and he knew what it was straight away (annoyingly). The Router had an ACL on its outside interface that was NOT allowing the peer in to establish a VPN.

On the router take a look a the outside interface, and look for an access-list;

[box]

Remote-Router#show run interface GigabitEthernet 0/0
Building configuration...

Current configuration : 214 bytes
!
interface GigabitEthernet0/0
 bandwidth 200000
 ip address 1.1.1.1 255.255.255.248
 ip access-group outside-in in
 ip nat outside
 ip virtual-reassembly in
 duplex auto
 speed auto
 crypto map CMAP-1
end

Remote-Router#

[/box]

So it’s called outside-in what’s it doing?

[box]

Remote-Router#show access-list outside-in
Extended IP access list outside-in
    10 permit udp any eq domain any
    20 permit icmp any any echo-reply (956 matches)
    30 permit ip 123.123.123.64 0.0.0.15 any (127341882 matches)
    40 permit ip 123.123.123.128 0.0.0.127 any (572 matches)
    50 permit ip 222.222.222.96 0.0.0.15 any (4 matches)
    60 permit ip host 123.123.123.68 any
    70 permit udp host 222.222.222.76 eq ntp any
    80 permit udp host 222.222.222.204 eq ntp any
    90 permit udp host 222.222.222.232 eq ntp any
    100 permit icmp any any (1320 matches)
    110 permit ip 223.223.233.0 0.0.0.255 any

[/box]

Add in our peer IP address;

[box]

Remote-Router#configure terminal
Enter configuration commands, one per line.  End with CNTL/Z.
Remote-Router(config)#ip access-list extended outside-in
Remote-Router(config-ext-nacl)# permit esp host 2.2.2.2 any
Remote-Router(config-ext-nacl)# permit udp host 2.2.2.2 any eq isakmp
Remote-Router(config-ext-nacl)# permit udp host 2.2.2.2 any eq non500-isakmp
Remote-Router(config-ext-nacl)#exit
Remote-Router(config)# 

[/box]

Let’s just make sure it’s there;

[box]

Remote-Router#show access-list outside-in
Extended IP access list outside-in
    10 permit udp any eq domain any
    20 permit icmp any any echo-reply (956 matches)
    30 permit ip 123.123.123.64 0.0.0.15 any (127341882 matches)
    40 permit ip 123.123.123.128 0.0.0.127 any (572 matches)
    50 permit ip 222.222.222.96 0.0.0.15 any (4 matches)
    60 permit ip host 123.123.123.68 any
    70 permit udp host 222.222.222.76 eq ntp any
    80 permit udp host 222.222.222.204 eq ntp any
    90 permit udp host 222.222.222.232 eq ntp any
    100 permit icmp any any (1320 matches)
    110 permit ip 223.223.233.0 0.0.0.255 any
    120 permit esp host 2.2.2.2 any
    130 permit udp host 2.2.2.2 any eq isakmp
    140 permit udp host 2.2.2.2 any eq non500-isakmp

[/box]

Don’t forget to save the changes with a ‘write mem’ command!

Related Articles, References, Credits, or External Links

Once again, thanks to SteveH for spotting, (in less than sixty seconds) what was wrong.

Cisco ASA EZVPN (Revisited)

Posted on by
19

KB ID 0001261

Problem

EZVPN is a technology that lets you form an ISAKMP/IPSEC VPN tunnel from a site with a dynamically assigned IP (EZVPN Client,) back to a device with a static IP (EZVPN Server).

I’ve called this EZVPN revisited, because this is a technology I’ve talked about before. So why am I here again? Well back then I used the ASDM. If you do that now, you need to go in and mess about with things to get it to work properly. Last week a client was asking me about buying a 5505 for his home, and putting a VPN into his place of work. Obviously he did not have a static IP at home, which was why I suggested EZVPN.

So it’s time to ‘Man Up’ and get to grips with the CLI. In the example below my corporate LAN is behind a Cisco ASA 5515-X, and my ‘Home Office’ is behind a Cisco ASA 5506-X, (you can use a 5508-X as well, or an old 5505).

 

Solution

So How does EZVPN Work? Well there’s no separate/special technology, it’s a good old fashioned Client IPSEC VPN. The one we used to use the OLD IPSEC VPN client for, (yes the one that went end of life – in 2011!)

But instead of using a piece of software to supply the username/password and the group/pre-shared-key, you configure a hardware device to supply those details. This enables the hardware device to bring up a software client VPN session. There are two methods of doing this, Client Mode and Network Extension Mode (NEM).

  • Client Mode: Works exactly like the VPN client software, and leases an IP address from a pool of IP addresses supplied by the ASA, (or a DHCP server).
  • Network Extension Mode: This works like a ‘proper’ site to site VPN, insofar as, all the IP addresses on the client/remote site can be addressed from the main site. 

I’m going to use Network Extension Mode for this example, I’m also going to enable ‘Split tunnelling’ so that only VPN traffic goes over the VPN.

Remote EZVPN Client WARNING

The client that ‘dials in’ cannot be running any other VPN solution. In fact it can’t even have IKE policies defined, (even if they are not in use).

Configure the EZVPN Server

The bulk of the work is on the main site ASA.

[box]

!
crypto ikev1 policy 65535
 authentication pre-share
 encryption 3des
 hash sha
 group 2
 lifetime 86400
!
crypto ipsec ikev1 transform-set TS-IPSEC-VPN esp-3des esp-sha-hmac
!
access-list SPLIT-TUNNEL standard permit 192.168.100.0 255.255.255.0
!
group-policy IPSEC-VPN internal
group-policy IPSEC-VPN attributes
 password-storage enable
 nem enable
 vpn-tunnel-protocol ikev1
 split-tunnel-policy tunnelspecified
 split-tunnel-network-list value SPLIT-TUNNEL
 vpn-simultaneous-logins 3
!
tunnel-group IPSEC-VPN type remote-access
tunnel-group IPSEC-VPN general-attributes
 default-group-policy IPSEC-VPN
 authentication-server-group LOCAL
tunnel-group IPSEC-VPN ipsec-attributes
 ikev1 pre-shared-key Cisco123456
!
crypto dynamic-map DYNAMIC-CRYPTO-MAP 65535 set ikev1 transform-set TS-IPSEC-VPN
!
crypto map CRYPTO-MAP 65535 ipsec-isakmp dynamic DYNAMIC-CRYPTO-MAP
!
crypto map CRYPTO-MAP interface outside
!
crypto ikev1 enable outside
!
object network OBJ-EZVPN-SUBNET
 subnet 10.254.254.0 255.255.255.0
!
nat (inside,outside) source static any any destination static OBJ-EZVPN-SUBNET OBJ-EZVPN-SUBNET no-proxy-arp route-lookup
!
username EZVPNSite1 password P@ssword123
!

[/box]

Points to Note:

  • I’m using 3DES and SHA1 for Phase 1 (ISAKMP,) and phase 2 (IPSEC).
  • The Network behind my main site ASA is 192.168.100.0/24.
  • The Network behind my remote site ASA is 10.254.254.0/24.
  • I’ve enabled split tunnelling.
  • My interfaces are called inside and outside, yours might be different!
  • Crypto Map Warning: If you already have a crypto map applied to the outside interface use the name of the existing one (i.e NOT CRYPTO-MAP), or your exiting VPN’s will stop working! Issue a ‘show run crypto map‘ command to check.
  • I have not enabled PFS. (If I had it would have been in the crypto map).

Configure the EZVPN Client (Remote Site)

The remote site(s) are easy.

[box]

!
vpnclient server 198.100.51.1
vpnclient mode network-extension-mode
vpnclient nem-st-autoconnect
vpnclient vpngroup IPSEC-VPN password Cisco123456
vpnclient username EZVPNSite1 password P@ssword123
vpnclient enable
!

[/box]

Adding Additional EZVPN Sites

To add another site in Client Mode you would simply add another username and password, on the EZVPN server. With Network Extension Mode then you would add an object and NAT exemption on the main site, then setup a new username and password for that site like so;

New Site EZVPN Server Config

[box]

!
object network OBJ-EZVPN-SUBNET-2
 subnet 10.254.254.0 255.255.255.0
!
nat (inside,outside) source static any any destination static OBJ-EZVPN-SUBNET-2 OBJ-EZVPN-SUBNET-2 no-proxy-arp route-lookup
!
username EZVPNSite2 password P@ssword456
!

[/box]

New Site EZVPN Client Config

You just need the new username and password;

[box]

!
vpnclient server 198.100.51.1
vpnclient mode network-extension-mode
vpnclient nem-st-autoconnect
vpnclient vpngroup IPSEC-VPN password Cisco123456
vpnclient username EZVPNSite2 password P@ssword456
vpnclient enable
!

[/box]

 

Related Articles, References, Credits, or External Links

NA

Cisco Error ‘%PHY-4-SFP_NOT_SUPPORTED’

Posted on by
Reply

KB ID 0001347 

Problem

This is another question I see getting asked a lot in forums!

You see something like the following;

[box]

000032: *Sep 28 09:35:32.507 UTC: %PHY-4-SFP_NOT_SUPPORTED: The SFP in Gi3/0/50 is not supported (PNL-3750-Stack)
000033: *Sep 28 09:35:32.507 UTC: %PM-4-ERR_DISABLE: gbic-invalid error detected  on Gi3/0/50, putting Gi3/0/50 in err-disable state (PNL-3750-Stack)

[/box]

The usual response is ‘Enable unsupported SFP’s’, and while that sometimes is the answer, it’s not always the answer!

 

Solution

1. Firstly Check the Modules and the Switches, Are you tying to plug a 10GB SFP+ into a slot that only supports SFP, (that includes plugging a twinax cable into an old switch!) In your ‘show run’ you should see TenGigabitEthernet (if your using SFP+ modules). Some switches with network modules list the same interface twice (once as 10GB interfaces and once at 1GB modules, I’ve blogged about that before see THIS ARTICLE, and to confuse things even further, the four interface versions, are grouped as two pairs with each pair consisting of one SFP slot and one SFP+ slot.)

2. Make sure your cable is NOT a CAB-SFP-50CM, (unless you are connecting a 3560 to ANOTHER 3560).

3. Are you using a 2960-S? If so you may need to update the IOS to use SFP+ (assuming your model supports SFP+ not all 2960-S models do).

4. Are you plugging into a Nexus switch with a 1GB connection? If so check the other end for the following error;

Description: Gi1/1/15: This port has been disabled because Non Compliant Gigabit Interface Converter (GBIC) connector detected.

If so, you may need to Manually set the speed on the 5K to 1000, (it wont auto-sense).

5. Is it a non-cisco branded SFP? If so it may still work, (but you will get no joy if you log a TAC call) with the following commands;

[box]

Petes-SW(config)#service internal
Petes-SW(config)#no errdisable detect cause gbic-invalid
Petes-SW(config)#service unsupported-transceiver

[/box]

If you are still in doubt check the Compatibility Matrix.

Related Articles, References, Credits, or External Links

NA

Setup RANCID and ViewVC (Part One)

Posted on by
17

KB ID 0001331

Problem

There are couple of good posts out there on setting up Rancid (Really Awesome New Cisco Config Differ). Some even show you how to set it up with ViewVC (Formally ViewVCS, basically a nice web based GUI front end, that does version control and highlights differences). It does this using a system called CVS (Concurrent Version System, hence the original name.)

Then I had to do some more searching to get it to back up my Cisco ASA firewalls and get the whole thing automated, which Ill cover in Part Two.

OS: CentOS 7 x64

Rancid: 3.4.1

ViewVC: 1.1.24

Solution

Once it’s built, let’s update our server;

[box]

yum upgrade
yum update
Answer any questions with 'y' for yes.

[/box]

Lets install all the components we need;

[box]

yum install nano wget ftp telnet mariadb-server mariadb perl tcl expect gcc cvs rcs httpd autoconf php-common php-gd php-pear php-pecl-memcache php-mysql php-xml mod_ssl MySQL-python

[/box]

We only need a simple web server, so set the firewall accordingly (I’m opening http (TCP 80) and https (TCP 443) but we will only configure http in this example);

[box]

firewall-cmd --permanent --zone=public --add-service=http
firewall-cmd --permanent --zone=public --add-service=https
firewall-cmd --reload

[/box]

We have already installed Apache (web server), so we need to start it up, then set it to ‘auto-start’ with the server. Then create a group, and a user (rancid), and place that user in the group we created. Finally create a directory for us to ‘unzip’ rancid into.

[box]

systemctl enable httpd.service
systemctl start httpd.service
groupadd netadm
useradd -g netadm -c "Networking Backups" -d /home/rancid rancid
mkdir /home/rancid/tar

[/box]

Move into the directory, we just created then download RANCID, unzip it, then install it.

[box]

cd /home/rancid/tar/
wget ftp://ftp.shrubbery.net/pub/rancid/rancid-3.4.1.tar.gz
tar -zxvf rancid-3.4.1.tar.gz
cd rancid-3.4.1
./configure --prefix=/usr/local/rancid
make install

[/box]

Copy over the ‘clogin’ file, (more on this later). Then set the ownership and and permissions on the rancid files and directories.

[box]

cp cloginrc.sample /home/rancid/.cloginrc
chmod 0640 /home/rancid/.cloginrc
chown -R rancid:netadm /home/rancid/.cloginrc
chown -R rancid:netadm /usr/local/rancid/
chmod 775 /usr/local/rancid/

[/box]

Now to set the ‘top level’ groups. Above I’ve got Firewalls and Switches, you might want to create a group for each customer you are backing up, or each site. (the design is up to you), but these are the ‘folder names’ you will see when you first log into the ViewVC web front end. Next you need to edit the main Rancid configuration file;

[box]

nano /usr/local/rancid/etc/rancid.conf

[/box]

Uncomment and add your groups, i.e.
LIST_OF_GROUPS=”Firewalls Switches“

Save and Exit the file, (in nano that’s CTRL+X, then Y to save, and finally {Enter}.) Now we need to change to the rancid user we created earlier, (remembering to swap back to ‘root’ afterwards). We do this to create the folder structure for the groups we just created. Most importantly this creates the router.db file(s) which lives in each group/folder. This file specifies what the devices are, and where they are, that you will be backing up.

[box]

su -rancid

Or 'su rancid' if you are logged in as root

/usr/local/rancid/bin/rancid-cvs
su

[/box]

Download and install ViewVC.

[box]

cd /home/rancid/tar/
wget http://viewvc.tigris.org/files/documents/3330/49471/viewvc-1.1.24.tar.gz
tar -zxvf viewvc-1.1.24.tar.gz
cd viewvc-1.1.24
./viewvc-install

[/box]

Open the ViewVC config file;

 

[box]

nano /usr/local/viewvc-1.1.24/viewvc.conf

[/box]

Uncomment and change the values, (as shown above).

root_parents = /usr/local/rancid/var/CVS : cvs
rcs_dir = /usr/local/bin
use_rcsparse = 1

Now to enable ViewVC to work with Apache, we need to copy over some CGI, and set some permissions.

[box]

cp /usr/local/viewvc-1.1.24/bin/cgi/*.cgi /var/www/cgi-bin
chmod +x /var/www/cgi-bin/*.cgi
chown apache:apache /var/www/cgi-bin/*.cgi

[/box]

Then in the Apache config file, scroll to the end and add the following text, (don’t forget to save and exit the file).

[box]

nano /etc/httpd/conf/httpd.conf

Paste the following (at the bottom)

# Custom Rancid Config

<VirtualHost>
        DocumentRoot /var/www
        ScriptAlias /cgi-bin/ "/var/www/cgi-bin"
        ScriptAlias /viewvc /var/www/cgi-bin/viewvc.cgi
        ScriptAlias /query /var/www/cgi-bin/query.cgi
<Directory "/var/www/cgi-bin">
    AllowOverride None
    Options None
    Order allow,deny
    Allow from all
</Directory>
</VirtualHost>

[/box]

We already installed MariaDB, (similar to MySQL) earlier, we just need to start it up, then set it to auto start with the server. Out of the box, MariaDB is a little insecure, so there’s a process of hardening it called ‘mysql_secure_installation’.

Note: the first time you run the secure installation it will ask for a password, this is the root password for MariaDB NOT the Linux root password, this will be {blank} so just hit {Enter}, then Yes to set the password, and set the MySQL/MariaDB root password (NEVER LOSE OR FORGET THIS). You will need it in a minute anyway, for all the other questions simply press {Enter} to accept the defaults.

[box]

systemctl enable mariadb
systemctl start mariadb
sudo mysql_secure_installation

[/box]

Now that MariaDB is installed, we need to create a user in SQL that ViewVC will use, to do that we need to log into SQL using the root password you just setup.

[box]

mysql -u root -p
Enter your SQL root password
CREATE USER 'VIEWVC'@'localhost' IDENTIFIED BY ‘Password123’;
GRANT ALL PRIVILEGES ON *.* TO 'VIEWVC'@'localhost' WITH GRANT OPTION;
FLUSH PRIVILEGES;
quit

[/box]

Note: sometime it does not like you copying and pasting the first ‘CREATE USER…’ line, (I don’t know why.) If it complains, type it out manually!

Now that is set up we can get ViewVC to create its database.

[box]

cd /usr/local/viewvc-1.1.24/bin
./make-database

[/box]

Use the following settings when prompted;

MySQL Hostname (leave blank for default):{Enter}

MySQL Port (leave blank for default):{Enter}

MySQL User: VIEWVC

MySQL Password: Password123

ViewVC Database Name [default: ViewVC]:{Enter}

Then, (the same as you did earlier,) create another user in MariaDB, that will be a ‘read-only’ user.

[box]

mysql -u root -p
Enter your SQL root password
CREATE USER 'VIEWVCRO'@'localhost' IDENTIFIED BY ‘Password456’;
GRANT SELECT ON ViewVC.* TO 'VIEWVCRO'@'localhost' WITH GRANT OPTION;
FLUSH PRIVILEGES;
quit

[/box]

Edit the ViewVC configuration so that it uses all the parameters you have setup;

[box]

nano /usr/local/viewvc-1.1.24/viewvc.conf

[/box]

 

Un

Scroll down to the [csvdb] section, then uncomment and change the values shown (above,) so that look like.

enabled = 1
host = localhost
port = 3306
database_name = ViewVC
user = VIEWVC
passwd = Password123
readonly_user = VIEWVCRO
readonly_passwd = Password456

Then get ViewVC to ‘rebuild’ the database .

[box]

/usr/local/viewvc-1.1.24/bin/cvsdbadmin rebuild /usr/local/rancid/var/CVS/CVSROOT/

[/box]

Reboot the server, and if you browse to http://{ip-or-hostname}/viewvc you should see something like this;

 

In Part Two we will add some Cisco Devices (Switches and ASA Firewalls) and schedule the backups.

Related Articles, References, Credits, or External Links

NA