macOS – SSH Error ‘No Matching Exchange Method Found’

Posted on 22/10/2023 by PeteLong

138

Mac SSH Error KB ID 0001245

Problem

Certified working all the way up to macOS Ventura version 13.6
Certified working all the way up to macOS Sonoma version 14.1

I thought my RoyalTSX had broken today, I upgraded it a couple of weeks ago, and I upgraded to macOS Catalina 10.15 the other day. After this, all my SSH sessions refused to connect with this error;

Unable to negotiate with x.x.x.x port 22: no matching key exchange found. Their offer diffie-hellman-group1-sha1

Note: You may also see the following error;

Unable to negotiate with x.x.x.x port 22: no matching cipher found. Their offer: aes128-cbc,3des-cbc,aes192-cbc,aes256-cbc

Update: 10/04/20: With newer equipment you may see the following error;

Unable to negotiate with x.x.x.x port 22: no matching MAC found Their offer: hmac-sha2-256

Update 08/12/21: With newer Cisco equipment you may see the following error;

Unable to negotiate with x.x.x.x.1 port 22: no matching key exchange method found. Their offer: diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1

Mac SSH Error – Fix

WARNING: My usual fix for this is to edit the macs ssh_config file directly and allow the older (less secure) connection methods. It has been (correctly) pointed out, that this is the ‘least preferred‘ method, as it will allow these less sure methods for ALL SSH connections. The preferred fix, is to create a new file in your own home folder, with the connection settings for EACH SSH CONNECTION. However, I manage hundreds of firewalls. routers and switches so that method is a little unwieldy for me. But I will include it at the end of the article, for the more security conscious of you.

Not Apple’s fault to be fair, it’s OpenSSH version 7. SHA1 is weak, so support for it has been removed. Which is fine, but all my clients Cisco Firewalls, Routers, Switches are probably all using RSA/SHA1. So until they re all updated I’m going to need to re-enable SHA1.

Open a terminal windows and execute the following;

[box]

sudo nano /etc/ssh/ssh_config
ENTER YOUR PASSWORD

[/box]

Locate the line ‘ # MACs hmac-md5,hmac-sha1,hmac-sha2-256,umac-64@openssh.com,hmac-ripemd160′ and remove the Hash/Pound sight from the beginning, and add the extra hashing algorithm that I’ve shown above in red.

Locate the line ‘ # Ciphers aes128-ctr,aes192-ctr,aes256-ctr,aes128-cbc,3des-cbc’ and remove the Hash/Pound sight from the beginning.

Then paste the following on the end;

HostkeyAlgorithms ssh-dss,ssh-rsa

KexAlgorithms +diffie-hellman-group1-sha1,diffie-hellman-group14-sha1

Like so;

Mac SSH Error – Quitting Nano

To quit nano, use the Ctrl-X key combination. because you are working on has been modified since the last time you saved it, you will be prompted to save the file first. Type y to save the file.

Theres no reason to reboot, it should work straight away.

But Wait – Theres More!

It’s going to happen every time you upgrade your mac, I’ve started taking a backup of the ssh_config file, then I can just restore it back again, like so;

Backup macOS SSH Settings & Ciphers

[box]

sudo cp /etc/ssh/ssh_config /etc/ssh/ssh_config.bak

[/box]

Restore macOS SSH Settings & Ciphers

[box]

sudo rm /etc/ssh/ssh_config
sudo mv /etc/ssh/ssh_config.bak /etc/ssh/ssh_config
sudo cp /etc/ssh/ssh_config /etc/ssh/ssh_config.bak

[/box]

Mac SSH Error (The More Secure & Preferred) Fix

Using this method you will create a connection (entry) for each asset you want to connect to. This will be saved in your Home directory so if multiple people use the Mac you will need to do this for each user.

Firstly create a config file in you home directory.

[box]

sudo nano ~/ssh/config

[/box]

What this does is create a blank file that you can then create an entry for each of the assets you want to connect to. Firstly specify the hostname, then the settings for that particular asset, like so:

[box]

Host YOUR-SWITCH
User YOUR-USER
port 22
KexAlgorithms +diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
Ciphers +aes128-cbc,3des-cbc,aes192-cbc,aes256-cbc
PubkeyAcceptedAlgorithms +ssh-rsa
HostkeyAlgorithms +ssh-rsa

[/box]

Note: Simply add new host entries at a new line in the SAME config file.

Use Azure MFA With Microsoft NPS (RADIUS) Server

Posted on 20/07/2021 by PeteLong

KB ID 0001759

Problem

I was in a forum last week and someone asked, “Can I enable Azure MFA, on my RADIUS server, to secure access to my switches and routers etc”. It turns out if you want to enable Azure MFA with Microsoft NPS it’s actually quite simple.

So, I’m using RADIUS auth (above) on my NPS server, and it’s simply checking the authenticating user is a member of a domain security group. Once it has satisfied that requirement, it will authenticate against my Azure AD, which will trigger an MFA event, (in my case send a request to the Microsoft Authenticator Application on my Android Phone).

Azure MFA With Microsoft NPS Pre-Requisites

The remote user needs EITHER an Azure P1 License, or a Microsoft 365 license.

“But I can use the Authenticator App with my Office 365 subscription?”

Well yes you can, but we are not authenticating to office 365 are we?

Below you can prove the licence is allocated in Office 365

And the same in Azure AD.

Now your user needs to have MFA enabled, (this should be pretty obvious), to use the Microsoft authenticator application the USER chooses that method of authentication, when you enable MFA for them (the first time they login). You can re-force that, from the following screen if you wish.

Azure MFA With Microsoft NPS: Deploying NPS

So I’ve pretty much covered this half a dozen times before, but for completeness I’ll quickly run though setting up NPS / NPAS. The quickest simplest method is to use PowerShell.

[box]

Install-WindowsFeature NPAS -IncludeManagmentTools

[/box]

From administrative tools open > Network Policy Server >Right click (Top Level) > Register Server in Active Directory > OK > OK

Execute the following PowerShell command to create a registry key

[box]

New-Item 'HKLM:\SOFTWARE\Microsoft\AzureMfa' -Force | New-ItemProperty -Name REQUIRE_USER_MATCH -Value TRUE -Force | Out-Null

[/box]

Enable NPS RADIUS on Windows Firewall

Now for some reason installing NPS does not open the correct ports on the Windows Firewall? So issue the following command;

[box]

Get-NetFirewallRule -DisplayGroup "Network Policy Server" | where DisplayName -like "*RADIUS*" | Set-NetFirewallRule -Service Any

[/box]

Azure MFA With Microsoft NPS: Domain (on Premises and Azure AD)

You will need to know what your Azure Tenant ID is, keep a copy of this handy either in notepad or on the clipboard because you will need it in a minute.

Below you can see I’ve got my domain user, their remote access (Dial In Tab) is set to control access though policy, and I’ve placed them in a security group called SG-Azure-MFA.

Configure NPS for RADIUS Access

Note: You may already have this configured, if so please skip to the next section.

The first task is to define the RADIUS CLIENT, in my case it will be a Cisco firewall, yours could be any device that requires RADIUS authentication. Locate REDIUS Clients > New > Provide a ‘Friendly Name’ (REMEMBER WHAT IT IS) > Enter its IP address > Then provide and confirm a shared secret (think of it like a password, you will need to add this to the radius clients config) > OK

Policies > Network Policies > New > Give it a sensible name > Next.

Add in a ‘Condition‘ for User Group, then add in the user group you created/used above.

Add in another ‘Condition‘ > Set the friendly name to the one you used when you created your RADIUS client.

Accepts all the defaults until you get to Configure Authentication Methods > Tick ‘Unencrypted Authentication (PAP, SPAP)’> Click yes if you want to read the warning > Next > Accept all the defaults from this point forward.

Enable Azure MFA With Microsoft NPS

Download the ‘NPS Extension For Azure MFA‘ software form Microsoft, and install it on your NPS server.

To actually enable it against your Azure AD, Execute the following PowerShell commands;

[box]

cd "c:\Program Files\Microsoft\AzureMfa\Config"
.\AzureMfaNpsExtnConfigSetup.ps1

[/box]

Eventually you will be asked to authenticate to Azure, do so with an administrative account.

You will be asked to provide your Azure Tennant ID.

When complete REBOOT THE NPS SERVER!

Testing Azure MFA With NPS

Again for Cisco ASA I’ve already blogged about this, but for completeness here’s me making sure it works;

Remember to RAISE the RADIUS timeout, by default its 10 seconds, I raised it to 30 seconds.

And on my phone I get prompted to allow

Authentication successful!

Troubleshooting (NPS Azure MFA Not Working)

Event ID 6274: The Request Was Discarded by a third-party extension DLL file.

This happens when the user you are authenticating does not have the correct license in Azure (or you have just allocated the license and have not waited for a while).

Full Error

[box]

Log Name:      Security
Source:        Microsoft-Windows-Security-Auditing
Date:          15/07/2021 16:42:58
Event ID:      6274
Task Category: Network Policy Server
Level:         Information
Keywords:      Audit Failure
User:          N/A
Computer:      PKI-02.pnl.com
Description:
Network Policy Server discarded the request for a user.

Contact the Network Policy Server administrator for more information.

User:
	Security ID:			PNL\tanya.long
	Account Name:			tanya.long
	Account Domain:			PNL
	Fully Qualified Account Name:	pnl.com/PNL/Users/Tanya Long

Client Machine:
	Security ID:			NULL SID
	Account Name:			-
	Fully Qualified Account Name:	-
	Called Station Identifier:		-
	Calling Station Identifier:		-

NAS:
	NAS IPv4 Address:		192.168.254.254
	NAS IPv6 Address:		-
	NAS Identifier:			-
	NAS Port-Type:			Virtual
	NAS Port:			6

RADIUS Client:
	Client Friendly Name:		Firewall
	Client IP Address:			192.168.254.254

Authentication Details:
	Connection Request Policy Name:	Use Windows authentication for all users
	Network Policy Name:		NP-Azure-MFA
	Authentication Provider:		Windows
	Authentication Server:		PKI-02.pnl.com
	Authentication Type:		PAP
	EAP Type:			-
	Account Session Identifier:		-
	Reason Code:			9
	Reason:				The request was discarded by a third-party extension DLL file.

[/box]

Event ID 6273: An NPS extension dynamic link library (DLL) that is installed on the NPS server rejected the connection

In my case I had re-install the NPS Azure extension.

Full Error

[box]

Log Name:      Security
Source:        Microsoft-Windows-Security-Auditing
Date:          15/07/2021 17:24:39
Event ID:      6273
Task Category: Network Policy Server
Level:         Information
Keywords:      Audit Failure
User:          N/A
Computer:      PKI-02.pnl.com
Description:
Network Policy Server denied access to a user.

Contact the Network Policy Server administrator for more information.

User:
	Security ID:			NULL SID
	Account Name:			tanya.long
	Account Domain:			PNL
	Fully Qualified Account Name:	PNL\tanya.long

Client Machine:
	Security ID:			NULL SID
	Account Name:			-
	Fully Qualified Account Name:	-
	Called Station Identifier:		-
	Calling Station Identifier:		-

NAS:
	NAS IPv4 Address:		192.168.254.254
	NAS IPv6 Address:		-
	NAS Identifier:			-
	NAS Port-Type:			Virtual
	NAS Port:			10

RADIUS Client:
	Client Friendly Name:		Firewall
	Client IP Address:			192.168.254.254

Authentication Details:
	Connection Request Policy Name:	Use Windows authentication for all users
	Network Policy Name:		-
	Authentication Provider:		Windows
	Authentication Server:		PKI-02.pnl.com
	Authentication Type:		Extension
	EAP Type:			-
	Account Session Identifier:		-
	Logging Results:			Accounting information was written to the local log file.
	Reason Code:			21
	Reason:				An NPS extension dynamic link library (DLL) that is installed on the NPS server rejected the connection request.

[/box]

Cisco Catalyst 9200 / 9300 DNA Licensing

Posted on 06/05/2021 by PeteLong

KB ID 0001750

Problem

I get asked this at least once a month, “What’s the score with this DNA Licensing?” It took long enough for everyone to get used to Lan Base, IP Base, and IP Services!

The cynic in me would say, Cisco have learned from Meraki that selling subscription licences is much better than selling products that you don’t get any recurring revenue from. But I’ll try an give you the short answer so you can get the correct license.

Solution: Buying Cisco Catalyst 9K Switches

Firstly: Not sure who decided that Cisco would release 9000 series Catalyst switches, when they had 9000 series Nexus switches? (Thanks for that!)

Catalyst 9200 or 9300?

As a rule of thumb 9200 series are typically used as access switches i.e. replacements for things like the Catalyst 2960, 2960-X, and 2960-XR). And the 9300 series are a replacement for things like Cisco Catalyst 3750G, 3750-X, and 3850.

Note: There’s also a Catalyst 9400 switch, which is a modular (line card) based chassis switch to replace the Catalyst 4500 and 6000 series.
Note2: There’s also a Catalyst 9500 switch that replaced the 10Gbps catalyst 3850 models (traditional 1U size).
Note3: There’s also a Catalyst 9600 switch which is modular (line card) based chassis switch to replace the Catalyst 6000 Series.
Note4: There’s also a Catalyst 9800 series which, just to confuse everyone further, is a range of wireless controllers?

So which switch to buy? Cisco keep adding models to both ranges so the first thing to do is decide 9200 or 9300, then look at the current Cisco Data Sheet for that range.

9200 Series Data Sheet

9300 Series Data Sheet

Then decide

How many ports (access/downlink) do you need, and what speed/type do they need to be?
Do you need PoE?
What uplink ports do you need? (Some models have fixed (built in) uplinks, others need a network module (modular) uplink. Remember modular uplinks have their own part number (SKU), and will need to be ordered separately. (Note: 9200L and 9300L have fixed uplinks)
Do you need additional (redundant) power supplies?
Do you need to ‘Stack’ your switches, if so don’t forget to get a stack cable (theres no separate stacking modules).

They were cheaper than you expected right?

That’s because now we need to add on a DNA licence as well.

DNA Licensing

Cisco DNA (Digital Network Architecture) is the name given collectively to a suite of products that are aimed towards being software driven, automated, with built in security.

There’s three types;

DNA Essentials: (Lan Base in old money) Basically Layer 2 functionality and static routing.
DNA Advantage: (Combines IP Base and IP Services in old money) Basically full Layer 3 functionality, (and all the functionality of DNA Essentials).
DNA Premier: Combines all the functionality of DNA Essentials and DNA Advantage, and adds on ISE integration and Cisco Secure Network Analytics (formerly Stealthwatch) support.

Each licence comes in either a 3 Year, 5 Year, or 7 Year subscription model.

Example DNA Licensing SKU: C9200- DNA-E-24-3Y

C9200 – for a Cisco Catalyst 9200 series switch.

DNA – Digital Network Architecture licence.

E – Essentials (A would be advantage, and P would be premier).

24 – For a 24 port switch

3Y – 3 Year Subscription

Cisco: Getting a SKU (Product ID) From a Serial Number

Posted on 28/04/2020 by PeteLong

KB ID 0001674

Problem

I had a situation a couple of weeks ago where I had the serial numbers for a bunch of Cisco switches, I needed to get some extended cover for them, but what I didn’t have were the Cisco SKU (Stock Keeping Unit) codes.

Solution

You will need to have a Cisco CCO login, once you have that go here > Add devices.

Give the device a name, (it does not matter what) > Paste in the serial number > Add.

Boom, there’s your SKU (Product ID)

Repeat as required.

Configuring Cisco HSRP

Posted on 08/02/2020 by Migrated

KB ID 0000946

Problem

Cisco HSRP: Normally your client machines have one route off the network, (their default gateway). But what if that goes down? HSRP aims to solve this problem by assigning a ‘Virtual IP address’ to your default gateway (or default route). So that IP can be shared amongst two or more possible devices (routers, or layer 3 switches).

Above, we have a client 192.168.1.10 that has two possible routes off the network, (.254 and .253). We will setup a virtual IP of .250 and both routers can use that IP, (if they are the active gateway). Below is a brief overview of how to set it up.

Deploy Cisco HSRP

Setting up Cisco HSRP

1. On the first router (Router0), add the standby IP address (192.168.1.250) the ‘1’ denotes the standby group (a number from 0 to 4096). It comes up as standby, then after it has checked (via multicast address 224.0.0.2 on UDP port 1985). It finds no other live HSRP devices using that IP address so it becomes ‘Active’.

[box]

Router0>enable
Router0#configure terminal 
Enter configuration commands, one per line. End with CNTL/Z.
Router0(config)#interface GigabitEthernet0/0
Router0(config-if)#standby 1 ip 192.168.1.250
Router0(config-if)#

%HSRP-6-STATECHANGE: GigabitEthernet0/0 Grp 1 state Speak -> Standby

%HSRP-6-STATECHANGE: GigabitEthernet0/0 Grp 1 state Standby -> Active

Router0(config-if)#

[/box]

2. Repeat this on the second Router, this one discovers the ‘Active’ router and sets itself up as ‘Standby’.

[box]

Router1>
Router1>enable
Router1#configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
Router1(config)#interface GigabitEthernet0/0
Router1(config-if)#standby 1 ip 192.168.1.250
Router1(config-if)#
%HSRP-6-STATECHANGE: GigabitEthernet0/0 Grp 1 state Speak -> Standby

Router1(config-if)#

[/box]

3. You can prove this by running show standby (or do show standby in configure terminal mode).

[box]

Router0

Router0#show standby
GigabitEthernet0/0 - Group 1 (version 2)
State is Active
8 state changes, last state change 00:02:02
Virtual IP address is 192.168.1.250
Active virtual MAC address is 0000.0C9F.F001
Local virtual MAC address is 0000.0C9F.F001 (v2 default)
Hello time 3 sec, hold time 10 sec
Next hello sent in 0.055 secs
Preemption disabled
Active router is local
Standby router is 192.168.1.253
Priority 100 (default 100)
Group name is hsrp-Gig0/0-1 (default)
Router0#

Router1

Router1#show standby
GigabitEthernet0/0 - Group 1 (version 2)
State is Standby
3 state changes, last state change 00:10:44
Virtual IP address is 192.168.1.250
Active virtual MAC address is unknown
Local virtual MAC address is 0000.0C9F.F001 (v2 default)
Hello time 3 sec, hold time 10 sec
Next hello sent in 2.125 secs
Preemption disabled
Active router is 192.168.1.254
Standby router is local
Priority 100 (default 100)
Group name is hsrp-Gig0/0-1 (default)
Router1#

[/box]

4. That is HSRP configured! However there are a few changes you might want to make, for example, what if one router had a 100MB leased line, and the other was a 2MB ADSL line, you would want the fastest one to be in use, (as long as it was up). To achieve that, give the router with the fastest connection a higher priority (you may notice above, that by default the priority is 100). Be aware, even if a router has the highest priority, it wont ‘seize’ the virtual IP, it just sits and waits until it’s available. For our 100MB and 2MB example that’s not good. We would want Router1 to seize the virtual IP as soon as it can. To do that we need to set it to preempt. (Note: This process is called ‘launching a coup’).

[box]

Router1(config)#interface GigabitEthernet0/0
Router1(config-if)#standby 1 priority 105
Router1(config-if)#standby 1 preempt

[/box]

5. At this point it’s important to say, that in our scenario we would also need to setup a virtual IP for the ‘other side’ of the routers (i.e their GigabitEthernet 0/1 interfaces), or the remote client (172.16.1.10) would not be able to return our ‘pings’ or get any traffic back to us. So lets setup a virtual HSRP address on that side as well. Notice I just use another standby group number.

Note: To work the remote host 172.16.1.10 will need its default gateway changing to the HSRP Virtual IP of 172.16.1.250.

[box]

Router0 

Router0(config)#interface GigabitEthernet0/1
Router0(config-if)#standby 2 ip 172.16.1.250
Router0(config-if)#

Router1

Router1(config)#interface GigabitEthernet0/1
Router1(config-if)#standby 2 ip 172.16.1.250
Router1(config-if)#

[/box]

6. Finally we have set Router0 with the highest priority and set it to seize the virtual IP as soon as it can. But what if another interface on Router1 goes down? e.g. If the GigabitEthernet 0/1 interface were to go down, HSRP would not do anything because it’s tracking both the GigabitEthernet 0/0 interfaces, so communications would fail.

To solve the problem we need to tell it which interfaces to ‘Track’. In our example we need to track GigabitEthernet 0/1, if that goes down we need to give the virtual IP address to the standby ‘router’. This works because once we tell it to ‘track’ the GigabitEthernet 0/1 interface, if that were to fail it will DECREMEMT the routers priority by 10. So for Router0 its priority would drop to 95, this is five less than the default value of 100 (on Router1). But Remember, at the moment that fail-over would still fail, unless you allow Router1 to preempt and launch a coup.

[box]

Router0 

Router0(config)#interface GigabitEthernet0/0
Router0(config-if)#standby 1 track GigabitEthernet0/1
Router0(config-if)#

Router1

Router1(config)#interface GigabitEthernet0/0
Router1(config-if)#standby 1 preempt
Router1(config-if)#

[/box]

7. We can see that by running a ‘show standby’ on Router0.

[box]

Router0#show standby
GigabitEthernet0/0 - Group 1 (version 2)
State is Active
7 state changes, last state change 00:00:31
Virtual IP address is 192.168.1.250
Active virtual MAC address is 0000.0C9F.F001
Local virtual MAC address is 0000.0C9F.F001 (v2 default)
Hello time 3 sec, hold time 10 sec
Next hello sent in 2.81 secs
Preemption enabled
Active router is local
Standby router is 192.168.1.253, priority 115 (expires in 7 sec)
Priority 115 (configured 115)
Track interface GigabitEthernet0/1 state Up decrement 10
Group name is hsrp-Gig0/0-1 (default)
GigabitEthernet0/1 - Group 2 (version 2)
State is Active
6 state changes, last state change 00:00:28
Virtual IP address is 172.16.1.250
Active virtual MAC address is 0000.0C9F.F002
Local virtual MAC address is 0000.0C9F.F002 (v2 default)
Hello time 3 sec, hold time 10 sec
Next hello sent in 0.81 secs
Preemption disabled
Active router is local
Standby router is 172.16.1.253
Priority 100 (default 100)
Group name is hsrp-Gig0/1-2 (default)
Router0#

[/box]

DHCP Scope: Full of BAD_ADDRESS Entries

Posted on 05/02/2020 by PeteLong

KB ID 0001651

Problem

I had a client machine struggling to get an DHCP address, and when I looked in DHCP the scope it was full of this;

BAD_ADDRESS This address Is Already in Use

Solution

A tour of Google and forums is full of posts by people with this problem, and other than, ‘Oh I looked in the logs and fixed it’ (with no mention of what log, or where this log was), or ‘Yeah I used Wireshark and located a problem client‘, then no follow up on what they did, or scanned for. So I pretty much had to slog through and work it out for myself. I’ll detail each step I took below, most of which didn’t help, or sent me in the wrong direction, but for you that may be a better solution.

And I will give you enough information to at least be helpful!

Firstly Common Sense Check: If this has just happened what have you changed? Have you added any Wireless Controllers, or Access Points? Have you deployed any new Switches or Firewalls. In my case, it was my test network so it could have been happening for months!

The most frequent cause of this error is simply because someone has setup another DHCP server on the network. That will be easy to diagnose, simply ‘Stop’ your DHCP Server;

Then on a DHCP client, issue an ifconfig /release and ifconfig /renew, If it gets an IP address, issue an ifconfig /all and look for the IP of the DHCP server it’s using, that’s your culprit. However as you can see, mine didn’t get an IP address so this wasn’t my problem.

The next most popular suggestion is to enable ‘Conflict Detection‘, though in most places the information on where to find this, is incorrect, (as it’s been copied and pasted around the forums without actually checking it!) See below, you locate it on the properties of the Protocol not the Server > Advanced Tab > You are supposed to set it between 1 and 6 so I went for 5, (but after deleting all the BAD_ADDRESS entries, they were all back after 30 minutes or so, so this didn’t work for me either).

Look in the logs: Well they were useless also, DHCP creates a new log every day in C:\Windows\System32\Dhcp called ‘DHCPSrvLog-DAY.log’ as you can see it was not helpful.

At this point I put my networking head on, and ‘thought outside the box’, If DHCP is detecting these as BAD ADDRESSES, then they must be in the arp cache on the DHCP server right? Well look at this;

[box]

arp -a

[/box]

Well that’s encouraging, at least now I’ve got a suspect MAC address, lookup up that MAC address online, and it comes back as VMWare (which sent me off in the wrong direction, it was not a VMware virtual machine in my vSphere in the end). Ive got a decent Cisco Switch so I thought I’d see which interface it was connected to, (but it wasn’t there).

[box]

show mac address-table

[/box]

At this point I was still thinking it was a VMware virtual machine, so I used PowerCLI (Thats PowerShell for VMware), to query for that MAC address, but that revealed nothing.

So, my last hope was Wireshark, I fired it up on the DHCP server, and set the filter to;

[box]

bootp.option.type == 53

[/box]

Then I deleted all the BAD_ADDRESS entries, left Wireshark ‘sniffing’, and went for lunch. I returned to this (see below). Now 192,168,100,107 was one of the BAD_ADDRESS entries, and I did not know what it was. The other entries on there for 192.168.100.3 are understandable, (that’s my DHCP server!) So now I had a Layer 3 address to hunt.

When I RDP connected to it, I got prompted for a password, so now I know it’s a Windows box! I hunted all through my VMware virtual machines, it was not there. Just as an afterthought I remembered I have a Hyper-V server, could that be running a virtual machine? BOOM! There is a SCVMM server, I was using for some Zerto testing a couple of months ago! Turned it off, problem solved!

Hope you find your culprit quicker than I did!

Cisco Catalist Upgrading 2900, 5500 and 3700 Stacks

Posted on 23/12/2019 by PeteLong

KB ID 0001630

Problem

People are often nervous about doing this, I’m not sure why because Cisco have made it painfully simple now. That’s because instead of the old /bin files we used to use, you can now upgrade a switch (or a switch stack) using a .tar file with one command, (and it will also upgrade all the stack members and the firmware on any other network modules you have in the switches at the same time).

Yes it does take a while*, and for long periods of time theres no updated output on the screen, which is worrying if you’ve never done it before.

*Note: The procedure below was updating two 2960-X switches and took about 45-50 minutes. If anyone wants to post any further timings below as a help to others, state the switch types and quantities, and versions you used, etc.

Solution

First things first, BACK UP YOUR SWITCH CONFIG. I also have a habit of copying out the original .bin file from the flash to my TFTP server as an extra ‘belt and braces’ precaution, in case everything ‘Goes to hell in a hand cart!’

I find it easier to do this with the update file on a USB Drive, (format the drive as Fat32). If you dont have a USB Drive, or the switch does not have a working USB port then don’t panic, you can use ftp or tftp to upgrade also.

Place your new upgrade .tar file on your USB Drive and insert it into the master switch, you should see the following;

[box]

Dec 19 13:13:18.466: %USBFLASH-5-CHANGE: usbflash0 has been inserted!

[/box]

Note: If yours says usbflash1, or usbflash2 etc. Then that’s just the switch numbering in the stack, use the number it tells you!

Make sure the switch can see your upgrade file;

[box]

Petes-Switch# dir usbflash1:
Dec 19 16:56:45.712: %USBFLASH-5-CHANGE: usbflash0 has been inserted!

Directory of usbflash0:/
 -rw- 37488640 Nov 25 2019 10:08:34 +00:00 c2960x-universalk9-tar.152-7.E0a.tar

8036286464 bytes total (7997743104 bytes free)

[/box]

You can execute the entire upgrade with this one command;

[box]

Petes-Switch# archive download-sw /overwrite usbflash0:/c2960x-universalk9-tar.152-7.E0a.tar

[/box]

Note: If using tftp then use archive download-sw /overwrite tftp:/{ip-of-tftp-server}/{image-name}.tar instead.

It will take quite a long time, as soon as it says extracting xyz….go and have a coffee, wait until it says ‘All software images installed.’

[box]

---LOTS OF OUTPUT OMITTED FOR THE SAKE OF BREVITY---
New software image installed in flash2:/c2960x-universalk9-mz.152-7.E0a
Deleting old files from dc profile dir "flash:/dc_profile_dir"
extracting dc profile file from "flash:/c2960x-universalk9-mz.152-7.E0a/dc_default_profiles.txt" to "flash:/dc_profile_dir/dc_default_profiles.txt"
Deleting old files from dc profile dir "flash2:/dc_profile_dir"
extracting dc profile file from "flash2:/c2960x-universalk9-mz.152-7.E0a/dc_default_profiles.txt" to "flash2:/dc_profile_dir/dc_default_profiles.txt"
All software images installed.

[/box]

Now let’s do a couple of checks just for our ‘peace of mind‘, first make sure the images are in all the relevant switches flash storage;

[box]

Petes-Switch#dir flash1:
Directory of flash:/

    2  -rwx        5486  Dec 19 2019 16:55:40 +00:00  private-config.text
    3  -rwx          33   Aug 7 2019 08:28:12 +00:00  pnp-tech-time
    4  -rwx       11114   Aug 7 2019 08:28:14 +00:00  pnp-tech-discovery-summary
    5  -rwx        3096  Dec 19 2019 16:55:40 +00:00  multiple-fs
  699  drwx         512  Dec 19 2019 17:35:25 +00:00  c2960x-universalk9-mz.152-7.E0a
  480  drwx         512  Dec 19 2019 17:35:28 +00:00  dc_profile_dir
  696  -rwx         796   Aug 9 2019 09:48:30 +00:00  vlan.dat
  698  -rwx        7539  Dec 19 2019 16:55:40 +00:00  config.text

122185728 bytes total (84392960 bytes free)
Petes-Switch#dir flash2:
Directory of flash2:/

    2  -rwx        5486  Dec 19 2019 16:55:40 +00:00  private-config.text
    3  -rwx          33   Aug 7 2019 08:32:38 +00:00  pnp-tech-time
    4  -rwx       11126   Aug 7 2019 08:32:40 +00:00  pnp-tech-discovery-summary
    5  -rwx        7539  Dec 19 2019 16:55:40 +00:00  config.text
    6  drwx         512  Dec 19 2019 17:35:26 +00:00  c2960x-universalk9-mz.152-7.E0a
  481  drwx         512  Dec 19 2019 17:35:28 +00:00  dc_profile_dir
  696  -rwx        3096   Aug 8 2019 10:21:29 +00:00  multiple-fs
  697  -rwx         796  Dec 11 2019 10:55:22 +00:00  vlan.dat
  698  -rwx        7514  Dec 19 2019 16:55:40 +00:00  config.text.backup
  699  -rwx        5486  Dec 19 2019 16:55:40 +00:00  private-config.text.backup

122185728 bytes total (84378624 bytes free)

[/box]

Note: Repeat for each switch in the stack, if you have further switches.

Why does it not have .tar or .bin on the end? Because it’s a folder 🙂

The let’s make sure the ‘boot variable‘ in the device is set to use the new image;

[box]

Petes-Switch# show boot
BOOT path-list      : flash:/c2960x-universalk9-mz.152-7.E0a/c2960x-universalk9-mz.152-7.E0a.bin
Config file         : flash:/config.text
Private Config file : flash:/private-config.text
Enable Break        : yes
Manual Boot         : no
Allow Dev Key         : yes
HELPER path-list    :
Auto upgrade        : yes
Auto upgrade path   :
Boot optimization   : disabled
NVRAM/Config file
      buffer size:   524288
Timeout for Config
          Download:    0 seconds
Config Download
       via DHCP:       disabled (next boot: disabled)
-------------------
Switch 2
-------------------
BOOT path-list      : flash:/c2960x-universalk9-mz.152-7.E0a/c2960x-universalk9-mz.152-7.E0a.bin
Config file         : flash:/config.text
Private Config file : flash:/private-config.text
Enable Break        : yes
Manual Boot         : no
Allow Dev Key         : yes
HELPER path-list    :
Auto upgrade        : no
Auto upgrade path   :

[/box]

All looks good save the config and reload the stack.

[box]

Petes-Switch# write mem
Petes-Switch# reload
Proceed with reload? [confirm] {Enter}

Dec 19 17:38:50.952: %SYS-5-RELOAD: Reload requested by console. Reload Reason: Reload command.

[/box]

Time for another coffee while it’s reloading the stack, when it’s back up you can check it was successful like so;

[box]

Petes-Switch# show version
---LOTS OF OUTPUT OMITTED FOR THE SAKE OF BREVITY---
Switch Ports Model                     SW Version            SW Image
------ ----- -----                     ----------            ----------
*    1 54    WS-C2960X-48TS-L          15.2(7)E0a            C2960X-UNIVERSALK9-M
     2 54    WS-C2960X-48TS-L          15.2(7)E0a            C2960X-UNIVERSALK9-M

[/box]

Cisco IOS: Ether-Channel Trunks

Posted on 08/04/2019 by PeteLong

KB ID 0001533

Problem

This is a subject that every time I need to create an Ether-Channel I end up checking beforehand, so it’s about time I wrote it up. We are combining two different things, an Ether-channel, (an aggregation of links) and a Trunk (the ability to carry many VLANS). If you are NOT from a Cisco background then you might want to read though the following post first to avoid confusion about the world ‘Trunk‘.

HP and Cisco – VLANs and Trunks Confusion!

So this is what I’m going to create;

I will combine the TWO links between the switches to act at one link (Ether-Channel). An Ether-Channel can have up to eight links.

Note: I’m only concentrating on the Ether-Channel setup so VLANs/VTP and Routing are not covered.

Solution

You can use two types of Ether-Channels PAgP (Port Aggregation Protocol), but WHY when it only works on Cisco switches. LACP (Link Aggregation control Protocol) which is supported by just about everything else, so let’s stick with that! By default a ‘Trunk’ will pass ALL VLANS, you might not want that, I’ll cover filtering VLANs a bit further down.

WARNING: If you simply connect two switches with two cables you will create a LOOP, if you have STP enabled the network will recover and block one of the links, but your colleagues will shake their heads and pull a ‘frowny face’. For that reason ‘SHUT THE PORTS DOWN BEFORE YOU CABLE / CONFIGURE THEM“.

Starting on Switch1 make sure there’s is no existing Ether-Channels configured;

[box]

SW-1#show etherchannel
                Channel-group listing:
                ----------------------

Group: 1
----------
Group state = L2
Ports: 2   Maxports = 4
Port-channels: 1 Max Port-channels = 4
Protocol:   LACP
Minimum Links: 0

[/box]

Above there is already an Ether-Channel (port-channel) on the switch (group 1) so you would have to use group 2. For arguments sake we will say I don’t have one, so I can use group 1.

Note: ‘show etherchannel summary’ is also a handy command to remember!

[box]

First shut down the uplinks, Note the syntax for the 'range interfaces', may differ from device to device, 
so use the TAB key.

SW-1#configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
SW-1(config)#interface range ethernet 0/1 - 2
SW-1(config-if-range)#shutdown

Add the ports to channel group 1, Note 'Active' denotes use LACP, (Passive also works, but one (or both) ends
should be active.)

SW-1(config-if-range)#channel-group 1 mode active
Creating a port-channel interface Port-channel 1

Create a 'trunk' with 802.1q encapsulation.

SW-1(config-if-range)#switchport trunk encapsulation dot1q
SW-1(config-if-range)#switchport mode trunk

[/box]

Then configure the other end the same, (assuming the port numbers are the same!) As mentioned above you can use LACP mode ‘passive‘ but I tend to set both ends active.

Once you have both ends configured and the cables in place, enable the interfaces with a ‘no shutdown‘ command, on both ends!

[box]

SW-1(config)#interface range ethernet 0/1 - 2
SW-1(config-if-range)#no shutdown

[/box]

Filtering VLANs on an Ether-Channel Trunk

Any further port-channel changes need to be done on the port-channel interface, so if you want to filter what’s allowed you simply use the following syntax;

[box]

SW-1#configure terminal
Enter configuration commands, one per line.  End with CNTL/Z.
SW-1(config)#interface Port-channel1
SW-1(config-if)#switchport trunk allowed vlan 1,10,100,200

[/box]

Note: When adding any future VLANS check the syntax, if you simply add a new one it will overwrite all the others, and things will break!

Do all Ether-Channels need to be Trunks? No! Not at all, they can be access ports (as long as they are all in the same VLAN), and they can also be routed uplinks, with an IP address at both ends, (specified in the port-channel interface).

Cisco Stacking 2960-X Catalyst Switches

Posted on 05/06/2018 by PeteLong

KB ID 0001444

Problem

You can stack up to 8 2960-X Switches*, you will require the stack modules and cables, (shown below).

*Note: If you are studying for an exam, and the question is StackWize the answer is 9.

Solution

Stack Modules: Power down the switch, remove the blanking plate and fit the module, then when powered on you can use a show inventory command to make sure the module has been detected correctly.

[box]

Switch#show inventory
NAME: "1", DESCR: "WS-C2960X-48FPD-L"
PID: WS-C2960X-48FPD-L , VID: V07 , SN: FCW2213B2XX

NAME: "Switch 1 - FlexStackPlus Module", DESCR: "Stacking Module"
PID: C2960X-STACK , VID: V02 , SN: FOC221410XX

Switch#

[/box]

Setting a Stack Master: If you do nothing and simply connect up a stack, then power it on, the switches have an ‘election’ and the one with the lowest MAC address becomes the master. This is a bit bobbins, so I prefer to configure a switch to be master by increasing its priority. This is a number between 1 and 15 (Highest wins) and all switches have a priority of 1 out of the box.

I typically set the ‘top’ switch as the stack master and set its priority to 15.

[box]

Switch#configure terminal
Switch(config)#switch 1 priority 15
Changing the Switch Priority of Switch Number 1 to 15
Do you want to continue?[confirm]{Enter}	
New Priority has been set successfully

TO CONFIRM
Switch#show switch
Switch/Stack Mac Address : 5061.bf51.dd80
                                           H/W   Current
Switch#  Role   Mac Address     Priority Version  State
----------------------------------------------------------
*1       Master 5061.bf51.dd80     15     4       Ready


Switch#

[/box]

Then cable the switches together with the stack cables. If you are cabling two switches together then cable like so;

If cabling more switches , then cable stack port 1 of a switch, to port 2 of the switch below it, and keep going, then on the bottom switch cable port 1 back up to port 2 on the top switch (to make a ring).

Then power everything on, check the stack is up

[box]

Switch#show switch detail
Switch/Stack Mac Address : 5061.bf51.dd80
                                           H/W   Current
Switch#  Role   Mac Address     Priority Version  State
----------------------------------------------------------
*1       Master 5061.bf51.dd80     15     4       Ready
 2       Member 5061.bf51.de00     1      4       Ready

         Stack Port Status             Neighbors
Switch#  Port 1     Port 2           Port 1   Port 2
--------------------------------------------------------
  1        Ok         Ok                2        2
  2        Ok         Ok                1        1

[/box]

Note: You can use ‘show switch stack-ports‘ to get the second half of that output only.

Check the switch stack ring;

[box]

Switch#show switch stack-ring speed

Stack Ring Speed        : 20G
Stack Ring Configuration: Full
Stack Ring Protocol     : FlexStack

[/box]

My Ring speed is only 20G? Yes thats correct 🙂

Cisco Say it’s supposed to be 80G? Yeah, marketing piffle sorry, with two rings at full duplex it’s rated at four times the bandwidth that’s where they get this figure from.

You can now configure all the interfaces from one management IP, note to select all the interfaces use the following syntax

[box]

Switch(config)#interface range gi1/0/1-48,gi2/0/1-48

[/box]

Cisco Error ‘%PHY-4-SFP_NOT_SUPPORTED’

Posted on 13/09/2017 by PeteLong

KB ID 0001347

Problem

This is another question I see getting asked a lot in forums!

You see something like the following;

[box]

000032: *Sep 28 09:35:32.507 UTC: %PHY-4-SFP_NOT_SUPPORTED: The SFP in Gi3/0/50 is not supported (PNL-3750-Stack)
000033: *Sep 28 09:35:32.507 UTC: %PM-4-ERR_DISABLE: gbic-invalid error detected  on Gi3/0/50, putting Gi3/0/50 in err-disable state (PNL-3750-Stack)

[/box]

The usual response is ‘Enable unsupported SFP’s’, and while that sometimes is the answer, it’s not always the answer!

Solution

1. Firstly Check the Modules and the Switches, Are you tying to plug a 10GB SFP+ into a slot that only supports SFP, (that includes plugging a twinax cable into an old switch!) In your ‘show run’ you should see TenGigabitEthernet (if your using SFP+ modules). Some switches with network modules list the same interface twice (once as 10GB interfaces and once at 1GB modules, I’ve blogged about that before see THIS ARTICLE, and to confuse things even further, the four interface versions, are grouped as two pairs with each pair consisting of one SFP slot and one SFP+ slot.)

2. Make sure your cable is NOT a CAB-SFP-50CM, (unless you are connecting a 3560 to ANOTHER 3560).

3. Are you using a 2960-S? If so you may need to update the IOS to use SFP+ (assuming your model supports SFP+ not all 2960-S models do).

4. Are you plugging into a Nexus switch with a 1GB connection? If so check the other end for the following error;

Description: Gi1/1/15: This port has been disabled because Non Compliant Gigabit Interface Converter (GBIC) connector detected.

If so, you may need to Manually set the speed on the 5K to 1000, (it wont auto-sense).

5. Is it a non-cisco branded SFP? If so it may still work, (but you will get no joy if you log a TAC call) with the following commands;

[box]

Petes-SW(config)#service internal
Petes-SW(config)#no errdisable detect cause gbic-invalid
Petes-SW(config)#service unsupported-transceiver

[/box]

If you are still in doubt check the Compatibility Matrix.

Problem

Mac SSH Error – Fix

Mac SSH Error – Quitting Nano

But Wait – Theres More!

Backup macOS SSH Settings & Ciphers

Restore macOS SSH Settings & Ciphers

Mac SSH Error (The More Secure & Preferred) Fix

Related Articles, References, Credits, or External Links

Problem

Azure MFA With Microsoft NPS Pre-Requisites

Azure MFA With Microsoft NPS: Deploying NPS

Enable NPS RADIUS on Windows Firewall

Azure MFA With Microsoft NPS: Domain (on Premises and Azure AD)

Configure NPS for RADIUS Access

Enable Azure MFA With Microsoft NPS

Testing Azure MFA With NPS

Troubleshooting (NPS Azure MFA Not Working)

Event ID 6274: The Request Was Discarded by a third-party extension DLL file.

Event ID 6273: An NPS extension dynamic link library (DLL) that is installed on the NPS server rejected the connection

Related Articles, References, Credits, or External Links

Problem

Solution: Buying Cisco Catalyst 9K Switches

Catalyst 9200 or 9300?

DNA Licensing

Example DNA Licensing SKU: C9200- DNA-E-24-3Y

Related Articles, References, Credits, or External Links

Problem

Solution

Related Articles, References, Credits, or External Links

Problem

Deploy Cisco HSRP

Setting up Cisco HSRP

Related Articles, References, Credits, or External Links

Problem

Solution

Related Articles, References, Credits, or External Links

Problem

Solution

Related Articles, References, Credits, or External Links

Problem

Solution

Filtering VLANs on an Ether-Channel Trunk

Related Articles, References, Credits, or External Links

Problem

Solution

Related Articles, References, Credits, or External Links

Problem

Solution

Related Articles, References, Credits, or External Links