DHCP Scope: Full of BAD_ADDRESS Entries

KB ID 0001651

Problem

I had a client machine struggling to get an DHCP address, and when I looked in DHCP the scope it was full of this;

BAD_ADDRESS This address Is Already in Use

Solution

A tour of Google and forums is full of posts by people with this problem, and other than, ‘Oh I looked in the logs and fixed it’ (with no mention of what log, or where this log was), or ‘Yeah I used Wireshark and located a problem client‘, then no follow up on what they did, or scanned for. So I pretty much had to slog through and work it out for myself. I’ll detail each step I took below, most of which didn’t help, or sent me in the wrong direction, but for you that may be a better solution.

And I will give you enough information to at least be helpful!

Firstly Common Sense Check: If this has just happened what have you changed? Have you added any Wireless Controllers, or Access Points? Have you deployed any new Switches or Firewalls. In my case, it was my test network so it could have been happening for months!

The most frequent cause of this error is simply because someone has setup another DHCP server on the network. That will be easy to diagnose, simply ‘Stop’ your DHCP Server;

Then on a DHCP client, issue an ifconfig /release and ifconfig /renew, If it gets an IP address, issue an ifconfig /all and look for the IP of the DHCP server it’s using, that’s your culprit. However as you can see, mine didn’t get an IP address so this wasn’t my problem.

The next most popular suggestion is to enable ‘Conflict Detection‘, though in most places the information on where to find this, is incorrect, (as it’s been copied and pasted around the forums without actually checking it!) See below, you locate it on the properties of the Protocol not the Server > Advanced Tab > You are supposed to set it between 1 and 6 so I went for 5, (but after deleting all the BAD_ADDRESS entries, they were all back after 30 minutes or so, so this didn’t work for me either).

Look in the logs: Well they were useless also, DHCP creates a new log every day in C:\Windows\System32\Dhcp called ‘DHCPSrvLog-DAY.log’ as you can see it was not helpful.

At this point I put my networking head on, and ‘thought outside the box’, If DHCP is detecting these as BAD ADDRESSES, then they must be in the arp cache on the DHCP server right? Well look at this;

[box]

arp -a

[/box]

Well that’s encouraging, at least now I’ve got a suspect MAC address, lookup up that MAC address online, and it comes back as VMWare (which sent me off in the wrong direction, it was not a VMware virtual machine in my vSphere in the end). Ive got a decent Cisco Switch so I thought I’d see which interface it was connected to, (but it wasn’t there).

[box]

show mac address-table

[/box]

At this point I was still thinking it was a VMware virtual machine, so I used PowerCLI (Thats PowerShell for VMware), to query for that MAC address, but that revealed nothing.

So, my last hope was Wireshark, I fired it up on the DHCP server, and set the filter to;

[box]

bootp.option.type == 53

[/box]

Then I deleted all the BAD_ADDRESS entries, left Wireshark ‘sniffing’, and went for lunch. I returned to this (see below). Now 192,168,100,107 was one of the BAD_ADDRESS entries, and I did not know what it was. The other entries on there for 192.168.100.3 are understandable, (that’s my DHCP server!) So now I had a Layer 3 address to hunt.

When I RDP connected to it, I got prompted for a password, so now I know it’s a Windows box! I hunted all through my VMware virtual machines, it was not there. Just as an afterthought I remembered I have a Hyper-V server, could that be running a virtual machine? BOOM! There is a SCVMM server, I was using for some Zerto testing a couple of months ago! Turned it off, problem solved!

Hope you find your culprit quicker than I did!

Related Articles, References, Credits, or External Links

NA

Veeam Backup Error – ‘Unable to release guest. Error: Unfreeze error:’

KB ID 0000763 

Problem

Yesterday morning, I walked into the office, the boss told me a client’s Exchange was running slowly and they had had a Veeam backup fail. I know this client well enough to know if it was something simple he would have fixed it himself, so while my laptop booted I armed myself with a coffee.

By the time I got on remotely, Ben on the help desk had also got online and was giving me the heads up on what the NAble proactive system had flagged up during the night, one drive was practically full, and it had filled up quickly.

I rang the client who told me the drive in question was the transaction log volume for Exchange, (which with failing backups would make sense). Also he had added 1500 iPad clients to the network in the last few weeks, and the transaction logs were going up by about 20GB a day. Without a good backup to flush the logs things had steadily got worse.

I connected to the Veeam backup server and this was the error.

Unable to release guest. Error: Unfreeze error: [Backup job failed. Cannot create a shadow copy of the volumes containing writer’s data. A VSS critical writer has failed. Writer name: [Microsoft Exchange Writer]. Class ID: [{76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}]. Instance ID: [{65ec880f-7b6a-402f-baf1-14d4de7f6fb9}]. Writer’s state: [VSS_WS_FAILED_AT_FREEZE]. Error code: [0x800423f2].]
Error: Unfreeze error: [Backup job failed. Cannot create a shadow copy of the volumes containing writer’s data. A VSS critical writer has failed. Writer name: [Microsoft Exchange Writer]. Class ID: [{76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}]. Instance ID: [{65ec880f-7b6a-402f-baf1-14d4de7f6fb9}]. Writer’s state: [VSS_WS_FAILED_AT_FREEZE]. Error code: [0x800423f2].]

Solution

1. OK that’s a huge error, but essentially it’s complaining about the VSS writer on the Exchange server. Log onto the Exchange server, drop to command line and issue the following command;

[box]
vssadmin list writers[/box]

Chances are you will see the following;

If you look in the Event Log you will probably also see Event ID 2007.

Information Store (2544) Shadow copy instance 1 aborted.

2. To fix that you need to restart the Microsoft Exchange information store service.

3. Check again to make sure you are back up.

4. Note: We are backing up using Veeam, make sure there is no instance of the Symantec Backup Exec Remote Agent for Exchange, if it’s there remove it.

5. Finally, I’ve got over 120GB of transaction logs to contend with, for the problem mail store, I’m going to enable circular logging to free up some room. (Note: You can disable this again once you have a decent backup if you wish).

6. At this point I rebooted both the Exchange server and the Veeam Backup server it then performed a backup of the Exchange server without error.

Update 170114

We had an issue with this again this week and the above resolution did not work. In the end we had to do the following;

1. Exchange shouldn’t be more than 20 sec.
So please kindly inspect closely the following article: http://www.veeam.com/kb1680

2. Please make sure that Exchange server isn’t running on a snapshot.

3. Troubleshoot the VSS service. i.e.

[box]

To check for unnecessary providers

vssadmin list providers

Check that there are no shadow copies running

vssadmin list shadows

Check the writers state. Probably there will be a Failed/Timed out writer. 

vssadmin list writers

[/box]

4. To get a failed writer to a normal state restart following services: “COM+ Event System”, “Microsoft Software Shadow Copy Provider”, “Volume Shadow Copy”.

5. Next you can manually create a shadow copy of an Exchange db volume:  (Note: This assumes Exchange is on C:).

[box]vssadmin create shadow /for=c:[/box]

6. Manual creation should report that is completed successfully.

7. Delete the shadow copy that you have just created.

[box]vssadmin delete shadows /for=c: /all [/box]

8. Finally make sure all shadows have been removed with the following command

[box]vssadmin list shadows[/box]

9. Attempt to re-run the backups

Related Articles, References, Credits, or External Links

Thanks to Steve Morrison and Dimitri from Veeam

Veeam Backup and Recovery Download

Veeam Availability Suite Download

 

Exchange 2016 (&2013) – How To Move a Mailbox Database

KB ID 0000725 

Problem

I like to keep Exchange databases in their own volume, so one of the first things I do is move them.

Warning: Moving a mail database will make it unavailable while it is being moved, warn your users, or plan in some down time.

Solution

1. First see where the database is now. From within the Exchange admin center (https://localhost/ecp) > Servers > Databases > Select the database to be moved.

Note: Take a note of the database path, and the database filename (filename.edb). Copy the complete path to Notepad.

2. Create a folder on your ‘target’ volume, for the database to live in.

3. Launch the Exchange Management Shell.

4. Execute the the following PowerShell command;

[box] Move-DatabasePath -Identity “Database Name” -EdbFilePath “E:Folder NameDatabase name.edb” [/box]

Answer Y to the questions (or A for all).

Process on Exchange 2016

Process on Exchange 2013 (same)

5. Now you can check that the database has mounted, and is in its new location.

 

Related Articles, References, Credits, or External Links

NA