EVERY VMware Job in Veeam Backup and Recovery failed with the following error, (for every Guest VM);
Processing SERVER-NAME Error: Failed to open VDDK disk [[DATASTORE-NAME] SERVER-NAME/SERVER-NAME.vmdk] ( is read-only mode – [true] ) Logon attempt with parameters [VC/ESX: [ESXi-SERVER-NAME.DOMAIN-NAME.COM];Port: 443;Login: [administrator@.vsphere.local];VMX Spec: [moref=vm-231];Snapshot mor: [snapshot-288575];Transports: [san];Read Only: [true]] failed because of the following errors: @Failed to open disk for read. Failed to upload disk. Agent failed to process method {DataTransfer.SyncDisk}.
Solution
The fact that I had upgraded the vCenter from 6.5 to 6.7 the same day is probably a clue! reconnecting and rescanning the vCenter did NOT resolve the problem.
Fix: Update Veeam Backup and Recovery, (In my case to 9.5 update 4b.)
Stop and disable the Veeam Services.
Reboot the server, (or go into task manager and kill and running Veeam processes)
Mount the install ISO and carry out the upgrade.
Reboot.
Problem solved!
Related Articles, References, Credits, or External Links
It’s been such a long time since I touched any backup software, I setup Arcserve UDP this week for a client to backup their servers to a NAS drive, then they wanted to back that data off to tape for an offsite backup.
I installed Arcserve no problem, it looks much the same as it did last time I used it. When I expanded the server-name only the local drives were shown, as I only had a basic licence adding the NAS drive was not an option.
Solution
A quick call to the boys in the data center who look after all our backups pointed me in the right direction. Instead of adding the mapped drive letter, you simply add the UNC path to the share as a “Preferred Shared/Machines” source.
Related Articles, References, Credits, or External Links
Seen today while installing Veeam 6.5 Backup and Recovery on Windows Server 2008 R2
The setup was unable to install new instance of Microsoft SQL server. Make sure this server meets minimum system requirements, or try installing it manually.
Solution
1. Press Windows Key+R > appwiz.cpl {enter} > Uninstall ALL references to SQL Server and Veeam (Warning: Obviously if the server is using SQL, SQL Express or MSDE for another reason then DON’T DO THIS).
2. Download and install SQL Express (SQLEXPR_x64_ENU.exe). Run the install and select default instance (accept the default name). I set the database for mixed mode authentication but you will probably be OK with Windows authentication.
3. This time it should complete without error.
4. Now run the Veeam setup again, and it will detect the already installed instance and run through without error.
Related Articles, References, Credits, or External Links
Error
Task Failed Error: No connections could be made because the target machine actively refused it
Also when trying to connect to VMware VCenter from the Backup and Replication management console;
Error
Failed to connect to “Host-name” by SOAP, port 443, user “User-Name”, proxy srv: port:0
Unable to connect to the remote server No connections could be made because the target machine actively refused it {IP-Address}:443
Solution
There are a few things that might cause this, make sure the Veeam Server can “Ping” the VCenter and the hosts. Also make sure if you use a proxy server there is an exception for traffic going to this IP address.
In my case the problem was simply that the VMware Server service was not running on the Virtual Center.
Related Articles, References, Credits, or External Links
Yesterday morning, I walked into the office, the boss told me a client’s Exchange was running slowly and they had had a Veeam backup fail. I know this client well enough to know if it was something simple he would have fixed it himself, so while my laptop booted I armed myself with a coffee.
By the time I got on remotely, Ben on the help desk had also got online and was giving me the heads up on what the NAble proactive system had flagged up during the night, one drive was practically full, and it had filled up quickly.
I rang the client who told me the drive in question was the transaction log volume for Exchange, (which with failing backups would make sense). Also he had added 1500 iPad clients to the network in the last few weeks, and the transaction logs were going up by about 20GB a day. Without a good backup to flush the logs things had steadily got worse.
I connected to the Veeam backup server and this was the error.
Unable to release guest. Error: Unfreeze error: [Backup job failed. Cannot create a shadow copy of the volumes containing writer’s data. A VSS critical writer has failed. Writer name: [Microsoft Exchange Writer]. Class ID: [{76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}]. Instance ID: [{65ec880f-7b6a-402f-baf1-14d4de7f6fb9}]. Writer’s state: [VSS_WS_FAILED_AT_FREEZE]. Error code: [0x800423f2].]
Error: Unfreeze error: [Backup job failed. Cannot create a shadow copy of the volumes containing writer’s data. A VSS critical writer has failed. Writer name: [Microsoft Exchange Writer]. Class ID: [{76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}]. Instance ID: [{65ec880f-7b6a-402f-baf1-14d4de7f6fb9}]. Writer’s state: [VSS_WS_FAILED_AT_FREEZE]. Error code: [0x800423f2].]
Solution
1. OK that’s a huge error, but essentially it’s complaining about the VSS writer on the Exchange server. Log onto the Exchange server, drop to command line and issue the following command;
[box]
vssadmin list writers[/box]
Chances are you will see the following;
If you look in the Event Log you will probably also see Event ID 2007.
Information Store (2544) Shadow copy instance 1 aborted.
2. To fix that you need to restart the Microsoft Exchange information store service.
3. Check again to make sure you are back up.
4. Note: We are backing up using Veeam, make sure there is no instance of the Symantec Backup Exec Remote Agent for Exchange, if it’s there remove it.
5. Finally, I’ve got over 120GB of transaction logs to contend with, for the problem mail store, I’m going to enable circular logging to free up some room. (Note: You can disable this again once you have a decent backup if you wish).
6. At this point I rebooted both the Exchange server and the Veeam Backup server it then performed a backup of the Exchange server without error.
Update 170114
We had an issue with this again this week and the above resolution did not work. In the end we had to do the following;
1. Exchange shouldn’t be more than 20 sec.
So please kindly inspect closely the following article: http://www.veeam.com/kb1680
2. Please make sure that Exchange server isn’t running on a snapshot.
3. Troubleshoot the VSS service. i.e.
[box]
To check for unnecessary providers
vssadmin list providers
Check that there are no shadow copies running
vssadmin list shadows
Check the writers state. Probably there will be a Failed/Timed out writer.
vssadmin list writers
[/box]
4. To get a failed writer to a normal state restart following services: “COM+ Event System”, “Microsoft Software Shadow Copy Provider”, “Volume Shadow Copy”.
5. Next you can manually create a shadow copy of an Exchange db volume:(Note: This assumes Exchange is on C:).
[box]vssadmin create shadow /for=c:[/box]
6. Manual creation should report that is completed successfully.
7. Delete the shadow copy that you have just created.
[box]vssadmin delete shadows /for=c: /all [/box]
8. Finally make sure all shadows have been removed with the following command
[box]vssadmin list shadows[/box]
9. Attempt to re-run the backups
Related Articles, References, Credits, or External Links
Error: Client error: Traffic block header is corrupted, traffic block index ‘14537045829806812737’. Unable to retrieve next block transmission command. Number of already processed blocks: [188511].
Solution
I struggled to find any decent info on this error on the internet, so I picked up the phone and called Veeam. First line asked me to apply the latest update for version 6.5. That did not resolve the error. Their next suggestion did however.
From the Veeam Backup and Replication Console > Backup Infrastructure > Backup Proxies > Select the Proxy that you are using (I only had one) > Properties > Traffic Section > Click the blue hyperlink called ‘Network Traffic Throttling Rules’ > Untick ‘Use multiple upload streams per job’ > OK > Finish.
Then re-run the job, or allow it to run as scheduled.
Related Articles, References, Credits, or External Links
If you have a slow connection, and you are trying to replicate servers from one site to another you may struggle to do the initial replication. I’ve had an ongoing problem with a client who was trying to do this, we set it up, and the link was too slow. The client upgraded his internet connections at both sites, still the replication window would have been longer than 24 hours. In the end we chose to ‘seed’ the replication. Using this process we take a backup on the servers at the source location, then take the backup to the target location. Finally we setup the replication task and tell it to use the backup as a ‘seed’. Using this method is preferable because only the changes then get replicated over the slow link.
In the following scenario Im using Veeam 6.5 but the process is the same for Veeam 7. As a backup target Im going to host a backup repository on a Buffalo NAS Box (via iSCSI), that I can transport to the other site easily. I’ve also got a Veeam server at both locations, if you do not you may need to setup a temporary server at the source location to do the initial backup.
Because I’ve got a Veeam server at both locations I can utilise them BOTH as backup proxies, If you are only going to have a Veeam box at the target location, then I strongly suggest you setup a backup proxy on another server at the source site.
At this point I’ve added the iSCSI box as a backup repository (If you are unsure on how to do this, I do the same thing again to present the iSCSI box at the target site below.
1. Im not going to run through how to setup a simple backup job, Veeam is refreshingly easy to use.
2. So now I have the backup on my iSCSI device, I can turn it off and move the files to the target location.
Present the Backed Up files to the Veeam Server at the Target Location
3. Here I’m pointing my Veeam Server directly at the iSCSI server.
4. Now I can bring the new ‘drive’ online and make sure it gets a drive letter in Windows.
8. This Server > Populate > Select the iSCSI drive letter.
9. Browse to the folder that contains your backup data > Next.
10. I’ve already configured vPower NFS so I’ll just click Next.
11. Tick ‘Import existing backups automatically’, and ‘Import guest file system index’ > Next.
12. Finish.
How Do I Setup a Veeam ‘Seeded’ Replication Job?
13. Launch Veeam > Backup & Replication > Replication Job > Give the job a name > Tick ‘Low connection bandwidth (enable replica seeding). At this point I also want to tick the next two options so that if I need to failover the Virtual machines it will connect them to the correct VMware Port group on the target host. Also the IP addresses of the failed over machines will be changed to match the subnet of the target network > Next.
14. Add > Browse to the VM(s) you want to replicate and select them > Next.
15. Choose the host that you want to replicate the virtual machine to >Set the resource pool if you use them > Select the datastore where you will be hosting the replica files > Next.
16. Add > Locate the ‘Port Groups’ on the source and the target virtual networks. (Note: Here the port groups have the same name, they are NOT the same port group) > Next.
17. Add > Add in the IP address details from the source network and the network you will want to bring up the replicas on in the event of a failover > OK > Next.
18. Add in the source and destination proxies (make sure you have one at both ends!) > Select a local repository (this is just for the metadata not the actual replica) > Here I’m going to store seven restore points (handy because you can restore single files from a replica if you need to). DONT click Next.
19. Advanced > Traffic Tab > Set Optimize for to ‘WAN target’ > OK >Next.
20. Enable seeding and select your new repository > If you have ran the job successfully before you may have an existing replica mapping you can use, I do not > Next.
21. Enable application aware image processing (in case you ever want to restore a single file, or mail attachment, or SQL table for example) > Enter and administrative account and password > Next.
22. Set the schedule for the job > Create.
23. Finish, (if you want to start the job immediately tick the box, and it will run now, and then run again as scheduled).
24. Now when the job runs it scans the ‘seed’ first, creates the replica, and finally replicates the difference.
25. You will notice whenever the replication tasks run in future, it only replicates the differences. For example, here on a subsequent run, it only took twenty six and a half minutes to do the job.
Related Articles, References, Credits, or External Links
You are trying to perform a snapshot using Backup Exec Open File Option and there is already a snapshot file there.
Error: V-79-57344-34110 – AOFO: Initialization failure on: “C:”. Advanced Open File Option used: Symantec Volume Snapshot Provider (VSP).
Snapshot provider error (0xE0008520): Device could not be snapped because it contains an active cache file.
Check the Windows Event Viewer for details.
Solution
1. Firstly and most importantly make sure there is no backup running at the moment.
2. Go the the server that you were backing up (i.e. if its a remote server, go and logon to that one NOT the media server).
3. Navigate to C:Backup Exec AOFO Store, Delete any files you find there, (Note: Its possible your OFO store may be in a different location, if so look for files called _BEVspCacheFile_0.*)
4. When you have located the file(s) delete them.
5. If it will not let you delete the files because they are in use you can “unlock” them with Unlocker.
Related Articles, References, Credits, or External Links
Backup Exec Exchange Backup fails with the following error,
Final error: 0xe000fed1 – A failure occurred querying the Writer status. Final error category: Resource ErrorsWriter Name: Exchange Server, Writer ID: {76FE1AC4-15F7-4BCD-987E-8E1ACB462FB7}, Last error: The VSS Writer failed, but the operation can be retried (0x800423f3), State: Failed during freeze
Solution
The Microsoft VSS writer that Backup Exec is using is in a failed state.
1. Drop to command line > Start > run > cmd.
2. Issue the following command,
[box]vssadmin list writers[/box]
3. Look for the Exchange writer – and see what state its in.
4. As you can see this one has failed – Reboot the server, 99% of the time that wil fix the error, if not see here.
Related Articles, References, Credits, or External Links
