Proxmox HA Cluster

Proxmox HA Cluster KB ID 0001872

Problem

Proxmox HA Cluster: As the name implies this allows the cluster to ‘restart’ failed virtual machines on another host in the cluster in the event of a host failure. Unlike VMware (that has a vCenter appliance to do this) in Proxmox it can be configured from any host. Which may seem like an advantage – but the GUI can get a little ‘clunky’ whilst configuring HA.

Proxmox used ‘Quorum based” or “Majority node clustering” If you’ve ever done clustering on Microsoft SQL or with Hyper-V or Set up a Microsoft Exchange DAG this should be a familiar concept. In those cases you needed a ‘witness’ if there were only two nodes in the cluster (to have the extra ‘vote’ in a fail scenario). That does not mean in Proxmox you cannot have a TWO NODE cluster, but it does mean if the nodes lose sight of each other, (to avoid a split-brain scenario) the hosts will continue to function but only in read only mode. The only way to rectify that is to manually give a host an additional vote with the following command.

pvecm expected 1

You can also have a ‘witness node’ which would be another machine that was running corosync-qdevice but that’s outside the scope of this article

So aim for a cluster of at least THREE NODES, to keep things simple.

Solution : Proxmox HA Cluster

Proxmox HA Cluster Prerequisites

1. Proxmox VE hosts built (with Static IP addresses)

2. SSH Access (TCP port 22) to/between all hosts.

3. If there’s a firewall between the hosts UDP Port range 5405 to 5412 needs to be open between hosts.

4. Some shared storage i.e. NFS, iSCSI etc.

5. Optional: A dedicated NIC for replication, for small deployment this is not necessary.

Creating a Proxmox HA Cluster

Log onto your first host, Datacenter > Cluster > Create Cluster > Give the cluster a name > Create.

Clunky Alert! We are waiting for it to say TASK OK, which it will say quite quickly, but in the background, it replaced the digital certificate that was being used for the very web session you are using! So if you don’t want to stare at this screen for ages wondering what is going on, have a cup of coffee, then hit ‘refresh’ and log back in again.

Whilst in the Cluster section > Join Information > Copy Information.

Proceed to your SECOND HOST > Datacenter > Cluster > Join Cluster > Paste in the information from the previous step > Enter the Root password for the first host > Join HA Cluster. Again things can get a bit clunky – I prefer to have a management window open to host 1 so I can see what’s happening.

Again it will look like its ‘hung‘ > Wait a while and hit refresh.

Now we have a two node cluster.

Repeat those steps to add a third host.

Adding Shared Storage Proxmox HA Cluster

This is all being done in my test lab so I’ve simply created an NFS Share on a Windows Server to present as shared storage. Datacenter > Storage > Add > NFS > Add the relevant information (ENSURE you select both Disc image AND ISO Image) Add,

Adding a VM to Proxmox HA Cluster

First we need a VM, (if you already have one skip this step) Right click a host > Create VM.

Give the host a name > Next > Select the Datastore > Select the ISO* > Set the guest OS type > Next > Select you Datastore again (twice) > Bus Device set to IDE** > Next

*Note: If you can’t see you ISO file make sure it’s in the //nfs-root/template/iso folder!

**Note: If you want SCSI then read the following article.

Proxmox Windows Drive Missing

Set the number of sockets and cores > Next > Set the amount vRAM > Next > Select Network requirements > Next > Finish.

Allow HA on a Proxmox VM

Once we have a VM created, then to protect it with HA > Datacenter > HA > Add > Select VM > Add.

Once its status reads ‘Started” it will be protected.

Testing Proxmox HA Cluster

As you can see (above) my Windows VM is running on host 1, I’m going to simulate a failure of this host (by disconnecting its network card). As you can see the constant ping I have running to that VM begins to fail.

But after a while the VM is restarted on Host 2 and my pings start getting replied to. Remember folks: On Windows you need to enable ICMP on the firewall, or it won’t respond at all!

Proxmox HA Cluster ‘HA Groups’

The final thing to be aware of are ‘HA Groups’ which you can use to enforce certain rules on hosts and guest VMs. Datacenter > HA > Groups > Create > Give the group a name.

Restricted: Think of this like a VMware affinity rule – you can define which hosts a virtual machine ‘can’ run on.
Nofallback: If there is no requirement for VMs to return to the original node they resided on then select this option.