xenserver ha improvements

XenServer HA Improvements

Geoff HigginbottomCTO ShapeBlue

geoff.higginbottom@shapeblue.comTwitter: @CloudStackGuru

Cloud Architect & ShapeBlue CTO

Specialise in….

Designing & Building Clouds based on Apache CloudStack / Citrix CloudPlatform

Developing CloudStack training

Blogging and sharing CloudStack knowledge

Involved with CloudStack before donation to Apache

Designed Clouds for Cloudera, Everyware, SunGard, Ascenty, BskyB, Trader Media, M5 Hosting, Team Cymru, Interoute, University of Pennsylvania.…

CloudStack Committer

Who Am I?

How does VM HA work in CloudStack?

VMware – vCenter handles it

XenServer, KVM – CloudStack handles it

Hyper-V – Not currently supported

CloudStack VM HA

Hotfix XS62ESP1004 (and XenServer 6.5.0)

Improved mechanism for detecting a failed Host and electing a new Pool Master

Requires Pool HA to be enabled

Prior to XS62ESP1004, correct procedure was to not enable HA on the XenServer Pool when using with CloudStack

So What's the Problem?

If you dot not setup Pool HA a new Pool Master will NOT be elected in the event of a Pool Master failure

All Hosts in the Cluster will go offline

CloudStack will not trigger VM HA, and will report all VMs as still running

No control of VMs in the affected Cluster

So What's the Problem?

Configure Pool HA but only for the Pool Master and not for VMs

How? Create and mount new SR – must not be used for VMs

Enable Pool HA via UI

XE Commands

Script it

Resolution

Use the UI…

Use the UI…

Use the UI…

Use the UI…

Set ‘Failures tolerated’ to Zero

Use the UI…

Use the UI…

Note: It may take two minutes to enable

HA Enabled

Run following two commands to enable Pool HA

xe sr-list name-label=<SR NAME> --minimal (to get the SR ID)

xe pool-ha-enable heartbeat-sr-uuids=<SR ID>

XE Commands

You will need to disable and re-enable Pool HA as some point in the future, so in true DevOps tradition, if you do it more than once, script it

Create a script called enable-ha.sh on any XenServer and populate with the following :

#!/bin/bash

MGMTHAUUID=`xe sr-list name-label=MGMT-HA --minimal`

xe pool-ha-enable heartbeat-sr-uuids=$MGMTHAUUID

echo "Pool HA is now enabled"

Note the script assumes the SR is called ‘MGMT-HA’

Script it…

To perform a controlled restart of a Pool Master Host: Place into Maintenance Mode within CloudStack

Disable Pool HA

Place the Host into Maintenance Mode with XenCenter to force the promotion of a Slave to Pool Master

Restart the Host

Once Host has fully booted and is online in XenCenter re-enable Pool HA

Exit Maintenance Mode in XenCenter

Exit Maintenance Mode in CloudStack

Managing HA Enabled Pools