How to automatically shut down VMware ESXi gracefully during power failure using an APC UPS.

# VMware

VMware ESX ESXi - How to shut down using APC UPSIn this latest post by TechHead guest contributor James Pearce he covers a topic near and dear to many of us – how to get VMware ESX/ESXi and its VMs to shut down gracefully upon power failure to the host.  Tighter integration between a UPS and a VMware ESX/ESXi host is no doubt something that will become more mature over time though for now it can be an issue for many administrators especially those running the free version of ESXi.  So read on to find out how James overcame this issue in his virtualization lab.

 

A Nearly Free UPS

APC UPSI recently acquired an APC Smart UPS that was being chucked out from work (having never worked) for my home lab along with an ancient AP9606 management card. With the batteries changed the UPS burst into life – but after some messing about getting the right firmware on it, I was disappointed to find no easy way to get it to shutdown my ESXi box when it needed to.

 

VMware License Restriction

The VMware management appliance (vima) can shutdown only paid-for installations of ESXi (using apcupsd and VMware community member lamw’s scripts) – the necessary interfaces on the free version have been made read-only since ESXi v3.5 U3.

 

Burp Suite

VMware ESX ESXi - How to shut down using APC UPS I’ve been finding a lot of use recently for network sniffers, so thought I’d have a look at how the VMware vSphere Client works, as obviously that can shut down the host. As luck would have it, the client is nothing more than a glorified web browser with the slight complication that it’s talking over SSL – but that’s no problem for PortSwigger’s Burp suite in its transparent proxy mode.

The traffic captures revealed that only three frames would be needed to perform the shutdown (hello, authenticate, and shutdown). A little manipulation is needed to get the session keys in, but that is basically it. ESXi’s startup and shutdown policy will do the work suspending or shutting down individual VMs, as configured through the vSphere Client.

 

The Script – shutdown.bat

Using this newly found knowledge I’ve created a Windows batch file (with a few supporting text files which are basically HTTP requests) that takes the hostname, username and password as parameters and will then shut down the host cleanly. The script needs something to launch it – APC PowerChute Network Shutdown in my case – and a utility to send the commands over SSL, for which I’ve used Nmap ncat (which just needs to be installed).

I have put all the necessary script files into a single convenient zipped file which you can download from here – the scripts are fairly well commented so you should be able to follow what is happening.

 

APC PowerChute

A potential issue is that APC’s PowerChute Network Shutdown utility will always shut down the Windows machine it’s running on. I’ve therefore used a separate Windows management VM to host PowerChute and my script, since I wanted everything else just suspended.

PowerChute has an option to ‘run this command’ but it’s limited to 8.3 paths and won’t accept command line parameters. A separate batch file is needed (poweroff.bat) that runs the shutdown script with the parameters – but that could shut down other ESXi boxes as well if required. Also the PowerChute service needs to be run as local Administrator as the default Local System account doesn’t have sufficient rights.

 

Testing the Scripts

VMware ESX ESXi - How to shut down using APC UPS

Download the ZIP and extract the files – I’ve assumed the package will be extracted to c:\scripts\esxi; update the path in poweroff.bat otherwise. Also the hostname, username and password also need to be specified in poweroff.bat.

Next install and configure PowerChute (in particular change the service user account) and enter the script in the ‘run this command’ box – I also increased the time allowed, but in practice it runs in a few seconds.

VMware ESX ESXi - How to shut down using APC UPS

Some waiting around can be avoided when testing by setting the UPS low-battery duration as high as it will go – just remember to change it back.

Next open up vSphere Client from a real machine, pull the UPS plug and once the battery get’s down to the specified number of minutes remaining, the script should run and the tasks will appear in vSphere Client.  Shortly afterwards the VM used to launch the script will itself shutdown under the control of PowerChute!

 

In Summary

The complete set of files can be downloaded here, and nMap ncat installation for Windows from here. Then a UPS management application is needed, for APC Smart UPSs use PowerChute for Windows.

The shutdown script includes logging and should report most errors. Bear in mind though that once a host is shutdown, it probably won’t be restarted when utility power is restored.

Burp Suite is a handy utility to bypass device limitations by enabling the scripting of management tasks that are only usually available through a web interface. I’ve used it to build scripts to regularly reboot home-spec routers every couple of weeks to keep them stable, and to set the time on the APC AP9606 management card daily since it doesn’t support NTP – and here to build a UPS shutdown script for ESXi; functionality that should really be built into ESXi in the first place.

 

VN:F [1.8.4_1055]
Rating: 0.0/5 (0 votes cast)
VN:F [1.8.4_1055]
Rating: 0 (from 0 votes)

Related posts:

  1. Upgrading VMware ESX or ESXi using vSphere Host Update Utility The vSphere Host Update Utility 4.0 is something of...
  2. Saving Power with VMware vSphere ESX – Dynamic Voltage and Frequency Scaling (DVFS).   With green computing, the introduction of the concept...
  3. How to view your VMware ESXi Host’s System Log, Config and Datastore via a web browser. Looking for an easy way to view your VMware...
  4. Running VMware vSphere Client on Windows 7 Many of you, like myself, have started running Windows...
  5. VMware ESXi with HP Management Agents ISO Download Returns to VMware.com Site After a brief period of absence the version of...

You can leave a response, or trackback from your own site.

18 Responses to “How to automatically shut down VMware ESXi gracefully during power failure using an APC UPS.”

  1. chris says:

    Clever.

    UN:F [1.8.4_1055]
    Rating: 0.0/5 (0 votes cast)
    UN:F [1.8.4_1055]
    Rating: 0 (from 0 votes)

    [Reply]

  2. James Pearce says:

    Update: vmware communities member William Lam (lamw) has used this to create a new Pearl script that provides this functionality in a neater form, although without the logging – check out his post on it at http://communities.vmware.com/docs/DOC-11623

    UA:F [1.8.4_1055]
    Rating: 0.0/5 (0 votes cast)
    UA:F [1.8.4_1055]
    Rating: 0 (from 0 votes)

    [Reply]

  3. Social comments and analytics for this post…

    This post was mentioned on Twitter by mars_seven: RT @PlanetV12n: How to automatically shut down VMware ESXi gracefully during power failure using an APC UPS. (TechHead) http://bit.ly/8dRJQj...

    UN:F [1.8.4_1055]
    Rating: 0.0/5 (0 votes cast)
    UN:F [1.8.4_1055]
    Rating: 0 (from 0 votes)
  4. Aben says:

    Hi James,

    Many thanks for the post. Well written and clear.

    Hope to see more clever tricks!

    UN:F [1.8.4_1055]
    Rating: 0.0/5 (0 votes cast)
    UN:F [1.8.4_1055]
    Rating: 0 (from 0 votes)

    [Reply]

  5. David says:

    Hi,

    Great job James, I’ve been looking for this for some time.

    I’m having a problem where the guest/s don’t get time to shut down gracefully before the host powers down. This is testing the command only, I have taken the ups out of the equation for now.

    NB: Esxi 4 running sbs 2008 and openfiler on a HP ML115 G5 w/ 8GB RAM. (There is a vcenter 4 server in the mix, but I’m directing the shutdown cmd to the host itself).

    Anyone have any ideas/suggestions as to why I cannot shutdown the sbs guest gracefully?

    Thanks in advance.

    UN:F [1.8.4_1055]
    Rating: 0.0/5 (0 votes cast)
    UN:F [1.8.4_1055]
    Rating: 0 (from 0 votes)

    [Reply]

    James Pearce Reply:

    Hi David

    Two things I’d suggest – first ensure vmware tools are installed in the guests. Second configure the ESXi startup / shutdown order to automatically suspend guests, openfiler starting first (and hence closing last).

    Lamw’s interpretation of this code will work against vcentre and there are other options too for licensed ESXi, like lamw’s other scripts.

    Thanks for reading!

    UA:F [1.8.4_1055]
    Rating: 0.0/5 (0 votes cast)
    UA:F [1.8.4_1055]
    Rating: 0 (from 0 votes)

    [Reply]

  6. Fred says:

    What to change if I only want my host to reboot ?

    UN:F [1.8.4_1055]
    Rating: 0.0/5 (0 votes cast)
    UN:F [1.8.4_1055]
    Rating: 0 (from 0 votes)

    [Reply]

    James Pearce Reply:

    Hello, I assume that you really just want to script a host reboot, in which case the best bet is to use Burp to capture that request from the VI Client.

    Hope that helps!

    UA:F [1.8.4_1055]
    Rating: 0.0/5 (0 votes cast)
    UA:F [1.8.4_1055]
    Rating: 0 (from 0 votes)

    [Reply]

    Fred Reply:

    Hi,

    I replaced “ShutdownHost_Task” with “RebootHost_Task” in the shutdown-3.txt file and it worked.

    I never think it will be so simple.

    Thank

    UN:F [1.8.4_1055]
    Rating: 0.0/5 (0 votes cast)
    UN:F [1.8.4_1055]
    Rating: 0 (from 0 votes)

    [Reply]

    James Pearce Reply:

    Excellent – thanks for sharing this!

    UA:F [1.8.4_1055]
    Rating: 0.0/5 (0 votes cast)
    UA:F [1.8.4_1055]
    Rating: 0 (from 0 votes)
  7. Federico says:

    Hi James,

    I am facing a similar situation like the one you describe.
    I have an ESXi 4.0 free licensed installation running some VMs.
    I have a UPS without a network management card. This UPS is connected against the host via USB or serial port. Need to shutdown VMs and finally the host in case of power outage.

    and need to avoid the following:

    1) license the ESXi 4.0
    2) buy a network management card
    3) buy APC software (not sure if cames for free when you buy network management card)

    Any ideas about how to do this?

    Any guide will be much appreciated

    Thanks
    Federico

    UN:F [1.8.4_1055]
    Rating: 0.0/5 (0 votes cast)
    UN:F [1.8.4_1055]
    Rating: 0 (from 0 votes)

    [Reply]

    James Pearce Reply:

    Hi, to get a USB UPS working you will need either a physical host with a USB port (an old laptop maybe?), a network USB hub (http://www.digi.com/products/usb/anywhereusb.jsp#overview), or an ESXi server that supports vmDirectPath and pass through a port in the host.

    To be honest, if it is an APC Smart UPS I would definitely hunt out a cheap old AP9606 web management card (seem to be about £25 on eBay at the moment, but they do come up cheaper sometimes). It doesn’t matter what it was from (i.e. PDU, UPS, whatever) – the firmware I’ve linked to above can be installed to get it working with a UPS.

    The APC software is freely downloaded from apcc.com, or there are freeware UPS utilities as linked to by the lamw scripts that are more generic.

    Hope that helps!

    UA:F [1.8.4_1055]
    Rating: 0.0/5 (0 votes cast)
    UA:F [1.8.4_1055]
    Rating: 0 (from 0 votes)

    [Reply]

  8. Roberto says:

    Thanks,
    excellent solution for a “strange” limitation on the free version. It works very well. Just for info (I’m not so far expert in VmWare management), if an ESXi machine is powered off with this script, are the hosted VMs gracefully closed before?
    TIA
    Regards
    Roberto

    UN:F [1.8.4_1055]
    Rating: 0.0/5 (0 votes cast)
    UN:F [1.8.4_1055]
    Rating: 0 (from 0 votes)

    [Reply]

    James Pearce Reply:

    Hi Roberto,

    Thanks for the question.

    What happens to running VMs depends on how the auto startup/shutdown policy is configured through the vSphere Client. I set all my VMs to ’suspend’, in a particular order, except the management VM running the APC software which is set to ’shutdown’. Since the script just asks ESXi to shutdown, these policies are adhered to in the process.

    Hope that helps.
    James.

    UA:F [1.8.4_1055]
    Rating: 0.0/5 (0 votes cast)
    UA:F [1.8.4_1055]
    Rating: 0 (from 0 votes)

    [Reply]

    Roberto Reply:

    James,
    Thanks a lot!
    All the VMs are configured now for “ShutDown Guest”.
    Best regards
    Roberto

    UN:F [1.8.4_1055]
    Rating: 0.0/5 (0 votes cast)
    UN:F [1.8.4_1055]
    Rating: 0 (from 0 votes)

    [Reply]

  9. Jordi says:

    First of all, congratulations for this post is excelent. Just a point when you talk about the local rights in the power chute service I belive if you put the path of ncat in the variable PATH of the global enviorment it will works in any account.

    UN:F [1.8.4_1055]
    Rating: 0.0/5 (0 votes cast)
    UN:F [1.8.4_1055]
    Rating: 0 (from 0 votes)

    [Reply]

    James Pearce Reply:

    Hi Jordi, glad you find it useful, and thanks for posting the permissions tip.

    Cheers,
    James.

    UA:F [1.8.4_1055]
    Rating: 0.0/5 (0 votes cast)
    UA:F [1.8.4_1055]
    Rating: 0 (from 0 votes)

    [Reply]

  10. Brilliant article James!
    With regards to the SBS shutdown question above, you definetly need to configure the startup/shuwdown sequence. Put your SBS shutdown timeout at 5-10 minutes. If SBS is done after 4 minutes, ESXi will continue with the next vm. If you keep everything default, ESXi will use the default timeout to continue to the next vm. After the last poweroff command, ESXi will shutdown itself, even if your SBS is still waiting to shutdown!

    UN:F [1.8.4_1055]
    Rating: 0.0/5 (0 votes cast)
    UN:F [1.8.4_1055]
    Rating: 0 (from 0 votes)

    [Reply]

Leave a Reply

You can add images to your comment by clicking here.

PHD Virtual - esXpress
StarWind Software
Virtu-Al.net
TrainSignal - vSphere Pro
Veeam - Backup & Replication
Virtualization Pro Summit 2010