ZEN and the Art of VMware ESX Storage VMotion (or ‘A Quick Guide to Storage VMotion’).

October 17th, 2008

Kiwi Si

Ok, so there’s not much Zen happening here but it sounded like a cool title for a posting and I like the Zen pics I found…

ESX Storage VMotion is probably something you’ve heard of before, if not used.

The posting is really a guide to performing a basic Storage VMotion using the provided command line method from VMware. There are also other useful bits of information along the way.

What is it?

As VMware explain in their ‘new features storage guide’:

"Storage VMotion (SVM) enables live migration of virtual machine disks from one datastore to another with no disruption or downtime. Just as VMware VMotion allows IT administrators to minimize service disruption due to planned server downtime, Storage VMotion allows them to minimize disruption by reducing the planned storage downtime previously required for rebalancing or retiring storage arrays. Storage VMotion simplifies array migration and upgrade tasks, and reduces I/O bottlenecks by moving virtual machines while the VM remains up and running. It provides a hot migration of the storage location on which the vmhome resides."

One of the common complaints with Storage VMotion is that it isn’t provided out of the box with a GUI interface – as such VMware received some criticism that this feature felt like it had been rushed out the door so as to meet the ESX 3.5 release.

Even if this was the case, which it sort of is, the fact remains that it is a great feature and does do what it says on the side of the tin (albeit a command line tin).

Historically Storage VMotion was only supported over Fibre connections but with the release of ESX version 3.5 Update 1 Storage VMotion is now officially supported by VMware over iSCSI. This is good news as with the arrival of 10Gb Ethernet iSCSI is most likely going to increase in popularity. Click here for more details on this.

That said, a potential fly in the ointment for iSCSI will be if Fibre Channel over Ethernet (FCoE) gains the momentum it is expected to over the next couple of years. Check out this article from The Register for a little background on what is happening in the world of FCoE.

I should point out that much of the information on what I am about to demonstrate can be found in the ESX/ESXi Basic Administration Guide’ (pdf) that can be downloaded here.

The Requirements.

The Administration Guide outlines these following requirements for VMotion:

Virtual machines with snapshots cannot be migrated using Storage VMotion.
Virtual machine disks must be in persistent mode or be raw device maps.
The host on which the virtual machine is running must have sufficient resources to support two instances of the virtual machine running concurrently for a brief time.
The host on which the virtual machine is running must have a VMotion license, and be correctly configured for VMotion.
The host on which the virtual machine is running must have access to both the source and target datastores.
VMware Infrastructure 3 supports a maximum of four simultaneous VMotion or Storage VMotion accesses to a single datastore. A migration with VMotion involves two simultaneous accesses to the datastore, by the source and destination hosts. A migration with Storage VMotion involves one access to the source datastore and one access to the destination datastore. Therefore, if no other migrations are occurring, up to four concurrent Storage VMotion migrations involving the datastore can occur simultaneously.

How to do it.

Storage VMotion can be run one of two modes: interactive and non-interactive. Basically the interactive mode prompts you to enter the required parameters one at a time and as you can probably guess, non-interactive requires the full syntax to be entered in one go. This is what the non-interactive syntax looks like:

svmotion [common VI Perl options]
        –datacenter=<datacenter name>
        –vm <VM config datastore path>:
              <new datastore name>
        [--disks <virtual disk datastore path>:
              <new datastore>,
              <virtual disk datastore path>:
              <new datastore>]

Unless I’m doing repetitive Storage VMotions (eg: via script, scheduled, etc) I tend to use the interactive mode as this also jogs my memory as to what parameters and syntax are required.

There is a popular Perl based script by Dominic at VMProfessional.com that simplifies performing a Storage vMotion. All that you need to do is update the script with some of the details relating to your VM environment (data store names, etc). I would recommend using this over using the ‘Interactive’ method as why reinvent the wheel? This is well worth a look and can be found here.

To start using Storage vMotion start the VMware VI Remote CLI (Command Prompt) from the start menu.

This will open a familiar looking Command Prompt box. From the prompt change the directory to ‘bin’ (ie: cd bin). It is from here you’ll be able to run the svmotion.pl script.

As mentioned above there are two modes in which you can run Storage VMotion, Interactive and, wait for it… Non-Interactive.

Let’s walk through an example of an Interactive Storage vMotion. First of all to make things easier, let’s clarify some of the names in this examples environment.

VirtualCenter URL: https://vmserver

User Name: Administrator

Password: <secretpassword>

DataCenter Name: TechHead

DataStore where the VM is currently located: [VMImages]

Target VM File name and location: W2K3_ADM01/W2K3_ADM01.vmx

Name of Destination Data Store: ml110g4ESX35:storage1

So, lets kick off this Storage VMotion!

Type in: svmotion.pl –interactive

Now type in the VirtualCenter URL : https://virtualserver

Enter in the ‘username’ and ‘password’

A connection is now made through to VirtualCenter. Once it has connected through successfully you will receive a confirmation: ‘Connected to server.’

Now enter the name of the ‘datacenter’: TechHead

We are now prompted for the target datastore name and the path to the vmx file associated with the VM. The datastore name must be put inside square brackets (ie: [<datastore name>]).

We type in: [VMImages] W2K3_ADM01/W2K3_ADM01.vmx

We’ll now enter in the name of the destination datastore, which in this instance is called ‘storage1′. Note: You don’t need to add the square brackets around the datastore name like we did in the earlier step.

We are now prompted to whether we want to ‘individually place the disks’. At this stage you have the option to separate the disks from the virtual machine itself. In most instances you would want to keep them together for ease of management. So to do this we enter ‘no’ as we don’t want to individually place the disks.

The Storage VMotion now starts to do it’s thing and replicate the files to the new destination datastore (whilst the VM is still up and running). At this stage depending on the size of the VM and the speed of the disks at both the target and destination datastore you will want to go and make yourself a cup of tea (or two).

As the Storage VMotion progresses additional #’s appear in the command prompt.

To give you an idea of how long a Storage VMotion takes – on my home lab environment I am copying a 43GB (approx) VM from my iSCSI SAN (e200 Controller with 2 x 7200rpm 320GB SATA hard disks in a RAID 1 configuration) to local storage on my ESX 3.5 server (non-RAIDed 7200rpm 160GB SATA disk) in about 30-35 minutes.

When using this with a proper Enterprise level SAN and faster disks you would, as expected, receive much quicker times.

Once Storage VMotion has been completed we are notified in the command prompt window. The VM’s files are now located in the new location without any downtime to the VM and any of the services it is providing.

Looking in the Datastore Browser we can see all the files that make up and are associated with the VM are now located in the new Datastore location. Hooray!

GUI Plug-In’s

Due to the lack of any GUI around Storage VMotion there have been a few attempts by individuals to provide such an interface/plug-in. Some better than others. One that stands out, and is popular in the VMWare community is by Andrew Kutz.

It can be downloaded from here and a good step by step guide by David Davis on how to install it can be found from here.