One of the most disruptive events for customers is often a storage platform upgrade, since everything is consolidated - ergo everything is potentially affected. EMC has a multitude of ways to do this... lemme think... right now I can think of 4 (non-disruptive platform upgrade in many CX cases, PowerPath Migration Enabler, Invista/Rainfinity, and OpenMigrator/SANCopy). Every one has a specific use case (every customer is different). But a new tool in the toolbox is making a strong entrance on the migration scene for customers using any VI 3.5 release - Storage VMotion.
As you'd imagine, EMC does a lot of these data migrations, and recently there was a big internal thread on this I thought it would be good to share, as well as sharing customer experiences....
Storage VMotion is obviously only useful when the data is VMware, and (for now) iSCSI and FC only. It works generally with NFS datastores , it's not supported - I've seen some flaky cases so I don't think VMware is being arbitrary. NFS as a source and a target is expected to be officially supported in the future.
That caveat list is pretty short. If it is VMware data (one more great reason to Virtualize?!), svmotion can be a fantastic option, and joins the other mechanisms as a key tool.
Read on for a couple of my svmotion "rules of thumb" and a couple customer examples....
Ok - under the covers, svmotion:
- Takes an ESX snapshot
- Then file copies the closed VDMK (which is the large majority of the VM) to the target datastore
- Then "reparents" the open (and small) VDMK to the target (synchronizing)
- Then "deletes" the snapshot (in effect merging the large VDMK with the small reparented VMDK)
Ok, common questions:
Q: Is Storage VMotion really non-disruptive?
A: Well, if you're a bit of a propeller-head/nit picker (as I'm wont to be) - it's "non-disruptive" (i.e. doesn't require an outage) but can have a "performance impact". During the vast majority of the operation, the source datastore needs to continue to support the production workload and the additional workload of being the source of the filecopy process. This filecopy isn't governed or throttled in any way, so it WILL become the heavy IO consumer at the expense of the production IO. My suggestion: even though it involves NO downtime, still schedule svmotion operations. They'll complete faster, too! BTW - this is one of the aspects of the vStorage API set that's being worked on - it's possible to offload the copy/move task from the ESX host on a VM per VM basis, for all types of datastores (VMFS - where it needs to involve a blocklist-type mechanism, and NFS where it doesn't).
Q: Why doesn't EMC and VMware coordinate better - after all EMC has SAN/NAS virtualization, doesn't that compete with Storage Vmotion?
A: If you ask me, Storage VMotion and SAN/NAS virtualization are complementary, not competitive. First of all, I've said it before, and I'll say it again, I think that vendors who equate storage virtualization (in the classic HDS USP, IBM SVC, EMC Invista/Rainfinity sense) with VMware are making a technical error. The core value prop of server virtualization is threefold: 1) Consolidation for efficiency; 2) flexibility via non-disruptive mobility; 3) hardware independence via software. Most customers only have one array per major site - and those customers get 1+2 (but not 3) in any modern array (to varying degress). SAN/NAS is inherently consolidated and highly utilized relative to DAS (the physical->VM server analogy). Most modern arrays have the ability to add/move/change almost every attribute non-disruptively. What SAN/NAS virtualization add is the ability to do that over multiple heterogenous SAN/NAS arrays. None of these provide "hardware independence via software" (although it's arguable whether any of these things get you independence or simply a transfer of vendor control points).
SAN/NAS virtualization approaches typically have a data mobility rate that's still an order of magnitude faster than a really efficient Storage VMotion. So - they are adjacent use cases: i) Storage VMotion is a laser (you can Storage Vmotion a single VM, or even part of a VM), conversely Storage Virtualization is a hammer - fast, larger volume, but not as focused; ii) Storage Virtualization might be a better answer if you have many SAN/NAS devices, and they are heterogenous; iii) Storage Virtualization can do it's function with host independence - Storage VMotion is VMware-only (take that Hyper-V).... Long and short: Storage VMotion is a useful thing for almost everyone, SAN/NAS virtualization is useful for a subset of customers.
Q: How fast is a a Storage VMotion operation?
A: It is primarily governed by the performance of the source and target datastores, but in my personal experience, 0.5GB/min is a good rule of thumb for a decently configured system.
Q: Can I do multiple Storage VMotion operations in parallel to make it work faster?
A: YES! You can do them both in parallel in the sense of simultaneous operations on the same datastore, as well as in parallel across datastores.
First - simultaneous operations on the same datastore.... Regarding If you look at this guide page 246.
"VMware Infrastructure 3 supports a maximum of four simultaneous VMotion or Storage VMotion accesses to a single datastore. A migration with VMotion involves two simultaneous accesses to the datastore, by the source and destination hosts. A migration with Storage VMotion involves one access to the source datastore and one access to the destination datastore. Therefore, if no other migrations are occurring, up to four concurrent Storage VMotion migrations involving the datastore can occur simultaneously."
BUT - in my experience, you don't net a lot from this. Normally with a single filecopy, there's some benefit of simultaneous multi-threaded copy processes, but remember, it's likely that this datastore already has a bunch of IO that it's doing. Can it help? Sometimes - but personally, I'm all for the next thing....
Second - parallel across datastores.... Absolutely leverage this. If you do it - you can get N jobs at once, where N is the number of datastores you have on the cluster, but cannot exceed 32 (because you can only have one SVmotion per ESX host, and a max of 32 per cluster with VI 3.5)
Q: I don't like the CLI, I really wish there was a GUI....
A: The CLI isn't that bad, and if you ask me is the right way to do this en masse (we've done engagements were we get a list of VMs from the VC DB and use that to automate the overall svmotion exercise). BUT - if you want a GUI - I've had OK experiences with this third party VIC plug-in: http://sourceforge.net/projects/vip-svmotion/ Good news is that the next VMware major release will add a formal GUI.
Q: Are you guys working on something in Storage Vmotion using the vStorage API efforts?
A: Yes. We think that if we have the block-list (or file data) to identify a VM exclusively, we can reduce the performance impact of the Storage VMotion filecopy step dramatically, within an array for all our arrays, and across arrays using Invista for Block and Rainfinity for NAS. More to come - but work is underway.
A couple customer Examples:
These are recent customer migration projects I know of....
- "We recently (September) did a svmotion migration of 156 VMs comprising 3.2TB of data (dozen luns) from a CX700 to a CX3-80 and finished in 16 hours .. Averaged 200GB/hour (33GB/10mins), and that's with management overhead of manually managing the svmotions (no automation)."
- "They have migrated most of the 600 to one of the new CX3-40s and are only seeing around 50% SP utilization. That is during backups and simultaneous migrations. The customer moved a large (40,000 iops oracle box). It took 7.5 hours to storage vmotion across 1.7TB worth of machine\storage."
Customers, VMware/EMC Partners - what has YOUR svmotion experience been?