A few technical threads from the inside - Part 1: RDMs

OK - from one day of insider EMC threads on VMware topics - three posts.

1) "What VMware features work on RDMs and which don't?'"

Raw Device Mapping is something which VMware shops need to know about.   They come in two modes - physical mode (pRDM), and virtual mode (vRDM).   

IMHO, RDMs should be used sparingly, for a couple reasons:

• They make management a little more difficult, and a little more difficult x many times = lot more difficult
• They make the LUN count maximums for ESX clusters come much faster (since you consume several LUNs per host by definition, as opposed to many in a single LUN using VMFS)
• They break the encapsulation model (in the sense that the VM is no longer completely self contained and portable across hardware)

Then again, they have a couple important super-powers in the VMware context:

• You can move a pRDM (but not a vRDM) from a physical host to a virtual and back.   Useful in weird P-to-V-to-P use cases, useful in "repro that in a physical" use cases.   PtoV should be a one way street, and every ISV starts embracing VMware support, so I don't think that value is a long-term use case, but important today.
• pRDMs can be used with array replication that provides application integration and quiescence in the application guest.   Bit by bit, we're all figuring out how to do this with VMFS (EMC can already do it with Exchange, SQL Server and Oracle to day with Replication Manager 5.1.2), so again, not a sustainable use case, but important today.

So, if they are rare, and the use cases are not sustainable, why mention them?   I keep getting the "pRDMs don't support VM HA/VMotion" comment.

Gang to be clear here:  RDMs in physical mode can be VMotioned, or work with VM HA .   All storage types (VMFS, NFS, iSCSI in the guest, pRDM, vRDM) can be VMotioned.  The only gotcha with pRDMs is that you can't do ESX snapshots, and then features that depend on this (VCB for example) don't work.   From an EMC tools standpoint, what works with Replication Manager?  Answer is easy:

• EMC Replication Manager supports the Microsoft iSCSI intitiator (MSI) to the guest - simple, easy, great for test and dev, full app integration and support
• EMC Replication Manager supports pRDM - ideal when the mount host is a phyiscal, it's a test/dev P-to-V use case, or when the mount host needs insanely high backup throughput (and where MSI may not be the right choice)
• EMC Replication Manager supports VMFS volumes and file-level consistency for VMs, as well as application-integrated snapshots for apps running in the VMs.

The sources of the misinformation are usually non-authoritative links for the “RDMs can’t be vmotioned” claim (techtarget – HA!)

Read this: http://www.vmware.com/pdf/vi3_35/esx_3/r35/vi3_35_25_3_server_config.pdf

That is authoritative.

• See page 144 (VMotion is supported, pRDM can’t take a native ESX snapshot, vRDM can)   Features that depend on ESX snapshots (like VCB) also don’t work with pRDM, but VCB doesn’t currently support VSS backup, so they’re not going to use that for Exchange (they will use RM in this case).
• See page 146 for details on vmotion of an RDM
• See page 147 for limitations (vmotion is not one of them).

Moral of that story: The VMware docs are great reference material, pRDMs can be vmotioned willy nilly, and Replication Manager integration with VMware is matched only by SnapManager for Virtual Infrastructure from Netapp (each have a couple minor things over each other).

Comments

[Duncan]

Maybe I don't understand the VSS remark, but as far as I know as of 3.5 U2 and VCB 1.5 VSS is supported: http://www.vmware.com/support/vi3/doc/vi3_vcb15_rel_notes.html

I haven't been able to test it though.

[Chad sakac]

Right - translating what VSS support means has been difficult since it's introduction.

VSS is often used referring to BOTH:

- Volume Shadow Copy services for NTFS filesystem-consistent backup using a freeze/thaw mechanism with Window's system provider, or 3rd party provider

- Volume Shadow Copy services as a 3-way API operation including a writer (think "application integration module"), a requestor (think "backup application running on the host"), and the same provider mechanism used for generic NTFS file-consistent backups.

For example on the latter - Exchange 2003 and 2007 ship with VSS writers (as does SQL server). A VSS requestor calls those VSS writer APIs to initiate application specific behavior during backup and restore. For exmaple in Exchange's case flushing and closing the current open log during backup, handling validation of the backup, and during restore, triggering and handling log replay. In otherwords "application level consistency actions". Then the VSS provider kicks in.

What happens in 3.5 u2 is that now, just like array-based replicas who's vendors created VSS providers that operate inplace of Microsoft's system provider, ESX snapshots now act as a VSS provider, and you get NTFS-consistent backups.

BTW - homework excercise for anyone with XP, or W2K3 or W2K8 - right click on any NTFS volume, go to "properties", then "previous versions" - that's native VSS.

Supporting VSS (and the storport driver stack, which is the big change in u2) is a mandatory pre-requisite to application-integrated VSS.

So, now people who have created VSS requestors for various applications can operate inside the guest, and leverage the native ESX snapshots for the underlying point-in-time copy. Examples of this are most backup applications (EMC's included), EMC Replication Manager and the NetApp SnapManager family (both of which are designed to integrate array-based replicas). EMC Replication Manager supported app-consistent replicas for SQL Server and Oracle in VMs before this change, because there you have other APIs for the same thing (SQL can use VSS and VDI - virtual device interface - not Virtual Desktop Infrastructure, Oracle has hot-backup mode), but this was mandatory for us to support Exchange 2003 and 2007 in VMs with app-integrated backup and recovery.

Does that make sense?

[Doug Michalko]

RDM's will work with vmotion except if RDMs are used with Microsoft Cluster Services in which case SCSI bus sharing is enabled and the vm cannot be vmotioned.