So, everyone wants more performance, always.
As I mentioned in yesterday’s post, it’s REALLY hard to generate a “VMware IO workload”, and most benchmarks out there aren’t that good at the IO part of a VMware benchmark.
Even VMmark 2.0 which is a great CPU/memory tile-based, mixed workload benchmark doesn’t really stress the IO subsystems particularly hard.
This means that optimizing IO performance around that workload is tough. And, storage subsystem performance is always very workload-centric.
For example – we demonstrated face-melting, record-setting NFS performance in the SPEC SFS benchmark that I pointed out here. So surely we have ROCKING NFS performance for NFS datastores on VMware on that same config, right?
Not necessarily. VMware NFS datastores (and Oracle on NFS) have a very different IO profile from a general purpose NAS use cases. They are much more transactional, so have a much higher random write mix, with vSCSIstats generally showing skews towards small IO sizes than you would see with “normal” NFS use cases that SPEC SFS simulates.
While VNX (and the previous EMC NAS and Unified generation) has generally decent VMware NFS datastore performance, but we know it can be even better. Engineering has made a series of optimizations that seem to have have a very material effect on VMware NFS datastore performance, but the best way is to check that against workloads that LOOK like customers (and check for regression in other workloads). We need to do that BEFORE we make it a maintenance release (which needs to go through a testing cycle), and check it into the next major release (which will of course go though a beta cycle). For that internal test, customer workload profiles are GOLDEN.
Beyond that we’re constantly working on VMware-focused features and use cases – for both VMFS and NFS use cases – and for that, customer workload profiles are GOLDEN.
BUT – believe it or not, we don’t have a good grasp on a broad set of “normal” customer workload characterizations in engineering proper (beyond support cases where something is by definition wonky).
So – we’re going to try to use stuff Clint Kitson, one of our vSpecialist rockstars (Clint, you are a star, I love you!) developed on a series of customer engagements to help not only EMC, but the community as a whole, and make it simple and rewarding for you.
This is the “Powershell toolkit” for storage IO profiling we’re using all over the place now. Useful in many contexts.
- Powershell scripts for collecting, processing, and federating ESXtop data (useful for any VMware customer)
- Powershell scripts for collecting and processing vSCSIstats (useful for any VMware customer)
- Powershell scripts for EMC performance statistics block/NAS (useful for any EMC customer)
Here’s the FIRST ask: If you’re a vSphere customer (of any shape/size, and EMC or non-EMC storage), please use these tools to gather a profile of the storage workload in your environment and share it on the Everything VMware @ EMC community (please zip the files, they are highly compressible).
I will provide an iomega IX2 to any customer who shares the details on their environment, including this raw data. The data will be public, which means it can be used by EMC and non EMC folks – and that means we’ll be able to better test, innovate, refine around the VMware use case, perhaps even built a VMware IO-centric benchmark.
Please post your environment info and the output from the scripts on this thread.
Here’s the SECOND ask: If you are a customer using EMC Celerra, EMC Unified, or VNX storage with NFS datastores, and have the ability to test an optimization in a non-production (but in use) NFS datastore environment, please let us know. If you are willing/able to do this please comment on this virtualgeek post, I’m sure I can find some additional iomega IX2s for anyone who provides feedback on the results they see from this optimization, I’m pretty pumped about the data I’ve seen.