VMworld 2014 – Open SDS Data Plane Transactional update (ScaleIO)

Lots of updates on the topic of “Software Defined Data Planes”!   Since the last VMworld, where VSAN and ScaleIO were “new products” – they’ve had several quarters in the marketplace together (really, VSAN was “new” as it GAed at the beginning of the year, and ScaleIO has been GA for about a year longer).

As they both became generally available, there was a LOT of discussion between VMware and EMC at the Federation level – because, to be frank, these two products have some material overlap, and we knew that they would inevitably compete in the marketplace to some extent.   So – where are we a few months in?   Read on for updates, demos, and a sneak peek at something really, really cool!

First – some background:

The story behind the story on this starts several years earlier – where both EMC and VMware were coming to the conclusion that “software data planes” (in both the networking domain and the storage domain) which were decoupled from hardware were going to be strategically important – and we started .   People often wonder how this sort of stuff “goes down”, and the answer cuts to the core of how our federation works: strategic alignment across the federation, collaboration and IP sharing where it’s right, with unfettered freedom of movement on innovation/execution where it’s necessary.

VMware took the idea (“build software-only storage stacks that leverage commodity hardware”) started down the path of developing VSAN organically (i.e. internal development team).   The design center of the VSAN design (which you can see so elegantly in the VSAN 1.0 design) was to hyper-couple it to the vmkernel, and make management an integral part of vCenter.  

EMC took the same idea (“build software-only storage stacks that leverage commodity hardware”) and starts organically building software-only versions of various data planes (some of which are GA now/soon – VPLEX/ve, Recoverpoint for VMs, vVNX aka “Project Liberty”, vOneFS), and kicked off the organic ViPR development project as a next-generation control abstraction, but just as importantly, a next-generation webscale Object/HDFS software storage stack.

But – EMC didn’t have any distributed transactional storage stacks (“type 3” in my taxonomy - see “understanding storage architectures” here).  It begs the question - why not simply use the VSAN stack, or both EMC/VMware co-develop VSAN and use it where needed?   Answer is simple – here’s what I mean:  

1. From the VMware world-view (and many customers!), a storage stack that is completely and tightly coupled to the vmkernel is logical… even though this means it could not be used with KVM, physical hosts, Hyper-V, etc.  
2. From the VMware world-view (and many customers!), if a storage stack was completely focused on the VM-object as the unit of storage and storage policy – that’s awesome, because you can optimize for that (awesome!).. even though that means that it couldn’t be used in cases where perhaps the use case demands physical hosts – for example, many “platform 3” use cases that use physical hosts and linux containers, or other “hyperscale” common design points.
3. From the VMware world-view (and many customers!), “should we scale above the number of nodes in a vSphere cluster?” is a somewhat nonsensical question.  If all you are targeting are vSphere use cases, and storing VMs – the vSphere cluster scale (however it changes with each vSphere release) IS the maximum scale in creating/placing VMs.  
4. From the VMware world-view (and many customers!), “should we be able to aggregate the performance of all the participating nodes to to a SINGLE  VM or host” isn’t as important as “deliver great aggregate performance across a set of VMs, and support the needs of average VMs”

Now, answer all these design requirements from the EMC perspective, and it comes out completely the opposite way: 1) no, we absolutely need to support non-VMware use cases; 2) no, we absolutely need to support hyper-scale use cases where perhaps the aggregate capacity and performance of hundreds, maybe thousands of nodes would be needed; 3) no, we absolutely need to be able to deliver any variation from all the aggregate performance/capacity in the storage pool to many hosts/VMs or focus it on a single massive consumer of IOps/GBs – and any variation in between.   BTW – this is ALSO the view of many customers (no “one size fits all” is the right answer)

With those answers, EMC’s approach cannot be the same as VMware’s approach to the same generalized architecture of “software that creates a pool of transactional storage from commodity hardware”.

So – while VSAN and ScaleIO are superficially the same (they both are software stacks that aggregate the storage of many hosts together), they are different in a ton of ways (architecturally – and how those differences manifest in operational behavior and characteristics).

To make this clear, we’ve pushed to the field, our partners, and to the market a very simple positioning guide for these two products, that can be summarized in this slide:

… So – now that they have been in the market for a while, what have I (personally – and I want to hear others’ perspective) learned?

Lots of interest in VSAN and ScaleIO all over the place!   VSAN is getting great adoption and so is ScaleIO.  Both also have a great list of logos/customers and the number is increasing at a great clip.   In my experience, in general people are “getting” the use cases.  Many people are realizing there are places they want to use appliances, but many are realizing that a lot of use cases can be well served by the SDS model.

Ok – kumbaya, but what about the conflict topic?   You can see the overlap in the above: what to recommend for a customer who has a LOT of vSphere, but also other important stuff?    Sadly, sometimes what a customer gets told starts with what people across the table are focused on.  That creates some weirdness, no doubt.   I have to say, it’s been the exception vs. the rule (and I’ve sampled the field teams pretty broadly).

What HAS been interesting is (at least from what I see) one of the points I’ve been discussing with the VSAN team for some time.  

VSAN absolutely scales up (performance, capacities, etc), but at larger scale customers, they do tend to “lean” towards heterogeneity (and this isn’t a “storage team vs. VMware admin team”.   Instead – it’s a “I don’t like parts of my architecture that pin me down in other parts of my architecture”.

This isn’t so much a “architectural” about VSAN or ScaleIO scaling – or “node counts”, but more a “business thing” that happens with customers:

• If you’re a customer who has everything in your environment running happily on a couple  vSphere hosts, VSAN is perfect.
• If you’re a massive enterprise with thousands of remote sites and looking for something to deploy in those remote sites, VSAN can be perfect.
• If you have 4 (or hundreds!) vSphere 16-32 node clusters in your datacenters, well you also tend to have other things that are also really important.   Maybe you have an OpenStack deployment that also supports a platform 3 app stack using Docker.   Maybe it’s a huge Oracle cluster that you want to virtualize/have virtualized – but need to present a 200,000 IOps to that one set of hosts/VMs (i.e. the flash on one VSAN storage node just can’t cut it)…   Then you start to ask “should I deploy SDS on commodity ONE way for vSphere, and ANOTHER way for those other things?”    Those customers tend to lean towards ScaleIO.
• If you’re or are starting a Horizon pilot at smaller scale, you love VSAN – technically and economically (low entry cost), but…
• …In more than a few cases, I’ve seen that as VDI projects (Horizon or Citrix) scale up north of a thousand clients, customers tend to find non-hyper-converged designs (blades + deduped AFAs + deduped NAS) a lot more cost effective.  This isn’t just true of VSAN (due to dedupe), but is true of the economic scaling model of the hyper-converged model itself so also applies to appliances (EVO:RAIL and the others out there, you know who I mean).

BTW – EVERYONE (me included) suffers from “sample bias” (you see what YOU see, and extrapolate in error) – does the above jibe with what YOU see?   AND – I want to be 110% clear – I think VSAN is awesome!

This has led to a TON of ScaleIO activity not only where we fully expected it (service providers, hyper-scale players, hyper-performance use cases, OpenStack deployments, etc) – but in vSphere use cases too (where a customer chooses heterogeneity over tight integration).

So – it’s great that ScaleIO 1.3 is here beacuse there is a ton of goodies in general, but also specifically for the people that use it with vSphere!

For vSphere – this demo shows the absolutely HUGE new vCenter plugin.  This makes installation and operational tasks super easy (ease of install was never a ScaleIO forte in the past).   The end of the demo shows the ALPHA (this is pre-beta!) of something else really, really cool.   Watch the video and then come back after the break…

 

Ok, back?   Isn’t that prototype vmkernel loadable module cool?   It makes the architecture much simpler.  One of the world’s biggest (and happy – but very demanding) VMware and EMC customers went through the logical flow I outlined above (weighing tightly coupled VMware-only vs. less integrated but open SDS) with both teams speaking to the customer as one (they are a Federation customer).   That particular customer ultimately picked ScaleIO (and they are going down the SDS and SDN path in a big way).

BUT… they really didn’t like (for operational and security reasons) having a Linux guest running the SDC (the ScaleIO client) as a VM or a physical and then presenting storage back to the vSphere cluster.   They asked (nicely, but firmly) that the VMware and EMC engineering teams work together to make a vmkernel loadable VIB – this way the vSphere host can directly access a ScaleIO SDS cluster (just like a Linux host could for example).  Cool eh?   Note that the SDS (the server component) doesn’t run on the vmkernel – so unlike VSAN, this wouldn’t “consume” the HDD/SSD in the ESX hosts – really making the model more for “ESX hosts run on blades and accesses SDS runs on dense rackmount”.

Again, this is totally a prototype demo – don’t ask me for target delivery dates!  As we get closer, and things firm up – I’ll spill the beans.    Also – highlights VMware and EMC working as partner – both companies share the Federation “customer first” motto.

That’s not all!  ScaleIO 1.3 has big leaps in the usual “bigger/stronger/faster” category (increase volume/snap counts as an example), but some other neat things:   The ability to configure a RAM read-cache (and like all read-caches – this can have a huge performance impact!); much improved manageability via the UI (previously many tasks needed the CLI).   Check them out in this demo!

ScaleIO and VSAN are both awesome!  Hopefully both the historical context, and the customer examples help understand why at the Federation level we are pursuing both “tightly integrated” and “open” SDS transactional data plane stacks.   Everytime I’ve seen a customer try ScaleIO, their reaction tends to be positive – I’m trying to make it easily accessible the same way we do all EMC software stacks (www.emc.com/getvipr, www.emc.com/getisilon as examples) – and pushing (along with VPLEX/VE, Recoverpoint for VM)… stay tuned there!

Thanks to the great EMC team that made these demos: Jase McCarty, Rafael Novo, Saqib Ghani and of course the awesome EMC ScaleIO team!

Are you using ScaleIO?  Are you using VSAN?   What has your experience been?  What do you want to see next?   Do your thoughts echo mine on “when one vs. the other”?

Comments

[SDSMafia]

Looks like the EMC rednecks sure hate VSAN. With EMC shitting all over VSAN looks like Nutanix is going to win.

[Chad Sakac]

@SDSMafia - are you willing to disclose who you are?

I guess I don't get how from my article you would think I'm shitting on VSAN. I think VSAN is an awesome choice for the target use cases where it's the right fit.

I think I'm a pragmatist - unless in "SDSMafia" terms, VSAN is the right answer, all the time, for all use cases, for all workloads.

That seems to me to be the worldview of - well... perhaps a fanatic?

[Amy]

Captain VSAN wrote this sooo.. there goes that theory @SDSMafia

[Chad Sakac]

@Amy - did they see me in my "I'm Captain VSAN" duds? I love VSAN. Seriously. I just don't think it's the answer all the time. Do you?