ScaleIO Node - what's the scoop, and what's up?

[Updated - 10/29/15]   This blog post has generated interesting dialog, both internal and external.   I’ve also seen some articles in the press that have gotten important things wrong and I want to add a couple comments for readers.  In this post, I was very emphatic, the ScaleIO is not a "Hyper-converged Infrastructure thing”, and many people have asked for a clarification.   Is that tap dancing, or does it reflect an important, real technical point?

To make this really clear:

1) if you want a Hyper-Converged rack-scale integrated system (DEFINED: designed to scale big, think about the domain of rack-scale management, ToR switch and SDN, and the full M&O for that whole domain) - we have one and only one offer only: VxRack.   VxRack comes with different “software personalities" - some that are open (bring anything - bare metal), some that are vSphere only (EVO SDDC), some that are focused on pure cloud native application stacks.

2) If you want a simple, turn-key hyper-converged appliance (DEFINED: designed to start small, focus on simplicity) - we have one and only one offer: VSPEX Blue.

We’ve realized that people confused what was a “storage thing” (ScaleIO node) - which bundles SDS + server with a hyper-converged infrastructure thing - so to make it clear - you can no longer get the compute-dense configurations and “DIY”.   If you want DIY - buy ScaleIO software.  If you want a hyper converged rack-scale infrastructure - buy VxRack.

If you’re here for the first time, read the blog and come back - if you’re here for the second time, this is clarification.

ScaleIO is software-defined transactional storage that can be deployed in:

• a two tier fashion (general compute/applications workloads run on separate nodes that access a pool of storage via separate SDS nodes over some form of network - common with dense blades for compute, dense rack-mount for storage pools)
• a hyper-converged fashion (general compute/application workload runs on the same nodes that the SDS stack pools the storage).

The ScaleIO node is a case where we are bundling the SDS layer with a server platform (and optionally) network with a single support model.   It can be in either of those deployment models.   And - it’s something customers are demanding (read the blog below for why) - but in short, you ain’t Facebook.  You can leverage software + commodity hardware models - but you ain’t Facebook :-)

 

That does not make ScaleIO node a generalized hyper-converged infrastructure appliance or rack scale system by my (or EMC’s) definition (and that’s not marketing, or positioning) - and this is important to understand.  

 

When customers want converged/hyper-converged infrastructure appliances (or rack scale systems) - what customers ask for is this:

• They want a management and orchestration stack that spans the domains of compute/storage/virtualization (in appliances), and this extends to the top-of-rack network and spine/leaf network (in rack-scale systems that must scale to and beyond rack boundaries operationally).   This needs to do configuration, fault, telemetry and more.
• They want the operational and support model to fully include up to the application workload - and customers think the lifecycle, reporting, support are all part of that design.  HINT: anyone who has an appliance that doesn’t engineer/manage the top of rack switch and SDN layer ain’t designed to scale big.
• They want it to be fully packaged/priced with that “all in” model in appliances - which are more fixed hardware variations and leave the network domain out of thinking.   Likewise, they want it to be fully packaged/priced/engineered (in rack scale systems - which require consideration for the network design based on plans for horizontal scaling, and more variant disaggregated hardware types) likewise “all in”.

These only look similar if you think the only thing that matters in a system-level design is a server choice and an SDS layer (common for people with a storage-centric world-view) - which is an incorrect view (even if you hear it from an EMCer)

 

If you are wondering if we can add a GPU in a ScaleIO Node, or if how you would update/patch the OS on a ScaleIO node, or manage the update of your abstraction stack, or how it would manage and balance application resources across the pool of kit…. well, hint: you’re not in the market for an SDS + server thing.   You’re in the market for a Hyper-Converged Infrastructure Appliance (HCIA) or a Hyper-Converged Rack Scale system...    

 

… And If you think you REALLY want a hyper-converged infrastructure appliance or rack scale system - why build it yourself, when there are choices in the market (VSPEX Blue = Appliance, VxRack = Rack-Scale System - and competitors of course) that do this better, and cheaper than you can, and will reduce your time to value, time to outcome?   

 

That’s why I’m so emphatic (and this is authoritative):

• ScaleIO = SDS Software.   
• ScaleIO Node = SDS bundled with Servers (awesome), which is != Hyper-Converged Appliance/Rack Scale System.   
• VSPEX Blue =  Hyper-Converged Infrastructure Appliance; 
• VxRack = Hyper-Converged Rack Scale System (that can have different IaaS stacks and other workloads) on it.

Returning to regularly scheduled broadcasting… :-)

---------------

Today is the launch (directed availability - general availability in Q1) of the ScaleIO Node - the ScaleIO software, bundled with a range of of server hardware, and if needed a ToR switch - delivered as an appliance with a single clear support model:  EMC supports the SDS software and the hardware it runs on.

Hmmm:

• So - is this thing a hyper-converged appliance?  NO.
• So - does this compete with VSPEX Blue?  NO.
• So - does this compete with VxRack?  NO.

What is it?  It’s MOSTLY a storage thing.   Scratch that - it’s a GREAT storage thing :-)   To be even more clear - it can be deployed in a hyper-converged way, but it’s not equivalent to a hyper-converged appliance, or a Rack-Scale System.   It’s closest relative isn’t a VxRack, but rather an Isilon node.   Another less close relative is a VSAN-Ready Node. 

What do I mean?  Why have we created the ScaleIO Node - and what’s it used for?  Read on!

First of all - the ScaleIO node is all about the ScaleIO SDS software, so if you want to stop reading right now, I would encourage it.

1. go to http://www.emc.com/getscaleio   Download the bits.   Install if you want to just do a few nodes, but....
2. … If you REALLY want to see what it’s capable of, go to http://emccode.github.io  There you’ll find vagrant, ansible, puppet, and other tools to help automate at scale deployment of ScaleIO.

ScaleIO is at it core quite simple.  It’s goal is to be an extremely scaleable, extremely simple (the simplicity is to some degree the key to performance and scaling) that can deployed for a broad heterogenous set of transactional use cases.    Randy Bias did a post comparing (with a ton of performance data) ScaleIO to the use of Ceph (commonly deployed as a transactional SDS stack with Openstack) here.   The point is that ScaleIO is a laser, and does what it does really well.   But remember - DON’T LISTEN TO ME.  DON’T LISTEN TO RANDY.   Download and give it a whirl.

People have taken the freely available and frictionless (I’ll say it again: no time bomb + no feature limits + no capacity limits + we don’t even ask for your email address :-) bits and the infra as code tools and created simple automation packages to deploy into AWS, Azure, vCloud Air and more.  

They have played with it at huge scale (hundreds/thousands of nodes) and massive performance levels for a few hours for a few dollars.   I’d encourage anyone to download, play, learn and share.  

What makes ScaleIO great is:

• It’s simple.
• It works.
• It’s performance (latency, bandwidth, system-wide IOps) is great - it’s a function of the hardware you use of course - but it’s great.
• It’s transactional.  Object stores are great - but their use cases are more “new”.  Transactional use cases are everything most people use storage for today.
• It’s disruptive.  It can be used in a ton of cases where people use EMC stuff (and non-EMC stuff) for today.
• It’s available in a simple, free and frictionless way.
• It’s super-flexible, and open to a ton of use cases.  You can deploy and use it in a million ways. 

The ScaleIO node, at it’s core is simple - all the things that are solid, and for customers that want it with hardware - it’s a great, simple answer.

While you’re at it, for your vSphere 6.x environment, try downloading VSAN here: http://www.vmware.com/go/vsan.   If you’re focused on vSphere uniquely, VSAN needs to be on your evaluation list.   The VSAN  6.x bits are a huge leap forward from the 1.x bits - and the VSAN roadmap is strong.  Expect more to come on VSAN and ScaleIO - my two cents - customers should evaluate and come to their own conlusions.

I did a blog a little while back that it’s worth checking out here: Is the dress white and gold – or blue and black? SDS + Server, or Node?   This captures the essence of what today’s announcement is about.   It’s captured in this “Software+Hardware” vs. “Software only” crazy illogical circle.

 

Interestingly as we do more and more with pure software-only stacks, I’m finding I’m navigating this circle with more customers.   They think they want a pure “software only” solution (starting at the 1 o’clock position), and then the dialog goes in a strange circle that ends with them choosing an software + hardware combo.   I’ve found that as much as I want to - I can’t “short circuit” the dialog - because then they think I care whether it’s a software + hardware combo (if you want more, read the blog post above).  I **don’t** care.

Customers that fancy themselves hyper-scale (hint, odds are good that you aren’t) take longer to go around the circle than those who don’t.   It’s a core operational and economic question.  Operationally: do you have (or do you want to have) a “bare-metal as a service” function?   Economics: can you actually save money by procuring the servers (which by definition are cheaper at first glance - but not as dense, or as built for purpose), particularly when you take on managment/sparing/fault management of said hardware.   

• For some customers - the answer is “yes”.  
• For many, the answer is “no”.  
• For many the answer is “I don’t care - the options software-only give me are worth a trade-off in support/density/….”.

We’ve discovered (as VMware has with “VSAN Ready Nodes”) is that supported/qualified hardware accelerates adoption of SDS stacks.

So - what does a ScaleIO node include?   1) ScaleIO software (specifically v1.32 as of this writing); 2) industry-standard servers; 3) optionally, the top-of-rack switch that we’ve tested with and support.

What does the server look like?   Well - the answer is that there’s a broad set.   Here’s one.

 

This is actually a performance oriented node (low storage, high CPU/memory).   So far - SINCE THIS IS A STORAGE THING -  the vast majority of the demand is for the capacity-oriented nodes.   

The premise here is simple.  

1. Start with the software-only.   That means you can use it in a ton of flexible ways.
2. Figure out whether it’s something you dig (easy - since the bits are right there for you, no need to listen to ANYONE - just download and go for it). 
3. Decide whether you prefer to build your own, or want storage node (bundle of the hardware).

Now - why do I keep reinforcing this as a storage thing?   After all - can you run compute on one of the nodes?   Can you?  Yes.   Should you?  probably not.

For those of you following closely, for a while we have demoed Isilon clusters that run compute workloads (even VMAX3 running general purpose workloads).   We’ve discovered that just because you CAN, doesn’t mean you SHOULD.

Since the ScaleIO Node is completely missing the management and orchestration stack to manage that, update it, and otherwise make it a Hyper-Converged compute thing (including the support model) - it is a storage thing, not a hyper-converged compute thing.  

BTW - get used to this idea.   Expect a OneFS software-only variation (choose your model).  No surprise there - that’s the exact same model as ECS (our Object/HDFS SDS stack).   Each are offered in “software only” and “software + hardware” models - and the software + hardware models will have nothing that stops running compute, but will not have the M&O stacks and engineering to make them a hyper-converged thing vs. a storage thing.   I suspect that other will (if they aren’t already) do this option for choice in packaging.

BTW - if what you need is a hyper-converged compute thing… If that’s what you want - it’s VxRack or VSPEX Blue depending on scale.

Here’s the continuum - from ScaleIO = software only (use however you want) -> Scale IO Node = software + hardware node (just like an Isilon node - which is software packaged with an industry standard server) -> VxRack = Hyper-Converged Rack Scale Infrastructure.

What’s going on with VSPEX Blue?   Building momentum and commitment.  

My personal view is that you cannot simultaneously design for “start small” and “scale big”.  

• When it comes to turnkey Hyper-Converged appliances, VSPEX Blue and it’s roadmap is our answer.  It’s simple, it’s turnkey, and it’s performant and feature rich.  A Total focus on vSphere and VSAN are unparalleled when you are focused on simplicity…. And VMware and EMC won’t stop here - we will keep pushing this Hyper-Converged Infrastructure Appliance (HCIA) market forward, faster, and faster, and faster.
• If you want a rack-scale model that can scale to thousands of nodes and you’re and Enterprise Datacenter (which is typically pretty heterogenous), VxRack (including, but not limited to the EVO SDDC Suite persona - and the higher level curated workflows and ecosystem in the Federation Enterprise Hybrid Cloud stack) is the answer.
• If you want a rack-scale model that can scale to thousands of nodes and it’s for a pure Cloud Native App use case VxRack with the Photon Platform and Pivotal Cloud Foundry is the answer.

So - with another example today of our model of SDS Data Planes are real, but we will give you the choice of package that fits you best….

 

...I’m insanely curious:

 

a) have you download the ScaleIO and VSAN bits?  What do you think?

b) where are YOU on the “circle of illogical choice?”  Do YOU think it’s illogical?

 

 

 

Comments

[Antonio Romeo]

I am not sure to understand one of your points (“it’s a storage thing”) and the implication in your post (“you can run workload on it but you really should not”).
In the “continuum” picture it is clear to me that customers can choose anything from DiY to fully engineered systems (vpex blue/vxRack) and in the spectrum of functionalities scaleio nodes are more on the DiY side. But then, if the customer is more on the DiY side of things, there is really no difference in running workloads using ScaleIO SW on customer-provided HW vs using a ScaleIO Node to do it. No difference, of course, a part from support: the fear of buying the wrong servers with the “sw only” version is common to everybody who has ever built its own PC in the past (forget whole server racks): a single component/driver/expansion can cause your preferred sw not to run, and you will go back and forth between the sw and the hw vendors forever.
So you have a DiY environment that you can use (as any ScaleIO SW install) for both converged and hyper-converged configurations. Is this a hyper-converged appliance? Definitely not. Can you run your applications on it? Just choose the right ScaleIO nodes for your application workload and the answer is Yes. You can and should run your workloads on it. Like on any server with ScaleIO SW on.

To respond to your question b: yes it is illogical. ScaleIO Node is the answer :-)

[Chad Sakac]

@Antonio, thanks for the comment, and I love your passion :-)

I hope my comments up front (decided to update based on your and other questions) about why I continue to be emphatic on the simplified answer: ScaleIO SW = SDS. ScaleIO Node = Storage Appliance. VSPEX Blue = HCIA. VxRack = Hyper-Converged Rack Scale System.

The dangerous line that I hope our field follows is not to delude that ScaleIO Nodes are "almost a hyper converged system" (in the hopes of rapidly pushing a sale). It is not. It is very, very close to DIY - but one where we provide the hardware/OS. No more (and there's a lot more in real hyper-converged appliances and rack scale systems), but also no less.

... and I'm sure you would say that the ScaleIO node is the answer based on your role :-)

ScaleIO is great, and customers should really try it. Then decide the best way to use it.

[Chet Walters]

I'm a partner. I have deployed ScaleIO software on commodity servers with PCIe flash... What a fantastic product! One of the few things I am really proud to represent from the EMC portfolio.

I get it... "Free and friction-less, Free and friction-less, Free and friction-less" I hear the marketing terms. The reality is that enterprise customers aren't going to run this thing unsupported. It's free like a puppy is free. :)

So why is the software only more expensive than XtremIO? I can't imagine with the cost of the physical nodes is on top of that.

[Fenton]

Could the VxRack Nutrino become an HPC monster.
Compute dense nodes with Nvidia Tesla GPUs + DSSD + ScaleIO + 40/100Gb networking.