Today, we unveiled what is perhaps (certainly IMO) the biggest thing at EMC World: ViPR 2.0 Data Plane/Data Services + ViPR 2.0 Controller + Scale IO integration.
Why is this a big deal?
- If you look at my DSSD post here, you’ll note that the new app stack has at one end of the spectrum in memory and extremely low latency distributed databse models – but at the other end, there are hyper-scaled tubs of object, HDFS. These are at scales that are a little mind-boogling (think many hundreds of PB and even EB-scale).
- When it comes to these “huge object tubs” EMC has actually been an early commercial leader in this space with Atmos over the last few years, which has more than 400 customers (with 60 huge service providers), more than 1.5 EB deployed, and more than 1800 Atmos platforms deployed. It’s one of the most widely used commercially available web-scale object stores – and I would wager that many of us “touch” an Atmos object store almost every day. Atmos continues to grow and be deployed…
- … But you inevitably learn through any experience, and when we wanted to create a “next-generation” SDS object model (one that would also support the Atmos APIs), the team in Seattle (led by Amitabh Shrivistava, one of the pioneers behind Azure) – there were things we learned through experience.
OK – CONTEXT… When people look at “what’s going on in storage” – well, we have a lot of disruption going on! You have:
- AFAs cannibalizing “Enterprise Hybrids” for “high IOps, linear behaviors, moderate-to-low latency” category. (EMC XtremIO!)
- Next generation Hybrids winning in the “swiss-army knife” category (EMC VNXe/VNX!)
- Hyper-converged/”Common building block” and Converged models being the way people increasingly consume infrastructure (VMware VSAN, EMC ScaleIO!)
- In-Memory and funky distributed NoSQL picking up “ultra low latency” category (DSSD!)
- NAS moving to “scale-out – for real – including powering data lakes that power a huge swath of the enterprise (NAS that gets presented as HDFS) (EMC Isilon!)
- Cold Data getting “peeled off” SAN and NAS into object tubs that scale like crazy (Hyper-Scale at Web 2.0 and Service Providers) - and sometimes getting consumed as HDFS, sometimes as object, and rarely, as NAS (EMC ViPR Data Services!).
- At Hyper-Scale, people wanting total software/hardware independence, but with hardware appliance options if they choose it (EMC Elastic Cloud Storage!).
- … And people wanting simpler ways to deal with 1-7 (EMC ViPR Controller!)
The ViPR Data Services are focused on 6 + 7, and the ViPR Controller on 8.
Sidebar: Some people have told me that they think that EMC apply the same name (“ViPR”) to apply to the BOTH the SDS controller and the SDS data plane layer. Yup, well, I get that - it can be confusing. We compounded that at the first unveiling of ViPR a year ago, when the focus was very much on the Controller (with almost no talk of data services) – step one was to “abstract, pool and automate” the existing storage layers. Build a “harmonizer”, a “simplifier” for storage models.
We were intentionally quiet about the ViPR Data Services – we wanted to harden the storage engine and more importantly, the persistence layer which is necessary to hit what is a critical milestone: supporting broad “Commodity Off the Shelf” (COTS) hardware along with existing SAN/NAS models. This support for COTS is now in ViPR 2.0.
Let’s see if I can help clarify – read on!
The ViPR 2.0 Object/HDFS model (dramatically simplified) looks like this:
Lessons learnt by Amitabh’s experiences building Azure, and EMC’s experiences with Atmos:
- Every major functional part of the system is an abstracted, independent layer.
- Each layer is completely horizontally distributed across all nodes in the cluster
- Each layer is is independently scalable, highly available with no SPOF.
- The Storage Engine itself uses “Chunk” model (128MB Chunks used for all operations, including geo dispersion) and a simple “Append only” operational model (note that this is true internally – externally there are byte-range updates via S3 and other object APIs.)
- It includes northbound HDFS and Object (Swift, S3, Atmos) presentation models now, but will also get NAS in the near future. I want to call this out (lots of diversity in NAS stack implementations and use cases!)
- This will be the NAS use case where you need “a little bit of NAS access to your giant object/HDFS tub).
- It will not be for people who don’t want/need “heavy duty NAS”, from a full scale-out software-only OneFS, or the rich transactional data services of software-only VNX (Project Liberty)
- Also – ScaleIO is effectively immediately the ViPR SDS block data service, and has been integrated with the ViPR controller.
For those of you that want a comparison in the “product category”, the closest thing out there to the ViPR 2.0 data services would be Ceph.
Note how when people need a feature rich filesystem or transactional one they don’t use Ceph’s approaches, but rather Gluster?
While ViPR will have things like NAS on TOP of it’s Object layer, our customers demand more – and that’s an important difference, one that I’ll come back to in a bit* (look for the asterisk).
In the ViPR 2.0 release, there is HDFS 2.0 API support, done via an API head. It uses a custom client/server protocol optimized for high scale, and uses the same backing unstructured storage engine as ViPR Object data service. You can literally just load this library and it’s completely API compatible.
There is an interesting upside to this model – single bucket for HDFS and Object – and in fact, simultaneous object/HDFS access to the SAME content:
This means it’s possible to handle the same objects in different ways – handy dandy for many operations!
The ViPR 2.0 data services release also dramatically improves geo dispersion and global namespace behaviors.
First, let’s talk geo-dispersion – where there are generally 2 approaches, and ViPR takes a new one:
- Mirrored Copy (left): had the downside of poor utilization (2.66x) - but hey, can always rebuild locally.
- Distributed Erasure Coding (middle): great utilization (1.6x) - but oh vey, geo distributed rebuild sucks.
- ViPR (right): best of both worlds – the storage engine ensures you have local copies for doing multiple node/disk reconstruction locally, coupled with geo dispersed erasure coding for full site failure survivability gives you a great overall utilzation (1.8x with four sites). When you’re operating at hyper-scale, the efficiency factor of everything matters a lot.
Second, let’s talk about a namespace – where just like data protection, there are generally 2 approaches, and ViPR takes a new one:
This data protection and global accessibility is useful for the broad “object” class use cases. I think it’s particularly interesting when looking at HDFS, where we support erasure encoding protection models (on both ViPR and Isilon for that matter) for a much more efficient storage model, and a stronger model for remote replication than many native options included with distributions. We need to drive harder to partner with (be embedded in?) the big Hadoop distributions (and rest assured we are working this furiously) to get a full embrace – much more can be done!
What does a ViPR Object use case look like? Here’s an example (and highlights the wonderful
Let’s switch gears: what does ViPR 2.0 Data Services run on? Well – almost anything – it’s SOFTWARE. There is broad OS support. In fact, there is no specific OS affinity. At the GA here in Q2, there will be support for RHEL 7, Oracle Linux 6.5, and we will certify SLES 11 and CentOS 7.
What about hardware? Again, almost anything. We will specifically certify hardware platforms and out of the gate, will support Atmos Gen 4 hardware, HP SL4540 (below) – but will rapidly follow with specific things our customers are asking for (Open Compute being a high priority).
Now, we’re not sitting still with the ViPR Controller either! Lots of news here – particularly when it comes to supporting more platforms. The ViPR Native Integration has expanded – very notably with ScaleIO (ScaleIO just got a massive boost in terms of management/operation/automation at scale!), Vblocks, and also HDS (both HUS and VSP).
But, there’s another big piece of news: We’ve broadened industry support via Cinder integration (the list is getting long!): IBM XIV, SVC, DS8000, 3PAR, EqualLogic, Lefthand, Solidfire, ZFS, and NetApp E-series are supported in this fashion.
*Coming back to the point I raised earlier (about SDS models that ONLY layer block and NAS services on top of object models, or treat them as completely unrelated things):
In general, while it’s possible to layer block and NAS services on top of object stores, they inherit the underlying behaviors of the object store. This is why people who love Ceph for what it’s good at, don’t typically use it for NAS or block services (which layer on top) except where it’s fast and dirty. When you layer one stack on top of another – you inherit the behaviors (good and bad) of the stack underneath.
It will be VERY interesting (over the fullness of time) to see file and block services layered on top of the decidedly low latency, and epic system IOps (but not hyper-scale distributed) object model of DSSD, but that’s entirely a different story.
When you put it all together:
- Shipping shortly in Q2, EMC ViPR 2.0 is one of the industries strongest software-only hyper-scale object/HDFS storage models.
- We are shipping EMC ScaleIO NOW, and it delivers OPEN, hyper-scale high performing, feature rich (QoS, snapshots and more) software only transactional block. I’ve seen nothing else like it. Think almost 15M IOps and 30GBps from 48 all flash servers, and you’re getting the impact.
- We are shipping software-only EMC VPLEX – today, though limited to VNXe (but this is not a hard limit, and I’m fighting to lift it). There is nothing like it. If you pair it with virtual Recoverpoint appliances, you have the industries strongest software only replication and continuous availability stacks. I’ve seen nothing else like it.
- At EMC World, we have committed software-only Isilon (vOneFS) that some of Isilon’s hyper-scale customer have demanded (on their own hardware and/or OpenCompute). The closest thing out there is Gluster, or MAYBE NDFS.
- At EMC World, committed software-only VNX, which makes the operational model and data services that hundreds of thousands of customers. The closest thing to this are the ZFS variants out there.
- And we can wrap that in a software-only control plane abstraction via the ViPR controller that simplifies abstraction, pooling and automation of it all…
… and you have the STRONGEST SDS portfolio in the industry (which looks like this):
There’s only one thing missing and a couple unknowns:
Here’s the one thing “missing”:
- Why would people still use Gluster if EMC Isilon, which is defensibly a MUCH stronger, more feature-rich scale-out loosely coupled NAS stack is available?
- Why would people still use Ceph if EMC ViPR Object/HDFS, which is a defensibly a MUCH stronger, more feature-rich distributed shared nothing hyper-scale Object stack is available?
- Why would people use DAS/SSD/PCIe storage if EMC ScaleIO which can “liberate, pool, automate, virtualize, and amplify server-attached HDDs, SSDs and PCIe storage” was available?
Why? Because it’s EASY those alternatives. Easy to get. Easy to play with. Easy to learn. Because they are opensource. Because they have a rich community. And… Because you don’t need to have sales rep engage with you to get started – and who likes being sold to (well, sometimes it can be fun :-)?
So – we’re making it easy. I’m happy to say that the ViPR 2.0 data services bits will be available in the same way that we’ve made the ViPR controller available – open, and freely for non-production use. You can get the current bits here, and it will be also be where the 2.0 (controller and data services) bits will be:
We will continue to make EVERYTHING that can be a software-only thing (remember that “tightly coupled scale-out” storage architectures – aka “Type 2” like XtremIO and VMAX have a lot of hardware dependencies – so are less likely) is delivered in this open, easy to access way.
We’ve even moved the virtual Isilon from the current location (available to any EMCer, EMC customer, and EMC partner) to that model here (where it’s available to all).
… do you see a change happening at EMC?
Now, here’s are the things that are “unknown”:
Look, EMC is CLEARLY the leader in the persistence hardware appliance market – whether it’s high end hybrid (VMAX), mid-range “swiss army knife” (VNX), AFA (XtremIO), or Purpose Built Backup Appliances (Data Domain). It’s clear we’re good (not perfect) at helping create new appliance categories (like DSSD).
It’s also CLEAR that we have defensibly strongest the Intellectual Property (both organic and in-organic) portfolio for SDS control and data planes TODAY.
BUT, there are some unknown things – that have less to do with tech, and more to do with business:
- Will EMC be able to overcome our “brand” (which no longer represents us – and hasn’t for years) as a company that is associated with “big, well made, and not low cost, iron” such that our customers look to us for SDS models? (while we continue to also make the best appliances in the market?)
- Will EMC’s incredibly successful go-to-market structure (sales, SEs, channels, support) be able to adapt to fully embrace what is surely a different way of: 1) engaging customers (likely doesn’t start with a sales call – but by giving them the product!); and 2) represents a shifting economic model that SDS models bring (lower bookings, but higher profitability) that are inevitable in the SDS model. Can we do that at SAME TIME expanding the hardware appliance market (with XtremIO, DSSD, VNX, VMAX, DD and more)?
I personally love the fact that I’m part of the fight (along with the other 60,000 EMCers and our partners beside me) to ensure that the answer to fixing the missing thing (free and open access), and ensuring the answer to the two “unknowns” is a FIRM, LOUD, PASSIONATE, AND CLEAR YES.
As always – comments welcome!
Very exciting! Glad to see us bringing the new EMC to the forefront!
Posted by: Derrel | May 07, 2014 at 08:51 AM