VMworld 2012: Flash, Flash, everywhere!

Flash is changing the world.

• It’s changing the world of consumer devices – from my Macbook Pro Retina (which I love), iPhones, iPads and more.
• It’s redefining the world of server-side caching using PCIe-based server caches, led by Fusion-IO, EMC VFcache and others.
• It’s creating new categories of “networked server-side caches”, led by EMC Project Thunder.
• It has made a new storage storage requirement mandatory – the “Hybrid Array” – with some mechanism to leverage some Flash as cache, as a tier, and be able to use that as transparently and easily as possible.
• It has created one of the hottest areas in storage startup land – the “All Flash Array” (AFA) – with folks like Nimble, Pure, Tintri, Violin, etc. all playing.   It is a fact that if you assume that EVERYTHING will write eventually to flash media, you can create whole new categories of capability.  EMC selected what we think is the “pick of the litter” in XtremeIO – and are proudly showing it at VMworld.   We listened to feedback from customers, and did a lot of engineering diligence in making our selection.   We also think that scale-out will become a critical requirement in this space, and have learnt, as others have, that “bolting on” scale-out is really, REALLY hard. 

Time will tell what happens in the AFA category, but I fully suspect that just like EMC’s entry into the server-side cache space with VFCache triggered actions at EMC’s competitors, I suspect that XtremeIO will trigger a second wave of acquisitions of the remaining players.

For detail, updates on what is the latest on VFcache, and secretive stuff on “Project X” – read on!

I’m sure that like always, at VMworld there will be ton of noise from people in one part of this pantheon or another that their part/implementation is the only one that matters!   It will be said with vigor, with passion, and heck, in some cases even with grains of truth and moderate conviction :-)   It is possible that over time, these things will shift – but for the foreseeable future – at least EMC’s view, and my personal view – is that Flash will appear all over the place in the datacenter and beyond.

I think the EMC strategy of trying to innovate and provide value in each of the distinct use cases is right – they are so distinct, they are almost orthogonal.

We’re seeing huge acceleration of customer interest and acquistion of VFCache – and in the category of PCIe-based flash in general.   At VMworld we’re showing the latest coolness that is in VFcache (v1.5 – which is now GA) – and it’s very impressive:

• Orchestrated vMotion – in fact, I believe this is an industry first in a way that VMware supports, and works with a ton of guest OSes.
• Cache Deduplication
• Multi-card support
• Larger configurations (700GB card)
• UCS blade support
• Support for active/passive clusters like SQL Server 2012 using WSFC

Check out the new code in action in the demo below:

You can download this demo in high-rez here in MP4 format and WMV format.

…That said – the one where there is the most overlap are the use cases for the All-Flash Array (AFA) and the Hybrid Array .   If you look at the demonstrations I covered in SPO3338 and you can see in this Project X blog post here – it is REALLY compelling.

So – let’s put it on the table?    Why not just go all Isilon/Xtreme IO if you’re EMC?  Or put another way - why do I think that the AFA market will be still relatively small in 2013?

Well – it’s pretty straightforward:

• It’s impossible.   Here’s some math: in 2011, ~ 1.8 Zettabytes of information was created.   In 2011, the total flash fab capacity globally was 21 Exabytes (~100 times less).  Oh, and 19 Exabytes got sucked up by consumer devices like iPhones, iPads, and Samsung Galaxy phones :-)   That means that the GLOBAL flash manufacturing capacity could cover, oh… about 0.01% of the total information storage need that was created (not to mention info that was created in years past).    Now, the AFA folks like XtremeIO can do some amazing things to optimize around their unique assumptions – like inline deduplication – but even if you assumed a nuts dedupe ratio like 100:1 (which is nuts for general purpose data – where 5-20:1 is more normal), you’re still talking about 1% of the total data that needs storage – that was created in year.   Therefore – hybrid arrays will continue to dominate by share/market etc, even while AFA become more an more prevalent.   Even over time – there will be a need for huge TB dumps, often in backup and NAS land.
• AFA will continue to demand a price premium.  The largest, fastest growing market is what we call the “S75” (sub $75K) market segment.   In this segment, people have very generalized requirements.  They need a swiss-army knife, because that’s all they will have.   If people are wondering why the VNX is doing well – its’ because it is a killer "swiss army knife”.   In that market segment, simple, efficient, general purpose/“do it all”, and frankly a set of entry price configurations leads to things that “look and feel” like EMC VNX or NetApp FAS 7-mode (this isn’t the post to battle out pros and cons) platforms to be the thing of choice.   Why couldn’t AFA arrays get into this segment?  Well, for some use cases, they can be economically viable right now – specifically those where all the data is hot (some weird database use cases) and PCIe read caches and array tiering don’t cut it, or latencies need to be really low (finance vertical), or deduplication is overwhelming (VDI).   I still expect that over years (2015?) flash manufacturing capacity, chip density, or alternate memory technologies will come into play – and at that point AFA will start to eat materially into the Hybrid array use cases.   Why couldn’t it happen faster?   Well, that’s easy: increased scale manufacturing isn’t easy or cheap.   The new technologies (phase-change, etc) aren’t ready yet, and the mature technologies (flash) have a steep price tag.  The price tag of a 3.75 Exabyte Flash Fab is oh, about $10 Billion.   That means if someone decided they wanted to manufacture the world need for flash – it would cost $5 Trillion.  Will it happen eventually?  You bet.  Tomorrow?  Nope.
• Storage stacks “burn in” time to mature.  I’ll hit on the specifics for XtremeIO in a separate post here.

That said – the emergence of Flash is changing everything, and is pervasive in the datacenter… just like x86, just like virtualization, just like low-cost memory, just like merchant silicon in networking, just like mass automation, just like new database and data management models…

Interesting times we live in, eh?!   I’m curious – what do YOU think about all this?  How are YOU using Flash and what are you seeing?  What are you hearing from the new players in the market?

Comments

[Barry]

Interesting thoughts Chad. I am interested in average lifespans people keep their arrays for. My previous company may run an enterprise array for 7 years subject to extended support being available. The same can be said of our customers. With that in mind what are the expected lifespans of some of this FLASH technology? I would certainly struggle to recommend a very expensive FLASH array to someone that will need replacing in three years.

[Don]

Barry, I have to disagree with your assessment of Flash and the longevity issue.
You stated that you used to keep arrays around for 7 years ... 7 years on an array is actually very inefficient use of power, space and cooling...almost all vendors see a doubling of capacity drives every year for the same price, this means that after 3 years, the old 72 GB drives you bought can be replaced with 600 GB drives at the same price, 1 TB after 4 years, 2 TB after 5 years, 4 TB and evntually 8TB after 7years.
In the last 3 years, we've seen the Cache on the arrays 5 to 20 times larger than previous whic immensely increases your response times.
If typical storage cabinet is approximately 128 drives, with Raid 5 and some hot spares, your looking at 100 drives at 72 GB so 7.2 TB array, after 7 years, that array could be replaced with 1 drive.

I would consider changing out your hardware more often than you do. I've also seen a large increase in drive replacement usually at 4 year period. Not much different from the SSD drives.

We've recently added a tiered storage array and 3% SSD allowed for a 4 times increase in IO and response times under 1 ms. Our DBAs are drooling ... Can't argue with results.

Cheers

[Chad Sakac]

@Barry - at EMC we've put the drives through the WRINGER. They have the exact same warranty and support model as the magnetic media. In consumer land, loads of the issues that people saw with SSD drives have never been an issue with the drives used in the enterprise arrays. In the early days of flash (EMC started shipping Flash in 2008), this was really only with SLC drives (which come at a large price premium), but these days, we support eMLC (enterprise-class MLC) drives - which just represents the ongoing downward price movement on flash (8x lower cost over the last few years).

Short version - move forward with confidence!

@Don - always great to have a customer chime in! I totally agree with your assessment. Thanks for the comment!

[Barry]

@Don thanks for the input Don. Trust me I fully understand efficiency and drive capacity arguments etc. I work in a mainly Education market where the limited funds have to be focused on delivering the teaching and also funding models change depending on which Government is in office. A typical establishment who have invested heavily in an array have data growth. This growth means they need to purchase additional shelves year on year so, you could argue, increasing their technical debt on the legacy array. As much as I agree about replacement after 3 or 4 years in the real (non corporate) world that is challenging when in the last 24 months the customer may have purchased $50k worth of shelves and drives.