Understanding Project “Lightning”

This is one of the bigger pieces of news coming out from EMC World 2011 in my opinion.

For those following the industry, it shouldn’t be too much of a surprise – but it strangely will be..  Flash is going to change everything, and exist everwhere.   It’s no different than any fundamental technology innovation.  When something is an order of magnitude (or more) faster, cheaper, smaller – it doesn’t get used in ONE way (as a tier, as a cache, as an extension of server memory), but tends to get used in MANY ways.

Flash is one of those examples.   It is literally 100x-1000x faster than magnetic media, and 10x-100x cheaper per unit capacity than DRAM.  

So – why would ANYONE think it would only replace high-performance magnetic media (like 10K and 15K RPM disks in the form of automated tiering like EMC FAST VP), or only as an extension of cache (like EMC FAST Cache), or only as an extension of host system memory (like Fusion/IO)?    We certainly don’t (and note how consistent we’ve been on this point – I’ve personally pointed it out over the last 2 years, here, here and here as examples).

I’ll quote myself (just to point out – we work/think about these things for a long time, so when we put something out there early, like Lightning, there’s a LOT behind it)

“Now you can see why I go on and on about Flash being transformative, and how it will be applied EVERYWHERE.   You’ll see it in servers, you’ll see it in array caches, you’ll see it in non-volatile storage tiers, you’ll see it in laptops, desktops, mobile devices – EVERYWHERE.   Just like “protocol passionistas”, I would tend to not listen to people who say “one way always”.   It’s just as likely that it’s simply what suits them.”

This is likely something everyone is going to need to do (like auto-tiering, like flash-assisted caches, like dedupe, compression, etc…)

So – what have we announced?  Well beyond making our broader flash strategy more clear (including: all-flash VMAX and VNX configurations; that we are going to do MLC in addition to SLC; and more) – the big news is outing the fact that for many moons, we’ve been quietly working on a project to not only deliver a server-side flash-based PCIe card, but more importantly – an end-to-end solution around applying the idea of “tiering” from the server all the way to the array.

If you’re interested, read on for more…

First thing – let’s briefly hit the other parts of the flash discussion:

• All flash VMAX and VNX configurations.   This may seem extreme, but there is surprising demand for this already.  We have formalized these configurations to make them easier to acquire.  Those of you that need this know you need this.
• Embracing Multi-Layer Cell (MLC) based Flash.   MLC based flash is what’s used in the majority of consumer SSDs.    When we designing for SSD and selling SSDs configurations with EMC storage 3 years ago, only Single Layer Cell (SLC) based SSDs could deliver the write performance and withstand the enterprise storage duty cycles.   But, through the inevitable march of progress, MLC has come a long way.   We think that MLC is ready for prime time, and we’ve announced that we will be qualifying MLC based SSDs for EMC use.   There’s a ton of innovation happening in this space, which is cool.    It’s also interesting that this (I know I sound like a broken record) portends the future of all-SSD based configurations being mainstream.   You’ll have a variety of tiers – but eventually ALL solid state (will take time to push out magnetic media, but to me it just seems inevitable).

But – the big news is Project Lightning.

So – Project Lightning is at it’s core – EMC server-side PCIe flash, but more importantly software that resolves one of the biggest downside of PCIe-based flash – which is by definition of current hardware-only models, it is captive to the server in which it resides.

The view driving this model (at least within EMC) that server-side PCIe-connected flash hardware is a commodity, or about to become a commodity.   Imagine the day that a future Intel Chipset natively supports both DRAM modules and little flash modules and you see how inevitable that is.   That’s not to say that PCIe server-side flash isn’t a good thing – it can be an order of magnitude faster in latency than accessing an array.

But – what if…

• there was a distributed cache model across not only all the server-side flash, but that extended to the array itself
• and workloads and data could move based on policy/SLA automatically?  

Wouldn’t that be cool?

Certain workloads are gated primarily by latency (milliseconds, microseconds, nanoseconds).   These are often ultra transactional systems.   Very often, those are against relatively small datasets (GB-TB).   Think of a trading application.   In these cases, moving the data up to be closer to the compute for the purpose of reducing latency matters.

Other workloads are gated primarily by bandwidth (MBps/GBps).   These are often “big data” systems – parallel workloads against large datasets (TB-PB), often coupled with distributed parallelized compute models.   Think of any MapReduce type job.  In those cases, it might be useful to move the compute down to be closer to the data matters.  

During Pat’s keynote, we demonstrated one way that this may work – moving workloads between ESX clusters via vMotion.   The starting cluster was a regular “joe blow” server cluster.  

We vMotioned a transactional workload to a vSphere cluster which was accelerated by the LIghtning hardware/software combination, lowering the workload’s latency.

Then, we vMotioned a bandwidth-constrained workload to a vSphere cluster which was running co-resident on the same hardware running Isilon, increasing the amount of bandwidth dramatically.   Yes, this idea (vSphere running on the arrays) does indeed exist within the walls of EMC, as does vSphere running co-resident on VMAX hardware.   If you think about a big Isilon cluster, with 100+ nodes of Intel x86 based power, or a future generation VMAX with 16+ similarly Intel-powered storage engines, it makes all the sense in the world – particularly for workloads where bandwidth and the parameters of the dataset make it easier to move the compute closer to the data rather than the other way around.

WARNING: this is directional folks – you cannot buy this yet.   Yes, we are previewing something before we are shipping it.   But, I think that this is what industry leaders do – they point the way.   We did it on SSDs.   EMC has been shipping SSD now for 3 years, and EMC has shipped 14PB of solid state storage – more than any other storage vendor.   almost 60% of the arrays are going out the door with SSDs.    We did it with Fully Automated Storage Tiering.   We said we would do it, and we did it.     While we could stumble (the price for putting it out there), I’ve been exposed to the Lightning team going at it for some time now – and I think it’s going to be the same thing here – we’re saying it, and we’re going to do it.

Pretty cool if you ask me.   What do you think – are we on the right/wrong track?

Comments

[Mark]

A very good idea. Seamlessly extending the SAN into the server, and the server's local storage into the SAN. It would allow you to use a smaller SSD in the server, while performing like a larger SSD. Also, you could leverage external, SAN-based SSD for high-performance writes, and the internal SSD as a very fast read buffer, providing write consistency and high availability (if that is required by the application). This could be big for big data apps.

[Michael Ryom]

Im wondering how this will work togehter with RPA(Recover Point). DCC seems smart... But how can you write simultaneously to two storage arrays via RPA, wont RPA be a huge bottleneck?

But it looking good :)

[Jcoleis]

First VPLEX Geo, now this? mind = blown.

[50mu]

SSD fill the gap ! Understanding Project “Lightning” #yam

[eprich]

Conceptually I love the idea. I've been asking for something like this for the last couple of years - I'm happy to see EMC has been listening. I can't wait to get some of these in my lab *hint-hint* ;)

[Rob]

I'm not feeling the DCC love. PrestoServe and TRAM went before, technology caught up in the form of SSD. A customer's end of day processing went from 3 hours to 5 minutes with SSD. I'm not seeing the win of additional layer of caching there. I suspect this is for
VMWare to take IO pressure off the storage path. Okay, but it will
be very limited in OS support otherwise? And if SSD is
at the heart of the SAN, how much added benefit?

Anyhow, MRAM to rule it all, eventually!

[John]

Any idea with whom EMC is working on the MLC solution and on lighting project?
There are rumors on small starup companies involved in these projects.. Is it true?

[Tim Kossack]

What about blade servers...?
will emc offer the cards for the servers themselves or will it license the ip to server vendors like ibm, dell, hp etc.?

[Tim Kossack]

Hi Chad,

will EMC itself offer these cards or just some sort of driver-software?

[Vaughn Stewart]

Very nice announcement. Is there any capabilities that Lightning will provide that are beyond what is available from FusionIO today? I'm a big fan of caching technologies and am hoping there's some additional magic bundled in.

[Mark Burgess]

Hi Chad,

I tried to get some more information on Project Lightning at EMC World but it was so top secret that no one knew anything about it!!!

Could you therefore help with the following:

1. How do the cards work (i.e. do they use FAST Cache with read and write)?

2. Will they work with any array and protocol (i.e. FC, iSCSI and NFS, EMC and non-EMC)?

3. Do they require FAST VP/Cache on the array (VNX/CX4)?

4. What is the typical use case (i.e. virtual desktops)?

5. What is the timescale for release?

One conclusion I came to this week is that Flash is rapidly becoming mainstream and it is not going to be long before SAS drives disappear.

So what is the timescale for MLC based EFDs (someone mentioned Q4)?

Also I saw from your great Chad's World demo that the Iomega devices are supporting flash.

Does this mean that the VNXe3100 will also support flash?

I know in the past we were only expecting flash on the 3300.

Many thanks as always
Mark