Another performance record… broken!

For what it’s worth, no one can say we didn’t say we were going be breaking records and that we’re not doing what we said we would :-)

This week, EMC VNX shattered the SPEC SFS 2008 CIFS record with a result of 661,951 Operations per second, and a latency of 0.81 ms.  That result is more than 4x more than the previous record – and the latency was more than 2x better than the next closest competitor (details here)

This is “the other shoe dropping” (it will be raining shoes, BTW) – the CIFS version of our NFS record breaker here.  The Register, covers it in the link below:

Putting the SPEC SFS NFS and CIFS results into a scatterplot visual representation, it looks like this for NFS:

…and like this for CIFS:

I suspect that EMCers and EMC partners might want to have these two slides I whipped up – so here they are in ppt format.

I intentionally didn’t label the “other guy” as I want to make this positive – i.e. we’re proud of our result, and we think it highlights some architectural strengths we have.   I will quickly hit the common QnA I see we got on these benchmarks (and the feedback/comments/rants I see on The Register threads).  If you’re interested, read on….

Q: Didn’t you use multiple VNX NAS and multiple VNX Block components?  Isn’t that not “one array”?

A: Yes, we did.   The definition of what an “array” is/isn’t is pretty darn tough to nail down in all those scatterplot points – as they are all wildly different architecturally.   I’ll give you a couple examples.  

• Some of the entries (I’m not going to use names to avoid getting into an accusation of being negative) are scale-out in the sense that they have many “brains” each attached to independent “disks” and can have a single filesystem span all the components.  Is that “one array”?   I would say yes. 
• Others are configurations with two independent brains, each supporting their own filesystems, separate licenses required for each brain, and attached to two sets of completely independent disks (except during “brain failure”).  Those two brains work independently, but support each other in case one fails.   Is that “one array”?  I would say yes. 

Architectural customer benefit (IMO): the VNX has an architecture that enables you to have as many or as few components as you need for block and NAS uses, and they are ALL managed by a single UI that’s built right into the platform.  The common UI goes beyond look and feel, most (but still not all) functions are common on NAS and Block storage models.   Point a browser at an IP, and you see/manage it all as one in Unisphere.   No additional management tools needed.   architecturally in VNX, storage capacity is shared completely by the NAS components, but a filesystem can’t span the heads.   Is that “one array”?  I would say yes, in analagous way that the different architectures above I would call “one array”.    Each of the folks with a dot on the scatterplot have arrays that are built differently.  

Any customer who wants to see just how easy it is to manage from small scale to these record-breaking scales as you add more block compute/NAS compute/SSD/large capacity HDD with Unisphere - we’re happy to show you :-)

Q: Isn’t this configuration loaded for bear with SSD?    Is that “real world”?

A: Yup, it sure is loaded with SSD!   We used several hundred SSDs.   Is that a “real world configuration”?  YES.   I literally saw a customer who ordered an EMC array 100% loaded with SSDs.  Hundreds, and hundreds of them.  

Architectural customer benefit (IMO): VNX and VMAX were completely designed to take advantage of, and scale to configurations with TONS of SSDs.   We saw how much SSDs were going to impact everything we in storage land, and started our own huge efforts over the last 3 years to be able to a) cope with it (i.e. eliminate architectural barriers/bottlenecks); b) leverage it using it as an extension of system cache via FAST Cache for NAS/Block use cases where the arrays didn’t already have huge global shared caches; c) use as a tier and deliver FAST VP to be able to deliver lower cost and higher performance at the same time.  

In the older gen EMC kit, if you had as much as we put into these configs, they would keel right over.  This is the march of progress.  Here’s an example: the 3 NS-960’s that powered the VMworld 2010 hands-on-labs (7,640,306,790 IOs served!!) only had 15 SSD each, but VMware is asking for 3 VNX 7500s with 90 SSDs each for VMworld 2011.   That means they will be able to do WAY MORE than last year using WAY LESS space, power, cooling.

This architectural advantage is rooted in the fact that with VNX you can scale out to many blades for NAS or block functions, each with many Intel westmere cores and PCIe gen 2 connectivity.   But, it’s as much about a lot of software R&D investments made over the last few years to be able to actually LEVERAGE both the SSD trend and the massive multicore trends.    BTW, in these tests, we also saw that we had enough headroom for some of the future capabilities we’ll be adding to VNX, like primary storage block-level dedupe to augment the existing file-level dedupe and compression (which Pat Gelsinger already outed, so I can’t get in trouble for saying it :-)   There’s a lot of coolness that will continue to come for EMC customers using VNX, and much Pat hasn’t disclosed, so I won’t either.

While most customers won’t be using all SSD configs, in EMC’s midrange, I looked at the stats for one of our divisions in the Americas, and they had SSD attach rates that were nearly 100%.  

• Heck, even if customers are using 10% or 20% SSDs and 90-80% large capacity magnetics spindles, doesn’t it matter to customers choosing architectures to know that they COULD scale as needed?  
• Wouldn’t you want an array that you could mix and match SSD however you need, and mix slower/larger capacity magnetic media in any ratio that you need?
• Wouldn’t you want to be able to hot-add as much of either as you need, and have the array automatically move everything around as the workloads changed?
• Wouldn’t you want to know that you had that kind of headroom (one of the purposes of crazy benchmarking is to show headroom)  Oh – and like I said earlier, manage it all as one in one simple, easy to use UI? 

I would challenge anyone pointing out the fact that we used a lot of SSD with this counter: “ummm, yeah.   SSD at scale is a really important thing for our customers, so go right ahead, respond – do an SSD-heavy response, nothing is stopping you.”   That is… unless you CAN’T scale up with larger amounts of SSD due to other things in your system.

Q: But wait, don’t you have multiple filesystems in this test? 

A: Yup, we sure do.  On the VNX today, while the underlying block storage is shared by the NAS components (so you don’t end up with “trapped” storage that is available but not to the NAS component that needs it), a filesystem cannot be more than 16TB in size, and exists on a single NAS component at a time.   BTW – the same is true of all the other posts from non-scale-out NAS offerings (most of the dots on the scatter chart).   All of one storage vendor’s entries for example, are not in cluster mode, but are the common 2 storage processor config and therefore has multiple filesystems (at least one per head).  P.S. I’m sure after all our benchmarking extravaganza, they will likely post a clustered filesystem SPEC SFS result – hey, I would.   BTW – while we’re on that note I saw some interesting threads in various places with comparisons with benchmark results from various competitors in earlier SPEC SFS versions.   That’s not accurate :-)  The SPEC SFS benchmark changed so you literally are comparing two different benchmarks, not just different configurations.   There are a much smaller number of SPEC SFS postings that are scale-out NAS clusters – where a single filesystem can span multiple components.   This is, in some customer’s use cases very, very important.

The VNX is EMC’s “swiss army knife” storage.   We think it is…  Simple.   Efficient.  Powerful.   It is very feature rich, has many enterprise use cases, does a lot of things really well, supporting all protocols, and all via one simple and easy UI – Unisphere.   It can scale to huge amounts of IOps for transactional workloads, and huge amounts of bandwidth (7+ GBps from a single VNX block brain).  Most of the time, customers are best served via “swiss-army knife” storage.   For most customers, a 16TB filesystem is immense.

Sometimes though, customer need a tactical nuke, not a swiss army knife.    In those cases, “doing a lot of things really well” is less important than “doing a smaller set of things amazingly well based on a fundamental architecture advantage”  In NAS land, some customers have requirements for storage platforms that can support a single giant (PBs) filesystem.  Sometimes it needs to be able to scale to 50GBps levels of bandwidth to that single filesystem.   Sometimes it needs many SSDs itself, sometimes it’s all about capacity.   Often, it’s a mix of both SSD and large capacity – just like all storage these days.   Often when those points reflect the customer storage challenge, “management at scale” is the singular largest challenge, and buying requirement.  …and after hearing it in many places, we acquired what we think is the best technology in that market with Isilon.   

More and more, storage vendors are realizing that as much as possible, simplicity is a good thing – but counterbalancing that is that for some use cases, the requirements DEMAND specific architectures.   We aren’t perfect (and we think we can/need to simplify further), but I feel good about EMC’s choices over the years about where to simplify, and where to embrace architectural diversity.   Seems like lately others might be coming to the same conclusions.

BTW – as a follow up to his The Register article, Chris Mellor asks an interesting additional question: “are these benchmarks broken?”.   I don’t know.   I kind of think that ALL benchmarks with single workload tests (ergo all official storage benchmarks I know of) are fundamentally busted in the sense that they typically don’t represent a customer (who usually have mixed workloads).

And – it’s only March of 2011.   There’s going to be a flood of records broken throughout the year.  

Oh – the VNX family and VNXe are generally available, and shipping in volume.   Have at it!

For those of you who aren’t having fun with this (hey I hated that we used to not participate in benchmarks, and you loved it, you can’t get mad now that we’re participating), and saying “EMC only is publishing results for SPEC SFS which suits them, and don’t do SPC-1 and SPC-2, and/or they don’t do it with layered data services”… All I can say is boy, I hope for your sake that isn’t your core argument.

As always, courteous comments are always welcome!

Comments

[Nazareno]

Brook this Record:
1,000,000 IOPS in a 3U package!
The RamSan-630 offers 10 TB SLC Flash storage, 1,000,000 IOPS (10 GB/s) random sustained throughput, and just 500 watts power consumption.

Storage
•Capacity: 10 TB
•LUNs: 1,024 LUNs
•Storage Medium: SLC Flash

Performance
•Maximum Bandwidth: 10 GB/s
•Read IOPS: 1 M IOPS
•Read Latency: 250 microseconds (Not MiLiseconds)
•Write IOPS: 1 M IOPS
•Write Latency: 80 microseconds (Not MiLiseconds)

Interfaces
•Expansion Slots: 5 slots
•Fibre Channel Ports per Card: 2 ports/card
•Fibre Channel Speed per Port: 8 Gb/s
•InfiniBand Channel Width: 4 X
•InfiniBand Channels per Card: 2 channels/card
•InfiniBand Data Rate Standard: QDR
•InfiniBand Speed per Channel: 40 Gb/s

http://www.ramsan.com/products/2

Disk Like SSDs aren't efficient. Maybe PCI-e...
Old Interfaces (SAS, SATA, SCSI, old Fibre Channel) can't solve the problem...

[SteveM]

Wasn't it the RamSan that took Virgin Blue (Navitaire) off-line for 21 hours? And disrupted flights for days costing VB about $15M

http://www.theregister.co.uk/2010/09/30/navitaire_cock_up/

Certainly NetApp have pointed the finger at the RamSan. Fast but not that available perhaps?

$15M starts to make any power bill cheap don't you think?

[Derek]

Chad,

You also need to provide $/operations a second. Speed is one thing, but most customers do not have bottomless pockets.

[Chad Sakac]

@Nazareno - Ramsan, Violin and others are all very interesting, but have NARROW use cases (but where that's the requirement, are often good choices). Not saying they are bad, but rather that they are narrow use cases (they are missing many of the broad/general use case features customers tend to look for in "swiss army knife storage").

@SteveM - yes, it is. It is worth noting that every vendor has a bad day. It's happened to EMC, it's happened to everyone.

@Derek - SPC benchmarks have $/ops as a pre-requisite. SPEC doesn't . The config however, was not expensive. A 5 VG8's are not expensive, and note that the VNX5700's are priced where the CX4-480 was (in fact a bit below). The most expensive part of the config were the SSDs, but even they are a fraction of what they were 6 months ago. if/when EMC does SPC results, there will be a $ figure as part of the benchmark. I'll give you another hint (would make for a fun blog post) - the price is no secret. Ping EMC or an EMC partner and ask for a quote :-)