EMC World Day 3: DSSD Tech Preview

Today, we also revealed more to the world about DSSD – which is positively face-melting :-)   What is DSSD?

• It is NOT a storage array in the traditional sense – but is a persistence target.
• It IS an insane dense NAND package – think order of magnitude more dense in both GB/RU and IOps/RU than everything else you see.
• It IS an insane low latency NVM thing – think tens of microseconds.  It’s not DRAM, but as close as you can get with NAND.

What the heck is it used for?   Think:

• modern hyper-transactional data fabrics that use HDFS as the underlying strata
• and modern hyper-transactional data fabric with a little bit of work to integrate
• key value stores with a little bit of work to integrate
• Insane HPC scenarioes (see the TACC example here: http://insidehpc.com/2015/04/taccs-wrangler-uses-dssd-technology-for-data-intensive-computing/)
• .. and if you REALLY need it to dress up as a storage target (a LUN/filesystem) that’s possible too… but frankly that’s like attaching a horse hitch to a Tesla.

I want to be really, really clear – DSSD’s magic is in both the software and the hardware – but there’s no way anyone could confuse this for “Software Defined Storage” – it’s as “hardware defined” as you can get.

In fact, it highlights something I believe – innovation in software and hardware are cyclical.   It’s not that software commoditizes hardware or vice versa – it’s that innovation in one creates an opportunity for innovation in the other and then this repeats.   For math majors, imagine that hardware innovation = SIN(X) and software innovation is COS(X).  They are waves – just phase-shifted by pi/2. 

A core hardware innovation triggers software innovation until it maximizes (and then commoditizes) the hardware, then software runs ahead, until the cycle starts again.

Rather than go on and on about DSSD – I wanted to give a bit of a “backstage tour”.  DSSD is in the directed availability period – we are taking on customers slowly – giving the engineering team an opportunity to keep hardening and developing the stack.

Read on for a “top secret tour” of DSSD :-)

So – I was in California a couple weeks back to visit customers, and do some EMC World prep.   There was lots getting me down (yup, happens to me too), and wanted to swing by the DSSD office to say hello, and get to know folks (and frankly, get a pick-me-up).

The DSSD campus is in Menlo Park – in an old warehouse that’s been reno’ed.  It’s tucked away, and when I arrived early in the morning it was already bustling.

Right off the lobby, there’s a conference room – and it was filled with new employees, all getting ramped up to support the installations that are starting to go on – in SE roles, in Customer support roles.

There was that palpable “startup feeling”, but what was great was that since they were already acquired, they were able to rapid ramp their staff faster than they would be able to naturally – the team is about 160 people and growing fast.

I ran into Bill Moore – the co-founder of DSSD, and all round good dude.   He has a nerd passion that echoes in me.  He was one of the earliest employees at 3PAR in may 1999 (wow – I can’t believe that was 16 years ago – the iPhone was unveiled 8 years later in 2007).   His other claim to fame was co-developing ZFS as a distinguished engineer later in his Sun career (was a staff engineer at Sun back in 1996).

During the acquisition party, Bill needed to be on vacation with his family, so they had this picture stand in for him :-)

During the acquisition process, a huge part of why we went “all-in” was the team.   It was very early in the startup life, but the idea and early execution was great, and the team – not just Bill, but the whole crew – were awesome.

He and I chit-chatted, and then went for a tour…

We walked over to the earliest system integration and hand-assembly lab.

It DIDcontain awesome :-)

There were some of the first elements, and one of the best soldering stations I’ve ever seen… It made me jealous.

You could see some of the earliest hand-assembled DSSD prototypes and components. 

All the components had nicknames silkscreened on the custom motherboards, and my favorite part was that they were all named after comic book/graphic novel villains :-)

Here is a very early DSSD prototype :-)  

I like the custom “power supply levitation system” on the left and the custom “high airflow cooling apparatus” used to keep it from cooking :-

Here on the left you can see a test harness for the host (client) PCIe adapter, where they could test out the software stack used to access the DSSD target via external PCIe gen3 4 lane interfaces.

On the right – you can see one of the early prototypes of the host client PCIe card aka “Isley” – aka the real name of “Poison Ivy”.  

It will be interesting to see how the emergence of these disaggregated architectures will drive extreme low latency and high-bandwidth networks as a new core server topology.

So, the DSSD appliance has a series of “Flash Modules”

That’s the silver thing in the picture on the left.   The packaging is CRAZY dense.  Think of 512 flash NAND die working in parallel in a stacked configuration.  

Coupled with the SRAM cache and controller, that dense NAND configuration mean insane parallelism, but it also means that each card can draw around 45-60W, and cooling is important.   The flash module package itself is a big honking heat sink.  This ain’t an SSD.

Peeling apart the flash module in the picture on the right, you can see the very, very dense NAND packaging.

This leads to one of the main design considerations at the system level – power and cooling.   This was one of the system-level airflow test units.  

The power draw of the fully populated DSSD gen 1 unit can we well north of 2000W/5U – so keeping the stuff cool needs a lot of airflow.   Suffice it to say it has custom-engineered cooling and fans – and standing behind it at even 30% use was like leaning into a hurricane :-)

In this picture, I’m literally leaning a little in, being buffeted by the airflow :)

So – how do you design a system for the necessary airflow?  Early in my career I did RF design, and that had an element of “art” to it – no matter how well you mathematically modeled it – the real world would always get you.   Airflow is like that – it’s a very chaotic system – small things create big unpredictable results.

Today, however, due to 3D printing, building prototypes and then literally testing the physical parts – the DSSD team can iterate.  In the lab they had a couple Makerbots working furiously.

You can see them in action on the left and the right here

It makes me smile with joy – the new tools that exist to create, they will drive new waves of innovation.   While Makerbot is having some difficulties, I think they deserve our support.

At least that’s what I’m telling my wife as I order my own Makerbot Replicator Z18 :-)

Moving out of the warehouse and into the other building across the street (an early former Sun office), we saw people furiously working on the software stacks and system-level integration and testing.   These pictures look like there were no people – but it was packed and people were working in small teams with great intensity.   On the right you can see right though the glass walls into their system-level test lab.

Speaking of the system-level test lab… 

 

Here you can see a DSSD D5 unit being tested under load in the way it’s intended to be used – at rack-scale, attached to around 48 hosts, and hammering away at a transactional workload that is either using a KV store, or one of the transactional data fabric tools that sits on top of HDFS.

Looking at the back, you can see that each host has a redundant connection to the PCIe fabric.

During the tour, there were numerous things that Bill showed me that highlighted the passion they have for the little things.   They didn’t like the way the LEDs on the PCIe ports worked.  They talked to the people that made them, and still weren’t happy with color intensity and variation.  So they built their own with light pipes :-

The other part of the building was cool from a history standpoint.  It was one of the earlier Sun buildings from their halcyon days. 

Of course, between Andy Bechtolsheim, Bill Moore, Jeff Bonwick and many others – there’s a lot of people whose history is linked to Sun.  

While they tore down walls, put in glass, made it hip and modern – they kept one wall (right) in the original Sun purple as a reminder.

Another little fun touch with good timing :-)   Remember each of the generations of components has an comic/graphic novel connection?   Check out

one of the earlier PCIe switch fabric prototypes – Ultron.   Bad-a$$ :-)

Today we demonstrated the power of DSSD on stage at EMC World.   While we don’t want to get people TOO excited (still needs bake time, working through directed availability), its soooo damn cool, it’s hard not too :-)

Near DRAM latencies.   Orders of magnitude higher densities, throughput, bandwidth – native support for KV store APIs, MemcacheD, transactional HDFS.   Really freaking cool.  Coming to a theater near you :-)

 

 

 

 

 

 

Comments

[Shaluka Perera]

How is this different from "bare metal" designs based on custom hot-plug NAND modules, redundant host port trays and fibre channel (or other) host connectivity? IMO, the key is to drive performance and low latency and keep it simple at Tier 0 storage. Looks like using off-the-shelf SSDs will not get one there.