An weird perspective on an incredible story

2 So – today, EMC’s Big Data Appliance is revealed.

To be competitive in the changing world – one needs to adapt constantly.  In fact reinvention, expansion, change – frankly “adaptability” is IMO one of the most important traits for a person, or a company.

The reality is that there are use cases where traditional shared storage models start to break down.    Extreme bandwidth workloads like the ETL portions of data analytics are one example that tend to crush the bandwidth limits of storage arrays (the big honkin’ scale-out ones included), as well as creating very weird (bad) price/performance equations when that’s the name of the game. 

We saw change happening fast and furious in this space, and the requirements were very orthogonal compared to the stuff we were seeing in others.   And, most of all – real customer need.

Let’s be clear – there are a couple different use cases for analytics – that need very different things:

1. IF the metric of success for the customer is “I need to do a little analytics, and my requirements are really squishy and elastic, and I’d like to use excess compute I have idle as needed” – scale-out, virtualize it, run it on a Vblock (or your own virtualized stack – it’s all about choice, baby!) as pure software.  Awesome.   We showed how you could use the linear scaling of GreenPlum 4.0 on vSphere and Vblock at VMworld this year – and it was pretty cool – you can see it here.  (Note – this is an example of what I mean when I say that within EMC we are big fans of the “sprinkle on new capabilities via Virtual Appliance models”)
2. IF the metric of success for the customer is “analytics are my lifeblood – there is an INSANE amount of data, and my requirements are elastic, but only in the sense that they vary from “INSANE” to “(INSANE)^insane” – you need to have a purpose-built appliance using DAS scale-out, shared nothing models.   There’s just no other way to get to the ETL times and the query performance you need.

Either way – not only is brute force important – but we think making the process of analytics a social process is very important.   This makes it easy to share, and collaboration and community gets wiki-economics working for you :-) 

That idea of easy access to information is very very important and a huge part of what I dig about GreenPlum.   It’s LESS about going to the proverbial “oracle” (ha!) on the mount and asking in a strange language known only by a few.   It’s MORE about than making everyone able to ask their own questions, get some answers, share your thinking and get other people talking.

OK – so what’s the new thing?  And, what’s this weird story, Chad?

If you’re interested – read on…

First of all – the appliance….

Yes - it has massive capability (we think it has 2x the ETL bandwidth of the competitors including their most recent stuff, and beats them all on query price/performance).  

Yes – it is comes as an appliance (here’s what it looks like).  Available in 1/2 rack configs, full rack configs, and scales up to 24 full cabinets if that’s what you need (2-5PB!)

Great quote from a customer:

“The new appliance gives us greater flexibility in how we approach our ever growing needs to apply sophisticated analytics to big data.” - Steve Hirsch, Chief Data Officer, NYSE Euronext

Yes – we have leveraged and integrated other EMC assets to extend DR (integrated Recoverpoint solution) and simple backup (DataDomain integrated solution with huge bandwidth capabilities that looks like this)

 

 

So – what’s the weird part of the story?   To me the standout was the crazy pace that this all happened.   The inside story tells you a little about the way EMC thinks and operates.   We have TONS of flaws, and TONS of things to work on.  But – when it comes time to make a decision, it happens fast.   And when the decision is made, the execution focus is a bit crazy.    As my blog title points out this is an “insider’s view” on an interesting episode.

The internal launch was about a month ago in early September (yup, we’re manufacturing and selling them), at which point one of the people on the project sent out a little email with the timeline (Mike Wytenus, hope you don’t mind me sharing and adding commentary).  Check it out:

• In January 2010 at the Exec staff (and also more broadly) there was a collective sense inside EMC that the DW/BI market must be both defended (we were seeing real customer choice driving towards appliance models in these “all about price/performance + appliance” use cases).   We could have “stuck our head in sand” and pretended that general purpose shared storage is ALWAYS right (which would have been delusional).   We decide we need to look at what it will take in our roadmaps to make general purpose shared storage more suited to this use case (massive bandwidth increases), but also to explore other options.  We decide that while a bit scary, if done right, it is also a growth opportunity for EMC.  
  • Moral?  We make mistakes all the time, but one thing we’re not is delusional.  There is surprisingly little koolaid drinking internally, and we tend (if anything) to be OVER self-critical. 
• By February 2010, executive ranks set in place sponsorship and provide investment to get us headed in right direction
  • Moral?  Listen – there are politics anytime you have more than 3 people.  And EVERYONE has some degree of ego.   But, one strength I like about EMC is when the decision is made, the support happens, and we start marching together.  
• Around March 2010, the Appliance POR gets set in 2Q10 for a July target date .. even though we don’t have all the pieces .. anxiety and intensity builds
  • Moral?  Sometimes you need to boldly go… in spite of not knowing all the answers :-) 
• Between March and June – the team rallies and agrees on competitive product specification and builds out cross functional activities to support imminent product availability and go-to-market plan
  • Moral?  Just like the earlier point on politics – there’s ALWAYS some bureaucracy (in fact, just the right amount can be a good thing – but not an OUNCE more than that :-)    But, in this sort of thing a small team rules + minimal bureaucracy the day.  I wasn’t directly part of this team, but was plugged in on the side – was amazing to see them go at it.
• July 29th, 2010 - EMC buys critical component in stack, Greenplum.   All hands on deck attitude and modus operandi accelerates.  Just like with Data Domain – this sort of acquisition (relatively big, something where the “acquiree” clearly knows more about the space than the “acquirer”, coupled with a loose linkage to existing businesses), our general way of working is we make the people who were just acquired the heads of a new operational division.   So – Greenplum, their leaders, in fact their location (in Silicon Valley) become ground zero of EMC’s Data Computing Division.
  • Moral?  Recognizing when you don’t have what it takes organically means you should exit, or acquire.   We’ve learned that acquiring someone with a less than killer product thinking we (EMC) can make it better is a very difficult proposition – particularly when it’s not something very, very close to something you’re already very good at.   So… acquire killer tech and people, and empower them to run with resourcs, rather than quickly forcing them to mesh with the machine.
• Sept 9th, 2010 - We build, test, confirm and release product …. with traditional EMC quality attitude … a greener, faster, lower overall cost alternative to competition … only 1 month and 1 week after original POR target.
  • Moral?  Put it all together – and you can literally go from 0-100 (no play in a market to a very, VERY competitive one) fast… VERY FAST.

Personally – I see this set of capabilities (make the “blink” decision; setup a structure to look at the challenge; setup and empower a small team to analyze in detail and solve; don’t be afraid to use M&A; execute) , as very important for our ever-changing world.  For individuals  as well as for companies.  For startups as well as FT500 behemoths (EMC being #166).   Fast change, and dealing with implications of fast change is very important.

I see this in many of our strong competitors – the ones who we tango with.   If you don’t have the organizational DNA to do the above, I have a hint for you – you had better get it.   There are competitors out there (ahem, Oracle with Larry at the helm), who will throw all partners under the bus and eat your collective lunches.    There are industry sea-changes (ones we see now, and others we still don’t see) that make startups into giants, and chew up and spit out the most successful enterprises.

In any case – of course the REAL execution test for EMC in this space starts now.  The challenges in front of us are vast, and ultimately – customers and the market will gauge our success.

But – I feel positive, have a spring in my step knowing that we CAN and WILL face this challenge and any others (the transition towards cloud models being another) that come our way.

Comments

[Jeremiah Cook]

This is an amazing story of the teamwork and innovation required to compete in today's market. Great job EMC. Very exciting. Can't wait to see more. So glad to be part of the extended team, as an EMC partner.