VMworld 2013: EMC Hadoop Starter Kit with VMware Serengeti

This is a really cool initiative that starts with a small group of folks (congrats guys!).   I've been so happy to see the vSpecialist crew we started 4 years ago go on to do so many awesome things:   Nick Weaver building Zombie at VMware (link here).  Clint Kitson and others in BRS building “Mater” and other automation tools to power the vCHS data protection capabilities (more on this later on Tuesday).  So many former vSpecialists are in various SE roles in the EMC and VMware family.   There are always things you wish you did differently, but overall it puts a smile on one's face :-)

This particular initiative is led by two vSpecialists (Ed Walsh and Jim Ruddy) who are now part of the EMC Office of the CTO, specifically in the Open Innovation Lab, who work with customers on funky things, and then figure out how to share it.   Here they are talking about it in code language (and a little on the why behind the project)

In this case, customers were looking to dramatically simplify and accelerate their Hadoop environments.   These guys packaged up a solution leveraging VMware Serengeti (vSphere Big Data Extensions) and Isilon (with cool HDFS mojo) into a repeatable cookbook.  Upside? 

• Rapid provisioning - Hadoop doesn't get easier than this.
• High availability - HA protection can be provided through the virtualization platform to protect the single points of failure (SPOF) in the Hadoop system, such as the NameNode for HDFS and JobTracker for MapReduce.
• Elasticity - easily scale up and down - and be more efficient about resource utilization and lowering costs.
• Multi-tenancy - stronger VM-grade resource and security isolation. With virtualization, mixed workloads that include non-Hadoop applications can run alongside Hadoop on the same physical cluster
• Portability - Use any Hadoop distribution througout the Big Data application lifecycle with zero data migration - Apache Open Source, Pivotal HD, Cloudera, Hortonworks

See it in action here (a demo is worth a 1000 words):

You can can find out more, and start contributing to the community here.

PLAY, LEARN, PARTICIPATE!

Comments

[Phil Utschig]

Chad-

This is GREAT stuff! Hopefully we can build this into a VSPEX RA: Hadoop Starter Kit + Pivotal HD.

I've got 3 partners asking for help with this.

Phil

[Kartik]

Hey Chad,
Thanks! A targeted Serengeti release towards Pivotal HD would also be great, ideally onto Cloud Foundry. I just met a customer that had great interest in this.
Kartik.