Fun with completely not supported, pre-alpha EMC tools: Part 3

Ok – this is saving (IMO) the best for last…

One of the things that makes me most happy about my job is meetings, planning, and budgeting… NOT.

In all seriousness – those are necessary evils. What DOES make me happy is working with my team, my fellow EMCers and EMC partners, with customers and with engineering. I’m really lucky to get exposed to a lot of ideas, a lot of technologies, and am surrounded by a lot of people who are constantly coming up with cool – and frankly FUN – ideas.

This is the third of a 3 part series on this topic. Each one will “out” a killer tool. Each of them will grow over time. Each of them is free. For now, all of them are not supported (and in one case not even in circulation).

I will be providing a pile of “Santa Chad” awards to people who innovate on top of the two of these ideas. Think iPads, Iomega PX4s, and VNXe 3300s. Remember, it COULD be throwaway work – since they are not supported products per se. But those of you that know me – there’s always methods to my madness :-)

The first one: “Hyperic/vCenter Operations/EMC VNX NFS Uber Mashup” is here.

The second one: “Beta of Analyzer Helper” is here.

This is the third one: “Project Orion”:

EMC’s greatest strength is an incredible breadth of portfolio and a “string of pearls” acquisition model.  We’re incredibly diverse as a company.   This diversity includes great technologies in the very independent, separate parts of the broader business (think VMware, think RSA, think Iomega, think Greenplum – not to imply these very different examples are similar except that they are a reflection of the broader, more diverse strategic view).

That technology diversity is also true in the more “classic” parts of EMC – Storage and Backup.  It’s a strength.  After all – when a customer needs the storage equivalent of a tactical nuke – they want/need something designed for purpose, like a VMAX or Isilon.  When they need a swiss-army knife, they want need a general purpose platform – like VNX.   When they need backup – the insane growth in purpose-built backup appliances like Data Domain and Avamar speaks to customer demand.

BUT – like all strengths – this is also a weakness.   It introduces complexity in building tools, in integrating with other partners (like Microsoft, SAP, VMware) – where we need to do it several times, and the other folks need to do it once or twice.  It makes scripting and automation harder to maintain.  Beyond that – it makes our GTM more complex (less about technology, more about business) – how do we scale how we support customers?  How do we best enable a channel – but the business thing is a discussion for another post.

Back to the technical question…  We’ve been working hard on the topic of “reduction of complexity where possible, embrace where it’s not possible” for the last few years:

1. In some cases where reducing complexity demands merged kernels, smaller footprint most – we started there (see more here).   We’ve shipped, and will continue to innovate down this path.
2. In other cases reducing complexity demanded merged management models most – we started there (see more here). We’ve shipped, and will continue to innovate down this path.

But even with simplified VNXe/VNX models – what about broader API-level unification across EMC?

So – a team started work on this not so long ago

Read on to see how this works, and where you can use it yourself….

 

It started a while back with this observation:

This is how people used to manage storage (at the highest, most abstract level):

… And today, that’s not how they want to manage storage (again, at the highest, most abstract level) – which tends to look more like this:

That’s what I mean when I keep saying “storage should be invisible”.   

Early versions of this idea (exposing into the application context) are the EMC Virtual Storage Integrator (VMware) and EMC Storage Integrator (Microsoft) – which customers and partners love.

But, solving this more broadly, and offering the proper programmatic northbound multitenant API model is important.   For us, it’s made harder by the below…

Every platform has a different API model. Think of what that means when we are developing VSI – we’ve managed to pull out in front of our competitors in spite of the fact that our breadth of portfolio means we have to develop/test against a LOT more. When it comes to APIs, SMI-S works on some platforms, but not all (and is remains block centric). We have a rich ECIM API on the newer VNX/VNXe platforms – but what do we do as the portfolio continues to expand like when we develop or acquire the next Isilon?

Ultimately – solving the macro problem will involve a lot from us – so stay tuned.   Also, to be fair – today, most customers and providers tend to optimize around the platform that best serves their aggregate needs – and all EMC platform have simple, open APIs.

BUT, there is a better way, a way to have your cake (great constant diversity) and eat it (API simplicity) too... one of the first steps is to make the most fundamental APIs to the storage platforms move to a simple, RESTful interface.  Something fast, simple and lightweight.   Something that you could quickly build a “southbound platform plugin” and a “northbound use case plugin”  The prototype for this was codenamed “Project Orion”.

We’ve hit a milestone where formal work is now going on on this, and we thought it would be cool to air out the idea – get feedback.   While this is absolutely pre-alpha (really, just prototype) code – and there’s nothing to say that if you use it, it isn’t throwaway work, I pushed hard – as I think it’s cool to see what the EMC Solution Group and Infromation Management Group is busy working away on.  Remember, if you ask for support, I’m just going to start laughing :-)

The prototype for Orion included:

• southbound adapters for VNXe, VNX, VMAX, Isilon, VPLEX (and could have easily included more).  
• northbound adapters for vCenter Orchestrator and for Microsoft System Center/Opalis.  

Part of the project was to see – how hard would it be to develop against it?   After all – if it’s not fast, simple and easy – it doesn’t help.

• So, as a proof point – Larry Grant (vSpecialist in NY/NJ, Oracle guru –and author of the great http://www.drvcloud.com/myblog/ blog was the developer of the southbound Isilon and VPLEX modules.
• Likewise, later in the project (umm, last week) the always incredible Clint created a Powershell northbound adapter – meaning, boom – now a single powershell API for the whole of EMC – that’s the power of the idea of Orion.  
• We’re working on a prototype SAP Landscape Virtualization Management module as we speak.

You can see a video demonstration of this in action here:

… So – what does this mean to you?

1. If you’re interested – feel free to download and play with Project Orion.   Remember – IT IS A PROTOTYPE – AND NOT SUPPORTED IN ANY WAY.   We’ve posted it on an ECN community, so people can download, but also comment, and play.   That community page is here:
2. Would LOVE your feedback.  Good idea?  Bad idea?   Implementation – what should we do differently?  Please quickly answer the 4 question survey below.   For more comprehensive feedback – PLEASE visit the ECN community page for this, and comment there, so it’s captured in a thread.

 

Comments

[Andrew Fidel]

Chad, I assume there were supposed to be links in this block that didn't make it to the post:

Back to the technical question… We’ve been working hard on the topic of “reduction of complexity where possible, embrace where it’s not possible” for the last few years:

In some cases where reducing complexity demands merged kernels, smaller footprint most – we started there [b](see more here)[/b]. We’ve shipped, and will continue to innovate down this path.
In other cases reducing complexity demanded merged management models most – we started there [b](see more here)[/b]. We’ve shipped, and will continue to innovate down this path.