EMC ProSphere 1.5-Play, Learn, Try!

EMC ProSphere 1.5 GAed recently – and I’ve been playing with it.  It’s really cool!

The first thing to understand about EMC ProSphere is, in my opinion – that it’s not Control Center.   Control Center is the Storage Resource Management tool in use in many, many large scale environments, and in my experience – I’ve never had a customer say they love it.  In Pat Gelsinger-land here at EMC – that’s the only acceptable bar :-)  

Some customers use Control Center.   Most complain about agents in deployment, scaling, and negative upgrade experiences.   Once up and running, they get value – but it can be painful to get there.

I think of ProSphere as the “Steve Austin” of the Storage Resource Management world – it’s been built from the ground up to be better, stronger, faster… and simpler, agent-less, scalable, federated – and so on.   The only difference from Steve Austin (the “$6M dollar man” BTW from the non americas audience :-) is that it isn’t based on the damaged body of it’s predecessor, and it doesn’t cost $6M :-)

The core design principles as the engineering team set out to make this “Storage Resource Management done right” capability were:

1. Make it simple – and really easy to install, and get to customer benefits quickly.
2. Make it scale and perform like a mofo.
3. Make it easy to integrate with – everything exposed via APIs – and integrate with all other EMC stuff
4. Use standard APIs to extend platform support.

They wanted to get the core architecture right and to get lots of customer feedback before adding features, or broadening support.  

To learn more, see a demo, and learn how to get it to play with, learn, and give us feedback – read on!

The ProSphere engineering team was very focused and very pragmatic on how to develop the platform.   While they started by nailing the core architecture (rather than exhaustive features/use cases), the goal is to, over time,  completely subsume Control Center use cases (in the meantime, there will be continuous Control Center updates for existing customers). 

ProSphere v1 had the goal to nail the performance-centric use cases and prove the architectural model (and get loads of customer feedback – which has been very positive).  v1.5 was to extend to cover the capacity-management use cases, introduce alerting and reporting – and the first parts of passive snooping on the fabric itself.  v2.0 will add a lot more (more on roadmap later).

To give you an idea of what they did:

• To make it simple and easy to deploy – the only way you get it is as a vApp.  It uses 75% less hardware than Control Center at the same scale.  Upgrading is crazy simple – it can auto-upgrade from the Internet.  To get to value quickly, it uses an agent-less design principle.  It’s the “anti Control Center” in all of these aspects.
• To make it scale and perform like a mofo – the whole architecture is based on functional elements that are designed around RESTful principles, and the back-end database uses the GreenPlum database.   If you want to scale, you can scale database, discovery modules – and if you want to federate between multiple sites, it’s a single click.  To understand the scale we’re talking about, imagine: more than 1.2 million volumes, 36,000 SAN ports, and 18,000 hosts – and that’s before you federate across instances.   But it’s simple and easy for a customer with a single array too.
• To make it easy to integrate with – all the EMC element managers share the same UI based on FLEX, everything is exposed via APIs, in the roadmap you can expect a ton of integration with EMC Storage Configuration Advisor and the Element Managers (Like Unisphere and SMC).   It is fully integrated with the vCenter and ESX APIs right now meaning like all EMC management stuff, you point at vCenter – and go.  You can see the power of that in the demo below.
• To use standard APIs to extend platform support – the team is really focused on not trying to be everything to everybody.  They integrate with EMC arrays today via SMI-S and that’s their plan to add third party array support.

Check out the video below (thank you Brian Dehn – great job!) to get an idea of how cool ProSphere is…

What can you expect in the near future for ProSphere in version 2.0?

• NAS Support (to me, this is the biggest gap in 1.x)
• Host capacity utilization (there is great capacity reporting in 1.5 – but from the array(s) standpoint)
• Broadened & consolidated alerting & policies (1.5 has this, but early capabilities)
• Performance Dashboards (1.5 has awesome performance info as you can see in the demo, but no performance dashboard like you see for capacity)
• Agentless improvements (1.5 introduced the first passive fabric-based construction of topology details – we think we can do a lot more in 2.0)
• NPIV & Virtual Fabrics
• Broadened array/target platform support.

EMC ProSphere is available at no cost to existing Control Center customers – and I would highly encourage them to check it out.

What if you’re NOT a Control Center customer – but want to give it a shot?  Or if you’re an EMC Partner?

I’ve been pushing the product team and I’m VERY happy to say that they share my enthusiasm and confidence in their baby – they’ve decided to make the Virtual Appliance available for download to the world in the same way we have access to it within EMC.   This is the same thing as the real deal.  It’s not supported in production, and the support will be provided on a best-effort SLA via the public EMC Community Network.   Expect to see another post soon with more details on how to get it soon.  

BTW – customers – use your voice!  Do you like it when we do make our products available to play/learn openly as virtual appliances?  Does it make you more likely to consider?  To acquire?  Do you cheat the system and risk putting them in production environments? 

I’m pushing ALL the EMC product teams hard on this topic internally.  I think the one who AREN’T doing this (you know who you are!) are on the losing side of the debate – more and more they are coming over to this way of thinking.   I also think and have experience that customers understand what “no supported in production” translates to.   Ultimately though – it’s YOUR opinion and actions that matter far far more than mine – please comment!

I’m doing a webcast next week on this broad topic (ProSphere architecture, capability, roadmap, HOWTO play) – it’s on March 29th, 8am PT/11am ET.   Register by clicking on the link below!

I think EMC ProSphere is really cool – but more than anything what I want – from customers, partners, EMCers – is feedback!  Are we on the right track?  What do you like/not like about ProSphere?  Do you like the fact that we’re putting it out there to play, learn and give feedback?

Comments

[Dan Libonati]

Chad.. Great post. where can i find help in deploying the OVF files? I have set up the Vm's but can not get to my SRM servers' web page.

[Otmar]

Thanks for that information about Prosphere. We're using Control Center, SMC and Cisco device/Fabric Manager at the moment and want to have a look at Prosphere soon.
Does Prosphere give one a view for VMs and their related ressources on ESXi?
Are other virtualization technologies supported like AIX LPAR?
Will NPIV on host adapters be supported in 2.0? (like CNAs in Cisco UCS blade systems or FC-HBAs in IBM power systems)
I already had a look at the new design as vapp. I think that was a very good idea, also the REST API is a huge benefit. Providing virtual appliances for testing is very welcome. If I have time and want to try out sth new it is nice to not make fuss with sales department first to get a test version.

[Dave Schebler]

With all respect...
We have looked at ProSphere and must compare it to ControlCenter because while we use the element managers for change, we use ControlCenter/StorageScope for data.

A few observations:
- Not all of us prefer the agentless approach which requires an account with priviledges (yes the priviledges can be managed via a sudo profile which must be maintained by the customer) to a master agent which for us is installed as part of the OS build.
- Since StorageScope is our tool of choice for historical storage data, we will not be able to adopt ProSphere until and unless it includes historical data (at least 5 years of monthly rollups) from CAS, NAS, and SAN.
- I am concerned about the use of a new SMI-S approach, maintained by the ProSphere team, versus Solutions Enabler for commincation with EMC block arrays. ControlCenter always lagged changes to SE badly. It seems to me that the PS team could have avoided this by adopting SE for EMC arrays - all PS would have to change is the PS code rather than the PS code and the connector. I'll grant it may not be industry standard, but SE is always current.
- Using a virtual appliance surely simplifies setup, but I see two issues here. First we are a regulated industry (pharmaceuticals) and our IT security folks are mighty interested in the contents of all virtual appliances. We may actually require more testing/validation for virtual appliances than simple installs (and just for the record, I hated ControlCenter installs/upgrades). The other issue is regarding virtual appliance replacement (upgrade) when there is historical data involved - how will this work?
- Regarding feedback to the ProSphere team ... When ProSphere was announced, we offered to provide feedback to the team based on our extensive use of ControlCenter and our needs for an EMC product to do more with storage data - never heard a word. After every subsequent ProSphere session at EMCWorld, I talked with the presenter regarding our concerns on the apparent direction of ProSphere relative to the issues above and again offering to provide feedback - same silence. After a recent introduction to ProSphere 1.5, we shared the observations above and again offered to work with the ProSpere team - we are still waiting...

All issues aside, I hope EMC does nail this product. We need it and we would rather not be required to use another vendor's product to manage and report the historical data for our heavily leveraged EMC storage.

[Chad Sakac]

@Dave - first of all THANK YOU. Thank you for being a great EMC customer. Thank you for your blunt, honest feedback, and thank you for your persistence and patience.

It will be many moons before ProSphere could dream of subsuming all Control Center/Storage Scope functionality - and in fact, if they were to try prematurely, it would likely (IMO) result in badness. Those are mature products, ones that support some of the largest environments in the world (like yours).

There are solid answers (in some cases, the answer is a "No", but in most it's a variation of "Yes") to your comments...

- Agentless - I hear you. There are some advantages of host agents. But, there are loads of disadvantages too. Right now, ProSphere doesn't have the host-side features of Storage Scope as an example - if it were to get them, some of the content can only be gathered via a host agent.

- The SE approach has both good and bad (which you name some).

- the vApp in some way can be viewed as a simplified package. There are lots of ways to pass regulations and gather compliance status of VMs - we can help there. The virtual appliance actually is a composite of several virtual machines - the core data is in the RDBMS based on Greenplum - and the whole vApp can be updated without a "redeploy".

- Feedback - I will correct this immediately!

Thank you once again - and we WILL nail this product, with your help!

[Mike Sampson]

Dave – to echo Chad – thanks for your feedback! Thank you for your persistence!
To expand on a few points:
- We know that agentless – particularly in its initial form – does not work for all situations. We’ve added a “passive” option in 1.5 and we’ll be adding more options going forward.
- One of our biggest goals with SMI-S is to be more current with support across platforms. We like a standardized approach, but if it results in more lag and not less, we’ll certainly re-evaluate. We understand platform support as the critical ask.
- I will be sure we close the loop with you directly on your request for a feedback session. We need your input to be sure this is successful!

Mike Sampson
ProSphere Product Management

[Jesse Tompkins]

A few things so far that I've run into with Prosphere 1.5.

1. The GUI is much improved over anything with ECC, but at the same time, sorely lacking. Number one, it actually is greatly hampered by having to fit into the web browser. Trying to look at my fabrics (with a mere 20-odd switches per fabric) is an exercise in looking at a ball of yarn (yes, I know I can zoom). I'd love to be able to condense all the extra information into just the switches for example, and be able to move things around to get the look I want.

2. It's too storage focused. The dashboard only shows a storage view. The explore "tab" only explores storage, to look at hosts/switches/VM I have to look through discovered objects. I want a infrastructure view, a host view, and a VM view. I want to be able to see end to end from host and be able to expand out each section easily. If I double click on the switch, I see all the zones. If I double click on the storage, I want to see all the luns/masking

3. What my client's executives want, is they want to be able to use a single tool to see all storage in their environment, from SAN to NAS to the space in the servers. They want to see it all.

4. Looking at VMs should be able to be able to be done at the Cluster level.

5. It may be agentless, but Windows servers still have to be reboot. You guys and MS need to get that worked out.

The installation was definately much easier. At no point was I fighting to make sure that even the db could install with regards to windows dlls. It was very fast. More importantly, the management of the system is hands down easier. Anyone who has to manage ECC usually ends up hating the program. Gone is the daily battle with keeping the agents up and running, keeping the database from doing something stupid. It's night and day better. I never want to deal with another ECC agent again.

I am also concerned with SMI-S for a variety of reasons. Obviously, it is nice to use industry standards. At the same time, discovery of large arrays (I have 8 bay DMXs) through SMI-S is very slow, and the issue with SMC's with names longer than x characters means that I don't get alerting through proshpere for my DMX, nor do I get performance data. In addition, our Brocade based fabrics aren't managed that well through SMI-S with regards to performance and reporting (This is 100% Brocade's fault in my mind though).

The one thing you have to be able to do though, is truly support third party arrays this time around. HDS support in ECC is awful. Thin Provisioning was never supported, and support of HDS Device Manager was several years behind. The same was true of NetApp. ECC demanded that NetApps not be in the mode that NetApp wanted all their filers to be in.

[Krupa Stephen Gadde]

Do we have VPLEX support in prosphere?

[vijay lathia]

Chad:

I work in ProSphere engineering team; you did a great job summarizing all the cool features and strengths of ProSphere1.5.

Vijay