Virtual Geek

EMC Unified Infrastructure Manager (UIM) v3.0–Hello World!

EMC Unified Infrastructure Manager has two parts – UIM Provisioning (UIM/P) and UIM Operations (UIM/O).  

What does UIM/P do?

• It turns all the elements in a Vblock into a single element – managed as one, single common API, single plane of glass.
• It changes the provisioning model from one of provisioning point physical elements, and into creating service catalogs of common templates – which accelerates deploying vSphere clusters and vCloud Director Provider Virtual Datacenter (pVDCs)

What does UIM/O do?

• Helps with Root Cause Analysis and correlation (impact on services of issues) in Vblocks.

Both are critical value propositions to customers who are choosing to focus higher up in the stack – and increasingly we’re finding this to be more and more the case.   In these cases, the same concepts that exist in the server/storage/network elements (create templates/profiles, deploy in “cookie cutter” fashion) simply need to be abstracted up one level – into the aggregate services that support virtualized use cases.

UIM has a strong roadmap – and is iterating furiously based on the continued demands of Vblock customers – and adoption continues to be fast and furious.  Both UIM/P 3.0 and the next UIM/O are being previewed at VMworld in Copenhagen, and the UIM/P 3.0 release is right around the corner.

What’s new in UIM/P 3.0?   A whackload – but here are the biggies:

• Elastic provisioning.  To be honest, this is a huge feature, and really bad on us that it took this long to introduce it.   This enables servers/VLANs/storage components to be decommissioned out of active, provisioned services (with deep integration with vCenter to ensure running workloads are not affected).  Until now, the only way to “shrink” was to decommission the full deployed service (think “vSphere cluster”)
• vSphere 5 and vCloud Director 5 integration.   There is broader automated configuration of the new services in vSphere 5, and also now completely automated creation of Provider VDCs in vCloud Director 1.5.    This means that all the steps required to create a pVDC are completely automated. You can imagine how much this simplifies things for customers and service provider customers.
• Simplified vApp deployment model.   The way UIM/P is deployed now is simple.  It’s a vApp.  Do you have any questions? :-)
• An awesome “demo mode”.  Very, very useful for EMCers, Cisco folks, and VMware folks – as well as all VCE partner/resellers.   UIM/P can operate completely standalone (including full API support for people who want to play with integration with things like Tidal, NewScale, BMC, vCO, and any other orchestration or other higher-level management tools) for a simple, turnkey way to play/learn/demo.

UIM/O will refresh soon also – and is AS impressive – we continue to deliver against the vision of simplifying the whole infrastructure proposition – which frankly is key so people can get on with the business of doing business, developing new apps, transforming their end-user experiences, and deploying their clouds.

Check it out below!

You can download this demonstration in high-rez offline MP4 and WMV versions.

“Chad’s Choice” Tools Webcast– PPT and Recording

The webcast we did regarding some fun, funky, beta/alpha/hacked tools got posted.

Chad's Choice - Fun with Face-melting (and totally non-production) EMC tools – ON DEMAND:

http://www.emc.com/events/2011/q3/09-29-11-chads-choice.htm

If you want to download the presentation for your own use (and for all the links), click on the link below.

I would highly recommend checking it out if you are VMware/EMC customer…

The webcast covers some tools I highlighted in a couple of blog posts – these are completely free, BTW:

1. the Hyperic/vCenter Ops tools for EMC VNX (which even since the webcast have been enhanced with more block capabilities in conjunction with the previous NAS performance metrics, and a tighter version that directly imports to vCenter Ops – just search on the “Everything VMware at EMC community)
2. Analyzer Helper (the only one which isn’t available to all yet, but working on it)
3. “Project Orion” (REST API interface for all EMC platforms) – NOTE: the bits and docs are now posted.

Have fun!

VMworld Copenhagen – my 2 cents on things to see and do…

VMworld Copenhagen is right around the corner (next week!!!)  I’m hoping to see you there!

Believe it or not – you can click on the link below and STILL get the EMC Early-Bird Registration rate…

I did a whole “Top 10” series for VMworld (1,2,3,4,5,6,7,8,9,10) – here, I’ll wrap it up in a single tight blog post :-)

One fun little factoid for people who read the blog – I run a half-marathon on Sunday, right before getting on the flight to Copenhagen.  So if I look a little… out of it… don’t assume that I drank too much the night before :-)

So – key things to see/do!

1.  The hands on labs

Same deal as in VMworld Vegas – I think the Hands-on-labs are one of the best ways to learn, and interact.   Why go to a session and HEAR about it, when you can go to the HoL and DRIVE IT.   Plus, as industry followers know, you never know when some vendor claiming to have “saved the HoL” (always a load of hoo-ha) and general in-fighting will result in a furious flame from an EMC executive on The Register (I caught some heat for that one) :-)

Like in Vegas – EMC will support Partner Hands-on-Labs in the VMware HoL, and also some vLabs for intimate EMC experiences – loads of chances to play with some fun kit!

2.  Sessions!

Here’s my recommended session calendar:

Tuesday

• 09.00 - 10.00 - Find out What’s New & What’s Next in Virtualization and Cloud Management
• 10:30 – 11:30 - BCO2479 – Understanding VMware vSphere Stretched Clusters, Disaster Recovery, and Workload Mobility (repeated on Thursday).   Lee and I were not allowed to talk about the new vSphere Metro Stretched Cluster support and HCL category at VMworld in Vegas, but now it’s public, so we’re adding it.
• 10:30 – 11:30 GD32 - vSphere and vCloud Director Group Discussion - I’m a big fan of the group discussion format – it’s more intimate and interactive than a breakout session. Plus, I think the world of Massimo. He’s been involved in a lot of the vCD projects I’ve seen in Europe, so brings a lot of real-world experience.
• 12.00 - 13.00 - Transitioning to ESXi
• 13:30 – 14:30 - SPO3977 – Next-Generation Storage and Backup for Your Cloud – the cat is out of the bag on this – it is a monster session, jam-packed with stuff.   Literally I’m showing the next couple of years of cool stuff.  If you’re an EMC customer (or using storage or backup of any kind with VMware – so all of you) this should be a good session.  Couple it with VSP3205 and you have the future of storage mapped out.
• 15.30 - 17.00: Steve Herrod, CTO and SVP of R&D

Wednesday

• 09.00 - 10.00 -  Raghu Raghuram, Senior VP and General Manager of Cloud Platforms
• 10:30 – 11:30 - BCA2320 – VMware vSphere 5: Best Practices for Oracle RAC Virtualization
• 12:00 – 1:00 – VSP3205 - Technology Preview: VMware vStorage APIs for VM and Application Granular Data Management (Vijay and Satyam from VMware).   I had to HINT at this before VMworld Vegas – but now the cat is out of the bag.  This is the “future of VMware storage” session, and is very interesting, and very eye-opening.  It’s what we’re working on in the more long-term view (as neither block nor NAS models suit virtualization particularly perfectly).   I would strongly suggest going.  And YES, I will be doing a post on this, I committed to hold off until after VMworld.   Do this and SPO3977 and you have the secret decoder ring on EMC’s next few years of R&D mapped out.
• 13:30-14:30 – GD10 - Site Recovery Manager Group Discussion - I’m a big fan of the group discussion format – it’s more intimate and interactive than a breakout session. Plus, I think the world of Lee. He rocks, and is, in essence, “mr. SRM” at VMware (and a good dude too).
• 13:30 – 14:30 - BCA1931 – Design, Deploy, and Optimize SharePoint 2010 on vSphere
• 15:00 – 16:00 - CIM2343 – Building a Real-Life High-Performance Financial Services Cloud
• 15:00 – 16:00 - VSP1926 – Getting Started in vSphere Design – with Scott Lowe!
• 15:00 – 16:00 – VSP1682 - vSphere Clustering QnA - Duncan, Frank, Chris Colotti – authors of one of the authoritative books on the topic (I have my copies) presenting on the topic. I’ll be there with books in hand to get signed :-)
• 16:30 – 17:30 - GD48 vFabric Data Director with June Yang.  Group discussions rock, and vFabric Data Director (vFDD) was a huge part of VMworld Vegas, and people need to see it. To drive people to this – let me say the following: “Hint: VMware is INTENTIONALLY trying to keep this topic quiet” (best way to get people to go :-)

Thursday

• 09.00 - 10.30- Paul Maritz, CEO
• 11:00 – 12:00 - SPO3978 – EMC IT: How We Accelerated Our Journey to the Cloud
• 11:00 – 12:00 – GD10 - Site Recovery Manager Group Discussion - I’m a big fan of the group discussion format – it’s more intimate and interactive than a breakout session. Plus, I think the world of Lee. He rocks, and is, in essence, “mr. SRM” at VMware (and a good dude too).
• 12:30 – 13:30 - BCO2479 – Understanding VMware vSphere Stretched Clusters, Disaster Recovery, and Workload Mobility
• 12:30 – 13:30 – CAP2154 - Powering database-as-a-service for your cloud with vFabric Data Director With June Yang.   vFabric Data Director (vFDD) was a huge part of VMworld Vegas, and people need to see it.  To drive people to this – let me say the following: “Hint: VMware is INTENTIONALLY trying to keep this topic quiet” (best way to get people to go :-)
• 14:00 – 15:00 – VSP2376 - Performance and Scalability enhancements in VMware vStorage VMFS-5
  Again – the content will rock, but so will the presenter. Mostafa is a senior support escalation engineer – so has the battlescars and lessons to share. He has presented at VMworld for years, and people like his content. I find I end up using Mostafa’s content for subsequent years.

3. Tweet ups!

Tweet-ups may sound dorky – well they kind of are :-)  But hey, it’s a pretty dorky (but fun!) event as a whole – we’re all dorks :-)  A tweetup is in essence a group session, with online folks asking questions too via Twitter.   They are a great way to interact, ask questions.  It’s a panel format, so people can come by the EMC Booth at noon every day for an intimate and fun group discussion…

• Tuesday 18 October

• Host: Tom Hayes
• Topic: VCE Simplifies and Speeds VMware Virtual Data Center Delivery with New Vblock Management Capabilities
• (Yes, it's a long title, but it's worth it, trust me...) Tom works in EMC's Infrastructure Management Group. I'm not (yet) allowed to divulge too many details about this topic, so, for now, suffice it to say that he'll be speaking about new ways to make deployment and management of the VCE Vblock even easier and faster than it already is today.

• Wednesday 19 October

• Host:Chad Sakac (@sakacc)
• Topic: The New vSphere 5 Metro Stretched Cluster: What It Is, What Does It Mean, When Should You Use It?
• Long-time members of our online community already know Chad, either from his quarterly "Chad's Choice" webcast, his Virtual Geek blog, or from episodes of Chad's World, so he needs no further introduction here. Chad will be speaking about a brand-new category on the VMware Hardware Compatibility List (HCL) -- the vSphere Metro Stretched Cluster. These new fully-supported-by-VMware stretched cluster configurations are made possible by changes to the VMware vSphere 5 behavior of VMware High Availability (HA) and VMware Dynamic Resource Scheduler (DRS). Chad will talk you through what this all means as well as answer any questions about when a Metro Stretched Cluster might be an ideal solution and when it might not be the best fit. You can get a preview of this meetup by reading Chad's blog post on this topic.

• Thursday 20 October

• Hosts:Simon Seagrave (@Kiwi_Si) and Chris Horn (@Horn_Chris)
• Topic: Building a Successful Hands-on Lab Environment
• Simon and Chris are both members of EMC"s vSpecialist Technical Enablement Team. They'll use their real-world experience creating virtual environments for the Hands-on vLabs at EMC World, EMC's part of the VMware Hands-on Lab at VMworld US, and the EMC Interactive Demo Area at VMworld USto explain what's involved in the creation of successful labs from first having the idea to running the environment live on-site and everything in-between. They'll answer any questions you have about how to create the total-immersion learning tool -- the Hands-on Lab.

4.  The PARTIES!

Work hard during the day – and celebrate at night!   

Monday… The vGEEK Party!  It’s from 7pm-11pm on Monday night.   Since this is before the conference starts it’s bound to be good :-) It’s actually become a bit of a tradition – we hold it for the VMware SEs, Cisco Datacenter folks, EMC competitors who want to raise a pint in celebration of the battle, and of course the EMC vSpecialists.   Basically – a celebration of our brothers/sisters in arms – those who are customer-facing nerds – we salute you!   It’s at the Appollo Brewery, which is right beside the Tivoli park – a famous Copenhagen landmark.  Technically invite only, but find Wade O’Harrow, Bertrand Lalanne, Paula Cuddy, Karen Agnew or I and we’ll work something out.

Tuesday… The EMC Customer appreciation party!  It starts at 7pm on Tuesday.  We figured “when in Rome…” so since it’s in Copenhagen, that would make the Carlsberg Brewery the natural place :-)

 

 

 

 

 

 

 

 

Wednesday… The mandatory party of them all of course is the VMware party – from 8pm to midnight!

Again – really hoping to see you there!!!

Chad’s World Episode #9 is live! (Heritage Auctions, VMAXe, Isilon, Lightning and more!)

This last episode was a lot of fun, and a great, great customer – Brian Carpenter from Heritage Auctions.  A great guy – met him for the first time in Vegas at VMworld – and he and his team and I had lots of fun at the craps table :-)

Beyond that though – a great user of EMC and VMware technologies.   What Brian has to say in this… is… welll… priceless.   The effects on their business they’ve been able to drive as an IT org – it’s worth hearing.

In this episode, we show more on EMC Project Lightning, talk about server/storage mashups – ergo running compute on Isilon and more.

On the more front, a funny insider story – Wade took the demonstration we did at OOW (with Pat Gelsinger/Joe Tucci/Larry Ellison in the starring roles) and put him and I in their place – which was funny… Check it out!

And – as we end that episode – I hope to see YOU next week in Copenhagen at VMworld!!!!

Next blog post will be a reminder of key things on the calendar that week to see/do!

Update on “Work Life of Chad” + EMC & Oracle/MSFT/SAP

Warning – I’ve done some technical posts lately, this one is personal and business.   Skip it if you want (though I think it’s interesting)…

When I did a blog post here advertising the new VMware Technology Partner Management (TPM) job opening, a lot of people said “whoa – is that Chad advertising his own job?”.   Not quite.

While there’s the “public face of the work life of Chad” (often doing silly things like Chad’s World or the Oracle Openworld keynote) here are the 3 key parts of my job over the last 4 years:

1. Responsible for the VMware/EMC technology alliance (which is engineering like the vMSC, VAAI, Virtual Volumes stuff + marketing stuff like pulling off VMworld and VMUGs, and vForums from an EMC standpoint)
2. Build and lead our now 200+ strong team of vSpecialists around the globe (which is field stuff).
3. Work personally with customers (which is field stuff – and remains my favorite :-)

While there is still a long way to go (and more and more every day) on the VMware/EMC technology alliance, and our ever-growing field focus – we’ve made a lot of progress.

EMC has asked me to help not only continue on the field front when it comes to VMware, but additional topics that matter a lot to our customers – SAP, Oracle, and Microsoft (and yet make sure that whatever I do, I don’t screw up the forward progress we’ve made re: VMware/EMC).

As part of that, it means opportunities galore:  

• It means I need a lieutenant to live on VMware’s campus on the alliance/engineering side (that’s the role I posted about – and we are interviewing, but please email if you think you might be right for it!)
• It means opportunities galore for vSpecialists as the team continues to expand, and we work to help our EMC brothers and sisters.
• And it means opportunities galore for  EMCers and EMC partners when it comes to Oracle, Microsoft and SAP.

That said, it’s a little tricky as I bleed VMware.

I’m also not going to make Virtual Geek become a “Oracle/SAP/Microsoft” place, because, after all – this blog isn’t just about my job, it’s about what I want to write about – and that hasn’t changed one bit. 

I think I’ve figured out the formula, though.   For Oracle and SAP it’s relatively easy.  EMC’s view (and certainly mine), is that Oracle and SAP work best on top of a virtualized, cloud infrastructure model – which is really led by VMware.   Larry Ellison may not agree with us (that’s clear – he thinks Sparc is going to win :-) but we’re pretty sure, and that’s what our customers as a whole are telling us.   Usually, listening to customers is a good guide (even considering the innovator’s dilemma problem).

BTW – while the blog won’t become an Oracle/Microsoft/SAP blog, I will (hopefully appropriately) link stuff.  For example:

Also, here’s a video discussing the latest on “why Oracle on vSphere and EMC”.

But, continuing on the thread…   So, we’ve got SAP/Oracle, but what about Microsoft?

Likewise, most customers virtualizing Microsoft apps like Sharepoint, SQL Server, Exchange and more do it on VMware.    But, here, the strategy needs to diverge a little.  There is absolutely a place for Hyper-V – and Windows 8 is looking pretty darn hot, so I need to have someone other than me be the very clear leader for our Microsoft-centric field community – who is a great guy named Adrian Simays.   He reports to me (so as EMC we have a coherent structure around enabling solutions communities), but we argue a lot – which is exactly right :-)

But, everyone of these will need a public community.   So We’ve kick-started new EMC communities that follow the “Everything VMware at EMC” community formula:

• Completely open.  You don’t even need to register.   The beautiful side effect is that google will crawl them thoroughly.    The measure of their success is they rise to the top when you Google “SAP EMC” or “Oracle EMC” or “Microsoft EMC”.
• Content that is favored is technical in nature.   This is what people want more than anything else.  I’ve asked them to encourage it several ways to keep it “goopey” (that’s a Chad term for nerdy, deep content).
• Dynamic and vibrant – including solution sizing support.  They are monitored by a centralized Solution Design Center team – they will respond in a 60 minute MTTR, and also will help with SAP/Oracle/Microsoft solution sizing for anyone who asks.
• Periodically reward people – regardless of who they are – for contributions to the communities.

Part of that process is shutting down old internal distribution lists and other forums.  I’m a big believer that we need to be open, be as transparent as possible, and in doing so – help our customers and partners.   You can’t be transparent if it’s internal only, and the great dialog doesn’t help our partners and customers.   Yes, every once in a while this “err on the side of public” punches me in the gut, but more often than not – it lifts me up.

People who really stand out over the coming months in those communities will become part of the future mSpecialists, the sSpecialists, and oSpecialists teams.  

If you’re part of an existing EMC Solutions Practice – that’s awesome (and likely means you’re a shoe in), but doesn’t guarantee you anything.   These communities are open to all – customers, partners, heck – competitors.   The people who contribute rise to the top.

They will get to go to TechEd, SAPPHIRE, OOW on my dime.  They will get to spend time with the engineering teams at EMC and the partners.   They will influence direction, priority and roadmap.  The price to have a front row seat is only to be willing to fight to get there – that’s it.

Q4 contests around each of these are about light up (fun, Iomega/iPads/SSDs, and more) to draw out the people who know WTF they are doing – so chime on in!

• “Everything SAP at EMC”: http://community.emc.com/community/connect/everything_sap
• “Everything Oracle at EMC”: https://community.emc.com/community/connect/everything_oracle
• “Everything Microsoft at EMC”: https://community.emc.com/community/connect/everything_microsoft
• “Everything VMware at EMC”: https://community.emc.com/community/connect/everything_vmware

VNXe update coming, Beta participation welcome!

[update – 10/6/11 – 10:56am] – I should have said “Award winning VNXe”!  The VNXe just won: “CRN’s Most Innovative Storage Technology” category for 2011.   Details here: http://www.emc.com/about/news/press/2011/20111006-01.htm

And that’s BEFORE this upcoming software update!

-------

There’s a lot of VMware-centric goodness in this one (some I know people have been eagerly awaiting), so if you’re a VNXe customer interested in participating in the Beta, please leave me a comment – I’ll followup with you offline…

Steve Jobs, my hat is off to you.

Steve – there will be many eulogies, and all from people more well suited than me.   Thank you for all you did for technology, and for the way humanity interacts with it.   Your passion and eccentricity were apparent and inspirational.

My hat is off to you – both in thanks, and in utmost respect.

Great study of multipathing and vSphere…

This came across my path yesterday and is awesome…    At a customer, someone did their own testing of PowerPath/VE relative to NMP as they refreshed their EVA to an EMC VMAX.

I want to highlight, EMC absolutely supports, and loves the NMP RR behavior, so it’s not that it’s bad, it’s just that the whole vStorage pluggable storage architecture gave us an opportunity to innovate and develop a “best” option over the “good” option of NMP round robin.

From the post:

“…it can be argued whether PowerPath provides benefits, but most people I have talked to in the real world swear that PowerPath is brilliant. But let’s face it, it HAS to be brilliant to justify the cost per socket. Before tallying up all my sockets and asking someone to write a check, I needed to do my own due diligence. There aren’t many comprehensive PowerPath VE vs. Round Robin papers out there, so I needed to create my own.

My assumption was that I’d see a slight performance edge on PowerPath VE, but not enough to justify the cost. Part of this prejudice comes from hearing the other storage guys out there say there’s no need for vendor specific SATP / PSP’s since VMware NMP is so great these days. Here’s hoping there’s no massive check to write! By the way, if you prefer to skip the beautiful full color screen shots, go ahead and scroll down to the scorecard for the results”

http://www.virtualinsanity.com/index.php/2011/10/03/powerpath-ve-versus-round-robin-on-vmax-round-1/

New VMware HCL category: vSphere Metro Stretched Cluster

It’s kind of funny – in our small but vibrant VMware community, there is totally a certain “zeitgeist” where people simultaneously think about topics.   Look at this:

1. I have a tickler on my calendar that said “do a post on the new vMSC category”
2. Today, I see that the always excellent Duncan Epping did a great post on vSphere 5.0 HA and metro/stretched cluster solutions.
3. There’s also Lee Dilworth from VMware and I presenting on this topic at VMworld 2011 (and updating for VMworld 2011 in Copenhagen Oct 18-20) here.
4. The always excellent Scott Lowe did a post on his Updated Stretched Cluster presentation
5. I get an email internally where a customer is confused on this topic…

So – what’s driving this “synchronized zeitgeist”?

Answer – a lot of internal work that’s coming to a head, more and more stretched cluster deployments = these use cases are now becoming much more mainstream, and simpler.

1) There is a new VMware HCL category “vSphere Metro Storage Cluster”.  This is the result of a LOT of work.   I’ve created a link to it here.   This is interesting – in the sense that prior to this, while there was a lot of vendor work, info, KB articles for this, there was no formal test harness for the configurations, and failure scenarioes associated with stretched vSphere clusters (look at Duncan’s post for a view of some of the major failure scenarioes).   Together, we built a whole test harness that is now the standard for this, and will be very useful going fwd.

2) The VMware KB articles around this use case are now a LOT simpler.   This is due to a ton of the changes in vSphere 5 HA and DRS behavior that Duncan points out.   You can see the udpated KB article by clicking below.   Remember, DRS host affinity rules are now supported in this use case when using vSphere 5, so go ahead an use ‘em.

3) Lots of work on hardening EMC VPLEX around this use case… (more combined zeitgeist – just in from the product team)

• GeoSynchrony 5.0 introduced the VPLEX Witness: Allows for discrimination between site failure and site partition which means that I/Os are automatically enabled on whichever side survives a failure event.  
• update: Note – VPLEX GeoSynchrony is the software that runs VPLEX.   VPLEX has 3 major operating modes for every device it presents  – Local (in one datacenter), Metro (stretched across synchronous distance) and Geo (stretched storage models across asynchronous distance).   As of right now, vMSC is not supported with Geo – only with Metro.   VMware/EMC continue to work on this, perhaps I’ll do a post on the technical challenges.   It works while there is no partition, but the partition scenarioes are very, very funky indeed, and EMC and VMware agreed that we want to keep it simple for now – Metro-distances only.
• VPLEX in GeoSynchrony 5.0 changed I/O Suspension behavior. vSphere 5.0 now recognizes this behavior and recognizes this as a PDL condition in vSphere 5 Net result when a split brain condition does occur, vSphere can now determine which side of a VPLEX is alive and therefore, restart VMs (and I/Os) on that side. 
  • Note - this still isn’t working 100% right, and will be updated in a future vSphere patch.  Short version, the "vm kill on PDL" doesn’t work perfectly, and still results in APD in some corner scenarioes.   We’re working on this very closely, but people can move forward with confidence without waiting.   BTW, before you ask (a good question that was asked internally) – does enabling VM Failure Monitoring in vSphere HA fix this behavior? = no

Short version – look at the “what way should you go forward” guidance in Scott’s presentation, or BCO2479 (Lee and I) to determine the right solution for disaster avoidance/disaster recovery (vMSC or SRM).  If vMSC is the right answer, move forward with confidence – it’s getting better and better every day.

Also, know we’re not stopping here.   Lee and I discuss some of these things in BCO2479…  Together, EMC and VMware are looking at how we could “surface” site bias via VASA, how we could make this automatically configure things like host affinity rules.   We’re working to make multi-site scenarioes (rather than 2-site) work better, including hybrid solutions of vMSC for two metro-separated sites and 3rd async sites with SRM, as well as future multi-site vMSC scenarioes.   Lots of great stuff!!!

Comments and input welcome – are you using or contemplating stretched vSphere clusters?

URGENT: VAAI–Thin Provision Reclaim.... on hold + workaround

[updated Oct 1st, 10:32am] – Question/answer and also script to automate, noted Jason Boche’s article

[updated Oct 1st, 5:37pm] – long answer to a posed question, digging even deeper into the “why”

[NOTE – added Oct 2nd, 7:28am] – while there’s a reasonable amount of urgency in this thread, it’s worth pointing out the core resilience in svmotion here. While the recommendations below are material, svmotions never caused any guest outage through all the tests, when they would fail, they would fail atomically when the stage of file deletion would take too long – ergo no data unavailability, and there is no risk of data corruption due to this issue]

So… Scott Lowe (EMC vSpecialist CTO, and all round good dude) made a weird out of the blue blog post two weeks ago called “Hidden VAAI command” – check it out here.

So… what would make him make a weird post like that?

Well – for the last couple months VMware and EMC (note this applies to larger storage vendor community) have been grappling with a tough issue, and we’ve come to an important conclusion (thinking these things through, measuring positive/negative impacts is not trivial).

On Friday, the call was made and communicated broadly (and Jason B picked it up almost immediately here)

There is a pretty big issue with the current implementation of VAAI TP reclaim. This shouldn’t be viewed as a “VMware issue” or a “Storage vendor issue”, but a “both” issue. This affects the broad vendor community, so while I’m an EMCer, and will reference EMC storage arrays, I strongly suggest talking to whatever vendor you use.

There’s a long version (which includes the why/how) and the short version…

• Short version – no storage vendors are getting certified with the TP reclaim offload, and you should disable this VAAI command.
• Long version – the core issue is that prior to the VAAI TP reclaim, deleting a file in VMFS was near instantaneous (VMFS filesystem inodes being updated). Conversely, with VAAI TP reclaim – even if the reclaims are superfast (milliseconds), they add up (as many thousands can occur on a large file). A large VMDK (or any VMFS file for that matter – though the VMDKs will generally be the big ones on a datastore) getting deleted can take seconds. In some cases, this means svmotions can fail (the file delete takes longer than the timeout value of one of the last steps in the svmotion, which is to delete the file)

Read on for details, history, KB articles, what is going on, and workarounds…