VMworld 2012: VDP– A 2 year secret project comes to life…

An exciting day!  

Today – the world is presented the culmination of about 2 years of hard work by a combined VMware and EMC engineering team.   Gang (and you know who you are), my hat is off to you!

Ok – rewinding the clock a couple years…  

• VMware was emphatic that they needed to deliver native backup/recovery functionality in vSphere, that it was something the customers really needed, particularly in the small-medium customer segment.
• They had developed their own capability (VDR), that had limited success for a variety of reasons.
• EMC was getting lots of great customer feedback on Avamar in the VMware context – based on the source-based variable length dedupe at Avamar’s foundation, but also for the deep use of VMware integration, from all the VADP elements (notably unique use of Changed Block Tracking not only for backup but also restore) and vCenter API integration.   But – it was “enterprise level” – in UI and product complexity, in entry price-points and more.
• EMC Avamar was available as a virtual appliance (Avamar/VE), but 3rd party virtual appliances (almost all of them) have limited commercial success.   Interestingly we have surveyed customers – and they like the idea – but almost universally if they want the function delivered as a virtual appliance – they also want it supported and delivered by VMware themselves – true of backup, storage and more.

The answer was obvious: 

1. VMware selects Avamar to embed in vSphere.
2. We embark on a joint engineering mission – 60 or so Avamar engineers start to “live in” VMware
3. We target a release window where this new capability would be included in vSphere natively.

In a nutshell and phrased as a mathematical expression:

(All the awesome simplicity of the vSphere 5.1 Web Client based on Flex) + (the core of Avamar’s variable-length inline dedupe and deep VMware integration in the form of Avamar/VE 6.1) = vSphere Data Protection.

Analysts started to sniff this out last week and jumped to all sorts of (to me at least) strange, hyperbolic conclusions.   For more detail, demos, advanced examples, and my view on the analyst commentary… Read on!

Analyst commentary fell into these two categories:

• “This changes the backup landscape” – this I agree with, particularly in the small and medium customer segments.
• “This represents a fundamental change in VMware/EMC relationship” – this I don’t agree with.   Journalists sometimes need to sensationalize – after all, who doesn’t like that sort of thing…   But, hey… VM HA for YEARS was based on EMC technology (re-written in vSphere 5).   Oh, want another example?   vShield DLP Sensitive Data Discovery is based on RSA technology embedded.  Oh, want another example?  Umm… vFabric Data Director was a completely jointly developed project.   Oh, another example?   The new Single Sign On technology in vSphere 5.1 is a joint VMware/EMC project.   You can expect VMware/EMC to share IP and continue to collaborate wherever and whenever it makes sense for the customers and the shareholders – and both companies have huge IP portfolios.

 

This is a demonstration of just how easy VDP is.  Create backup jobs in 5 clicks, and restore even faster.  Unbelievable efficiency in dedupe and use of changed-block tracking for both backup and restore.   Simple end-user self-restore at the file-level.

You can download a high-rez version of this demonstration here in MP4 format and WMV format.

We also showed how you can use VDP programmatically using PowerShell – check it out.   This could (and will) be used for accelerating integration with things like vCloud Director, VMware View for “Backup as a Service” for these other use cases…

You can download a high-rez version of this demonstration here in MP4 format and WMV format.

Interestingly – we’re not stopping here… 

• With all the things learnt from VDP, it is accelerating Avamar development.  You can expect the next major version of Avamar to have material leaps forward in terms of VMware integration (which it already does amazingly well – in fact this was one of the reasons that it was selected by VMware) and simplicity (lessons learnt from the vCenter Web Client FLEX work).
• There’s a clear spot in the market for something a little bigger than VDP, but still smaller than the full blown Avamar deployment (ergo replication, greater than 2TB deduped stores, more application integration – but still VMware centric as opposed to Avamar which is heterogeneous).
• If you look at this post on VM Granular Storage – you can see how in the future VDP will benefit from VM Granular Storage and the EMC platforms that will support it.

Also – what this highlights is how EMC views value being “mobile” in the land of the Software-Defined Datacenter.   Some of the functions people get today (backup, storage, security) can and will appear in multiple places in the stack over time – and that is A-OK – in fact we will help innovate and accelerate that.

So – what do you think?  Right track?  Wrong track?

Comments

[Matt]

This is really cool...is this something that is avalable right now? Is it included in the Enterprise VMWare license like the old VDR or is it a seperate purchase?

[Andrew Fidel]

Chad, if you can comment when can we expect VDP's big brother? A product with higher VM count support and perhaps a handful of non-VM licenses sounds like a potential fit for my organization if the cost is reasonable.

[Calle]

Andrew, whilst there is no public information today about anything bigger, you always have the choice of going with EMC Avamar, which has the same fantastic functionality, but also support for much higher VM count, physical machines, applications, replication, etc etc.

Avamar is available in single node versions up to 7.8TB (pre-dedupe) and grids up to 124TB (pre dedupe), both variants can be augumented with EMC DataDomain systems for very large capacity demands.

[Anil Bidari]

To Matt :my below blog should answer all of this to you..

“Avamar and VMware Data Protection (VDP) are not competition, but should know differences between both”

Vmware at VM World -2012 announced an exciting next generation backup and recovery solution into VSphere 5.1 termed as Vmware Data Protection( VDP) powered by EMC-Avamar. The VDP replaces the Vmware Data recovery feature which was available in VSphere 5. VDP will be offered as part of VSphere 5.1 at free of cost in all the Vmware Essentials+, Standard, Enterprise and Enterprise+ VSphere licenses. .It would enable customers to do image level backup, self-restore and deduplication.

EMC Avamar on other hand is an Purpose Built Backup Appliance (PPBA), which has capabilities like image level and Guest level backup ( allows application consistent backup) ,with DR capabilities to replicate your backup data to offsite location. Avamar PPBA in addition to do VM backups can address many other use cases like physical servers, Desktop /Laptop , Remote Office Branch office (ROBO) and NAS data backup. EMC Avamar along with Data Domain actually leads the PBBA market worldwide with 62 % market share as per IDC.

It is important to understand the difference between both VDP and Avamar PBBA

For image on differences on both feature wise...please click below
http://www.facebook.com/photo.php?fbid=453644338014001&set=a.376262395752196.90637.319208948124208&type=1

[grant albitz]

IS backing up vcenter server itself with vdp supported? If you lose a datastore with a few guest vms and also vcenter are you dead in the water? It seems as if you need vcenter in order to initiate a restore which makes me wonder how this will work for your average small customer who probably would have all vms and vcenter on one san and then the vdp appliance on another datastore.

[Tibo]

vDP is a great news because the Data Recovery appliance was quite disappointing.

I used to work on Avamar products but there is something I don't understand about vDP :

Why does it need 3 TB for only 2 TB of deduplicated data ???

[dCkO]

Any idea as to if and when those Avamar PowerCLI scripts will be out?

[xandre]

Where I can find avamar.psm1 ?

[xandre]

Where can I find avamar.psm1 ?

[Paul]

and I too am wondering where I would find avamar.psm1...

[Barry]

Can you please offer a download option for avamar.psm1?