EMC Atmos Virtual Appliance – both to play, and for production

Great news in Atmos land!

First of all – what is Atmos?

Atmos may seem like something new to many, but you use Atmos-like stuff every day.  If you think about YouTube, or your Facebook pictures, or Twitpics – or the type of storage used for these new funky scale-out noSQL based applications, it ain’t your usual SAN/NAS stuff.   Those are all examples of next gen apps that leverage this class of storage platform.

The largest example of cloud object storage is Amazon S3, so if you know that, imagine Atmos as similar, but different in the sense that:

1. it’s read-writeable without doing anything fancy
2. it has a more rich geo-dispersion capabilties and more rich metadata model
3. perhaps most importantly, you can use it in the cloud, but you can ALSO have one of your own (much like the vCloud value proposition is you can have a private vCloud inside your enterprise and federate to public vClouds)

Beyond Web 2.0 apps – there are all sorts of other applications.    Example – the vSpecialist squad is global and we distribute REAMs of information (including huge Virtual Appliances and large binaries).   The use case doesn’t need transactional storage.   We use FTP today.   We could use NAS (DFS type use case), but that wouldn’t be ideal. 

What would be ideal was a federated global object-accessed storage model.  We could be (and are) deploying an Atmos instance in several global cities so everyone gets their info local, but info is distributed if it needs to be (and can be controlled if county laws don’t allow that), and we can federate to service provider Atmos services.

Can you use Atmos for transactional storage (ergo IOps and MBps centric workloads where response times are measured in milliseconds) like those needed by Compute Cloud services like vCloud)?  No.   

Thank goodness EMC has a generally available scale-out transactional storage platform that leverages commodity server hardware and can still scale to petabytes, support active/active models, and can STILL hit price points of measured in tens of cents/GB/month (and be acquired on a “pay as you grow shared risk model”). 

People think of VMAX as “Cadilliac” (and it does run most mission-critical applications at many finance customers and the state/local/federal government).   But at the same time, at it’s most basic level – it’s a cluster of commodity x86 based servers that scale up to 16 servers in an active/active cluster, and is attached to a bunch of disk (up to 2400 disks).  Think about that.   That’s what powers many IaaS clouds.

The point is that in spite of having that (and killer midrange Unified platfom), there is a real, orthogonal need for the functional capability of cloud object storage.   Recently, the great news of the market and technology of Atmos was validated by the acquisition of Bycast by our friends at NetApp as a response to the EMC-developed Atmos. 

(does anyone remember all the “why do you need Atmos if you have Centera! Hah!” comments flung our way when Atmos was GAed?   Yeah.   I guess having SnapLock doesn’t mean they didn’t need Bycast.   Moral of the story, don’t give much credence to anything any vendor says about another – that goes for me too).

Today, there’s all sorts MORE goodness to the Atmos story:

• NEW: EMC Atmos is now available as a virtual appliance – Atmos Virtual Edition - for production use (you need to buy it :-).   This means you can use Atmos not only in the original deployment model (servers with DAS where Atmos turns it into an infinitely scalable cloud object storage model) but also run it on top of all sorts of traditional storage models like with EMC’s Unified platform.
  • Also note – you can automatically tier files from EMC NAS to Atmos via the EMC File Management Appliance, ALSO available as a virtual appliance for production use – do you see a theme?

This means it’s supported on any server and any storage with a pretty basic set of minimum requirements (on the VMware HCL, and only VMFS via FC and NFS).  It can be added to a Vblock as an example.   If anyone asks “is it supported on a Vblock?”, remember that a Vblock supports any VM that runs on a supported Guest OS, and Atmos VM qualifies.

The cost of adding Atmos Virtual Edition to an EMC Unified config is ludicrously low.  think low tens of thousands – and that includes 3 years support/maintenance.

• NEW: EMC Atmos is publicly available free of charge as “Atmos in a Box” for people to play with and learn.   Read on below for more on getting it and using it.
• NEW: More EMC Atmos based services are up and running.  AT&T’s is here, Peer 1’s here, Hosted Solutions here. 

UPDATE JULY 1st (Canada Day) 2010: EMC has decided to make things clearer by not being in the business of OFFERING Atmos as a service for any additional customers (it is staying up for existing users).    People had a hard mental time understanding whether EMC was positioning Atmos as a:

• PRODUCT - as a Virtual Appliance on top of traditional storage models, or as physical appliance designed for maximum scale and lowest cost)
• SERVICE – as Atmos Online.

Further – there was always a frustrating part of a conversation with any service provider potential Atmos customer where they would ask “are you competing with us?”   The answer was simple “NO” – Atmos Online was always intended to be a proof-point of the technology, but part of being a proof-point of a cloud object storage technology was you needed to really be up and running (since big parts of the technology are things like the end-user portal, multi-tenancy and usage/chargeback models )

Lastly – sometimes EMC’s own field would sometimes want to sell it as a service, which was a conflict of interest with the service provider.   

Things are more simple now:

Atmos is a technology.   EMC sells that technology to people who want to offer cloud storage services.   Service providers like the ones listed above are offering that service.   EMC directs customers who want Atmos as a service to those partners.    For customers looking for “private cloud” versions of Atmos (which can federate to the public cloud variants), Atmos can be deployed on their premise as a Virtual Appliance or as a Physical Appliance.

• Of course, EMC Atmos Online Storage Service is also up and running.   EMC’s own service based on the technology is there not because we want to be a service provider, but because it represents a real, living proof point.  Also, you can’t really claim to understand cloud storage models unless you have services up and running – it’s how we learn and continue to improve the product that we sell to service providers.    You can sign up with a credit card for EMC Atmos Online Storage Services below (2 months free!).   For those of you wondering about (monthly, no commit) pricing…
  • Storage capacity: $0.15 - $0.25 per gigabyte per month, based on storage class
  • Bandwidth into Atmos Online Storage Service: $0.10 - $0.20 per gigabyte per month, based on storage class
  • Bandwidth out of Atmos Online Storage Service: $0.20 - $0.25 per gigabyte per month, based on storage class

 

• NEW: Want to “cloud backup your iPod/iPad/iTouch?” – There’s an app for that :-)  It’s called Atmos iSafe.  It backs up your iPod/iPad/iTouch to any Atmos API compliant cloud (including AT&T’s, Hosted Solutions, Peer 1, or anyone – including the EMC Atmos Online Storage Service).   You can download it from the iTunes App Store – it’s free :-)

So… see the coolness involved in this?   Want an Atmos instance to call your own?  No problem :-)  Read on for the “Atmos-In-A-Box” instructions…

This is going to be a bit basic, but it really is this easy :-)    We’ll make it even easier over time, but sharing is fun :-)

Now, normally, the Atmos VM has one tiny ISO.  I’ve set it up, and it’s SUPER easy.  You install the first node, then subsequent RMG nodes PXE boot from that first one and get “Borg’ed” into “the collective” :-)

Now, I’m pushing the Atmos team to let that one into the wild (would be the same deal as the Celerra, NOT SUPPORTED IN PRODUCTION for the freely available one), but I’m not quite there yet (please comment if you would like it, that helps me with ammo :-)

This is almost as good….

The “Atmos-in-a-Box” (AIAB) is a set of Atmos VMs, all pre-constructed and pre-configured.  As easy as Atmos VE is, AIAB is even easier – though a little more fixed in configuration (as the 4 VMs are pre-built, pre-configured).    What you end up with is the following:

 

It’s a little restrictive in the sense that you have given set of nodes, with given names and given IP addresses.   With Atmos Virtual Edition of course, you can configure it as you see fit (yes, yes, I’m on it :-)

The AIAB package and installation instructions are available for download here:

• Install Tech Note
• Atmos 1.3 Doc Set
• VM Package 1
• VM Package 2
• VM Package 3
• VM Package 4
• Version
• Install

FYI – you’re downloading it from Atmos Online (which is staying up for any existing users) :-)

Have fun – and comments/feedback welcome!

Comments

[John]

way cool :) thnx

[Mark Burgess]

Hi Chad,

Where can you get the best practice guide from?

There is no link on the image above and I have searched on PowerLink and it does not appear to be there.

Many thanks
Mark

[naveen]

excellent info

[Frank Murphy]

Mark,

The Best Practice was posted here a few weeks back down this path:

Home > Support > Technical Documentation and Advisories > Software ~ A-B ~ Documentation > Atmos > General Reference

You can reach it by either of these links:

http://tiny.cc/8bnin

or:

https://powerlink.emc.com/nsepn/webapps/btg548664833igtcuup4826/km/live1/en_US/Offering_Technical/Technical_Documentation/EMC_Atmos_Virtual_Edition_Best_Practice_Guide_A01.pdf?mtcs=ZXZlbnRUeXBlPUttQ2xpY2tDb250ZW50RXZlbnQsZG9jdW1lbnRJZD0wOTAxNDA2NjgwNGZhNzQyLG5hdmVOb2RlPVNvZndhcmVEb3dubG9hZHMtMg__

While you're there, I recommend a great companion read in the Atmos Web Services Best Practices paper.

It covers some of the points mentioned above around using the API to directly influence the behavior of the underlying storage infrastructure, and the concepts are what's implemented under the hood in many of the apps and integrations available today.

The API is substantially the same whether you're coding to the reference architecture at www.atmosonline.com or your own Virtual Edition instance or to Atmos cabinets deployed in a Private Cloud, or to any of the Service Providers implementing Atmos in their services.

Web Services BP is here:
http://tiny.cc/x8czs

or here:

https://powerlink.emc.com/nsepn/webapps/btg548664833igtcuup4826/km/live1/en_US/Offering_Technical/White_Paper/h7067-atmos-cloud-optimized-storage-wp.pdf?mtcs=ZXZlbnRUeXBlPUttQ2xpY2tDb250ZW50RXZlbnQsZG9jdW1lbnRJZD0wOTAxNDA2NjgwNGUyN2E5LGRvY3VtZW50VHlwZT1wZGYsbmF2ZU5vZGU9MGIwMTQwNjY4MDRlMmYwOF9Hcmlk

Then,

If you really want to go down the rabbit hole, check out the code snippets, docs, sample apps, and other varied cool-ness that Aashish Patil and his team maintain over on the developer community page too:

https://community.emc.com/community/edn/atmos

Frank

[Paul]

What would you recommend when positioning EMC's value proposition for public cloud services, given AtmosOnline as a sales process has been terminated? Should we show prospects the AtmosOnline interface, and show value and gain mindshare with that method? Or does it make sense to engage directly with any of the cloud providers you listed in your blog?

[Theron Conrey]

Chad,

A SUPER easy to install Atmos VM to install from a tiny ISO would be awesome. For non-prod use I would be super eager to get it! Downloading the AIAB now though.

Thanks!
-Theron

[Mark Burgess]

Hi Guys,

Just to let you know the best practice guide on powerlink gives an access denied error.

This may explain why I am seeing very few Atmos documents on powerlink.

Best regards
Mark

[Edwinder]

Atmos no more (Chad, you're quoted in thereg)

http://www.theregister.co.uk/2010/0701/emc_cans_atmoss_online_service/

[Peter Goossens]

Hello Chad,

I'm having trouble downloading the big VM-files from Atmos Online.
Multiple disconnects when downloading, corrupt files, etc...
Is there another way to download/share the files?

Best Regards,

Peter

[Sho Funatsu]

I'm also facing problems downloading the VM files...

Thanks,
Sho

[Janakan Rajendran]

Can Atmos VE do network RAID if I'm using a commodity hardware without RAID controller and install the VE on it?

[Dean]

Is there a ESXi version yet?

[Burnaby Appliance Repair]

Before going to do the repair, you know what is and what is not covered by your plan. Many safeguards are the terms and conditions, such as where you can be contacted for repair. You paid a premium for the guarantee that it is non-existent coverage of the attempt to solve the work yourself or by contacting technician.To unauthorized repair to avoid an awkward and unnecessary spending that you have everything connected and plugged in. It may sound stupid , but it happens.

[Brian Feeny]

Chad, thanks for making this available. When I fire up Atmos and login as SysAdmin, the graphics box when you click Dashboard just shows "Internet Explorer cannot display the webpage". Any ideas what that may be? I have accepted the certificate, added it to my trusted root cert store, also added everything as a trusted host, and I am using https://cos1-001/mgmt_login.

The issue is that in /usr/local/maui/gui/app/views/maui_admin/site_dashboard.html.erb the line that displays this is:

/ganglia/index_maui.php?c=">

The ganglia_addr is set to be the same as the rms_addr earlier in the file:

And the RMS address is set to 172.20.1.21

Is there a way to "update" the atmos to tell it to use the public IP's on eth2 for the graph instead of the 172.20.1.21. That network is isolated and so not available to the client browser when hitting the Atmos management web interface.