Celerra Virtual Appliance HOWTO 101 - Basic Setup and Configuration

Glad to see that people are having downloading and success with the Celerra VM on VMware Workstation based on this original post.  I wanted to provide a quick "HOWTO" to help, and will publish a 201 (setting up Replication and a 301 - configuring SRM).  

Most folks are doing well if they just RTFM, but I thought a little "hand-crafted walkthrough" might help others.   The standard documentation is geared towards VMware Workstation, which is the officially supported target (BTW, you don't need to ask, we're making ESX OVF a standard package :-), so some people stumbled a bit when trying to use it with ESX.

I've updated the OVF (some of the problems people were having were a result of the actual previous OVF I posted - SORRY - sheepishly) .  Make sure you download the new one if you're going to follow along (again, you can get that here)

Ok - have you got it?   Ok... then read on...

Quickly then here - in this HOWTO:

1. Importing the Virtual Appliance
2. Get the Celerra Sim up on ESX, and configuring the VMX
3. Configuring network interfaces (and what they all mean)
4. Making the Celerra Sim unique after cloning
5. Licensing Celerra features
6. Configuring Datamover interfaces and iSCSI targets
7. Getting iSCSI LUNs to the ESX cluster
8. getting NFS exports to the ESX cluster

I will post followups shortly:

• 201 series (adding storage to the simulator, configuring snapshots, configuring remote replication, etc)
• 301 series (configuring Site Recovery Manager, VDI mass replicas, etc).

In a little bit, I'm going to start posting bit by bit other EMC VMs (they are legion - ADM, Control Center, Avamar, Replication Manager, Networker 7.4.x, IT Compliance Analyzer, EMC Backup Advisor, and that's just getting started) with similar guides, so let me know if this is useful - BTW - they are also going through the formal release and VM Appliance programs, but I'm notoriously impatient, and think many of you out there are smart enough to get some fun out of these without too much hand-holding :-)

Read on......

Step 1: Download  and import.  You can get the Celerra Simulator here (or if you are an EMC employee, partner, or customer, you can get it from Powerlink: http://powerlink.emc.com and search for "Celerra Simulator".   If you start with the official VMware Workstation build - you can use VMware Converter to import it, but make sure you revert the VM format to VMware Workstation 4 format before you do.    Otherwise, just start from my OVF.  Once you have the OVF , the first thing is to import the Celerra Simulator.

This video shows how to do Step 1:

You can download a high-rez version of this video here.

 

Step 2: Configure the VM.  Note that the VM needs a fair amount of RAM - 2GB if it's standalone, and 3GB if it's going to be replicating.   FYI - the SMALLEST real Celerra ships with 8GB between all it's components.  This is classic EMC - at least from our hardware divisions  :-)  Though I shouldn't say that.  Technically the smallest EMC hardware storage platform that WORKS (not on the HCL - yet) is the LIfeline, which costs about $500, though even smaller, cheaper stuff from iomega counts too..

This video shows how to do Step 2:

You can download a high-rez version of this video here.

 NOTE: if it fails to start the blackbird service, or stays at that point during boot sequence: 1) just reboot the VM; if that doesn't do it, disconnect the VM from the external network, then reconnect it after configuring the interfaces for your environment.  Sometimes this happens due to IP address conflicts on your network. 

Step 3: Configure IPs. Next, you have configure ethernet interfaces.  This is a one-time task, but one where a lot of people get stumped.  

They key to understanding is to understand the real architecture of a Celerra.    A real Celerra has 3 major architectural components: Control Station (s), Datamovers (filer heads), and Block back-end storage which is used by the Datamovers.  It’s architected this was for a couple reasons: 1) having a control station (rather than managing the Datamovers directly) means that simply scaling by adding Datamovers (up to 8 are supported in the larger Celerra configurations) is easy, and management stays simple.   This design also also keeps the control path separate from the data path, and means that the complex element (the datamover) never needs "GUI design work"; 2) the Block Storage being separate means that we can use CLARiiON HW to handle things like RAID, leaving the Datamover (filer head) to doing fileserving.   There are technical arguments either way on this architecture desing.   For comparison, the Celerra is most analogous to NetApp FAS devices, and in their case, the block storage is completely handled by the filer heads (including RAID), and you manage each of the two heads independently (unless you use their enterprise management tools).

Here's a picture of the front of a Celerra (specifically the NS22FC):

 

And how it looks when it arrives:

And here is a physical picture of the back, which is the most important for this dialog.   The reason for this design is essentially two-fold: 1) the control station behing "out of band" makes security a bit stronger, and means that scaling is easier (since Celerras scale up to 8 datamovers, the control station gives you a single management GUI, rather than 8).

Now, here are the key interfaces, and how they map to the VM.  They key thing to understand in the Celerra Simulator (not the real Celerra), the VM itself is the Control Station, and the Datamovers are a service running in the control station.   OK, in the diagram below, it shows how **I** have mine configured, but NOTE that by default, cge0 and cge1 BOTH bind to eth0.

So - VM to vSwitch (aka Guest to physical world) mapping: vNIC 1 = etho, vNIC 2 = eth1 and vNIC 3 = eth2 in the VM.

and Guest to the Celerra: eth0 = CGE0, eth1 = CGE1 (again, this is in **my** config - by default, CGE1 also binds to eth0), eth2 = control station management GUI.

So - this is what you have logically represented:

Phew! (not hard, but I'm a big believer in core understanding as the basis of learning, rather than rote learning - that way you can extrapolate)

This video shows how to do Step 3:

You can download a high-rez version of this video here.

Step 4: making your Celerra VM unique. Next, we've got to do some weird steps, so follow carefully.   These are a good idea always - but mandatory when using SRM (and replicating Celerra VMs).   I'll reiterate - you don't need to do this if you're just going to poke around or use the Celerra for simple shared storage.   So why are we doing this?   Well, in later HOWTO videos, we will configure replication and SRM.  Since the VM has been cloned - they have to be specified as unique.   There are three things that matter here: 1) the serial number of the array; 2) the control station name; 3) the MAC addresses of the CGE interfaces.   The "serial number" of the Celerra is generated from the VM UUID - and you just need to poke it to generate a new serial number.  

Here's how you do it.  There is a script in the /opt/blackbird/tools folder called init_storageID that updates the serial number of the Celerra Simulator. After you clone the VM and have the UID updated, just log into the Simulator as root and run this script. It will generate a new serial number based on the UID and reboot the system. When it comes back up, you should be able to replicate between the clone and the original

Then, you need to make sure the Control Station has a nice hostname, not the generic "localhost".   Lastly, you need to remove and re-add the CGE interfaces (which also gives you an opportunity to map them to whatever eth interfaces (and therefore vNIC and vSwitch you want).

This 2-part video (argh - the Youtube 10 minute limit) shows how to do Step 4:

You can download a high-rez version of this video here.

Step 5: Licensing.  Ok - weird steps done (we should work on making that simpler, dontcha think? :-), let's open the Celerra Manager GUI, and enable licenses.

This video shows how to do Step 5:

 

You can download a high-rez version of this video here.

Step 6: Configuring cge IPs and creating iSCSI targets.  Now we're cooking.   Quickly, and easily assign IPs to the cge interfaces and iSCSI targets.  

This video shows how to do Step 6:

You can download a high-rez version of this video here.

Step 7: Configuring iSCSI LUNs and presenting them to ESX as VMFS datastores.  Now we're cooking.   Simple - use the Celerra Manager wizard to create iSCSI LUNs, then configure the iSCSI initiator in ESX and get those LUNs loaded up with VMFS and ready to rumble!

This video shows how to do Step 7:

You can download a high-rez version of this video here.

Step 8: Configuring NFS exports and presenting them to ESX as NFS datastores.  Almost there.   Easily, use the Celerra Manager to create a filesystem, export it using NFS, and discover the power and simplicity of NFS datastores in ESX.  

This video shows how to do Step 8:

You can download a high-rez version of this video here.

OK - now what?

Well, you have a working shared storage config, without shared storage - so the world of advanced VMware features are at your fingertips.   You also have a fully functional (BUT NOT FOR PRODUCTION USE!!!) Celerra, one of the leading advanced unified storage platforms on the market - it is VERY powerful and has more features than you could imagine.   You can use the public support forums here: http://forums.emc.com  You'll need a powerlink login, but you can register.   You can literally do anything you can with a real Celerra.  I will be posting more HOWTOs as well as linking to others.   Post comments/questions, and let me know neat things you do with the sim!!

Comments

[Ryan B]

Great post! As a smallish shop we can't afford a lab unit, so we end up testing on the least critical DM of our production NS704G. With the simulator, we'll at least be able to do some basic feature testing. I'm really looking forward to setting up SRM using the simulator...

Thanks!

[John]

Loved the post. Wished I had something like this for the centera I was fiddling with.

Any shot you can do the AVE one next? I have to get 1.1 live ASAP.

[Alan Warwick]

Thanks Chad. I've been trying to setup up an SRM lab here at the office and this modified Sim will make it happen.

Two ESX servers on a real NS22, running one of the simulators, going to to another ESX server attached to an NS601G/CX400 running the other simulator. Each simulator will present iSCSI LUNs to the host they are running on. Looks a bit like Perpetual Motion or Renewable Energy!

[Mike Cu]

Fantastic Fantastic follow up post!!!

Finally something for us small IT shop personnel to get our hands on to see how the big boys do it.

A community blog site needs setting up ASAP.

However my children and wife would officially like to complain.
The summer vacation is now going out of the window!

[Alan Warwick]

Chad,

any idea how till 202? I wondering on the ways to add storage to the Celerra. I know I have the 5GB and 15GB built in and I can add storage with NFS (Looks difficult). Can I add vmdks to the VM and make it see them as Clariion LUNs?

[Chad Sakac]

Alan - just for you, I'm going to try to pop out the 201 tomorrow am. It's easy, you can just present more VMDK volumes to the VM, and with a little config jiggery-pokery, get more storage. Only thing easier would be a real Celerra :-)

[Chad Sakac]

John - I'm going to try to get AVE 1.1 ASAP. Do me a favor and if you've got some pull with the AVE team, remind them they have a great product, and should let me go hog-wild :-)

[Attila Bognar]

Hi Chad,

first: thank you for your blog.

I installed/configured the sim following your post, I have two questions:

- /etc/hosts seems to be incorrect (hostname -f does not work correctly). You type:
192.168.1.98 csprod csprod.esxdomain.local csprod
Shouldn't it be:
192.168.1.98 csprod.esxdomain.local csprod
(the localhost line has the same problem)

- why are you setting and exporting NAS_DB=/nas just before rebooting?

And a tip: you can easily and quickly logout with CTRL-D.

Attila

ps: the sound card support is funny :-)

[Stan Dorsett]

Great tutorial Chad. Any idea when you'll release the next in this series? Specifically I'm looking for the "config jiggery-pokery" to allow my simulator to see added vmdk files.

[Earl Hinkle]

This is great, especially creating all the other products in VM's for test dev environments. This will definitely give EMC a leg up in the storage area.

[Earl Hinkle]

Are they going to come up with a sim for the CX line of products? ie cx3 and cx4?

[Doug Baer]

Great stuff. I've got replication setup and it seems to work, however, the nas_replicate command hangs and trying to list the replications in the GUI does the same.

I'm replicating to a real NS20 and the commands work from that side. Any ideas about where I can look to get more information on what nas_replicate is trying to do so I can try to trace the problem? CPU isn't spiking, Memory use is minimal... I can't find a bottleneck.

Incidentally, this command is used by the Celerra SRA, so testing with SRM doesn't work for me, either.

[Mike Kolcun]

Thanks very much for this post. Very very helpful!

[Mike Kolcun]

Hi.

I was wondering about monitoring the Celerra (simulator) via SNMP. I've been able to use the RFC1213 MIB to get some basic data, but haven't been able to find another mib which might provide more.

I found a MIB at http://www.oidview.com/mibs/1139/EMC-CELERRA.html but it did not seem to work.

Any suggestions? Thanks a lot

[Murali Raju]

Hi Chad,
Thanks for this great appliance! I thought I'd address a small issue in regards to VMware Tools. I had a little bit of trouble upgrading VMware Tools on the Celerra VSA and ran into some issues with building vmxnet.

I put up a new blog and my first crack on a video upgrading VMware Tools on the Celerra VSA or Linux in general :)

http://appliv.com/?p=10

Regards,

Murali Raju

[marin]

Hi again

I was wondering if this it is normal that this the workstation template iéported as Version 4 i have only eth0 and eth1, no matter what in tried
If I enter ifconfig eth2, it shows that the card is here but as down. Bring it to UP do not change anything : No IP even thought eth0 and eth1 work like a charm. More over at start eth0 and eth1 are seen to be brought on line but not eth2

Tried it as bridged and host only : same result

Tried on workstation 6, 6.5 and player very exact same result

Not tried on ESX Thought

Any thought and idea ?

Best regards, Marin

[marin]

Hold On

Putting Eth2 UP dans using netconfig -d strait afterwards seems to do the trick... Weird
Any way iany idea or comment on this behaviors is welcomed :-)

Marin

[marin]

Hi (again)
Just one question : is the celerra in a VM is supposed to work with other iSCSI initiator than the ESX one ? The installation goes without a glitsh following the very efficient vids... But a the very last moment using the MS initiator as "quick test" it fails.
No matter the the config (bridget, host only.. ) when I try to connect to the portal (step one of the MS client) I get the message : "Add Target : Target Error".One more thing : so far i don't have any problem with other ISCSI Target Ie : Free NAS, Open filer....)
The strange thing is the ISCSI IP port is reachable from the iscsi initiator so there is no firewall problem there. I Just did the very precise installation a iscsi target creation, ie no chap or ip sec (nothing more, nothing less). Any thought or did this already happen to someone ?
Net step i will try on the ESXi server.. .and keep you informed

Best regards

[Brain]

How to install the Celerra simulator on a linux machine..?

[Paul Wegiel]

WOW - this stuff is amazing.
I finally managed to install and configure the Celerra simulator in my SRM lab in the basement. I replaced the SAN I have been using so far with Celerra and it works great.

Chad, I want to thank you for providing this awesome learning resource!!!

[corrugatedbox]

Just curious..do you all use VSA for production or just demo purpose. the lowend storage is like 5000 and this vsa cost 4000..what is the selling point?

[www.facebook.com/profile.php?id=1164846918]

Thanks for all you do Chad! I'm looking for the procedure to delete an iSCSI LUN. Using the SIM I created 2 iSCSI LUNs backed by 1 target. Connected two ESX hosts to them. Now I'm trying to delete 1 of the LUNs but I'm getting complaints that the 2 ESX IQN's are still connected to the LUN/Target. I don't want to delete the other target because the other LUN has data on it. I registered with the EMC forums yesterday to look for an answer but haven't received my login credentials yet possibly due to registering with a new company name. The EMC forums appear to be not browsable without an active account.

Thanks, Jas

[Sarang]

I did entire configuration as per steps but I am getting error while creating NFS in step 8
NFS Does not support MOUNT version 3 over TCP

Any help in this matter will get me closer to resolution.
Any other way to configure this?

I just need to use shares as test.
Thanks in advance.

[Rainer]

anyone able to sucessfully start the OneDM Simulator from Powerlink with Workstation 7 ?

I keep getting coredumps with NOT_IMPLMENTED pollVMX.c errors

thanks
Rainer

[Eric Moore]

Thanks so much for the wonderful work.

Have you details/instructions on setting this up for a single NIC? I have the EMC Celerra Simulator Guide however, it still isn't quite clear how to use this with one NIC.

All the best,
Eric

[Phil Lawrence]

I'm going to probably not be that popular on this one but this does seem to be a whole load of work to get a simulator working to test an interface and capabilities of a storage appliance. Is there not an easier way to get it working???

I've downloaded and played with the Sun S7000 simulator and it took me a few minutes to download 500MB and NOT 1.5GB); I can run it in VMware player or Virtual Box and it took me about 5 minutes to get it up and running.

I get iSCSI, CIFS, NFS, HTTP, WebDAV and a whole bunch of other stuff in the S7000 simulator, I can update to the latest firmware and can do a whole bunch of things with it (just like the real version) like snapshots, replication, shares, alerts, analytics (you have to see what this can do for you...) etc., etc.

Regards,

Phil

[TBone A]

ESXi 4 and the lastest VSA. I do not have an ETH2 in the VSA? Can someone help me out here? I'm a bit lost.

[Philip Leighton]

TBone

I have the same issue - no ETH2 - I have also posted oon Celerra forums about this and in the other blog post. You are not alone !! :)

(And I also agree somewhat with Phil above - getting this up and running is a bit cumbersome)

P

[a13x212]

Sarang,

Where you able to get pass the NFS Does not support MOUNT version 3 over TCP error? I'm encountering the same issue.

[ThomasT74]

I found a little mistake in the init_storageID script.
In the lower bit there is a while loop and it increases WAIT instead of WAITS.

I was discovering this as the script took very long to complete.
Stopping it with CTRL-C missed a needed step in the script and I was not able to create a file system.

[Anonymous]

hi,
thanks for the info on celerra.
I would like to ask is there a way i can get CISCO MDS fabric manager simulator???
It would be a great help if anybody can mail me or provide me with the link to download one.

[Account Deleted]

Hi Chad,
Thanks for all the information here. I was wondering when we could get 201 and 301 series to get our hands even more dirtier.

[Brad Brunner]

Having issues getting it to boot past blackbird, on Workstation 7. Do we need to run converter on it, so it works with later version of VMWorkstation?

[CraigT]

Hi Chad, Your information has been priceless in setting up my Celerra VM SIM (I am/was purely a Windows Guy). The only problem I have run into is after creating a CIFS server, I cannot get it to join to the Domain. I have set the date and time manually for the Server_2 DM and when I run server_date server_2 from the GUI CLI it returns the proper date/time now but the GUI (which for some reason does not allow the option to change it(its fixed)) does not show the correct time (4 hours off). I am not sure if that is the problem as the logs state " SMB: 3: dns_DCLookup::process:: Unable to get any DC from DNS for domain op.corp.lab
2010-08-24 13:23:44: DNS: 3: 1: Unable to connect to name server 172.xx.xxx.64 : Protocol not available

DNS is configured properly and I am able to ping from my DM IP to the DC (and vice versa)so I am not sure if it is the Clock skew in the GUI or the something else? Is there a reason the date and time is fixed for the DM but not for the Control Station Properties?

[Andy Lawrence]

Can you show how to get iSCSI working in the EMC Unisphere with the version 6 of the simulator?

[Robert]

Hi Chad,

I cannot figure out how to download Celerra Virtual appliance.
Can you let me know how to download the file?

Thank You,
Robert

[bha]

hi can you help me with the celera simulator download

[Chad Sakac]

@Robert, @BHA - it's pretty straightforward (unless a link is busted). If you look at the top, this post fans out from a "master link". Follow that master link, and you'll find the latest Celerra download.

OR

Google "UBER Celerra VSA"....

[Eric]

Hi,

How to solve the error after a fresh setup saying "Box Monitor Recover slot 3 failed."

Thanks in advance,
Eric

[Eric]

Hi Bah,

You can download the latest simulator from nick's website.

http://nickapedia.com/2010/09/12/ubertastic-celerra-uber-vsa-v3-unisphere/

Regards,
Eric

[Cheap Chanel Bags]

frustration and regret that I didn't pay closer attention in geometry class.
It appears that she's only made thirteen movie and/or television appearances during her career, the last one something called Buddy Boy in 1999. My exhaustive internet research turned up little more than squat.

[Brajesh]

I am using latest simulator with two data movers - downloaded from power link. I am able to bind CGE0 & CGE1 to eth0 & eth1 respectively. I can create a data mover interface using CGE0. I can ping from this interface to outside server like NTP, my test servers and vice versa. Not a showstopper issue but a wired message is scrolling in my vm console. Specially whenever any traffic is hitting this interface. "Protocol 0000 is buggy, dev eth0"

[Mario]

Just got to this article by google after the "MOUNT version 3 over TCP error".

I found the problem: This error message appears if you try to connect to the managment interface of the VNX. After you create (or before) a share / mount etc. you need to create a new interface (Settings -> Network -> Interface -> Create).

After i created the interface it worked directly for me.

[Priitz]

Rajesh,

I had the same problem; "Protocol 0000 is buggy, dev eth0".

Cause: Appears when there is a conflict.

Here is the fix:

create an interface on "cge1 mapped to eth1" and not "cge0 mapped to eth0". Use the interface for creating CIFS server etc.

Illustration: (interface new on cge1 mapped to eth1)

new protocol=IP device=cge1
inet=190.10.11.14 netmask=255.255.255.0 broadcast=190.10.11.255
UP, ethernet, mtu=1500, vlan=0, macaddr=0:c:29:6f:71:98

"cge0 mapped to eth0" is used for the internal communication between control station and datamover. When we create an interface on this physical device and use it for CIFS; it creates a conflict.

illustration: (default interfaces e131 & e130 for internal comm.)

el31 protocol=IP device=el31
inet=128.221.253.2 netmask=255.255.255.0 broadcast=128.221.253.255
UP, ethernet, mtu=1500, vlan=0, macaddr=44:41:52:54:0:6 netname=localhost

el30 protocol=IP device=el30
inet=128.221.252.2 netmask=255.255.255.0 broadcast=128.221.252.255
UP, ethernet, mtu=1500, vlan=0, macaddr=44:41:52:54:0:5 netname=localhost

Thank you.

[Virtual Pilot 3d Review]

Most folks are doing well if they just RTFM, but I thought a little "hand-crafted walkthrough" might help others. The standard documentation is geared towards VMware Workstation, which is the officially supported target (BTW, you don't need to ask, we're making ESX OVF a standard package :-), so some people stumbled a bit when trying to use it with ESX.
Virtual Pilot 3d Review

[Godgive]

I have problem when i want to join a domain during the creation of CIFS server. What should be done to go through.