One of the most popular 2008 articles I've posted was this one on how to build a home-brew ESX server. Within EMC it's a game now for our nerd elite to come up with creative ways to do it cheaper, faster, smaller, and cooler :-)
Keep reading for the new (at least for now!) record holder, from Adam Jones a Sr. TC, VCP (and more acronyms behind his sig than I've ever seen :-) in Kansas City with a dual core, x86-64 (ergo future ready), 8GB machine....
Do you have your own home-brew story? I'd love to hear it!
Ok, so... the name of the game is cheap hardware that works with VI3.x and also the current builds of future VMware products. Bypass the most sensitive part (SATA controller support) by using the ESXi installable and dd to a bootable USB stick (though I can confirm my own success with Intel P35 chipset (ICH9) SATA controllers and VI3.x - will report back on the G/P33 and G/P35 chipset's SATA controllers and future ESX builds in a couple days).
UPDATE (Jan 5th, 2009) - yup, both the P33 (ICH9) and P35 chipset (ICH9) SATA controllers work out of the box with those future builds - odd, since the P33 doesn't with VI3.x - must use a different PCI ID....
This is a great place if you're looking to build your own home-brew and aren't using the shopping list links below and want to check on other parts and people's experiences.
The one thing I WOULD stay away from (if you are planning on moving beyond ESX 3.5 eventually) are the desktop intel cards - man, they've given me a lot of grief recently.. the Intel server cards (i.e. instead of the GT or PT, use the MT) have worked consistently. They are all good with ESX/ESXi 3.x, but for now at least, I've only had success with the MT with future builds - though that says nothing of future functionality or timing.
Heck, if you really want to have it easy, get something on the HCL :-)
Use either internal SATA drives in the ESX hosts, or low-cost external storage options and then leverage VSAs (Virtual Storage Appliances - like the Celerra VSA here) which can provide iSCSI and NFS shared storage to create clusters, and also to emulate the high-end array functions (snapshots, replication, SRM). You CAN run VC as a VM on these avoiding the need for a 3rd physical box. Do this by install VIC somewhere, point it at the first ESXi box, then create a VM, and install VC there (though I still prefer VC as a physical or a VM on another cluster, personally).
Just REMEMBER (this has given me grief lately - another post on that coming up), the ESXi build needs to have the scratch space (user world) specified (advanced settings on the host in the screenshot below) and enabled for VM HA to work (clear error message on this if you don't do it right), AND this needs to be a separate datastore location for each host in the cluster (no clear error message on this - it just doesn't enable on the subsequent hosts booting - the bottom checkmark is always not checked). VMware - a nit - there is a typo in the dialog box "user wold" (this is update 3)
This isn't hard to resolve (just create a bunch of iSCSI VMFS datastores or different NFS exports), but the fact that they MUST be different is not obvious...
Without further ado, Adam's h/w list:
ESX 3i Server with 8 GB RAM and 2 GB IP NICs - $326.93 Before RAM Rebate
- Case - Count: 1 - IN WIN IW-BT611T.300SL Silver Best Performed 0.8mm SECC Japanese Steel metal MicroATX Desktop Computer Case 300W Power Supply - Retail - http://www.newegg.com/Product/Product.aspx?Item=N82E16811108060 - $60.99
- Motherboard - Count: 1 - ASUS M2A-VM AM2 AMD 690G Micro ATX AMD Motherboard (note that this doesn't have ESX/ESXi SATA support, so you will need the ESXi Installable USB technique, and external storage) - Retail - http://www.newegg.com/Product/Product.aspx?Item=N82E16813131172 - $64.99
- Processor - Count: 1 - AMD Athlon 64 X2 5000+ Brisbane 2.6GHz Socket AM2 65W Dual-Core Processor Model ADO5000DOBOX - Retail - http://www.newegg.com/Product/Product.aspx?Item=N82E16819103211 - $56.99
- RAM - Count: 2 - OCZ Gold 4GB (2 x 2GB) 240-Pin DDR2 SDRAM DDR2 800 (PC2 6400) Dual Channel Kit Desktop Memory Model OCZ2G8004GK - Retail - http://www.newegg.com/Product/Product.aspx?Item=N82E16820227199 - $83.98 for 8 GB Before Rebate
- NIC - Count: 1or 2 - Intel PWLA8391GTL 10/ 100/ 1000Mbps PCI PRO/1000 GT Low Profile Desktop Adapter - OEM - http://www.newegg.com/Product/Product.aspx?Item=N82E16833106122 - $29.99 Each
- Optional for GB IP Network
Switch - Count: 1 - NETGEAR GS605 10/100/1000Mbps Desktop Switch - Retail - http://www.newegg.com/Product/Product.aspx?Item=N82E16833122140 - $38.99 Before Rebate
Storage - Adam used the Intel Entry Storage System SS4200-E - $353.55 (before you add your own SATA drives - up to 4 drives)
UPDATE (Feb 2, 2009) - The SS4200E is a fine choice, and so is the ix2. If you can wait a little bit... I would hold on. iSCSI work is in the lifeline codebase, but still pushing to make it available in all the platforms - which have varying CPUs, and varying iSCSI performance. for now, consider the shipping SS4200E and ix2 as NAS-only until I give a firm update. There are other reasons also to wait for about 1 month....
- This is one of the storage platforms (in this case Intel OEM) that runs EMC Lifeline (our SMB stack for NAS and increasingly iSCSI). The iomega StorCenter is using this same codebase in the 2 drive iomega StorCenter ix2 (the ix2 is actually on the VMware HCL here).
- WARNING the Intel unit isn't on the HCL, and whereas I know the iomega HCL roadmap (There are more of those SMB "VMware Ready" EMC iomega storage devices coming in larger drive configs... ), I don't for the Intel HW platform - so don't assume it will be on the HCL. remember - this is a home-brew!
- http://www.smallnetbuilder.com/content/view/30482/75/ - Review
- http://www.intel.com/products/server/storage-systems/ss4200-e/ss4200-e-overview.htm - Intel Product Main
- Now, the Intel unit has more CPU and spindle horsepower than the really low-end iomega devices (Adam measured 90MBps write and 50MBps read performance) and is a good buy at that price (could you go even cheaper, yes with things like openfiler), IMHO.