What Isilon means from my viewpoint…

I’m going to stay silent until the Isilon deal closes – as people have seen/learnt over the last year, these things aren’t done until they are done, so this will be my only post on it for a while.  

So… note – I wrote this on Nov 15th, right after the deal was public, but before it closed. 

So…

1. For what it’s worth – I’ve been a big supporter of Isilon in the EMC family through the M&A dance.  I think it’s an important category (very scalable scale-out NAS – categories where SONAS, Ibrix, Exanet and others play) – and one that Isilon are clearly very, very good at, and one that frankly, EMC and other NAS leaders on their own were not that good at.   Now they have more resources for engineering and go-to-market (ADDED Dec 21st - just look at this list of job openings!!)
2. There is a VMware angle to scale-out, extreme performance – aka “Big Data NAS”.  That angle falls into “some today, but even more over time” (more on this below)
3. This means EMC has great portfolio answers for:
   1. scale-out block (VMAX)
   2. scale out NAS (Isilon)
   3. scale-out object (Atmos)
   4. Best in breed inline dedupe for backup (Data Domain)
   5. These capabilities that are second-to-none.  They compliment “mainstream Unified Storage”  (with more exciting news on that front soon enough)
4. In my opinion, we will see that for a long time (perhaps perpetually), there will “mainstream Unified Storage” and “Big Data Transactional storage” as separate swim lanes..   To help understand this, think about it these two ways:
   • Many customers want a “swiss-army knife” when it comes to storage.  
     • They want something:
       • very feature rich
       • very available
       • very flexible
       • Supporting every multi-protocol (FC/FCoE/iSCSI/NFS/CIFS, with object as a bonus)
       • Supporting a performance envelope of many GBps of bandwidth and from thousands to hundreds of thousands of IOps.
     • Those requirements can be very well met with “traditional storage” non-scale out models.   They are also requirements  EMC supports very well with EMC Unified Storage today.  This is an area where we’re going to continue to innovate in.  While we need to find ways to leverage TONS of CPU cores in the commodity hardware we use in EMC Unified platforms, it’s not fundamentally about “scale-out”.  
     • Anything that adds complexity, cost unnecessarily can throw the “storage brains/ports/cache vs. IOps/GB” ratios out of whack.   EMC Unified does what the majority of the “Classic Swiss Army Knife enterprise storage requirements” very, very well TODAY, and it’s only going to get better. 
     • Many of those requirements are things that Isilon dreams of doing today.   
   • But,  sometimes a customer knows that they will have a “Big Data” problem from the get go.  
     • These are characterized by:
       • extreme scale-out transactional behavior and designs. 
       • In NAS land we’re talking 10PB or more, many tens of GBps (or 100s of Gbps), 100+ nodes – all delivering extreme performance, balancing cache/metadata/data/ports across every node.
       • Scale-out models require a different approach to resiliency – big failures can only partially affect overall system envelopes.
       • Similar (but not the same) requirements exist in scale-out block and object models. 
     • EMC Unified (and many of it’s competitors) dream of doing some of the things Isilon, VMAX, and Atmos do today.   

Long and short - expect to continue to see more and more EMC innovation, growth, development (both organic and M&A if that’s the right thing to do) down both categories from EMC:

1. “Simple, Efficient, Unified”
2. “Scale-Out Big Data”

Now – a little more detail on two areas that need more detail (VMware and scale-out NAS, Unified vs. scale-out)… Read on if interested…

Explaining #2 (VMware and scale-out NAS) a little more…  I’ve talked about the areas of scale-out NAS and VMware that are core challenges today (independent of the NFS server’s characteristics).   In particular, the limitations in the current vmkernel to NFS v3 – translate to all NFS traffic originating from an ESX host to a single datastore ending up on a single MAC interface (due to there being a single TCP session).  I’ve discussed this in detail here.

So – this limits scale-out NAS in VMware use cases in the sense that all the traffic for a given datastore hits a single “head”, a single set of cache, a single ethernet interface, no matter how you slice it.   For some customer use cases – this is not a big deal.  Others just balance across many datastores.  Others just brute-force it with 10GbE.  But for many, it’s… less than ideal.

That’s not to say that Isilon’s scale-out approach isn’t good TODAY, just that when it comes to VMware, you only get half the “scale out” equation (remember, this is always for a single datastore – multiple datastores can be distributed) 

If you had 3 Isilon nodes as an example – they would look something like this:

A bigger cluster would look like this:

The key to Isilon (at least from my firehose drinking over the last few months) is that the filesystem is spread out amongst all the nodes (along with the cache, metadata and all sorts of other stuff).   Any file query can come into any node, and be serviced by all the others.  You can have a big giant filesystem across all the nodes.

This means that in vSphere 4.1, only one node will service the NFS queries – but all the nodes will service the IO itself.   You get very scaleable backend performance, but for a single datastore, are still limited to the bandwidth of a single node (hundreds of MBps).   The green in the diagram below are “elements in a 3 node cluster which are actively servicing a given VMware NFS datastore”:

That diagram (thanks John!) is why you get Isilon customers noting:

“as I add more nodes, my VMware environment on NFS just gets faster!”

It is because behind the scenes, the data is being rebalanced across all the nodes, adding more and more backend bandwidth and more IOps.   You could achieve a similar effect (admittedly much more manually) on an EMC unified platform by spreading a filesystem across more and more spindles.

What about going forward?   In “future vSphere releases”, we’ve intimated that the current NFSv3 may be upped to NFSv4.1 (multiple sessions would be nice), and perhaps pNFS as well (read more on that here) 

But, there’s also something that could happen and would be an easier thing.   Future vSphere releases could also vary the NFS repository by not just referring to it by IP address and export, but by HOSTNAME, IP address and export.  This could easily make every ESX host access a datastore via a different IP, which means you would scale up everything up – CPU, ports, cache, and the backend all together.

That would mean that for this same Isilon cluster (and remember, 3 nodes is the SMALLEST they can be, the largest is 100+ nodes – we aren’t talking about 6, 12, 24 nodes), could have all three nodes both servicing NFS operations and also servicing backend IOs. 

It’s cool now, and will only get MORE cool.   BTW – remember - when we’re talking about “scale-out”, you can have hundreds of Isilon nodes in a scale-out cluster.

Explaining #4 (Unified vs. Scale-Out) a little more…  Before any acquisition (small or large) one of the most important things is talking to customers.    Isilon customers overwhelmingly had very positive things to say.   And they have 1400+ customers in every possible size, segment and vertical.

One of the most consistent was that the way that it scaled out was very simple.  Unlike other designs in this space, there aren’t dedicated metadata nodes, lock management nodes, caching nodes.   Every node shares common functions (although there are different node types that change the compute/storage ratio for various use cases), with: protocol services, metadata, caching, and back-end IO all being distributed across all nodes (and getting rebalanced as additional nodes are added).

Heck – there’s no traditional RAID model – there is file-level RAID which makes for crazy awesome rebuild characteristics.

This model makes for killer, simple scale-out NAS.   They even have the “Scale-Out NAS” equivalent of EMC FAST (fully automated tiering) – Isilon scale-out grids can be composed of different node types (IOps centered S-series nodes, bandwidth-centered X-series, or GB-centric NL nearline series), and their “smart pools” automatically move the dataset around as it’s needs and characteristics change.  

But – all that doesn’t (naturally or immediately) translate into simple Unified storage model, or killer scale-out block capabilities.  For example, there’s no FC or FCoE support today.   There’s a lot more, but that highlights what I’m talking about.

Obviously, over time the goal is always to further simplify (wherever possible – and embrace logical divergence where use cases are very divergent). 

Wrapping this up – I’ll restate what I said earlier…

• For those that need simple, efficient, unified storage – EMC has a killer answer in EMC Unified storage.  Think of the last category as the “swiss army knife”, not the “tactical nuke”.
• For customers that need scale out NAS storage – EMC has a killer “tactical nuke” answer in Isilon.  
• For those that need scale-out enterprise block storage – EMC has a “tactical nuke” killer answer in VMAX. 
• For those that need internet-scale scale-out object storage – EMC has a “tactical nuke” killer answer in Atmos. 

But… EVERY EMC platform are all x86-based commodity hardware platforms.   If only…

If only EMC had a extensible modern FLEX-based Unified UI (look at this partner feedback from twitter: @veverything “customer decided to go with the EMC storage solution based solely on Unisphere. Nice job to the engineers at EMC!”)

If only EMC had some killer encapsulation intellectual property that would help us, over time, bring Isilon tech into other packages, other form factors, and more completely into the family :-)

Not only do those massive engineering investments in UI homogenization and encapsulation enable us to accelerate ourselves by making merge internal assets (I’ve already pointed out how it accelerated our EMC Unified Thin abilities, our FAST VP capabilities, our compression capabilities) , it also makes EMC able better leverage M&A as a strategic option.    As much as we work to be innovative (as any company must), if you limit your rate of innovation to what you can do organically, you will be in trouble.

In conclusion… Welcome Isilon – I’m very happy to have you part of the family!   And I’ll say something I often say again…  Take a look at this slide I’ve been presenting since VMworld for ideas on the fun stuff going on – see the hint that has been in there for a while, highlighting so it’s obvious :-)

So consider this blog a partial answer to “What I think it means…”, and also a “more to come” :-)

Very exciting days!   Happy holidays all!

Comments

[Storagecritic]

Hi Chad,

Congrats on getting Isilon.
Just when the customers were thinking Celerra's were greatly improving and scaling up, EMC NAS gets rebooted. Seems to be that EMC is changing from "Where Information lives" to "Where Companies Lives". I had been a fan of EMC since early days as a customer. Need some real cool innovation like "Symmetrix" of the past from EMC. Understand the need for survival but would like something great from EMC innovation team.

Data is Data whether Big or Small. Why not have one Unified array for all type of data from 1 TB to 100 PB that can support federation, cloud, backups and so on? Hope I am right thinking something special may be cooking at EMC

[Jason Morris]

I am very excited about this - Isilon has always been amazing to work with and the scale out capability the way Isilon does it is very unique.

From my experience, I have to agree, the more you scale out the faster your NFS datastore get's - the throughput on our cluster was unbelievable!

I hope EMC does the right thing and keeps Colby, Isilon's Engineer on board - that guy made working in Isilon awesome!