Software-Defined Networks and Storage + Your Thoughts…

There's been an interesting idea percolating around EMC for the last two years - which is the application of SDN approaches to dramatically simplify the storage domain. 

• This could make at scale, heterogeneous environments (making the ability for persistence stacks - in all their forms, not just "arrays" - connect to each other, move things around) simpler and automated.
• This could make end-to-end concepts of multi-tenancy in storage (right now so hard that in practice, few do it) work - both in the VVOL domain and the basic VMFS or Nova->Cinder Volume mapping model.

Again - there's a tendency to look at the storage domain as "arrays" and server/software stacks - these are two different packages of the same thing.   The packaging matters - but from an architectural standpoint, it's the same.

Ergo - this topic applies to VNX/VMAX/XtremIO/Isilon customers in the same way it would ScaleIO customers (or people who wanted to have Object storage in specific tenant IP address domains, some that may overlap.

One of the engineers working on this (Erik Smith) has done a blog post on this topic here.

I also did a blog post on this back in August of 2013 (here) , and showed off a prototype then.

What this was is now more clear.  It was a VMAX3 prototype, using Hypermax (the platform OS) that has a hypervisor (more like containers than ESX) and running an Open vSwitch (OVS) instance locally as the VXLAN termination.

Here's another example - NFS services for a multi-tenant scenario:

The "WHY" on this is important (in the words of Erik):

Our goal is to enable dynamic connectivity by providing a storage friendly, network agnostic API that can be used by any storage service to establish connectivity to any other potentially connectable storage service.  This API could be provided by the VSN-CS (Virtual Storage Network Connectivity Service) that is described in the NFS Service demo. 

Here's my request - TELL US, ARE YOU INTERESTED?  Drop us an email at storageSDN@emc.com and tell us a) what you have; b) what you want :-)

Comments

[Briantwalter]

Personally, I think a number of these challenges would be solved if ESX supported native connections to HDFS and/or object stores as a persistent layer. NFS is simple, VMFS is robust, and vVol looks promising but why not something more standardized, modern, horizontally scalable, and open?

--EMCer

[Chad Sakac]

@BrianTWalter - reality is that HDFS and Object stores are just not good for the transactional load that you find under most VMDKs. The analogy is why AWS has both EBS and local storage (to boot EC2 images and for transactional load) and S3 (for object storage).

It just isn't converging. Models that layer transactional on top of object tend to not be particularly performant for transactional (small IO size, low latency).

Of course, HDFS and Object stores are supported the way they are normally accessed - through the network of the host (in this case a guest VM via the ESX host network stack).