Having trouble importing a big OVF/OVA?

Have been working on a couple long nerdy posts, so today’s is a short one…

Last week was the “Managers GeekWeek”, and one part of our Kobayashi Maru scenario involved doing some creative stuff with an “as of yet” not released into the wild EMC VSA.  

Importing the VSA kept failing for some reason, complaining about “not enough space on the datastore” vCenter error, when clearly the datastore was big enough.

Here’s me importing the VSA – it deploys fat right now as an engineering build – it’s 400GB when deployed on the datastore (which of course is thinly provisioned at the array level with VAAI, so it deploys super fast :-)

But – the datastore is 1.9TB in size, so why does it throw this error?

The answer to the question is the allocation size. 

You set this when you create the VMFS filesystem, and most people just click “next, next, finish”.   The default allocation size is 1MB, which correlates to a maximum single file of 256GB – and this is the maximum file in the filesystem, PERIOD (so any file, including any VMDK). In the example above, I was importing a virtual appliance which had a large, single VMDK (400GB).   The error message spells it out, but otherwise smart folks were missing it.

So – how do you resolve?   Easy.  You use a larger allocation size.  Since this is a destructive process on the datastore, if there are VMs on the datastore, use Storage vMotion to vacate the datastore, then delete the datastore and recreate it.   This dialog is where you set the VMFS allocation size.   Here, selecting the largest current allocation size (8MB) correlates with the largest filesize being 2048GB.

I haven’t seen a best practice on the topic, but it seems to me the downside of the larger allocation sizes (slightly less efficient for small files that are <8MB in size) seems to me to be a small one.   Every datastore I’ve every seen is dominated by a relatively small number of very large VDMKs (base VMDKs or snapshots and memory state snapshots) – so personally, the slight loss in efficiency I’ll take over having the ability to easily put in big appliances as I see fit. 

Personally, I’m just starting to default to the larger 4MB or 8MB size.   What say you?

Comments

[BradC]

I've never seen a reason to pick anything other than 8MB. It sounds like the only thing you'll lose is a little bit of disk space:

http://www.boche.net/blog/index.php/2010/10/29/vmware-vsphere-design-workshop-day-3/

You should also pick one size and stick with it within a single shared storage system or VAAI won't be able to do large copy offloads:

http://itzikr.wordpress.com/2010/07/13/19/

[ScottW]

I've started using the non-default block sizes as well... the convenience of larger single VMDKs outweighs the slight performance trade-off...

(Also, I had the opportunity to catch Chad's presentation at Sydney's vForum last month. I must say he is a very entertaining speaker, and if anyone has the chance to see him, then do so!!)

[Jason Boche]

There have been several debates on block size. All reputable discussions end with a few take aways:

1. VMFS block size, whether large or small, has no measurable performance impact on the file system or I/O (hence no impact to guest VMs)

2. Large VMFS block size has little to no notable impact on disk waste in the grand scheme of things. Sub Block Allocation helps out here when writing small files on large blocks (particularly with smaller LUN sizes & hence less VMs & less small files ie. logs, nvram, etc.) but whatever SBA doesn't cover is a trivial discussion. VAAI begins to counter this point a bit as larger LUNs without traditional SCSI protocol constratins become viable but even with that I think the dollars and cents discussion around a trivial mount of disk waste is counter productive.

3. By virtue of the above, create 8MB block sizes for VMFS volumes and move on. There is little to no measurable down side and everything to gain in terms of predictable and adaptable scalability for future VM growth which is an unknown at the time of VMFS volume creation.

[VMTyler]

Ha. I ran into the exact same issue with the exact same VSA. Took me a bit to figure it out.

[Rawley Burbridge]

Consistency eliminates many problems, be it naming conventions or configurations, keep them consistent and you will eliminate many problems. I've defaulted to 8MB for awhile and use that block size really regardless of the size of the VMFS volume, just so I know the environment is consistently configured.

[Derek]

I also standardize on 8MB blocks, regardless of VMFS size, for the sake of consistency. This also ensures VAAI works as expected since the XCOPY operation needs consistent block sizes.

[Dpironet]

I had the same issue once. Since then I always recommend customers to format with a 8MB block size whatever the datatstore size is.
And something very important, keep it consistent across the datastores for many reasons, several cited in above comments...

For those who don't understand interactions between block size, file size and sub block, there is a good blog post available at http://wp.me/pvGaC-wP (DeinosCloud)

Rgds,
Didier

[Cole]

I standardize on 8MB blocks, just so that everything in my environment is consistent.

[Deinoscloud.wordpress.com]

I had the same issue once. Since then I always recommend customers to format with a 8MB block size whatever the datatstore size is.
And something very important, keep it consistent across the datastores for many reasons, several cited in above comments...

For those who don't understand interactions between block size, file size and sub block, there is a good blog post available at http://wp.me/pvGaC-wP (DeinosCloud)

Rgds,
Didier

[Matt Meyer]

I also have standardized on 8MB blocks for a couple reasons.

1. I don't like designing with limitations that cannot be easily changed.
2. 8MB blocks will theoretically reduce the amount of locking on the LUN, especially for View or Lab Manager environments with linked clones. Every time a vmdk needs to grow or a log file grows beyond its block allocation, the file will grow 8MB instead of the default 1MB. This will, again in theory, reduce the number of LUNs locks while the file grows by 8x. Reduce the number of locks and you reduce the number of storage related issues that come up because of it.