VMFS-5 introduced in vSphere 5

One of the new features in vSphere 5  is VMFS-5.

In fact there is over 140 of new features in VMware vSphere 5 which has been announced last week.

VMFS-5 is one of the new features in vSphere 5. VMFS – Virtual Machine File System. VMware wants to make the management of storage easier for the customer, by having the possibilities to create larger LUNs with more VMs. So to manage less datastores, because the possibility to have 2TB+ VMFS volumes without using extents. The use of extents was necessary in vSphere 4.x to create larger than 2TB datastores.

The 2TB limit stays for the size of the largest VMDK, but you can have 2TB+ disk if you're using Physical RDMs.

There is a theoretical maximum number of powered on virtual machines that are supported per VMFS volume –  2048. But, there are many considerations to take into account. And one must take into consideration the necessary IOPS for the underlying storage array to handle the load.

The unifying block size (1Mb) is introduced, and so you're able to create a larger files (>256GB) by using those blocks. You're no longer asked to use larger blocks – 2,4 or 8MB file blocks in order to create large files. But note that if you upgrade the existing VMFS-3 to VMFS-5, the block size stays the same as it was in VMFS-3.

 What to choose. The transition (upgrade) or creation of new datastores with VMFS-5 format?

The upgrade is clearly possible and non disruptive, but you won't benefit from much of the new features that VMFS-5 provides until you reformat the datastore and precise that you would like to use VMFS-5 as a format. Please note, that if you choose the VMFS-5 as a format, the only ESXi 5.0 hosts will be able to access those datastores.

VMware recommends to go clean, if you can. Create new datastore and format with VMFS-5.

As you can see on the image below, the choice between the VMFS-3 and VMFS-5 is indicated with the legacy host support for the VMFS-3 option.

If you have the time and all your infrastructure is changing/migrating to ESXi 5.0, then you'd prefer the VMFS-5, otherwise you will face the following limitations when you upgrade by keeping your existing datas on the datastores.

 You will face the following limitations when you upgrade only (not clean format) to VMFS-5:

– VMFS-5 will continues to use the previous file block size which may be larger than the unified 1MB file block size.

– VMFS-5 continues to use 64KB sub-blocks and not new 8K sub-blocks.

– VMFS-5 continues to have a file limit of 30720 rather than new file limit of > 100000 for newly created VMFS-5.

– VMFS-5 continues to use MBR (Master Boot Record) partition type; when the VMFS-5 volume is grown above 2TB, it automatically & seamlessly switches from MBR to GPT (GUID Partition Table) with no impact to the running VMs.
– VMFS-5 continue to have its partition starting on sector 128; newly created VMFS5 partitions will have their partition starting at sector 2048.

When you format directly your Datastores with VMFS-5 you'll benefit of:

Unified 1MB File Block Size –  Previous versions of VMFS used 1,2,4 or 8MB file blocks. These larger blocks were needed to create  large files (>256GB). These large blocks are no longer needed for large files on VMFS-5. Very large files can now be created on VMFS-5 using 1MB file blocks.

Large Single Extent Volumes –  In previous versions of VMFS, the largest single extent was 2TB. With VMFS-5, this limit has been increased to ~ 60TB.

Smaller Sub-Block –  VMFS-5 introduces a smaller sub-block. This is now 8KB rather than the 64KB we had in previous versions. Now small files < 8KB (but > 1KB) in size will only consume 8KB rather than 64KB. This will reduce the amount of disk space being stranded by small files.

Small File Support – VMFS-5 introduces support for very small files. For files less than or equal to 1KB, VMFS-5 uses the file
descriptor location in the metadata for storage rather than file blocks. When they grow above 1KB, these files will then start to use the new 8KB sub blocks. This will again reduce the amount of disk space being stranded by very small files.

Increased File Count – VMFS-5 introduces support for greater than 100,000 files, a three-fold increase on the number of files supported on VMFS-3, which was ~ 30,000.

ATS Enhancement. This Hardware Acceleration primitive, Atomic Test & Set (ATS), is now used throughout VMFS-5 for file locking. ATS is part of the VAAI (vSphere Storage APIs for Array Integration). Atomic Test and Set (ATS) also known as Hardware Assisted Locking: a mechanism to modify a disk sector to improve the performance of ESX updating metadata.This enhancement improves the file locking performance over previous versions of VMFS. 

Source: VMware Blog;  Stephen Fosket's blog;