VAAI stands for vStorage APIs for Array Integration. It provides Hardware Acceleration functionality, enabling your ESXi hosts to offload certain virtual machines and storage operations to the storage array. When VAAI is in use, ESXi hosts are able to perform these operations faster and with less impact to the host itself. VAAI requires:
- Enterprise or Enterprise Plus licence
- Storage Array that supports VAAI storage-based hardware acceleration. You can determine whether your array supports VAAI by checking the VMware HCL.
The Hardware Acceleration available for block devices are:
- ATS (Atomic Test & Set) – Used during locking and creation of files
- Clone Blocks/Full Copy – Used to copy or migrate data within the same array
- Zero Blocks – Used to zero out disk regions
- Thin Provisioning – Allows the array to reclaim space on thin provisioned LUNs.
- Block Delete – Allows for space to be reclaimed using the SCSI UNMAP feature. See this article for more information.
VAAI is controlled by these advanced settings:
HardwareAcceleratedLocking (ATS)
HardwareAcceleratedMove (Full Copy)
HardwareAcceleratedInit (Zero Blocks)
Using the vSphere client, you can easily see whether Hardware Acceleration is supported for your storage device. To do so, select a host that has access to the storage device, then browse to Configuration | Storage:
You can also check whether it is supported using esxcli:
~ # esxcli storage core device vaai status get naa.60a980900646e3Cfa3346f3450576f59 VAAI Plugin Name: VMW_VAAIP_NETAPP ATS Status: supported Clone Status: supported Zero Status: supported Delete Status: supported
You can check to see whether VAAI is enabled by browsing to Configuration | Software | Advanced Settings, and checking the following features are enabled:
DataMover.HardwareAcceleratedMoveDataMover.HardwareAcceleratedInitVMFS3.HardwareAcceleratedLocking
If they are enabled, the value will be set to ‘1’, which is the default:
You can also check if VAAI is enabled by using the CLI. You can either use esx/vicfg-advcfg to check the advanced settings:
# esxcfg-advcfg -g /DataMover/HardwareAcceleratedMove # esxcfg-advcfg -g /DataMover/HardwareAcceleratedInit # esxcfg-advcfg -g /VMFS3/HardwareAcceleratedLocking
Or you can use esxcli:
# esxcli system settings advanced list -o /DataMover/HardwareAcceleratedMove # esxcli system settings advanced list -o /DataMover/HardwareAcceleratedInit # esxcli system settings advanced list -o /VMFS3/HardwareAcceleratedLocking
For example:
~ # esxcli system settings advanced list -o /DataMover/HardwareAcceleratedMove Path: /DataMover/HardwareAcceleratedMove Type: integer Int Value: 1 Default Int Value: 1 Min Value: 0 Max Value: 1 String Value: Default String Value: Valid Characters: Description: Enable hardware accelerated VMFS data movement (requires compliant hardware)
T10 based SCSI Commands
If your storage device is T10 compliant then it will use the T10 based SCSI commands, which allows hardware acceleration without the use of a VAAI plugin. If you storage device isn’t T10 compliant then VAAI is used to enable hardware acceleration. Read more about this here.
VAAI Plugins and Filters
In order to communicate with devices that do not support the T10 SCSI standard a VAAI filter and a vendor-specific VAAI plugin is used. To view the VAAI plugins you can run:
~ # esxcli storage core plugin list -N VAAI Plugin name Plugin class ---------------- ------------ VMW_VAAIP_NETAPP VAAI
And to display the filters you can run:
~ # esxcli storage core plugin list -N Filter Plugin name Plugin class ----------- ------------ VAAI_FILTER Filter
You can view the Hardware Acceleration support status of a device by running:
~ # esxcli storage core device list -d naa.xxxxxxxxxxxxxxxxxxxxxxxxxxx Display Name: NETAPP Fibre Channel Disk Has Settable Display Name: true Size: 1024111 Device Type: Direct-Access Multipath Plugin: NMP Vendor: NETAPP Model: LUN Attached Filters: VAAI_FILTER VAAI Status: supported
VAAI Claimrules
Every block storage device that is managed by VAAI needs two storage claim rules. It needs one that specifies the hardware acceleration filter and it needs one that specifies the hardware acceleration plugin for the device.
To list the filter claim rules:
~ # esxcli storage core claimrule list -c Filter Rule Class Rule Class Type Plugin Matches ---------- ----- ------- ------ ----------- ---------------------------- Filter 65429 runtime vendor VAAI_FILTER vendor=MSFT model=Virtual HD Filter 65429 file vendor VAAI_FILTER vendor=MSFT model=Virtual HD Filter 65430 runtime vendor VAAI_FILTER vendor=EMC model=SYMMETRIX Filter 65430 file vendor VAAI_FILTER vendor=EMC model=SYMMETRIX Filter 65431 runtime vendor VAAI_FILTER vendor=DGC model=* Filter 65431 file vendor VAAI_FILTER vendor=DGC model=* Filter 65432 runtime vendor VAAI_FILTER vendor=EQLOGIC model=* Filter 65432 file vendor VAAI_FILTER vendor=EQLOGIC model=* Filter 65433 runtime vendor VAAI_FILTER vendor=NETAPP model=* Filter 65433 file vendor VAAI_FILTER vendor=NETAPP model=* Filter 65434 runtime vendor VAAI_FILTER vendor=HITACHI model=* Filter 65434 file vendor VAAI_FILTER vendor=HITACHI model=* Filter 65435 runtime vendor VAAI_FILTER vendor=LEFTHAND model=* Filter 65435 file vendor VAAI_FILTER vendor=LEFTHAND model=*
And to list the plugin claim rules:
~ # esxcli storage core claimrule list -c VAAI Rule Class Rule Class Type Plugin Matches ---------- ----- ------- ------ ---------------- ---------------------------- VAAI 65429 runtime vendor VMW_VAAIP_MASK vendor=MSFT model=Virtual HD VAAI 65429 file vendor VMW_VAAIP_MASK vendor=MSFT model=Virtual HD VAAI 65430 runtime vendor VMW_VAAIP_SYMM vendor=EMC model=SYMMETRIX VAAI 65430 file vendor VMW_VAAIP_SYMM vendor=EMC model=SYMMETRIX VAAI 65431 runtime vendor VMW_VAAIP_CX vendor=DGC model=* VAAI 65431 file vendor VMW_VAAIP_CX vendor=DGC model=* VAAI 65432 runtime vendor VMW_VAAIP_EQL vendor=EQLOGIC model=* VAAI 65432 file vendor VMW_VAAIP_EQL vendor=EQLOGIC model=* VAAI 65433 runtime vendor VMW_VAAIP_NETAPP vendor=NETAPP model=* VAAI 65433 file vendor VMW_VAAIP_NETAPP vendor=NETAPP model=* VAAI 65434 runtime vendor VMW_VAAIP_HDS vendor=HITACHI model=* VAAI 65434 file vendor VMW_VAAIP_HDS vendor=HITACHI model=* VAAI 65435 runtime vendor VMW_VAAIP_LHN vendor=LEFTHAND model=* VAAI 65435 file vendor VMW_VAAIP_LHN vendor=LEFTHAND model=*
To configure hardware acceleration for a new array you need to add filter and plugin claimrules. You then need to load the rules into runtime. To create the claimrules, for example:
~ # esxcli storage core claimrule add --claimrule-class=Filter --plugin=VAAI_FILTER --type=vendor --vendor=EMC --autoassign ~ # esxcli storage core claimrule add --claimrule-class=VAAI --plugin=VMW_VAAIP_SYMM --type=vendor --vendor=EMC --autoassign
Then to load the claim rules into the system:
~ # esxcli storage core claimrule load --claimrule-class=Filter ~ # esxcli storage core claimrule load --claimrule-class=VAAI ~ # esxcli storage core claimrule run --claimrule-class=Filter
If you wanted to prevent a particular storage array from using VAAI you can do so by using the VMW_VAAIP_MASK VAAI plugin rule. For example:
esxcli storage core claimrule add -t vendor -P VMW_VAAIP_MASK -V EMC -c VAAI --autoassign
You would then need to load the rules, as shown earlier.
Useful Links and Resources
https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1021976
https://pubs.vmware.com/vsphere-51/index.jsp#com.vmware.vsphere.storage.doc/GUID-9933B376-2EC2-4862-9DD9-4C9314B71ED3.html