Administer Hardware Acceleration for VAAI

VAAI stands for vStorage APIs for Array Integration. It provides Hardware Acceleration functionality, enabling your ESXi hosts to offload certain virtual machines and storage operations to the storage array. When VAAI is in use, ESXi hosts are able to perform these operations faster and with less impact to the host itself. VAAI requires:

Enterprise or Enterprise Plus licence
Storage Array that supports VAAI storage-based hardware acceleration. You can determine whether your array supports VAAI by checking the VMware HCL.

The Hardware Acceleration available for block devices are:

ATS (Atomic Test & Set) – Used during locking and creation of files
Clone Blocks/Full Copy – Used to copy or migrate data within the same array
Zero Blocks – Used to zero out disk regions
Thin Provisioning – Allows the array to reclaim space on thin provisioned LUNs.
Block Delete – Allows for space to be reclaimed using the SCSI UNMAP feature. See this article for more information.

VAAI is controlled by these advanced settings:

HardwareAcceleratedLocking (ATS)
HardwareAcceleratedMove (Full Copy)
HardwareAcceleratedInit (Zero Blocks)

Using the vSphere client, you can easily see whether Hardware Acceleration is supported for your storage device. To do so, select a host that has access to the storage device, then browse to Configuration | Storage:

You can also check whether it is supported using esxcli:

~ # esxcli storage core device vaai status get
naa.60a980900646e3Cfa3346f3450576f59
   VAAI Plugin Name: VMW_VAAIP_NETAPP
   ATS Status: supported
   Clone Status: supported
   Zero Status: supported
   Delete Status: supported

You can check to see whether VAAI is enabled by browsing to Configuration | Software | Advanced Settings, and checking the following features are enabled:

DataMover.HardwareAcceleratedMove
DataMover.HardwareAcceleratedInit
VMFS3.HardwareAcceleratedLocking

If they are enabled, the value will be set to ‘1’, which is the default:

You can also check if VAAI is enabled by using the CLI. You can either use esx/vicfg-advcfg to check the advanced settings:

# esxcfg-advcfg -g /DataMover/HardwareAcceleratedMove
# esxcfg-advcfg -g /DataMover/HardwareAcceleratedInit
# esxcfg-advcfg -g /VMFS3/HardwareAcceleratedLocking

Or you can use esxcli:

# esxcli system settings advanced list -o /DataMover/HardwareAcceleratedMove
# esxcli system settings advanced list -o /DataMover/HardwareAcceleratedInit
# esxcli system settings advanced list -o /VMFS3/HardwareAcceleratedLocking

For example:

~ # esxcli system settings advanced list -o /DataMover/HardwareAcceleratedMove
   Path: /DataMover/HardwareAcceleratedMove
   Type: integer
   Int Value: 1
   Default Int Value: 1
   Min Value: 0
   Max Value: 1
   String Value:
   Default String Value:
   Valid Characters:
   Description: Enable hardware accelerated VMFS data movement (requires compliant hardware)

T10 based SCSI Commands

If your storage device is T10 compliant then it will use the T10 based SCSI commands, which allows hardware acceleration without the use of a VAAI plugin. If you storage device isn’t T10 compliant then VAAI is used to enable hardware acceleration. Read more about this here.

VAAI Plugins and Filters

In order to communicate with devices that do not support the T10 SCSI standard a VAAI filter and a vendor-specific VAAI plugin is used. To view the VAAI plugins you can run:

~ # esxcli storage core plugin list -N VAAI
Plugin name       Plugin class
----------------  ------------
VMW_VAAIP_NETAPP  VAAI

And to display the filters you can run:

~ # esxcli storage core plugin list -N Filter
Plugin name  Plugin class
-----------  ------------
VAAI_FILTER  Filter

You can view the Hardware Acceleration support status of a device by running:

~ # esxcli storage core device list -d naa.xxxxxxxxxxxxxxxxxxxxxxxxxxx
   Display Name: NETAPP Fibre Channel Disk 
   Has Settable Display Name: true
   Size: 1024111
   Device Type: Direct-Access
   Multipath Plugin: NMP
   Vendor: NETAPP
   Model: LUN
   Attached Filters: VAAI_FILTER
   VAAI Status: supported

VAAI Claimrules

Every block storage device that is managed by VAAI needs two storage claim rules. It needs one that specifies the hardware acceleration filter and it needs one that specifies the hardware acceleration plugin for the device.

To list the filter claim rules:

~ # esxcli storage core claimrule list -c Filter
Rule Class   Rule  Class    Type    Plugin       Matches
----------  -----  -------  ------  -----------  ----------------------------
Filter      65429  runtime  vendor  VAAI_FILTER  vendor=MSFT model=Virtual HD
Filter      65429  file     vendor  VAAI_FILTER  vendor=MSFT model=Virtual HD
Filter      65430  runtime  vendor  VAAI_FILTER  vendor=EMC model=SYMMETRIX
Filter      65430  file     vendor  VAAI_FILTER  vendor=EMC model=SYMMETRIX
Filter      65431  runtime  vendor  VAAI_FILTER  vendor=DGC model=*
Filter      65431  file     vendor  VAAI_FILTER  vendor=DGC model=*
Filter      65432  runtime  vendor  VAAI_FILTER  vendor=EQLOGIC model=*
Filter      65432  file     vendor  VAAI_FILTER  vendor=EQLOGIC model=*
Filter      65433  runtime  vendor  VAAI_FILTER  vendor=NETAPP model=*
Filter      65433  file     vendor  VAAI_FILTER  vendor=NETAPP model=*
Filter      65434  runtime  vendor  VAAI_FILTER  vendor=HITACHI model=*
Filter      65434  file     vendor  VAAI_FILTER  vendor=HITACHI model=*
Filter      65435  runtime  vendor  VAAI_FILTER  vendor=LEFTHAND model=*
Filter      65435  file     vendor  VAAI_FILTER  vendor=LEFTHAND model=*

And to list the plugin claim rules:

~ # esxcli storage core claimrule list -c VAAI
Rule Class   Rule  Class    Type    Plugin            Matches
----------  -----  -------  ------  ----------------  ----------------------------
VAAI        65429  runtime  vendor  VMW_VAAIP_MASK    vendor=MSFT model=Virtual HD
VAAI        65429  file     vendor  VMW_VAAIP_MASK    vendor=MSFT model=Virtual HD
VAAI        65430  runtime  vendor  VMW_VAAIP_SYMM    vendor=EMC model=SYMMETRIX
VAAI        65430  file     vendor  VMW_VAAIP_SYMM    vendor=EMC model=SYMMETRIX
VAAI        65431  runtime  vendor  VMW_VAAIP_CX      vendor=DGC model=*
VAAI        65431  file     vendor  VMW_VAAIP_CX      vendor=DGC model=*
VAAI        65432  runtime  vendor  VMW_VAAIP_EQL     vendor=EQLOGIC model=*
VAAI        65432  file     vendor  VMW_VAAIP_EQL     vendor=EQLOGIC model=*
VAAI        65433  runtime  vendor  VMW_VAAIP_NETAPP  vendor=NETAPP model=*
VAAI        65433  file     vendor  VMW_VAAIP_NETAPP  vendor=NETAPP model=*
VAAI        65434  runtime  vendor  VMW_VAAIP_HDS     vendor=HITACHI model=*
VAAI        65434  file     vendor  VMW_VAAIP_HDS     vendor=HITACHI model=*
VAAI        65435  runtime  vendor  VMW_VAAIP_LHN     vendor=LEFTHAND model=*
VAAI        65435  file     vendor  VMW_VAAIP_LHN     vendor=LEFTHAND model=*

To configure hardware acceleration for a new array you need to add filter and plugin claimrules. You then need to load the rules into runtime. To create the claimrules, for example:

~ # esxcli storage core claimrule add --claimrule-class=Filter --plugin=VAAI_FILTER --type=vendor --vendor=EMC --autoassign
~ # esxcli storage core claimrule add --claimrule-class=VAAI --plugin=VMW_VAAIP_SYMM --type=vendor --vendor=EMC --autoassign

Then to load the claim rules into the system:

~ # esxcli storage core claimrule load --claimrule-class=Filter
~ # esxcli storage core claimrule load --claimrule-class=VAAI
~ # esxcli storage core claimrule run --claimrule-class=Filter

If you wanted to prevent a particular storage array from using VAAI you can do so by using the VMW_VAAIP_MASK VAAI plugin rule. For example:

esxcli storage core claimrule add -t vendor -P VMW_VAAIP_MASK -V EMC -c VAAI --autoassign

You would then need to load the rules, as shown earlier.

Useful Links and Resources

https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1021976
https://pubs.vmware.com/vsphere-51/index.jsp#com.vmware.vsphere.storage.doc/GUID-9933B376-2EC2-4862-9DD9-4C9314B71ED3.html

Administer Hardware Acceleration for VAAI

T10 based SCSI Commands

VAAI Plugins and Filters

VAAI Claimrules

Useful Links and Resources

Upgrading Datastores From VMFS3 to VMFS5

Apply VMware Storage Best Practices