This post is intended to cover the topics on the VCAP-DCA blueprint relating to tuning virtual machine configurations, with a focus on memory, CPU, network and storage settings. As a result, this will be a fairly long post, so lets get started!
Tuning Virtual Machine Memory Configurations
Along with the allocation of memory resources and setting limits, reservations and shares, there are a few other things to note around the VCAP-DCA blueprint and tuning virtual machine memory configurations.
Memory Hot Add
The Hot Add feature allows you to increase the memory allocation for a virtual machine (on supported Guest OS) whilst it is powered on. It is only possible to do this, however, if you have already enabled Memory Hot Add in the virtual machine’s settings. This has to be done whilst the virtual machine is powered off: 
Swap File Location
You can change an individual virtual machine’s swap file location by changing the option in the virtual machine’s settings: 
Associate Memory Allocations with a NUMA Node
You can set it so that a virtual machine gets all its future memory allocations from a single NUMA node:
Tune Virtual Machine Networking Configuration
I don’t have a lot to say here other than that, where possible, use the VMXNET3 adapter for your virtual machines. Also, be aware of Split RX mode.
Tune Virtual Machine CPU Configuration
Rather than just allocating a number of virtual CPUs to a virtual machine, it’s possible to define how these are split up in terms of virtual sockets and cores. This is generally done due to Guest OS/Application licensing restrictions:
As shown earlier in this post, it’s possible to hot add vCPUs on supported guest OS’ so long as the option has been enabled whilst the virtual machine is in a powered off state. You should also be aware of the CPU Affinity settings and the Hyperthreading settings:
I’ve written about these settings in a previous post.
CPU Identification Mask
The CPU Identification Mask (CPUID Mask) setting controls the CPU features that are exposed to the guest OS. Masking features can increase vMotion compatibility as, when carrying out a vMotion the host will compare the CPU features available to the guest operating system with the features available on the destination host. This is essentially an earlier, per-VM, version of EVC (as described here). You can read more about configuring CPU identification masks here.
CPU identification mask settings are available under the Options tab, in the virtual machine’s settings:
CPU/MMU Virtualisation
An ESXi host will automatically choose whether a virtual machine should use hardware support for virtualisation based on the host’s processor type and the guest OS installed on the virtual machine. However, the automatic setting can be overridden.
As stated in the Performance Best Practices guide, on processors that support hardware-assisted CPU virtualisation but not hardware-assisted MMU virtualisation, the monitor mode that ESXi chooses will either be binary translation (BT) with software MMU (swMMU) virtual machine monitor (VMM) mode or the hardware virtualization (HV) with swMMU VMM mode.
When the host has processors that support hardware-assisted MMU virtualisation ESXi will choose from HV with hardware MMU (hwMMU) VMM mode, HV with swMMU VMM mode, and BT with swMMU VMM mode, depending on the guest operating system installed in the virtual machine.
To manually change the mode, if supported, edit the virtual machine settings, choose the Options tab, and select CPU/MMU Virtualisation:
The options available are as follows:
- Automatic allows ESXi to determine the best choice. This is the default setting.
- Use software for instruction set and MMU virtualization – disables both hardware-assisted CPU virtualization (VT-x/AMD-V) and hardware-assisted MMU virtualization (EPT/RVI).
- Use Intel® VT-x/AMD-V™ for instruction set virtualization and software for MMU virtualization enables hardware-assisted CPU virtualization (VT-x/AMD-V) but disables hardware-assisted MMU virtualization (EPT/RVI).
- Use Intel® VT-x/AMD-V™ for instruction set virtualization and Intel® EPT/AMD RVI for MMU virtualization enables both hardware-assisted CPU virtualization (VT-x/AMD-V) and hardware-assisted MMU virtualization (EPT/RVI).
If you want to confirm which mode a virtual machine is using, it is recorded in the vmware.log file. A quick way to check is to search for ‘MONITOR’ in the log file:
/vmfs/volumes/4f27b82e-3fc1540e-bf6b-000c295da2d9/XP # cat vmware.log | grep MONITOR 2012-09-07T10:38:15.306Z| vmx| MONITOR MODE: allowed modes : BT32 HV HWMMU 2012-09-07T10:38:15.306Z| vmx| MONITOR MODE: user requested modes : BT32 HV HWMMU 2012-09-07T10:38:15.306Z| vmx| MONITOR MODE: guestOS preferred modes: HWMMU HV BT32 2012-09-07T10:38:15.306Z| vmx| MONITOR MODE: filtered list : HWMMU HV BT32
The filtered list displays the valid modes for the VM, with the left most mode being utilized.
Tune Virtual Machine Storage Configurations
When it comes to tuning virtual machine storage configurations, the main topics I can think of (and I’ll link to them rather than re-create the content here…) are types of virtual disk and RDMs.