What is an RDM or Raw Device Mapping?
An RDM, or Raw Device Mapping, is a mechanism for giving a virtual machine direct access to a LUN. This LUN can then be formatted with any file system, such as NTFS for Windows virtual machines, which removes the need for creating a VMFS datastore and storing a virtual disk (vmdk) on it.
You could think of an RDM as a symbolic link from a VMFS volume to a raw LUN. A mapping file is created, and is referenced in the VM’s configuration file, which points to the raw LUN. This mapping file acts as a proxy for the physical device and contains metadata used for managing and redirecting access to the raw disk. A virtual machine reads the mapping file, then subsequently sends it’s read and write requests directly to the raw LUN.
There are two modes in which you can configure an RDM; Virtual Compatibility Mode and Physical Compatibility Mode, seen here in the virtual machine settings when adding an RDM:
Physical Compatibility Mode means that there will be minimal SCSI virtualisation of the mapped device. The host passes all SCSI commands, with the exception of the REPORT LUNS command, directly to the storage device. This can be useful when running SAN management software inside a virtual machine, as SAN management agents tend to need direct access to the device, without having the SCSI commands intercepted by the host/virtualised. An example of a SAN application that requires a physical mode RDM is NetApp’s SnapManager suite. A physical mode RDM also allows for Physical to Virtual clustering. The downside is that you cannot perform certain VM management functions on disks configured in physical mode. This includes VM snapshots, cloning and storage vMotion.
Virtual Compatibility Mode virtualises all SCSI commands between the virtual machine and the storage device. As far as the guest operating system is concerned, it operates in exactly the same way as a vmdk on a VMFS volume. Because of this, virtual compatibility mode enables you to use features such as snapshots and cloning. Though it is important to note, if a VM with a RDM in virtual mode is cloned, the contents of the RDM lun is copied into a vmdk (as the raw lun itself cannot be cloned by a host).
With VMFS5, you can have RDMs larger than 2TB, but only when the RDM is in physical compatibility mode. However, you cannot convert these larger RDMs to virtual disks as it would exceed the 2TB maximum vmdk size.
Benefits and Use Cases of Raw Device Mapping
Along with the benefits mentioned above around using SAN software and LUNs greater than 2TB, other benefits include User Friendly Persistent Names, vMotion, Snapshots (virtual compatibility mode only), and NPIV.
User Friendly Persistent Names provide a friendly name for RDM device. Once the RDM has been created you will not need to refer to the device by it’s device name. Instead you will be able to refer to the device using the name of the mapping file. For example:
/vmfs/volumes/myVolume/myVMDirectory/myRawDisk.vmdk
An RDM can be included when you carry out a svMotion or a cold migration. There are a few things to be aware of though. With RDMs in physical compatibility mode, only the mapping file will be migrated. With RDMs in virtual compatibility mode, the raw disk will be migrated to either a thin provisioned or thick vmdk. Once migrated into a vmdk file you cannot migrate back to a RDM (unless you create a new RDM and copy the data from within the guest..).
Guest OS Clustering RDM Considerations
A common use case for RDMs is guest OS clustering (think Microsoft Cluster Services) and physical to virtual clustering. There are a few different ways to build a cluster using virtual machines, with different requirements when it comes to using RDMs.
CIB – Cluster In a Box – Most often used for test and development scenarios. This is where you have two VMs running on the same host, configured as a cluster. When using Microsoft clustering it’s recommended to use an RDM in virtual compatibility mode.
CAB – Cluster Across Boxes – When the VMs making up the cluster are on different hosts, then either RDM mode will work, though it’s recommended to use virtual compatibility mode.
Physical-to-Virtual Cluster – Here it is necessary to use physical mode compatibility for the RDM.
This is summarised, along with other RDM features, here.
Limitations of Raw Device Mappings
- RDMs can not be created on directly attached block devices.
- Virtual machine snapshots cannot be used for RDM disks that are in physical compatibility mode, however they are available when using virtual mode. If you attempt to snapshot a virtual machine with an RDM in physical mode you will see the following error:
- You cannot map a RDM to a partition on a LUN. It has to be mapped to the whole LUN.
- NFS is not supported as a destination for an RDM disk migration.
How to Create a RDM
After assigning the storage to a host and performing a rescan, we can add the storage directly to a virtual machine as an RDM disk. In this example, we’ll add an RDM to a Windows 2008 R2 Server Core virtual machine. Running diskpart before beginning we can just see the one disk configured on the VM:
To add a raw disk mapping, open the virtual machines settings and add a new hard disk. Choose RDM when you get to the Select a Disk page:
On the next screen you should be able to select the LUN:
There will next be a prompt about where to store the RDM mapping file:
On the next page we can choose the compatibility mode of the RDM – Physical or Virtual:
On the next screen we can choose a SCSI virtual device node, and if the RDM is virtual mode we can choose whether or not we’d like the disk to be Independent. With physical RDMs there is no choice regarding the mode, as snapshots cannot be taken regardless.
Click next and review the summary of the chosen options, then click Finish, then exit the VM settings screen, committing the changes.
On the guest OS we can then do a rescan, and the disk should be available:
Looking in the VMs directory on the datastore we can see the RDM mapping files. This is an example of a virtual RDM. For physical RDMs the filename will be xxxx-rdmp.vmdk.
We can view the contents of the descriptor file by running ‘cat Testvm2_1.vmdk’. In this case we get the following:
The highlighted parts show where this differs from an ordinary vmdk descriptor file.
Provided we have the disk uuid, we can create RDMs from the command line using the following commands:
For virtual compatibility mode RDMs we can use:
vmkfstools -r /vmfs/devices/disks/<device> example.vmdk
And for physical mode RDMs we use:
vmkfstools -z /vmfs/devices/disks/<device> example.vmdk
We can also use Powercli to create both types of RDM:
$deviceName = ($host | Get-ScsiLun | Where {$_.CanonicalName -match “naa”})[0].ConsoleDeviceName
New-HardDisk -VM $vm -DiskType RawPhysical -DeviceName $deviceName
Finding RDMs Using PowerCLI
Get-VM | Get-HardDisk -DiskType “RawPhysical”,”RawVirtual” | Select Parent,Name,DiskType,ScsiCanonicalName,DeviceName | fl
References and Useful Links
Final Word
In this article you have learned about VMware RDMs, otherwise known as a Raw Device Mapping. This has included some backgroup information on what a raw disk mapping is and a tutorial on how to create an RDM.