Changing SATP Claimrules for specific storage configurations

At my current customer I was upgrading their vSphere 4.1 environment to vSphere 5.5. When installing and configuring the first ESXi 5.5 host, I noticed a difference in the Path Selection Policy (PSP) between the ESX 4.1 hosts and the ESXi 5.5 host. Each host is connected to two Hitachi storage arrays, each a different model. In the current 4.1 config a general setting is used to have both arrays use Round Robin:

esxcli nmp satp setdefaultpsp --psp VMW_PSP_RR --satp VMW_SATP_DEFAULT_AA

Because of the above rule in the ESX 4.1 hosts, all storage that is not captured by the existing rule sets, will be set to Round Robin (VMW_PSP_RR). Although applying this rule to my ESXi 5.5 host, would solve the problem for me on short term, I could imagine running into an issue when the customer would attach a third storage that didn’t required Round Robin, but would then somehow still default to it, because of this “catch-all” rule.

To write a very specific rule for this storage array I would need to find out what vendor and model this storage array is according to ESXi. This was actually the hardest part, since I was using the wrong command at first, but with some help of the VMware Community and specifically Tom Verhaeg, I found the right command. Let me take you through the steps.

Changing SATP Claimrule

Changing the SATP Claimrules is easiest when done on the command line of the ESXi host using the esxcli module. First we need to find out for a specific LUN, what the current path policy is by running the command below. The naa number used is the LUN that has the incorrect Path Selection Policy. You see that for this LUN the current policy is VMW_PSP_FIXED, since the Storage Array Type (SATP) is VMW_SATP_DEFAULT_AA according to ESXi.

COMMAND:

esxcli storage nmp device list -d naa.60060e80132892005020289200001001

RESULT:

 naa.60060e80132892005020289200001001
 Device Display Name: HITACHI Fibre Channel Disk (naa.60060e80132892005020289200001001)
 Storage Array Type: VMW_SATP_DEFAULT_AA
 Storage Array Type Device Config: SATP VMW_SATP_DEFAULT_AA does not support device configuration.
 Path Selection Policy: VMW_PSP_FIXED
 Path Selection Policy Device Config: {preferred=vmhba1:C0:T0:L37;current=vmhba1:C0:T0:L37}
 Path Selection Policy Device Custom Config:
 Working Paths: vmhba1:C0:T0:L37
 Is Local SAS Device: false
 Is Boot USB Device: false

The next step is to find out what storage array Vendor and Model type this LUN is coming from, because we need this info to create a new SATP claiming rule. Running the following command, shows that we’re working with Vendor = Hitachi and Model = OPEN-V.

COMMAND:

esxcli storage core device list -d naa.60060e80132892005020289200001001

RESULT:

naa.60060e80132892005020289200001001
 Display Name: HITACHI Fibre Channel Disk (naa.60060e80132892005020289200001001)
 Has Settable Display Name: true
 Size: 1048576
 Device Type: Direct-Access
 Multipath Plugin: NMP
 Devfs Path: /vmfs/devices/disks/naa.60060e80132892005020289200001001
 Vendor: HITACHI
 Model: OPEN-V
 Revision: 5001
 SCSI Level: 4
 Is Pseudo: false
 Status: on
 Is RDM Capable: true
 Is Local: false
 Is Removable: false
 Is SSD: false
 Is Offline: false
 Is Perennially Reserved: false
 Queue Full Sample Size: 0
 Queue Full Threshold: 0
 Thin Provisioning Status: yes
 Attached Filters: VAAI_FILTER
 VAAI Status: supported
 Other UIDs: vml.020025000060060e801328920050202892000010014f50454e2d56
 Is Local SAS Device: false
 Is Boot USB Device: false
 No of outstanding IOs with competing worlds: 32

Before creating the new rule, I wanted to check the current SATP rule list, so I could also check after creating the rule, if there was a change. For easy of reading I only display the results here that were important to me:

COMMAND:

esxcli storage nmp satp rule list

RESULT:

Name	Vendor	Model	Rule Group	Claim Options	Default PSP
VMW_SATP_DEFAULT_AA	HITACHI		system	inq_data[128]={0x44 0x46 0x30 0x30}	VMW_PSP_RR
VMW_SATP_DEFAULT_AA	HITACHI		system

The first line tells ESXi that if you find a storage of Vendor Hitachi with specific claim options “inq_data[128]                              ={0x44 0x46 0x30 0x30}” (which I don’t fully understand), then the VMW_PSP_RR policy should be used. The second line says to apply the system default connected to VMW_SATP_DEFAULT_AA for all Hitachi arrays. Let’s check what the default for VMW_SATP_DEFAULT_AA is, although we already know by what we’ve seen before.

COMMAND:

esxcli storage nmp satp list

RESULT:

Name                 Default PSP    Description
——————-  ————-  ——————————————
VMW_SATP_MSA         VMW_PSP_MRU    Placeholder (plugin not loaded)
VMW_SATP_ALUA        VMW_PSP_MRU    Placeholder (plugin not loaded)
VMW_SATP_DEFAULT_AP  VMW_PSP_MRU    Placeholder (plugin not loaded)
VMW_SATP_SVC         VMW_PSP_FIXED  Placeholder (plugin not loaded)
VMW_SATP_EQL         VMW_PSP_FIXED  Placeholder (plugin not loaded)
VMW_SATP_INV         VMW_PSP_FIXED  Placeholder (plugin not loaded)
VMW_SATP_EVA         VMW_PSP_FIXED  Placeholder (plugin not loaded)
VMW_SATP_ALUA_CX     VMW_PSP_RR     Placeholder (plugin not loaded)
VMW_SATP_SYMM        VMW_PSP_RR     Placeholder (plugin not loaded)
VMW_SATP_CX          VMW_PSP_MRU    Placeholder (plugin not loaded)
VMW_SATP_LSI         VMW_PSP_MRU    Placeholder (plugin not loaded)
VMW_SATP_DEFAULT_AA  VMW_PSP_FIXED  Supports non-specific active/active arrays
VMW_SATP_LOCAL       VMW_PSP_FIXED  Supports direct attached devices

What the admin of this environment did in the past, was changing this “overall” default to VMW_PSP_RR, which is not the approach I want to take. The above table explains why the ESXi 5.5 host is now working with the VMW_PSP_FIXED policy. We’re now ready to change the SATP rule using “HITACHI” as storage Vendor, “OPEN-V” as model type. We’re telling the ESXi to use VMW_SATP_DEFAULT_AA with a PSP of “VMW_PSP_RR” when Venor and Model match our specification:

COMMANDS:

esxcli storage nmp satp rule add -V HITACHI -M "OPEN-V" -P VMW_PSP_RR -s VMW_SATP_DEFAULT_AA
esxcli storage core claimrule load

To check how this worked out, we check the satp rule list again:
COMMAND:

esxcli storage nmp satp rule list

RESULT:  (filtered results)

Name	Vendor	Model	Rule Group	Claim Options	Default PSP
VMW_SATP_DEFAULT_AA	HITACHI		system	inq_data[128]={0x44 0x46 0x30 0x30}	VMW_PSP_RR
VMW_SATP_DEFAULT_AA	HITACHI	OPEN-V	user		VMW_PSP_RR
VMW_SATP_DEFAULT_AA	HITACHI		system

To check if this changed the way the policy was applied to the LUNs, run the command below. In the documentation I found that the host should check the claimrules every 5 minutes, but I decided to reboot the host after adding and loading the new SATP rule. Maybe I was just being impatient :-)

COMMAND:

esxcli storage nmp device list -d naa.60060e80132892005020289200001001

RESULT:
naa.60060e80132892005020289200001001
Device Display Name: HITACHI Fibre Channel Disk (naa.60060e80132892005020289200001001)
Storage Array Type: VMW_SATP_DEFAULT_AA
Storage Array Type Device Config: SATP VMW_SATP_DEFAULT_AA does not support device configuration.
Path Selection Policy: VMW_PSP_RR
Path Selection Policy Device Config: {policy=rr,iops=1000,bytes=10485760,useANO=0; lastPathIndex=0: NumIOsPending=0,numBytesPending=0}
Path Selection Policy Device Custom Config:
Working Paths: vmhba1:C0:T0:L37, vmhba0:C0:T0:L37
Is Local SAS Device: false
Is Boot USB Device: false

WARNING: Changing Path Selection Policies or SATP rules, should only be done when you are 100% sure of what you’re about to do. Always check the VMware HCL to see what the prefered policy is for your storage array in combination with firmware and ESXi version. Also check what your vendor’s documentation recommends.