FC

VMware T10 compliant VAAI integration and HP P2000 G3 FC storage

by

 

I’ve recently been updating firmware on some development and testing storage and found that the HP P2000 storage array firmware update TS251R004 and above enable the HP P2000 G3 FC enable T10 compliance for the hardware.

 

To quote VMware’s documentation on their VAAI implementation specific to T10 compliance:

The second required component can be referred to as a VAAI plug-in specific to the VAAI filter. It implements vendor-specific VAAI functions such as ATS, XCOPY and WRITE_SAME. There were different implementations of the VAAI block primitives in vSphere 4.1, but all of the primitives in vSphere 5.0 have been ratified by T10, so any array that is T10 compliant should be able to use VAAI.

 

This means that you no longer need to be running the HP P2000 VAAI plugin software directly on ESXi hosts. In fact, HP recommend you uninstall and remove the plugin software before you upgrade the firmware on these arrays, otherwise you could suffer from performance degradation and possible loss of access to datastores.

My process was to first of all login to all hosts and check for the presence of the VAAI plugin.

  • SSH into host as root, run find / -name hp_vaaip_p2000
  • Ensure that nothing comes up with the find command, if it does (you see something like this output: /usr/lib/vmware/vmkmod/hp_vaaip_p2000), then you should use this HP document to ensure it is removed correctly: https://h20566.www2.hpe.com/hpsc/doc/public/display?sp4ts.oid=4118559&docId=mmr_kc-0123414&docLocale=en_US – this will involve some setting changes, and removing claim rules as well as removal of the HP P2000 VAAI VIB itself.
  • After verifying nothing came up, check other hosts, and once happy all hosts are clear of the plugin, upgrade the firmware for the P2000 system.
  • Ideally reboot ESXi hosts after the firmware update and ensure access to datastores is still there. Check the hardware acceleration status of datastores – they should show up as “Supported”.

HP P2000 G3 FC MSA – troubleshooting a faulty Controller (blinking Fault/Service Required LED)

by

Setting up a new HP P2000 G3 FC MSA with dual controllers over the last couple of days for a small staging environment, I ran into issues from the word go. The device in question was loaded with 24 SFF disks and two Controllers (Controller A and B).

 

On the very first boot we noticed a fault (amber) LED on the front panel. Inspecting the back of the unit, I noticed that Controller A and B were both still flashing their green “FRU OK” LEDs, (which according to the manual means that the controllers were still booting up), even after waiting a number of hours. On Controller A, I could see a blinking amber “Fault/Service Required” LED. Following through the troubleshooting steps in the manuals lead nowhere as the end synopsis was to check the event logs. Even the Web interface was acting up – I could not see the controller’s listed, could not see any disks and the event logs were completely empty. Obviously there was a larger issue at hand preventing the MSA and even the Web interface from functioning properly. To further confuse matters, after shutting down and restarting the device, controller B starting blinking the amber LED instead of A this time, both still stuck in their “Booting up” state. Refer to the linked LED diagram below and you’ll see that the LED flashing green is labelled as 6, and the amber blinking LED is the one labelled as 7 on the top controller in the diagram.

LED Diagram

HP Official documentation

After powering the unit down completely, and then powering back up again, the MSA was still stuck in the same state. Powering down the unit once more, removing and reseating both controllers did not help either. Lastly, I powered it all off again, removed controller A completely, then powered up the device with just Controller B installed. Surprisingly the MSA booted up perfectly, and LED number 6 (FRU OK) went a nice solid green after a minute or so of booting up. No amber LEDs were to be seen. Good news then! Hot plugging controller A back in at this stage with the device powered on resulted in both controllers reporting a healthy status and all the disks and hardware being detected. A final test was done by powering off everything and powering it back up again as it should be from a cold start. Everything worked this time.

 

Here is a photo of the rear of the device once all was resolved showing the solid green FRU OK LEDs on both controllers.

 

 

Bit of an odd one, but it would seem that controllers together were preventing each other from starting up. Removing one then booting up with this seemed to solve the problem, and at the end of the day all hardware was indeed healthy. After this the 24 disks were assigned and carved up into some vdisks to be presented to our ESXi hosts!