Updating Mellanox ConnectX-3 NIC Firmware

Published

In a previous post, I provided a guide on configuring SR-IOV for a Mellanox ConnectX-3 NIC. I’ve since picked up a second one of these and was attempting to follow through on the same guide. However, when I attempted to “query” the device, I saw the following:

$ sudo mstconfig -d 02:00.0 query

Device #1:
----------

Device type:    ConnectX3
PCI device:     02:00.0

-E- Failed to query device: 02:00.0. Unsupported FW (version 2.31.5000 or above required for CX3/PRO)

Clearly we need to update something here, hence, here’s my guide to updating that firmware and making mstconfig happy again.

Prerequisites

As with the other post, you’re going to need a Mellanox ConnectX-3 card for this to be of any use. Once again, I’m using Ubuntu 16.04 because that’s what that OpenStack gate uses but I think most of this stuff is packaged on Fedora too.

Identify and Download Firmware

As you can see on the firmware download page, there are a variety of ConnectX-3 models available, each with different feature sets (Ethernet vs. Infiniband, two ports vs. four). You need to figure out which one you have. These models are identified by their OPN and PSID. I’ve still to figure out how to identify the former, but the latter is easily identified using the mstflint tool. For example:

$ sudo mstflint -d 02:00.0 q
Image type:      FS2
FW Version:      2.30.8000
Rom Info:        type=PXE version=3.4.151 devid=4099 proto=ETH
Device ID:       4099
Description:     Node             Port1            Port2            Sys image
GUIDs:           0002c90300056aa8 0002c90300056aa9 0002c90300056aaa 0002c90300056aab
MACs:                                 0002c921d6d0     0002c921d6d1
VSD:
PSID:            MT_108012002

So with our PSID to hand - MT_108012002 in this case - we can proceed to downloading the correct firmware from the firmware download page. There doesn’t seem to be anyway to download by just PSID so I had to click through the list of OPNs until I found the OPN with a PSID that matches. In my case, this was MCX312A-XCBT for firmware version 2.42.5000. Download the relevant firmware to your host. For example:

$ wget http://www.mellanox.com/downloads/firmware/fw-ConnectX3-rel-2_42_5000-MCX312A-XCB_A2-A6-FlexBoot-3.4.752.bin.zip

Update Firmware

With our firmware update to hand, it’s time to actually update the NIC. For this section, I initially followed the updating guide provided by Mellanox. However, as before, I didn’t want to install anything that wasn’t already provided with my distro. As such the instructions provided below are a suitable alternative.

Firstly, extract the firmware you downloaded previously. For example:

$ unzip fw-ConnectX3-rel-2_42_5000-MCX312A-XCB_A2-A6-FlexBoot-3.4.752.bin.zip

Once done, it’s time to flash the firmware. To do this, we once again use the mstflint tool::

$ sudo mstflint -d 02:00.0 -i fw-ConnectX3-rel-2_42_5000-MCX312A-XCB_A2-A6-FlexBoot-3.4.752.bin burn

Current FW version on flash:  2.30.8000
New FW version:               2.42.5000

Burning FS2 FW image without signatures - OK
Restoring signature                     - OK

Now reboot your machine so the new firmware is loaded::

$ sudo reboot

And with that, we have flashed out new firmware and can query the NIC configuration::

$ sudo mstconfig -d 02:00.0 query

Device #1:
----------

Device type:    ConnectX3
PCI device:     02:00.0

Configurations:                              Current
         SRIOV_EN                            0
         NUM_OF_VFS                          8
         LOG_BAR_SIZE                        3
         BOOT_OPTION_ROM_EN_P1               1
         BOOT_VLAN_EN_P1                     0
         BOOT_RETRY_CNT_P1                   0
         LEGACY_BOOT_PROTOCOL_P1             1
         BOOT_VLAN_P1                        1
         BOOT_OPTION_ROM_EN_P2               1
         BOOT_VLAN_EN_P2                     0
         BOOT_RETRY_CNT_P2                   0
         LEGACY_BOOT_PROTOCOL_P2             1
         BOOT_VLAN_P2                        1

Easy peasy. Now back to configuring SR-IOV.

comments powered by Disqus