1 Copyright and Version

© Copyright 2022-2024 Hewlett Packard Enterprise Development LP. All third-party marks are the property of their respective owners.

HFP-DOCS: 24.10.1-9

Doc git hash: 797fe3e0d7379a3dca10cb1fc0a8e6f77ffb9daa

Generated: Mon Nov 04 2024

2 Release Information

2.1 Overview

This document uses the terms “install”, “update”, and “upgrade” interchangeably as the HFP procedure is the same in each scenario.

When installing the HPC Firmware Pack (HFP) on a system managed by Cray System Management (CSM), refer to the HPE Cray EX System Software Stack Installation and Upgrade Guide for CSM for a high-level overview of the installation and upgrade workflow for all products. It also includes the software and hardware compatiblity matrix, third-party product documentation links, and cross-product dependencies.

Installing HFP is primarily accomplished by executing Install and Upgrade Framework (IUF) commands as documented in the Cray System Management documentation and described in this guide in sections Install or Upgrade HPC Firmware Pack and IUF Stage Details for HFP.

2.2 Product Details

The HPE Cray EX HPC Firmware Pack (HFP) provides firmware packages for HPE Cray EX system hardware. The HFP product distribution tar.gz file includes a NOTES.txt file which lists the recommended firmware packages to install.

NOTE: Firmware packages for certain hardware components are not included in HFP and must be acquired from the hardware vendor, for example Dell and Mellanox network switch firmware.

2.3 Firmware Details

This is a list of firmware files that are part of the HPC Firmware Pack.

2.4 Differences from Prior Release

Some previously omitted NVIDIA firmware was updated and is now included in this version of the HFP. Firmware available in this HFP includes:

• Nvidia A100 GPU (4 and 8 GPU models)

• NVIDIA Acc FPGA

Other NVIDIA firmware is still omitted from this HFP release and includes the following components. The omitted packages include:

• Tesla_A100_92.00.25.00.08.x86_64.rpm

• delta-cecv3.5-ota.bin.hpepkg.signed.ilo-3.5-1.fwpkg

• CEC-Enabled-fpga3v00-ota.bin.hpepkg.signed-3.00-1.fwpkg

• NVSwitch_92.10.18.00.01.scexe

• PEX88000_v3.0f.scexe

• NVIDIA_Tesla_V100_PCIE_32GB_88.00.7E.00.03.scexe

• Quadro_RTX6000_90.02.4A.00.02-1-0.x86_64.rpm

These components will be updated in a future release of HFP.

2.5 Performing Firmware Upgrades

HFP provides the firmware packages for HPE Cray EX systems, but HFP does not perform firmware upgrades. There are two methods to upgrade firmware: with and without the Firmware Action Service (FAS) which is only present on systems managed by CSM. The method used to install HFP depends on whether FAS is installed and operational, as described in the following subsections.

2.5.1 Updating BMC Firmware and BIOS for ncn-m001

Refer to this section in the Cray System Management Documentation, Updating BMC Firmware and BIOS for m001.

2.5.2 Upgrading Firmware With FAS

Systems managed by CSM most often perform firmware upgrades using FAS. The Install or Upgrade HPC Firmware Pack section of this document describes how to install HFP on a CSM-managed system with FAS installed and operational. The Install HPC Firmware Pack from PIT or LiveCD section of this document describes how to install HFP on a CSM-managed system booted into the Pre-Install Toolkit (PIT) or LiveCD environments (typically only the case when the system is being installed for the first time).

FAS details can be found in the Update Firmware with FAS section of the Cray System Management Documentation.

2.5.3 Upgrading Firmware Without FAS

On systems without FAS, firmware provided by HPF can be installed by following the instructions included in the directory of the HFP product distribution tar.gz file that contains the firmware package.

Each hardware product directory includes firmware packages (fwpkg, rpm, …) as well as a DOC directory with vendor documentation, including installation instructions. The following are example directory listings for HPE_XL675d-Gen10Plus (HPE ProLiant XL675d) and GB_SVR_1264UP_C17_C21 (Gigabyte 1264UP) hardware:

    HPE_XL675d-Gen10Plus/
        A47_2.40_02_23_2021.fwpkg
        DOC/
            HPCM-Firmware-Flash_v2021.03.04.pdf
            INSTALL.txt
            README.txt
        FAS-BIOS-HPE_XL675d-Gen10Plus-2.40-1-sles15sp1.x86_64.rpm
 
    GB_SVR_1264UP_C17_C21/
        DOC/
            BMCFirmwareUpdate.txt
            Gigabyte-Shasta-Firmware-Update.pdf
            README.txt
            Relnotes_MZ32-AR0-YF_C17_F01.pdf
            Relnotes_MZ32-AR0-YF_C17_Rome.pdf
            Relnotes_MZ32-AR0-YF_Naples.pdf
        sh-svr-1264up-bios-21.00.00-20210325025941_8df4708.x86_64.rpm

Focusing on the HPE_XL675d-Gen10Plus directory listing:

• The directory name HPE_XL675d-Gen10Plus reflects the type of hardware the firmware is for.

• The A47_2.40_02_23_2021.fwpkg file is used for manual installation without FAS.

• The DOC directory contains documentation files, including release and installation details.

• The FAS-BIOS-HPE_XL675d-Gen10Plus-2.40-1-sles15sp1.x86_64.rpm file is used for installation with FAS.

2.5.3.1 iLO Information

Documentation in some, but not all, of the DOC directories states that HPE Integrated Lights Out (iLO) server management software can be used to install the firmware. In those cases, the following documentation provides additional details on how to use iLO and may be of interest.

• HPE iLO 5 2.55 User Guide

  • The “Viewing and managing firmware and software” section documents how to use the iLO web interface to install firmware.

• RESTful Interface Tool 1.50 User Guide

  • The “Firmware update command” topic in the “iLO Commands” section documents how to install firmware using the RESTful Interface Tool.

NOTE
* iLO and BIOS firmware are only provided by HFP. Download remaining drivers and firmware from the Service Pack for ProLiant (SPP) and Apollo Servers.

• Both iLO5 2.78 and iLO5 3.01 are included in this release. iLO5 firmware version 2.78 needs to be included in ALL releases that have an iLO5 version > 2.78 due to needing to update to 2.78 before updating to the later version. Please note iLO5 v2.78 is positioned to be the base version for future iLO5 upgrades considering that upcoming versions of iLO5 binary files are expected to be larger than 32MB in size. Previous versions of iLO5 (prior to 2.78) will only accept 32MB size binary files. This restriction has been removed in iLO5 v2.78. For more information, please refer to this FAS section, Update iLO5 Firmware Above v2.78 in the Cray System Management Documentation.

• Models HPE EX254n and HPE EX255a are incompatible with the 1.8.X series of Cc/Nc firmware. While it is feasible to downgrade the Cc/Nc firmware from the current 1.9.X version to 1.8.X, it is crucial to note that the mentioned models will not be supported unless Cc/Nc is at version 1.9.X. CAUTION Exercise caution when proceeding with the downgrade to ensure compatibility with supported models.

3 Install or Upgrade HPC Firmware Pack

This section describes how to install HFP on a CSM-managed system with FAS installed and operational.

The Install and Upgrade Framework (IUF) provides commands which install, upgrade, and deploy products on systems managed by CSM. IUF capabilities are described in detail in the IUF Section of the Cray System Management Documentation. The initial install and upgrade workflows described in the HPE Cray EX System Software Stack Installation and Upgrade Guide for CSM (S-8052) detail when and how to use IUF with a new release of HFP or other HPE Cray EX products.

Read the Overview section of this document to understand what is and is not executed as part of the HFP install process. See the Upgrading Firmware Without FAS section of this document for instructions on systems not managed by CSM.

3.1 Install and Upgrade Framework

IUF will perform the following tasks for a release of HFP.

• process-media stage
  • extracts release distributions
• pre-install-check stage
  • pre-flight checks for services:
    • verify FAS is installed
    • verify rpms and zip files are loaded into FAS
    • removes the incompatible files from FAS and from s3 fw-bucket
• deliver-product stage:
  • Sets up Nexus with a product’s blobstores and repositories
  • Uploads RPMs to repositories in Nexus
  • Uploads Docker images to the Docker registry in Nexus
  • Uploads images into the Artifacts (S3) service
  • Runs post-install-test-nexus.sh (to check that rpms are uploaded to Nexus)
  • Runs post-install-test-fas.sh (to check that the firmware is loaded in FAS)

3.2 IUF Stage Details for HFP

This section describes any HFP details that an administrator needs to be aware of before running IUF stages. Entries are prefixed with Information if no administrative action is required or Action if an administrator needs to perform tasks outside of IUF.

• pre-install-check

Information: pre-flight check stage displays versions of HFP installed, verifies if nexus, FAS, and S3 are running, displays current FAS version, displays rpms/zip files which will be loaded in the FAS

• deliver-product

Action: If the command cray fas loader nexus create fails during the post-install-test-fas.sh check (which checks that the firmware is loaded in FAS), the user must manually run the command. The failure occurs when loaderStatus="$(cray fas loader list 2>&1 | grep loaderStatus) returns busy even after 200seconds of waiting for ready status.

Run the following command to check if the cray fas loader status is ready or busy.

ncn-mw# cray fas loader list --format json | grep loaderStatus

This will return a ready or busy status. Example: loaderStatus = "ready"

If it is busy, wait until fas loader status is ready by monitoring the status using the same command. When the cray fas loader status is ready, reinstall HFP or load the firmware directly from nexus using FAS commands. For more information, please refer to Load Firmware from Nexus.

4 Documentation for Each Firmware Unit

The documentation is used for manually installing firmware when not using FAS on HPE Cray EX.

Documentation for each firmware unit is alongside the firmware in the overall package.

├── HPE_XL675d-Gen10Plus                                           <----- Hardware type this firmware is for
│ ├── A47_2.40_02_23_2021.fwpkg                                    <----- File used for manual installation
│ ├── DOC                                                          <----- Documentation│ │
│ │ ├── INSTALL.txt
│ │ └── README.txt
│ └── FAS-BIOS-HPE_XL675d-Gen10Plus-2.40-1-sles15sp1.x86_64.rpm.   <----- rpm used by FAS for update

├── GB_SVR_1264UP_C17_C21
│ ├── DOC
│ │ ├── BMCFirmwareUpdate.txt
│ │ ├── Gigabyte-Shasta-Firmware-Update.pdf
│ │ ├── README.txt
│ │ ├── Relnotes_MZ32-AR0-YF_C17_F01.pdf
│ │ ├── Relnotes_MZ32-AR0-YF_C17_Rome.pdf
│ │ └── Relnotes_MZ32-AR0-YF_Naples.pdf
│ └── sh-svr-1264up-bios-21.00.00-20210325025941_8df4708.x86_64.rpm