Implementing a Fabric Upgrade

Netvisor ONE enables you to  implement a fabric-wide upgrade and reboot the switches at the same time or in a sequential order.

Starting the Fabric Upgrade

Upgrading to a newer version of Netvisor ONE  uses following steps:

• Establishing a serial console connection to the switch
• Performing disk checks and free disk space checks
• Copying the upgrade package to the switch
• Starting the upgrade process by using the command, fabric-upgrade-start.

The fabric-upgrade command establishes a controller node (primary node) to manage the fabric upgrade procedure.  This node is instantiated on the switch where the fabric-upgrade command is run.  

Copy the downloaded software to the switch (using sftp) that will be the controller or primary node.  The controller node monitors the progress of the upgrade on each node and can view the status of the upgrade using the fabric-upgrade-status-show command.  The controller node is identified by an “*” after its name in the status output.

The software package download is a relatively simple operation but it is important to consider the platforms that need to be upgraded in a fabric. The following package name examples reflect the platform name associated with each package.  

For each platform in a fabric, a corresponding package (nvOS-X.X.X-XXX-ONVL.pkg) is needed for the fabric upgrade process.  For example, if you are upgrading to 5.1.1 release on a F9372 platform, the package name is: nvOS-5.1.1-5010115297-onvl.pkg

Following are the commands that control the software or fabric upgrade process:

• software-upgrade – upgrade a single switch and reboot automatically

or

• fabric-upgrade-start –  assign the controller node and begin the upgrade process with options (see Fabric Upgrade Command Variables below)
• fabric-upgrade-status-show – monitor the progress of the upgrade for each node in the fabric
• fabric-upgrade-finish – assuming auto-finish option is not used, begin the reboot process based on start options
• fabric-upgrade-abort – abort the upgrade process and return switches to their prior state, no reboot needed
• fabric-upgrade-continue – continue the upgrade if an error occurs on a particular switch

The fabric-upgrade-start command defines all the future behavior of the upgrade process, meaning any optional settings need to be defined with the “start” command.  See optional settings in the next section.  In addition, the fabric-upgrade-start command acquires a configuration lock from all the members of the fabric.  No configuration changes are permitted during the upgrade process.

The fabric upgrade feature has two phases:

• Upgrade — start the upgrade which creates and updates Netvisor ONE to new boot environments but does not reboot the fabric.

• Reboot — reboots the entire fabric after all server-switches are upgraded to new boot environments. It is also possible during this phase to abort the process and discard the new boot environments.

Note: The fabric is locked during the entire process and you cannot change any configurations during the process.

Before You Begin the Fabric Upgrade

Before you begin, you may want to consider the following options for the fabric-upgrade-start command:

• auto-finish — you can specify to automatically reboot the entire fabric after the upgrade is complete. The default is no-auto-finish.
• abort-on-failure — specify if you want the upgrade to stop if there is a failure during the process.
• manual-reboot — specify if you want to manually reboot individual switches after the upgrade process. If you specify no-manual-reboot, all switches reboot automatically after the upgrade is complete.
• prepare — specify if you want to perform setup steps prior to performing the upgrade. This step copies the offline software package and then extracts and prepares it for the final upgrade process. Once you begin the prepare process, you cannot add new switches to the fabric.

You can check the status of the upgrade using the fabric-upgrade-status-show command:

CLI (network-admin@sw1) > fabric-upgrade-status-show

The first entry in the log is the duration of the upgrade process. It does not include waiting time. The switch with the asterisk (*) is the controller server-switch where the fabric-upgrade-start command was issued.

Additional commands for the fabric upgrade feature:

• fabric-upgrade-finish — you can issue this command at any time during the fabric upgrade to reboot all nodes in the fabric and complete the upgrade. Once the upgrade phase is complete, all server-switches display the “Upgrade complete” message in the log field. You can then safely reboot the fabric.
• fabric-upgrade-abort — aborts the software upgrade process. All changes to the server-switches are cleaned up and the server-switches do not reboot. The configuration lock on the fabric is also released.

If you issue the fabric-upgrade-abort command during the upgrade process, it may take some time before the process stops because the upgrade has to reach a logical completion point before the changes are rolled back on the fabric. This allows the proper cleanup of the changes.

• fabric-upgrade-prepare-cancel — cancels a fabric upgrade that was prepared earlier.
• fabric-upgrade-prepare-resume — resume a fabric upgrade that was prepared earlier.
• fabric-upgrade-prepare-show — displays the status of prepared upgrades on the fabric nodes.

Review bootenv

A new boot environment is built during the upgrade process.  Upon reboot this new boot environment becomes active and the new software is up-and-running on the switch.  Generally, it is not required to interact with the boot environments during the upgrade process.  It may be necessary to review the boot environments using the command bootenv-show if there is some failure during the upgrade process.