Upgrading the Netvisor ONE Software


Software upgrades are a routine maintenance procedure that must be completed every so often.  However, there are some guidelines to consider before you start the upgrade procedures.  


Before you start the upgrade process, obtain the required upgrade software.  You can download the software manually and copy it to a switch before beginning the upgrade procedures.


Note: If you are upgrading from Netvisor ONE version 3.0.4 or earlier to Netvisor ONE version 5.x.x, 6.x.x, or 7.x.x, then, you must first upgrade to Netvisor ONE version 3.1.1 before upgrading to a later release. The direct upgrade from version 3.0.4 or earlier to version 5.x.x/6.x.x/7.x.x is not supported.


Software and fabric upgrade process comprises of two distinct phases: (i) the installation (upgrade) of the new software and (ii) a switch reboot to activate the new software.


Read this section completely before starting the software upgrade procedure.


To upgrade the software on a switch, use the software-upgrade command.


CLI (network-admin@switch) > software-upgrade


software-upgrade                           

  package upgrade-package-name.pkg

Starts software upgrade on local switch using software bundle from /sftp/import directory.

software-upgrade                           

  package upgrade-package-name.pkg            

  auto-reboot|no-auto-reboot


Starts software upgrade on local switch using software bundle from /sftp/import directory. Use the auto-reboot or no-auto-reboot parameter to specify whether the switch needs to be automatically rebooted after software upgrade or whether admin will manually reboot the switch at a later time to start with new upgraded software. The default value is auto-reboot. 

This parameter is added in Netvisor ONE 6.1.0.

software-upgrade                           

  package upgrade-package-name.pkg            

  upload-server upload-server- name/ip:/path/to/upload/to 

Starts software upgrade on local switch using software bundle from /sftp/import directory  and uploads switch configuration backup file to the specified path on server using SCP.

software-upgrade                           

  package upgrade-package-name.pkg            

  auto-reboot|no-auto-reboot         

  upload-server upload-server-string 


Starts software upgrade on local switch using software bundle from /sftp/import directory. Use the auto-reboot or no-auto-reboot parameter to specify whether switch needs to be automatically rebooted after software upgrade or not. 

The default value is auto-reboot. The presence of upload-server parameter in software-upgrade command uploads switch configuration bundle to the specified path on server using SCP.


software-upgrade-abort                

Aborts an in progress software upgrade. Note that software upgrade will be aborted when upgrade process completes its current step and reaches logical next step.  

This command is added in Netvisor ONE 6.1.0.

software-upgrade-reboot                    

Indicates the software upgrade process to reboot the switch after software upgrade is complete. This command is used when no-auto-reboot parameter is specified when software upgrade is started. 

This command is added in Netvisor ONE 6.1.0.

software-upgrade-instant-status-show

Displays the current (most recent status of software upgrade during a fabric-wide upgrade process. This command accepts various format options of Netvisor CLI. 

This command is introduced in Netvisor ONE 6.1.0

software-upgrade-status-show               

Displays the current status of software upgrade that is in progress or completed.


Starting with Netvisor ONE 6.1.0 release, additional options such as auto-reboot|no-auto-reboot, abort, and manual reboot are supported on the software-upgrade command. By using these options, you can control the automatic reboot of switch after software upgrade completes or you can abort an upgrade that is in progress. 

If you start software upgrade by specifying auto-reboot parameter for software-upgrade command, switch automatically reboots after software upgrade completes and switch boots up with new software version. 

However, if you start the software upgrade by specifying the no-auto-reboot parameter for software-upgrade command, switch completes the software upgrade and waits for administrator to manually reboot the switch to boot into upgraded software. After determining that upgrade is complete,  you must issue a software-upgrade-reboot command to reboot the switch to boot with upgraded software. 

While software upgrade is in progress or software upgrade is complete and switch is waiting for administrator to issue software-upgrade-reboot command, if you want to abort the upgrade and keep switch in current software version, you can use software-upgrade-abort command to abort the upgrade process. This scenario is explained further in the examples below.


Note: The software-upgrade-abort and software-upgrade-reboot commands are supported only if upgrade is started using software-upgrade command. Do not run these commands if you started fabric wide upgrade using fabric-upgrade command. An error message is displayed if you do so.


To start a software upgrade follow the steps below:


  1. View the current version of Netvisor ONE on the switch by using command:

For example,

CLI network-admin@switch > software-show

version: 6.0.1-6000116966

  1. Identify the software package. Depending on the version of Netvisor ONE running on your switch, you should use appropriate software upgrade package.  Select the appropriate upgrade bundle from the following upgrade matrix table based on the current version on your switch:


Current Software

Target Release for Upgrade

Upgrade Type

Upgrade Software Package

3.x GA

7.0.0 GA

Software Upgrade using release upgrade bundle

nvOS-relupg-7.0.0-7000019033-onvl.pkg

5.x.x OR 

6.x.x OR 

7.x.x


7.0.0 GA

Software Upgrade using regular offline  upgrade bundle

nvOS-relupg-7.0.0-7000019033-onvl.pkg


  1. Copy the upgrade package to the switch, to do:
    1. Enable Secure File Transfer Protocol (SFTP) on the switch:

CLI (network-admin@switch)> admin-sftp-modify enable

sftp password:

confirm sftp password:

CLI (network-admin@switch)>


    1. Upload the Software package to the switch:

root@server-os-9:~/# sftp sftp@switch

The authenticity of host 'switch (10.0.0.02)' can't be established.

RSA key fingerprint is SHA256:SI8VQZgJCppbrF4sRcby36Fx7rz3Hh5EJllPPyScLZU.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'switch1, 10.0.0.02 (RSA) to the list of known hosts.

* Welcome to Pluribus Networks Inc. Netvisor(R). This is a monitored system. *

* ACCESS RESTRICTED TO AUTHORIZED USERS ONLY *

* By using the Netvisor(R) CLI,you agree to the terms of the Pluribus Networks *

* End User License Agreement (EULA). The EULA can be accessed via *

* http://www.pluribusnetworks.com/eula or by using the command "eula-show" *

Password:

Connected to switch

sftp> cd import

sftp> put nvOS- nvOS-6.1.0-6010018118-onvl.pkg

Uploading nvOS- nvOS-6.1.0-6010018118-onvl.pkg 

nvOS- nvOS-6.1.0-6010018118-onvl.pkg

nvOS- nvOS-6.1.0-6010018118-onvl.pkg 100% 332MB 7.5MB/s 04:00


  1. Start the upgrade process by using the software-upgrade command with package parameter, which allows you to specify the name of the upgrade file.  The switch gets automatically rebooted after software upgrade is complete. 

CLI (network-admin@switch) > software-upgrade package nvOS-6.1.0-6010018118-onvl.pkg

Scheduled background update. Use software-upgrade-status-show to check. Switch will reboot itself. DO NOT reboot manually.


Caution: Do not reboot or power off the switch during the upgrade procedure.  When software upgrade is complete, switch reboots automatically.


  1. Monitor the upgrade process using the software-upgrade-status-show command:


CLI (network-admin@switch)>software-upgrade-status-show show-interval 5


[Apr11.21:30:41] Starting software upgrade ...

[Apr11.21:30:42] Cleaning old package bundles

[Apr11.21:30:42] Checking available disk space...

[Apr11.21:30:42] Avbl free space: 12.69G, Required: 0.62G

[Apr11.21:30:42] Unpacking local package bundle...

[Apr11.21:30:42] Extracting initial bundle.

.

.

log

-----------------------------------------------------------------

[Apr11.21:30:41] Starting software upgrade ...

[Apr11.21:30:42] Cleaning old package bundles

[Apr11.21:30:42] Checking available disk space...

[Apr11.21:30:42] Avbl free space: 12.69G, Required: 0.62G

[Apr11.21:30:42] Unpacking local package bundle...

[Apr11.21:30:42] Extracting initial bundle.

[Apr11.21:30:50] Decrypting signed bundle.

[Apr11.21:30:52] Extracting signed bundle.

[Apr11.21:31:00] Extracting packages.

[Apr11.21:31:09] Fetching repository metadata.

[Apr11.21:31:09] Skipping dpkg update in current boot image

[Apr11.21:31:11] Computing package update requirements.

[Apr11.21:31:12] Upgrade agent version: 6.0.1-6000116966

[Apr11.21:31:12] Upgrading software upgrade framework

[Apr11.21:31:16] Fetching repository metadata.

[Apr11.21:31:17] Skipping dpkg update in current boot image

[Apr11.21:31:17] Computing package update requirements.

[Apr11.21:31:17] Upgrade agent version: 6.1.0-6010018118

[Apr11.21:31:17] Upgrading nvOS 6.0.1-6000116966 -> 6.1.0-6010018118

[Apr11.21:32:28] Cleaning up old BEs.

[Apr11.21:32:30] Upgrading nvOS 6.0.1-6000116966 -> 6.1.0-6010018118

[Apr11.21:32:30] Software upgrade completed. Rebooting.


After the switch reboots and you see following message in serial console of the switch, you can SSH to switch as the network-admin


nvOS system info:

serial number: 1XXXXXXX00059

hostid: 090XXX9d

device id:

[ OK ] Started NetVisor Operating System.

Starting nvOSd Monitor...

[ OK ] Started nvOSd Monitor.

[ OK ] Reached target Multi-User System.

[ OK ] Reached target Graphical Interface.

[ OK ] Started Stop ureadahead data collection 45s after completed startup.

Starting Update UTMP about System Runlevel Changes...

[ OK ] Started Update UTMP about System Runlevel Changes.

* Welcome to Pluribus Networks Inc. Netvisor(R). This is a monitored system. *

* ACCESS RESTRICTED TO AUTHORIZED USERS ONLY *

* By using the Netvisor(R) CLI,you agree to the terms of the Pluribus Networks *

* End User License Agreement (EULA). The EULA can be accessed via *

* http://www.pluribusnetworks.com/eula or by using the command "eula-show" *

switch1 login: network-admin

Password:

Netvisor OS Command Line Interface 6.1

Connected to Switch switch; nvOS Identifier:0x90XXXX9; Ver: 6.1.0-6010018118

CLI (network-admin@switch) > software-show

version: 6.1.0-6010018118


To verify that a standalone (non-cluster) switch is fully operational after the software upgrade, you can use the log-event-show command. The 'System is up for service' log message indicates that the switch is ready for forwarding. For example:

CLI (network-admin@Leaf1)> log-event-show


category time                       name           code  level event-type message

-------- -------------------------- -------------- ----- ----- ---------- ------------------------

event    2021-03-10,23:37:40.572119 systemup_alert 11513 note  system     System is up for service


Note: You can also use the 'System is up for service' log message to verify that a switch is functional after a reboot or restart triggered by the following operations: fabric-upgrade, switch-reboot, nvos-restart, fabric-join, cluster repeer, and config import commands.


Apart from upgrading Netvisor ONE on individual switches, starting from Netvisor ONE version 6.1.0, you can perform software upgrade on all switches in a fabric using software-upgrade from one switch and reboot the switches to perform  fabric-wide software upgrade (software upgrade on all the nodes in a fabric).


You can choose one of the below options to perform software upgrade on all switches from one switch:


  • Start software upgrade on entire fabric and reboot the switches sequentially

OR

  • Start software upgrade on entire fabric and reboot all switches, but not sequentially


Start software upgrade on entire fabric and reboot the switches sequentially


Follow the steps to perform software upgrade on entire fabric and reboot the switches sequentially:


  1. Enable SFTP on all switches and provide the credentials for each switch using the CLI command:

CLI (network-admin@switch)> switch * admin-sftp-modify enable


  1. Enable shell access for network-admin role on all switches in the fabric:

CLI (network-admin@switch)> switch * role-modify name network-admin shell


  1. Download the software upgrade bundle to each switch by entering the below command only on one switch: 

CLI (network-admin@switch)> switch * shell cd /sftp/import;wget http://sandy.pluribusnetworks.com//offline-pkgs/onvl/nvOS-6.1.0/nvOS-6.1.0-6010018118-onvl.pkg  


  1. Start the software upgrade process on all the switches using the command:

CLI (network-admin@switch)> switch * software-upgrade package nvOS-6.1.0-6010018118-onvl.pkg no-auto-reboot


  1. Verify that all nodes display the "Waiting for software-upgrade-abort/software-upgrade-reboot" message by using the command individually on each switch:

CLI (network-admin@switch)> software-upgrade-status-show


  1. Issue the software-upgrade-reboot command on each switch in the fabric one by one based on the reboot sequence that best works for your deployment. 


Start software upgrade on entire fabric and reboot all switches non-sequentially 


Follow the steps to perform fabric-wide software upgrade without reboot sequence:


  1. Enable SFTP on all switches and provide the credentials for each switch using the CLI command:

CLI (network-admin@switch)> switch * admin-sftp-modify enable


  1. Enable shell access for network-admin role on all switches in the fabric:

CLI (network-admin@switch)> switch * role-modify name network-admin shell


  1. Download the software upgrade bundle to each switch by entering the below command only on one switch: 

CLI (network-admin@switch)> switch * shell cd /sftp/import;wget http://sandy.pluribusnetworks.com//offline-pkgs/onvl/nvOS-6.1.0/nvOS-6.1.0-6010018118-onvl.pkg  


  1. Start the software upgrade process on all the switches with no-auto-reboot command parameter:

CLI (network-admin@switch)> switch * software-upgrade package nvOS-6.1.0-6010018118-onvl.pkg no-auto-reboot


  1. Verify that all nodes display the "Waiting for software-upgrade-abort/software-upgrade-reboot" message by using the command individually on each switch

CLI (network-admin@switch)> software-upgrade-status-show


  1. Reboot all the switches except the node where you are running the commands. This step ensures that the software-upgrade-reboot command is executed by all the switches before the node where you originally started the upgrade process goes down. 

CLI (network-admin@switch)> switch <comma separated non controller switches> software-upgrade-reboot


  1. Reboot the node where you started the upgrade and complete the upgrade process of the fabric: 

CLI (network-admin@switch)> switch <controller node> software-upgrade-reboot


Below are some examples of the software upgrade that can be used when you want the switch to not reboot automatically after upgrade is complete, but wants to reboot manually or if you want to abort the software upgrade that is in progress.

  • Usage of the software-upgrade command with no-auto-reboot option and later issuing an abort command to abort the upgrade process:


CLI (network-admin@switch) > software-upgrade package nvOS- 6.1.0-6010017911-onvl.pkg no-auto-reboot


CLI (network-admin@ switch) > software-upgrade-status-show

log

----------------------------------------------------------------------

[Mar11.07:47:21] Starting software upgrade ...

[Mar11.07:47:22] Checking available disk space...

[Mar11.07:47:22] Avbl free space: 82.67G, Required: 1.28G

[Mar11.07:47:22] Unpacking local package bundle...

[Mar11.07:47:22] Extracting initial bundle.

[Mar11.07:47:38] Decrypting signed bundle.

[Mar11.07:47:39] Extracting signed bundle.

[Mar11.07:47:56] Extracting packages.

[Mar11.07:48:13] Fetching repository metadata.

[Mar11.07:48:13] Skipping dpkg update in current boot image

[Mar11.07:48:13] Computing package update requirements.

[Mar11.07:48:13] Upgrade agent version: 6.1.0-6010017911

[Mar11.07:48:13] Upgrading nvOS 6.0.1-6000116966 -> 6.1.0-6010017911

[Mar11.07:49:01] Waiting for software-upgrade-abort/software-upgrade-reboot


CLI (network-admin@switch) > software-upgrade-abort

Upgrade running, scheduled abort


CLI (network-admin@switch) > software-upgrade-status-show

log

---------------------------------------------------------------------

[Mar11.07:47:21] Starting software upgrade ...

[Mar11.07:47:22] Checking available disk space...

[Mar11.07:47:22] Avbl free space: 82.67G, Required: 1.28G

[Mar11.07:47:22] Unpacking local package bundle...

[Mar11.07:47:22] Extracting initial bundle.

[Mar11.07:47:38] Decrypting signed bundle.

[Mar11.07:47:39] Extracting signed bundle.

[Mar11.07:47:56] Extracting packages.

[Mar11.07:48:13] Fetching repository metadata.

[Mar11.07:48:13] Skipping dpkg update in current boot image

[Mar11.07:48:13] Computing package update requirements.

[Mar11.07:48:13] Upgrade agent version: 6.1.0-6010017911

[Mar11.07:48:13] Upgrading nvOS 6.0.1-6000116966 -> 6.1.0-6010017911

[Mar11.07:49:01] Waiting for software-upgrade-abort/software-upgrade-reboot

[Mar11.07:50:39] User indicated software-upgrade-abort

[Mar11.07:50:40] User aborted the upgrade.

[Mar11.07:50:42] Software upgrade aborted

[Mar11.07:50:42] ERR: Upgrade failed: User aborted the upgrade.


  • Usage of the software-upgrade command with no-auto-reboot option and later issuing a software-upgrade-reboot command to complete the upgrade process:


CLI (network-admin@switch) > software-upgrade package nvOS-6.1.0-6010017911-onvl.pkg no-auto-reboot


CLI (network-admin@switch) > software-upgrade-status-show

log

--------------------------------------------------------------

[Mar11.07:52:59] Starting software upgrade ...

[Mar11.07:52:59] Checking available disk space...

[Mar11.07:52:59] Avbl free space: 82.67G, Required: 1.28G

[Mar11.07:52:59] Unpacking local package bundle...

[Mar11.07:52:59] Extracting initial bundle.

[Mar11.07:53:16] Decrypting signed bundle.

[Mar11.07:53:17] Extracting signed bundle.

[Mar11.07:53:33] Extracting packages.

[Mar11.07:53:50] Fetching repository metadata.

[Mar11.07:53:50] Skipping dpkg update in current boot image

[Mar11.07:53:50] Computing package update requirements.

[Mar11.07:53:51] Upgrade agent version: 6.1.0-6010017911

[Mar11.07:53:51] Upgrading nvOS 6.0.1-6000116966 -> 6.1.0-6010017911

[Mar11.07:54:38] Waiting for software-upgrade-abort/software-upgrade-reboot


CLI (network-admin@switch) > software-upgrade-reboot

Upgrade running, scheduled reboot


CLI (network-admin@switch) > Shared connection to switch closed.


Check the status on the new BE:

       

CLI (network-admin@switch) > software-upgrade-status-show

log

-------------------------------------------------------------

[Mar11.07:52:59] Starting software upgrade ...

[Mar11.07:52:59] Checking available disk space...

[Mar11.07:52:59] Avbl free space: 82.67G, Required: 1.28G

[Mar11.07:52:59] Unpacking local package bundle...

[Mar11.07:52:59] Extracting initial bundle.

[Mar11.07:53:16] Decrypting signed bundle.

[Mar11.07:53:17] Extracting signed bundle.

[Mar11.07:53:33] Extracting packages.

[Mar11.07:53:50] Fetching repository metadata.

[Mar11.07:53:50] Skipping dpkg update in current boot image

[Mar11.07:53:50] Computing package update requirements.

[Mar11.07:53:51] Upgrade agent version: 6.1.0-6010017911

[Mar11.07:53:51] Upgrading nvOS 6.0.1-6000116966 -> 6.1.0-6010017911

[Mar11.07:54:38] Waiting for software-upgrade-abort/software-upgrade-reboot

[Mar11.07:57:07] User indicated software-upgrade-reboot

[Mar11.07:57:08] User issued software-upgrade-reboot, rebooting the switch.

[Mar11.07:57:08] Upgrading nvOS 6.0.1-6000116966 -> 6.1.0-6010017911

[Mar11.07:57:08] Software upgrade completed. Rebooting.

[Mar11.07:59:35] Upgrade agent is listening.


  • Usage of the software-upgrade command in default mode:


CLI (network-admin@switch) > software-upgrade package nvOS-6.1.0-6010017911-onvl.pkg


log

--------------------------------------------------------------

[Mar11.09:32:10] Starting software upgrade ...

[Mar11.09:32:10] Checking available disk space...

[Mar11.09:32:10] Avbl free space: 81.89G, Required: 1.28G

[Mar11.09:32:10] Unpacking local package bundle...

[Mar11.09:32:10] Extracting initial bundle.

[Mar11.09:32:27] Decrypting signed bundle.

[Mar11.09:32:28] Extracting signed bundle.

[Mar11.09:32:44] Extracting packages.

[Mar11.09:33:01] Fetching repository metadata.

[Mar11.09:33:01] Skipping dpkg update in current boot image

[Mar11.09:33:02] Computing package update requirements.

[Mar11.09:33:02] Upgrade agent version: 6.1.0-6010017911

[Mar11.09:33:02] Upgrading nvOS 6.0.1-6000116966 -> 6.1.0-6010017911

[Mar11.09:33:51] Upgrading nvOS 6.0.1-6000116966 -> 6.1.0-6010017911

[Mar11.09:33:51] Software upgrade completed. Rebooting.

Shared connection to switch closed.


  • Usage of the software-upgrade command in default mode and later issuing an abort command to abort the upgrade process. The software-upgrade-abort command is also supported in the default mode (same as auto-reboot mode):


CLI (network-admin@switch) > software-upgrade package nvOS-6.1.0-6010017911-onvl.pkg


CLI (network-admin@switch) > software-upgrade-abort


CLI (network-admin@switch) > software-upgrade-status-show

log

-------------------------------------------------------------

[Mar11.09:43:04] Starting software upgrade ...

[Mar11.09:43:04] Checking available disk space...

[Mar11.09:43:04] Avbl free space: 81.15G, Required: 1.28G

[Mar11.09:43:04] Unpacking local package bundle...

[Mar11.09:43:04] Extracting initial bundle.

[Mar11.09:43:13] User indicated software-upgrade-reboot <============ time at which user initiated the command

[Mar11.09:43:21] Decrypting signed bundle.

[Mar11.09:43:22] Extracting signed bundle.

[Mar11.09:43:38] Extracting packages.

[Mar11.09:43:55] Fetching repository metadata.

[Mar11.09:43:56] Skipping dpkg update in current boot image

[Mar11.09:43:56] Computing package update requirements.

[Mar11.09:43:56] Upgrade agent version: 6.1.0-6010017911

[Mar11.09:43:56] User aborted the upgrade.  <================ place at which actual abort is performed (consistent state)

[Mar11.09:43:56] Software upgrade aborted

[Mar11.09:43:56] ERR: Upgrade failed: User aborted the upgrade.


Another new command added in Netvisor ONE version 6.1.0 is the software-upgrade-instant-status-show command, which displays the most current status of the upgrade process of all the switches in the fabric. This command is different from the software-upgrade-status-show command, where all details of the upgrade process is displayed.


Use this command to view the most recent status on each switch in the fabric when you perform the upgrade process on all switches of the fabric (fabric-wide upgrade):


CLI (network-admin@switch) >  switch * software-upgrade-instant-status-show show-interval 1


switch       log

------- --------------------------------------------------------------------

Switch1 [Mar11.07:48:13] Upgrading nvOS 6.0.1-6000116966 -> 6.1.0-6010017911

Switch2 [Mar11.07:48:13] Upgrading nvOS 6.0.1-6000116966 -> 6.1.0-6010017911


The switch * in the above command indicates all the switches in that fabric which is undergoing the software upgrade process.


north
    keyboard_arrow_up
    keyboard_arrow_down
    description
    print
    feedback
    support
    business
    rss_feed
    south