Configuring Uplink Groups



Certain features require that a switch monitors the total available aggregate uplink bandwidth compared to the amount of downlink traffic being generated by the attached devices. The requirement is to prevent oversubscription of the uplinks by traffic sourced from the downlinks (host-facing ports). 


For that purpose, starting from Netvisor ONE release 7.0.0, the software supports the configuration of uplink groups, which can be used with features like vLE and bidirectional vPG for uplink oversubscription avoidance. (Refer to the Configuring Virtual Link Extension and Configuring Pluribus Network Packet Broker sections of the Configuration Guide for more details on those features.) 


Uplink oversubscription is defined as: total downlink capacity > total uplink capacity (for a given switch). 


Note: The operational status of switch ports is not considered to calculate the uplink and downlink aggregated bandwidth capacities. Only the configuration/administrative state is considered. 


Note: Uplink groups support only bidirectional vPG-driven connections and not unidirectional vPG-driven connections. 


The configuration of an uplink group allows users to track the available aggregate bandwidth of the physical ports added to a group. (Trunks cannot be added.) Then, a system configuration allows the users to select which action to apply in case of aggregate bandwidth mismatch between the uplink ports and the downlink ports: 


  • No warning or error is generated (default behavior) 
  • Allow the creation of a connection across the fabric, but: 
    • Display an appropriate message to warn about oversubscription 
    • Log the event in system.log 
  • Refuse the creation of a connection across the fabric if its creation leads to oversubscription. 


Note: The software currently does not support separate uplink port groups per feature connection to monitor oversubscription more granularly. Hence this could lead to inaccuracies in reporting (i.e., to false positives). 


The software tracks port additions/modifications/deletions as well as port speed changes to calculate the aggregate bandwidth. Whenever a feature-driven connection across the fabric is created/deleted/disabled, the downlink port capacity is also modified (when applicable) and checked. 


Note: Uplink/downlink port speed changes when enabling/disabling auto-negotiation are not reflected in the respective group’s aggregated bandwidth. So, after auto-negotiation, if a port speed changes and leads to oversubscription, the event is not reported. 


Release 7.0.0 introduces a new set of commands to configure an uplink group, as shown below: 


CLI (network-admin@switch) > uplink-group-port-show

switch: switch

ports: none


CLI (network-admin@switch) > uplink-group-port-add ports 12-14


CLI (network-admin@switch) > uplink-group-port-show

switch: switch

ports: 12-14


Note: Only physical ports can be added to an uplink group. Configuring trunks will produce an error. 


You can remove ports from an uplink group like so: 


CLI (network-admin@switch) > uplink-group-port-show

switch: switch

ports: 12-14


CLI (network-admin@switch) > uplink-group-port-remove ports 12

CLI (network-admin@switch) > uplink-group-port-show

switch: switch

ports: 13-14


You can configure the action to take when uplink oversubscription is detected with the following command: 


CLI (network-admin@switch) > system-settings-modify uplink-oversubscription-action default | warn | error 


default means no warning or error is generated. 


warn allows the creation of a connection across the fabric, but a warning message is produced if the creation causes uplink oversubscription. 


If the switch is already in oversubscribed state, the system generates a specific warning message right when the option is configured: 


CLI (network-admin@switch) > system-settings-modify uplink-oversubscription-action warn 

Warning: The switch is already in uplink oversubscription state. Configure ports to uplink group 


You can configure the error option so that the software refuses the creation of a connection across the fabric when causing uplink oversubscription: 


CLI (network-admin@switch) > system-settings-modify uplink-oversubscription-action error 


CLI (network-admin@switch) > system-settings-show format uplink-oversubscription-action 

uplink-oversubscription: error 


If the switch is already in oversubscribed state, the command will fail right away with an error when the option is configured: 


CLI (network-admin@switch) > system-settings-modify uplink-oversubscription-action error 

system-setting-modify: failed to change to error action as the switch is already in uplink oversubscription state. Add ports to uplink group or remove existing VLE/VPG(bidir) connections. 


If you remove ports from an uplink group, oversubscription may occur due to the capacity change. The warn and error options trigger, respectively, the following messages (and action): 


CLI (network-admin@switch) > uplink-group-port-remove port 25 

Warning: removing ports from uplink group is causing oversubscription of uplink ports. 


CLI (network-admin@switch) > uplink-group-port-remove port 25 

uplink-group-port-remove: removing these ports from uplink group failed as it is leading to oversubscription of uplink ports. 


If you change uplink/downlink port speeds, that may lead to oversubscription. The warn and error options trigger, respectively, the following messages (and action): 


CLI (network-admin@switch) > port-config-modify port 45 speed 40g 

Warning: Change of speed for the port(s) 45 has caused uplink oversubscription. 


CLI (network-admin@switch) > port-config-modify port 45 speed 40g 

port-config-modify: change port speed failed for given port(s) 45 as leading to oversubscription of uplink ports. 


In case of the warn option, this is the warning message generated for a vLE configuration with uplink oversubscription: 


CLI (network-admin@switch) > vle-create node-1 switch node-1-port 12 node-2 switch2 node-2-port 13 name vle1 

Warning: The creation/enabling of vle vle1 is causing uplink oversubscription by 1000 mbps. 


In this scenario, the calculated difference between the downlink and uplink capacity is 1000 Mbps. 


In case of the error option, when oversubscription occurs locally to a vLE switch, this message is displayed: 


CLI (network-admin@switch) > vle-create node-1 switch node-1-port 12 node-2 switch2 node-2-port 13 name vle1 

vle-create: creation/enabling of VLE vle1 failed because it is causing uplink oversubscription by 1000 mbps. Remove associated transparent VLAN and VXLAN mapping for the VLE. 


Similarly, when oversubscription happens on a remote switch due to the vle-create command, this message is generated: 


CLI (network-admin@switch) > vle-create node-1 switch node-1-port 12 node-2 switch2 node-2-port 13 name vle1 

vle-create: error from switch2: creation/enabling of VLE vle1 failed because it is causing uplink oversubscription by 1000 mbps. Remove associated transparent VLAN and VXLAN mapping for the VLE. 


In case of the warn option, this is the warning message generated for a vPG configuration with uplink oversubscription: 


CLI (network-admin@switch) > vflow-create name v1 scope fabric bidir-vpg-1 vpg1 bidir-vpg-2 vpg2 

Warning: The creation/enabling of (bidir vpg)vflow v1 is causing uplink oversubscription by 1000 mbps. 


In case of the error option, when oversubscription occurs locally to a vPG switch, this message is displayed: 


CLI (network-admin@switch) > vflow-create name v1 scope fabric bidir-vpg-1 vpg1 bidir-vpg-2 vpg3 

vflow-create: The creation/enabling of (bidir vpg)vflow v1 is causing uplink  oversubscription by 1000 mbps. 


Similarly, when oversubscription happens on a remote switch due to the vflow-create command, this message is generated: 


CLI (network-admin@switch) > vflow-create name v1 scope fabric bidir-vpg-1 vpg1 bidir-vpg-2 vpg3 

vflow-create: error from switch2: The creation/enabling of (bidir vpg)vflow v1 is causing uplink oversubscription by 1000 mbps. 


With the warn option, when a vLE is created with endpoints that are local to the switch and have a speed mismatch, when oversubscription occurs this warning message is generated: 


CLI (network-admin@switch) > port-show port 9,49 format port,config,

port config

---- ------

9 fd,10g

49 fd,40g


CLI (network-admin@switch) > vle-create name v1 node-1 switch node-1-port 9

node-2 switch node-2-port 49

Warning: creation/enabling of VLE is causing endpt 9 being oversubscribed.


In the same scenario, but with the error option, this action is generated in case of oversubscription: 


CLI (network-admin@switch) > vle-create name v1 node-1 switch node-1-port 9 node-2 switch node-2-port 49 

vle-create: creation/enabling of local VLE failed because endpoint 9 is oversubscribed. 

Remove associated transparent VLAN and VXLAN mapping for the VLE. 


With the warn option, when vPGs are created with endpoints that are local to the switch (port 17,45 in the example below) and have a speed mismatch, this warning message is generated: 


CLI (network-admin@switch) > vpg-show

scope name type ports vni vlan

------ ---- ------------- ----- -------- ----

fabric vpg3 bidirectional 17 12500000 2750

fabric vpg4 bidirectional 22 12500001 2751

fabric vpg5 bidirectional 45 12500002 2752


CLI (network-admin@switch) > port-show port 17,45 format port,config,

port config

---- ------

17 fd,10g

45 fd,40g


CLI (network-admin@switch) > vflow-create name v3 scope local bidir-vpg-1 vpg3

bidir-vpg-2 vpg5

Warning: creation/enabling of local vflow is causing endpoint 17 to be

oversubscribed.


In the same scenario, but with the error option, this action is generated: 


CLI (network-admin@switch) > vflow-create name v3 scope local bidir-vpg-1 vpg3

bidir-vpg-2 vpg5

vflow-create: creation/enabling of local vflow failed because endpoint 17 is

oversubscribed.


With the warn option, when adding ports to a vPG with an active vFlow connection, the following warning message is generated when oversubscription occurs: 


CLI (network-admin@switch) > vflow-show format name,bidir-vpg-1,bidir-vpg-2,inport,

name bidir-vpg-1 bidir-vpg-2 in-port

---- ----------- ----------- -------

v6 vpg3 vpg2 12


If port 12 (10g) is removed and trunk port 273 (20g) is added to vpg3 of vFlow v6


CLI (network-admin@switch) > vpg-port-remove vpg-name vpg3 ports 12

CLI (network-admin@switch) > vpg-port-add vpg-name vpg3 ports 273

warning: creation/enabling of Vflow v6 is causing uplink oversubscription by

10000 mbps


In the same scenario, but with the error option, this action is generated in case of oversubscription: 


CLI (network-admin@switch) > vpg-port-remove vpg-name vpg3 ports 12

CLI (network-admin@switch) > vpg-port-add vpg-name vpg3 ports 273

vpg-port-add: creation/enabling of Vflow v6 failed because it is causing uplink

oversubscription by 1000 mbps. Consider adding ports to uplink group.


With the warn option, when trunk ports of a bidirectional vPG belonging to active vFlow connections are modified, the following warning message is generated when oversubscription occurs: 


CLI (network-admin@switch) > vflow-show format name,bidir-vpg-1,bidir-vpg-2,inport,

name bidir-vpg-1 bidir-vpg-2 in-port

---- ----------- ----------- -------

v6 vpg3 vpg2 27


CLI (network-admin@switch) > trunk-show name t1 format name,trunk-id,ports

name trunk-id ports

---- -------- -----

t1 273 21


If the uplink bandwidth is 10 Gbps and the trunk t1 is expanded to double its capacity, say, with ports 21-22 (20g), then oversubscription would occur: 


CLI (network-admin@switch) > trunk-modify ports 21-22 name t1 

Warning: modifying trunk parameters in use by bidir vpg is leading to uplink oversubscription. 


In the same scenario, but with the error option, this action is generated instead: 


CLI (network-admin@switch) > trunk-modify ports 21-22 name t1 

trunk-modify: modifying trunk parameters in use by bidir vpg failed because it is leading to uplink oversubscription. 


You can show the uplink oversubscription state with the following command: 


CLI (network-admin@switch) > uplink-group-port-oversubscription-state-show 

uplink-bw:               20000 

downlink-bw:             50000 

uplink-oversubscription: yes 


north
    keyboard_arrow_up
    keyboard_arrow_down
    description
    print
    feedback
    support
    business
    rss_feed
    south