Infrastructure Support for Transceiver Online Insertion and Removal (OIR)

---

The Transceiver OIR infrastructure supports the following three operations:

• Detecting the insertion or removal of optical devices and cables into or from the front-panel ports, thereby allowing subsequent actions to take place.
• Initializing the optical devices and cables at system start-up, and also as they are inserted into the chassis during run-time.
• Configuring key parameters into the hardware based on the required settings for the optical devices and cables as they are detected.

Note: The support for the Transceiver OIR infrastructure was provided on NRU03 and NRU-S0301 platforms in Netvisor ONE version 6.0.1 and on Wedge100-32X in version 6.1.0 . Starting with Netvisor ONE version 7.0.0, the OIR infrastructure is also supported on the following platforms:

• Dell: S5232F-ON, S5248F-ON, S5224F-ON, Z9264
• Edgecore: AS7816-64X, AS7726-32X, AS5835-54X, AS7326-56X, AS5835-54T 
• Freedom: F9664-C, F9432-C, F9460-X, F9480-V, F9460-T

The support for transceiver OIR infrastructure enhances Netvisor ONE to process the transceiver OIR events and apply appropriate configuration and startup sequencing based on the type of module inserted or removed. This infrastructure implementation improves the hardware monitor service to detect OIR events and notify Netvisor ONE.

Prior to version 6.0.1, Netvisor supported initialization of optical devices on switches only at power-on. Starting with Netvisor ONE version 6.0.1, with the Transceiver OIR infrastructure support, you can also facilitate run-time initialization of optical devices (hot-swappable) while the switch is powered-on. This provides scalable infrastructure for device-specific initialization requirements upon OIR of the transceiver on NRU03 and NRU-S0301 platforms, and on Wedge100-32X in version 6.1.0 . From Netvisor ONE version 7.0.0 onward, this capability is extended to other platforms listed in the Note above.

That is, you can remove, insert, swap, or replace the transceivers when the system is functioning. Each transceiver has unique characteristics, parameters, timing requirement, initialization sequence, and configuration impact to the switch port and Netvisor ONE adheres to these characteristics for the transceiver to operate correctly.

With transceiver OIR infrastructure  support, Netvisor ONE automatically detects the type of device installed in the physical port and the OIR determines the port settings if the eth-mode or pre-emphasis  (port-cable-type) settings are not configured already.

An important distinction between pre-OIR support release and releases with OIR support is the use of sys-default configuration option, as explained  in the table:

Note: Ensure to set the eth-mode settings in accordance with the switch vendor recommendation.

For example, releases prior to having support for OIR,  a 100G DAC cable and 100G SR4 optic cable have the same eth-mode configuration if both ports are configured for sys-default. However, with OIR support, these ports have eth-mode configured as CR4 and SR4, respectively. 

When you are upgrading Netvisor ONE from an earlier version which does not have support for OIR to a version which supports OIR, the original settings are retained unless you revert to the system default settings by using the port-config-modify port <port-num> eth-mode sys-default command or the port-cable-type-modify port <port-list> type sys-default command upon upgrade. The sys-default parameter ensures that the hardware settings are set in accordance to the specific device type installed after upgrading to a version which supports OIR.

Note: It is highly recommended to configure eth-mode and cable-type to sys-default for newly deployed ports. 

Below is an sample configuration after upgrading to a Netvisor version that features platform support for OIR:

CLI (network-admin@nru03-switch) > port-cable-type-modify port all type sys-default

Warning: updating SI settings will apply to all ports in a port group and will flap enabled ports

Please confirm y/n (Default: n):y

Updated SI settings for port(s) 1-125 to sys-default

CLI (network-admin@nru03-switch) > port-cable-type-show

switch       port  type        cable-sub-type

----------- ----- ----------- --------------

nru03-switch none  dac         ca-s

nru03-switch none  dac         ca-l

nru03-switch none  dac         ca-n

nru03-switch none  dac         default

nru03-switch none  optical     default

nru03-switch 1-125 sys-default default

The auto negotiation and the FEC are user-configurable features, that is, you can enable or disable these features; Transceiver OIR will not override those settings.

Note: When you upgrade from  an earlier version which does not have support for OIR to a version which supports OIR, all existing port configuration settings are preserved.

Usage Guidelines

• At system startup with a fully-populated switch, an initial insertion event is generated for each transceiver, but this has a minimal impact on Netvisor  startup time.
• The eth-mode, ER4 is not supported by Broadcom SDK.  Per Broadcom recommendation, the OIR infrastructure programs the hardware as SR4 when ER4 device is detected.
• For RDH 72/16 (PSM4) the recommended system default eth-mode is CAUI4  at 100G speed and LR4 for 4x25G speed as determined by the OIR.
• The transceiver OIR implementation in Netvisor ONE version, which supports OIR has minimal error handling capabilities. Failures in the execution of  OIR state machine events may cause port to enter into error state. If you encounter an error, you must remove and re-insert the transceiver to recover. 

Note: If for any reason you are unable to recover the transceiver, you must contact Pluribus TAC for support.

• The transceiver OIR infrastructure introduces a new SNMP event when entering and exiting the error state. Below is a sample SNMP event generated during an enter/exit error state as a reference:

2020-09-23 21:40:39 localhost [UDP: [127.0.0.1]:37190->[127.0.0.1]:162]:

DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (17946935) 2 days, 1:51:09.35 

SNMPv2-MIB::snmpTrapOID.0 = OID: PN-LOG-MIB::pnLogMatchNotification 

PN-LOG-MIB::pnLogMatchName.0 = STRING: PortOIRErr PN-LOG-MIB::pnLogFileName.0 = STRING: /nvOS/log/event.log PN-LOG-MIB::pnLogMatchCount.0 = Gauge32: 3 

PN-LOG-MIB::pnLogMatchData.0 = STRING: 2020-09-23,21:40:01.352234-07:00 

nru02-alpha2 nvOSd(3830) event port_oir_error_enter(11527) : level=warn 

event-type=port : port=1 : Port=1 enter xcvr oir error state

2020-09-23 21:41:39 localhost [UDP: [127.0.0.1]:37190->[127.0.0.1]:162]:

DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (17952935) 2 days, 1:52:09.35 

SNMPv2-MIB::snmpTrapOID.0 = OID: PN-LOG-MIB::pnLogMatchNotification 

PN-LOG-MIB::pnLogMatchName.0 = STRING: PortOIRErr PN-LOG-MIB::pnLogFileName.0 = STRING: /nvOS/log/event.log PN-LOG-MIB::pnLogMatchCount.0 = Gauge32: 4 

PN-LOG-MIB::pnLogMatchData.0 = STRING: 2020-09-23,21:40:46.408637-07:00 

nru02-alpha2 nvOSd(3830) event port_oir_error_exit(11528) : level=warn 

event-type=port : port=1 : Port=1 exit xcvr oir error state

• It is possible to miss an insertion or removal event if a transceiver is inserted and removed rapidly. 

Note: You must restrain from removing and inserting a transceiver rapidly because that could cause the port to enter error state in the OIR state machine, especially if a different type of transceiver is  inserted. 

• Transceiver OIR leverages the device characteristic information maintained in the device EEPROM.    

Note: The vendors that do not comply with MSA standard may not operate properly when used with OIR.  For best results, use only those devices that are listed in the Transceiver Compatibility Information, available on  Pluribus Networks Technical Documentation. 

• With OIR infrastructure:
• The pre-emphasis and the eth-mode settings are set in accordance with the device type
• The DAC pre-emphasis setting is set in accordance with the DAC cable length

Note: Currently the OIR infrastructure uses the 3 meter DAC setting when programming the hardware, regardless of the actual DAC cable length.

CLI Settings

Note: You should never modify the  OIR settings.

Use the following CLI commands to verify the OIR details on your switch. 

To display the XCVRs present on port in the switch, use:

CLI (network-admin@switch) > port-xcvr-show port <port number>

For example, to see the details:

CLI (network-admin@S5232F) > port-xcvr-show format switch,port,vendor-name,part-number,serial-number

switch  port vendor-name     part-number   serial-number 

------- ---- --------------- ------------- --------------

S5232F  49   Amphenol        603020002     APF16010026PM3                    

S5232F  51   FINISAR CORP    FCBN510QE2C03 WXG0EXM        

S5232F  55   Arista Networks CAB-Q-Q-3M    XHC1315DG10H   

S5232F  56   DELL            GMFC5         MY013602746004F

To check the port phy status, use:

CLI (network-admin@switch) > port-phy-show port <port number>

For example,

CLI (network-admin@S5232F) > port-phy-show

switch port state speed  eth-mode max-frame def-vlan

------ ---- ----- ------ -------- --------- --------

S5232F 1    down  100000 caui4    1540      1

S5232F 5    down  100000 caui4    1540      1

S5232F 57   up    100000 cr4      1540      1

S5232F 61   down  100000 caui4    1540      1

S5232F 117  up    100000 cr4      1540      1

S5232F 126  down  10000  xfi      1540      1

S5232F 127  down  10000  xfi      1540      1