Troubleshooting the Fabric


There may be instances when you need to troubleshoot the fabric. The following is a list of helpful port numbers, multicast information, and communication on the fabric.


Internal Keepalive


  • Multicast IP: 239.4.9.7
  • UDP Destination Port: 23399
  • This packet is sent from the CPU to the internal port to ensure that the CPU path to the switch is working and the internal port is up.


Fabric Keepalive


  • UDP Destination Port: 23394
  • Point to point UDP fabric keepalive
  • If these messages don't get through, the fabric node may go to offline state.


Global Discovery


  • Multicast IP: 239.4.9.3
  • UDP destination port: 23399
  • Each node periodically multi-casts a message about the fabric. This enables fabric-show on L2-connected nodes to show available packets and also enables fabric-join name name. It also enables you to join a fabric over Layer 3 connectivity by specify an IP address.

Proxy commands 


  • TCP Destination Port: 23397 SSL
  • Used for Netvisor One-to-Netvisor One communication. Used for internal purposes and also to implement commands executed on other switches from a local switch.


Status propagation


  • TCP Destination Port: 23398 SSL
  • Port changes and vport changes propagated to other nodes in the fabric.


TCP API clients


  • TCP Destination Port: 23396 SSL
  • TCP API clients connect to this port. Can be disabled using admin-service-modify if <mgmt/data> no-net-api command.


File System replication


  • TCP Destination Port: 23392 SSL
  • For ZFS send and ZFS receive messages when replicating file systems across the fabric.


L2 ARP/DMAC miss/Broadcast encapsulation


  • UDP Destination Port: 23389
  • These are VXLAN-encapsulated packets sent from CPU to CPU between two L2 connected switches.

L3 ARP/DMAC miss/Broadcast encapsulation


  • UDP Destination Port: 23388
  • These are VXLAN-encapsulated packets sent from CPU to CPU between two L3 connected switches.


vPORT status


  • Multicast IP: 239.4.9.4
  • UDP Destination Port: 23390
  • vPort updates from hypervisors or hosts in the fabric.


vFlow CPU packets


  • UDP Destination Port: 23398
  • These packets are sent point-to-point for vflow-snoop of a fabric-scoped vFlow.

 

All of these messages need to be able to get through in order to keep an L2 fabric healthy. The multicast messages don't propagate through routers so they aren't used for L3 fabrics.


The CLI command fabric-node-show displays information about Netvisor One internal data structures for each node in the fabric. If no keepalive or other messages are received from a fabric node for about 20 seconds, the node is marked as offline.


Anything that prevents keepalive or other kinds of messages from flowing freely between fabric nodes can cause problems for fabric connectivity.

If the fabric transaction IDs become unsynchronized, use the transaction commands to either roll forward or back the transaction IDs.