Troubleshooting the Fabric

There may be instances when you need to troubleshoot the fabric. The following is a list of helpful port numbers, multicast information, and communication on the fabric.

Multicast IP: 239.4.9.7

UDP Destination Port: 23399

This packet is sent from the CPU to the internal port to ensure that the CPU path to the switch is working and the internal port is up.

UDP Destination Port: 23394

Point to point UDP fabric keepalive

If these messages don't get through, the fabric node may go to offline state.

Multicast IP: 239.4.9.3

UDP destination port: 23399

Each node periodically multicasts a message about the fabric. This enables fabric-show on L2-connected nodes to show available packets and also enables fabric-join name name. It also enables you to join a fabric over Layer 3 connectivity by specify an IP address.

TCP Destination Port: 23397 SSL

Used for Netvisor OS-to-Netvisor OS communication. Used for internal purposes and also to implement commands executed on other switches from a local switch.

TCP Destination Port: 23398 SSL

Port changes and vport changes propagated to other nodes in the fabric.

TCP Destination Port: 23396 SSL

C API clients connect to this port. Can be disabled using admin-service-modify if <mgmt/data> no-net-api command.

TCP Destination Port: 23392 SSL

For ZFS send and ZFS receive messages when replicating file systems across the fabric.

UDP Destination Port: 23389

These are VXLAN-encapsulated packets sent from CPU to CPU between two L2 connected switches.

UDP Destination Port: 23388

These are VXLAN-encapsulated packets sent from CPU to CPU between two L3 connected switches.

Multicast IP: 239.4.9.4

UDP Destination Port: 23390

vPort updates from hypervisors or hosts in the fabric.

UDP Destination Port: 23398

These packets are sent point-to-point for vflow-snoop of a fabric-scoped vFlow.

 

All of these messages need to be able to get through in order to keep an L2 fabric healthy. The multicast messages don't propagate through routers so they aren't used for L3 fabrics.

fabric-node-show displays information about Netvisor internal data structures for each node in the fabric. If no keepalive or other messages are received from a fabric node for about 20 seconds, the node is marked as offline.

Anything that prevents keepalive or other kinds of messages from flowing freely between fabric nodes can cause problems for fabric connectivity.

If the fabric transaction IDs become unsynchronized, use the transaction commands to either roll forward or back the transaction IDs. See Fabric Administration<TextRegular> on page 208.