Configuring Active-Active vLAG Forwarding with Loopback Recirculation
In network designs in which vLAG load-balancing with fast failover is required to work in conjunction with dynamic routing protocol peering, the aforementioned vLAG forwarding rule may cause traffic drops.
Therefore, for certain switch models with spare bandwidth an additional configuration option has been implemented.
Let’s consider the example in Figure: Symmetric Routing over a vLAG with Loopback Recirculation below where a cluster pair uses standard OSPF peering over a Layer 2 domain represented by a redundant (v)LAG.
To the routing protocol running on PN-2 the (v)LAG looks like a share medium over which both cluster nodes, 10.10.10.1 and 10.10.10.2, are reachable. However, the (v)LAG performs traffic load-balancing: packets sent toward 10.10.10.1 can be steered to either PN-0 or PN-1; likewise, packets sent toward 10.10.10.2 can be steered to either PN-0 or PN-1. When packets are sent to the “wrong” next-hop, the latter will have to use the cluster links to steer the packets back to their “correct” destination.
However, packets traversing the cluster links that need to egress a vLAG (going downstream to reach one of the hosts) are dropped.
For this case, Netvisor ONE supports the configuration of internal loopback recirculation for routed packets entering the switch from cluster links in order to bypass the vLAG forwarding rule.
The approach consists in provisioning a set of physical ports in loopback mode without requiring external cabling. In particular, the Pluribus E28-Q model offers up to 12x10GE ports (that is, which are not exposed to users in the front panel) that can be configured in loopback mode for an additional forwarding capacity of 120 Gbps.
An internal forwarding rule can be manually installed on both cluster nodes to redirect the traffic to the internal loopback when it ingresses from the cluster links with a destination MAC address matching that of the next-hop. When a packet is recirculated by this rule over the loopback, it gets normally routed and hence is not subject to the vLAG forwarding rule (that would otherwise drop it).
Figure 7-15 -Symmetric Routing over a vLAG with Loopback Recirculation
Note that, when this option is configured, approximately 50% of the traffic traverses the cluster links, which is not ideal, hence it is not recommended as a general solution but can be useful in certain designs.
For each vRouter that needs redirection of the traffic to the internal loopback, first find the MAC addresses of the (two, in case of a cluster) corresponding interfaces using the vrouter-interface-show command:
CLI (network-admin@switch) > vrouter-interface-show vrouter-name UP1
vrouter-name: UP1
nic: eth11.200
ip: 200.200.200.1/24
assignment: static
mac: 66:0e:94:10:29:e1
vlan: 200
vxlan: 0
if: data
vm-nic-type: data
exclusive: no
nic-config: enable
nic-state: up
In this case the interface MAC address is 66:0e:94:10:29:e1.
Then configure a loopback (with a single port or a multi-port trunk):
CLI (network-admin@switch) > port-config-modify port 89 loopback
Or for the trunk case:
CLI (network-admin@switch) > port-config-modify port 89-100 loopback
CLI (network-admin@switch) > trunk-create name loopbackUP ports 89-100 loopback
Then create the special forwarding rule using the vFlow command:
CLI (network-admin@switch) > vflow-create name LB_vflow scope local in-port 129 dst-mac 66:0e:94:10:29:e1 action set-dmac-to-port action-value 89 action-set-mac-value 66:0e:94:10:29:e1
Or for the trunk case:
CLI (network-admin@switch) > vflow-create name LB_vflow scope local in-port 129 dst-mac 66:0e:94:10:29:e1 action set-dmac-to-port action-value 130 action-set-mac-value 66:0e:94:10:29:e1
For action-value use the loopback port number or the loopback trunk ID (130, in this example, assigned during creation).
For a cluster pair, you must configure each node using the above configuration steps.