Displaying VXLAN Features with EVPN
A multi-pod EVPN configuration requires the extension of the intra-pod features commonly used for single-fabric setups. Therefore, before proceeding, refer to the VXLAN commands described in the Configuring VXLAN chapter above for reference.
VTEPs are the VXLAN termination points: Netvisor ONE provides a CLI construct called VTEP object to automatically set up a mesh of bidirectional connections between the end points.
Multi-pod EVPN designs comprise two or more pods in which at least one node per pod is designated as border gateway, as explained earlier. Border gateways are interconnected by EVPN-derived automatic VXLAN tunnels, as shown in dark red color in the figure below for POD 1 and POD 2.
In the network design in Figure 10-6, for redundancy, two pairs of clustered leaf nodes are selected as border gateways (highlighted in yellow and green).
Figure 10-6: Border Gateways vs. Non-Border Gateways
Displaying VTEPs
VTEPs can be created normally in each pod on all the leaf node pairs to automatically set up the intra-pod tunnels (in blue color in the figure above).
However, when the border gateways are configured and actively exchanging EVPN messages a pair of automatic VTEP objects (and the corresponding tunnels) are created as displayed in the output below:
CLI (network-admin@bgw11) > vtep-show format switch,scope,name,location,vrouter-name,ip,virtual-ip,
switch scope name location vrouter-name ip virtual-ip
------ ------ -------------------- ------------- --------------- ------------ ------------
fabric vtep-leaf1 leaf1 vrouter-leaf-1 172.16.128.1 172.16.128.3
fabric vtep-leaf2 leaf2 vrouter-leaf-2 172.16.128.2 172.16.128.3
fabric vtep-leaf3 leaf3 vrouter-leaf-3 172.16.129.1 172.16.129.3
fabric vtep-leaf4 leaf4 vrouter-leaf-4 172.16.129.2 172.16.129.3
fabric vtep-leaf5 leaf5 vrouter-leaf-5 172.16.130.1 172.16.130.3
fabric vtep-leaf6 leaf6 vrouter-leaf-6 172.16.130.2 172.16.130.3
fabric vtep-bgw11 bgw11 vrouter-bgw-11 172.16.133.1 172.16.133.3
fabric vtep-bgw12 bgw12 vrouter-bgw-12 172.16.133.2 172.16.133.3
bgw11 local __evpn__172.16.134.3 host-external 172.16.134.3 ::
bgw12 local __evpn__172.16.134.3 host-external 172.16.134.3 ::
Each automatic VTEP is created with a local scope and the host-external location string on the two border gateways only. The name starts with the __evpn__ string followed by the (virtual) IP address of the neighboring border gateway (pair).
EVPN VTEPs can be displayed with the following command and identified from their name and from the vtep-source column:
CLI (network-admin@bgw11*) > vtep-show format name,vtep-source
name vtep-source
-------------------- -----------
vtep-bgw11 config
__evpn__172.16.134.3 evpn
Displaying Automatic Tunnels
Tunnels are created automatically between border gateways for inter-pod communication. They can be identified from the type column in the following command:
CLI (network-admin@bgw11*) > tunnel-show format name,type,
name type
------------------------------------- ----------
auto-tunnel-190.11.1.1_65.1.1.10 vxlan
auto-tunnel-172.16.133.3_172.16.134.3 vxlan-evpn
Displaying EVPN Border Gateways
The role of border gateway, as configured by the user, can be shown with the expanded vtep-show format all command that includes the evpn-border column:
CLI (network-admin@bgw11) > vtep-show format all
switch scope name location vrouter-name ip virtual-ip mac description mac-learning evpn-border
------ ------ -------------------- ------------- --------------- ------------ ------------ ----------------- ----------- ------------ -----------
fabric vtep-leaf1 leaf1 vrouter-leaf-1 172.16.128.1 172.16.128.3 00:00:5e:00:01:01 on false
fabric vtep-leaf2 leaf2 vrouter-leaf-2 172.16.128.2 172.16.128.3 00:00:5e:00:01:01 on false
fabric vtep-leaf3 leaf3 vrouter-leaf-3 172.16.129.1 172.16.129.3 00:00:5e:00:01:02 on false
fabric vtep-leaf4 leaf4 vrouter-leaf-4 172.16.129.2 172.16.129.3 00:00:5e:00:01:02 on false
fabric vtep-leaf5 leaf5 vrouter-leaf-5 172.16.130.1 172.16.130.3 00:00:5e:00:01:03 on false
fabric vtep-leaf6 leaf6 vrouter-leaf-6 172.16.130.2 172.16.130.3 00:00:5e:00:01:03 on false
fabric vtep-bgw11 bgw11 vrouter-bgw-11 172.16.133.1 172.16.133.3 00:00:5e:00:01:06 on true
fabric vtep-bgw12 bgw12 vrouter-bgw-12 172.16.133.2 172.16.133.3 00:00:5e:00:01:06 on true
bgw11 local __evpn__172.16.134.3 host-external 172.16.134.3 :: 00:00:00:00:00:00 on false
bgw12 local __evpn__172.16.134.3 host-external 172.16.134.3 :: 00:00:00:00:00:00 on false
The role of a border gateway can also be verified by using the vrouter-show command; however, the basic command may not be sufficient. To obtain additional info, you may use the format-all option. Alternatively, you can use the evpn-border filter to display only the border gateway node(s) for example like so:
CLI (network-admin@bgw11) > vrouter-show format name,hw-router-mac,evpn-border
name hw-router-mac evpn-border
--------------- ----------------- -----------
vrouter-bgw-11 66:0e:94:14:f3:5b enable
vrouter-bgw-12 66:0e:94:22:d7:1c enable
vrouter-leaf-1 66:0e:94:ad:d4:41 disable
vrouter-leaf-2 66:0e:94:95:f9:74 disable
<snip>
The border gateway role is also reflected in the BGP configuration, as discussed above and as shown in this command output:
CLI (network-admin@bgw11*) > vrouter-bgp-show multi-protocol l2vpn-evpn
vrouter-name neighbor remote-as next-hop-self prepend replace-as ebgp-multihop update-source route-reflector-client override-capability soft-reconfig-inbound max-prefix-warn-only bfd multi-protocol weight default-originate neighbor-keepalive-interval(s) neighbor-holdtime(s) dup-addr-max-moves(s) dup-addr-moves-duration(s) dup-addr-freeze connect-retry-interval(s) send-community allowas-in allowas-in-origin as-override advertisement-interval(s) description
--------------- ------------ --------- ------------- ------- ---------- ------------- ------------- ---------------------- ------------------- --------------------- -------------------- --- -------------- ------ ----------------- ------------------------------ -------------------- --------------------- -------------------------- --------------- ------------------------- -------------- ---------- ----------------- ----------- ------------------------- -----------
vrouter-bgw-11 172.16.134.1 66611 no yes no 4 172.16.133.1 no no yes no no l2vpn-evpn none no 60 180 5 180 180 15 yes no no no 0 none
vrouter-bgw-11 172.16.134.2 66611 no yes no 4 172.16.133.1 no no yes no no l2vpn-evpn none no 60 180 5 180 180 15 yes no no no 0 none
The above verbose command output can be compacted, for example like so, to get an abridged overview of the EVPN neighborship:
CLI (network-admin@bgw11*) > vrouter-bgp-show format ,neighbor,remote-as,update-source,multi-protocol
vrouter-name neighbor remote-as update-source multi-protocol
-------------- ------------ --------- ------------- --------------
vrouter-bgw-11 172.16.0.161 64311 ipv4-unicast
vrouter-bgw-11 172.16.2.161 64312 ipv4-unicast
vrouter-bgw-11 172.16.4.161 64313 ipv4-unicast
vrouter-bgw-11 172.16.6.161 64314 ipv4-unicast
vrouter-bgw-11 172.16.133.2 64518 ipv4-unicast
vrouter-bgw-11 101.1.1.1 60000 ipv4-unicast
vrouter-bgw-11 172.16.134.1 66611 172.16.133.1 l2vpn-evpn
vrouter-bgw-11 172.16.134.2 66611 172.16.133.1 l2vpn-evpn
Displaying VNIs
VTEPs’ VNI associations can be displayed with the vtep-vxlan-show command locally on the border gateway(s):
CLI (network-admin@bgw11*) > vtep-vxlan-show
name vxlan isolated
-------------------- ------ --------
vtep-leaf1 10201 no
vtep-leaf1 103501 no
vtep-leaf1 10403 no
vtep-leaf2 10201 no
vtep-leaf2 103501 no
vtep-leaf2 10403 no
vtep-leaf3 10201 no
vtep-leaf3 103501 no
vtep-leaf3 10403 no
vtep-leaf4 10201 no
vtep-leaf4 103501 no
vtep-leaf4 10403 no
vtep-leaf5 10201 no
vtep-leaf5 103501 no
vtep-leaf5 10403 no
vtep-leaf6 10201 no
vtep-leaf6 103501 no
vtep-leaf6 10403 no
vtep-bgw11 10201 no
vtep-bgw11 103501 no
vtep-bgw11 10403 no
vtep-bgw12 10201 no
vtep-bgw12 103501 no
vtep-bgw12 10403 no
__evpn__172.16.134.3 10201 no
__evpn__172.16.134.3 103501 no
In the above example, note that VNIs 10201 and 103501 are present on all the leaf nodes including the border gateways’ VTEPs and the automatically created host-external ones.
In other words, this command can be used to verify that the inter-pod VNIs are configured consistently on each pod’s VTEPs and that, as a consequence, the common VNIs get added to the automatic inter-pod tunnels too.
Let’s suppose the number of VNIs grows pretty large. Then, you can limit the output of the display for example to a particular EVPN VTEP of interest, like so:
CLI (network-admin@bgw11*) > vtep-vxlan-show name __evpn__172.16.134.3
name vxlan isolated
-------------------- ------ --------
__evpn__172.16.134.3 10201 no
__evpn__172.16.134.3 10202 no
__evpn__172.16.134.3 10203 no
__evpn__172.16.134.3 10204 no
__evpn__172.16.134.3 10205 no
…
Displaying EVPN Route Types
In order to display the supported EVPN route types for VXLAN (encapsulation type 8) received by a border gateway node, you can use the following command:
CLI (network-admin@bgw11*) > vrouter-evpn-bgp-routes-show format vrouter-name,rd,vni,network,route-type,next-hop,path,extended-community
vrouter-name rd vni network route-type next-hop path extended-community
-------------- -------------- ------ ------------------------ ---------- ------------ ---- -------------------------------------------
vrouter-bgw-11 0.0.0.0:2 103501 [24]:[10.2.3.0] 5 172.16.133.3 ET:8 RT:64518:103501 Rmac:00:00:5e:00:01:06
vrouter-bgw-11 0.0.0.0:2 103501 [24]:[10.2.4.0] 5 172.16.134.3 RT:1075:103501 ET:8 Rmac:00:00:5e:00:01:01
vrouter-bgw-11 0.0.0.0:2 103501 [24]:[10.2.4.0] 5 172.16.134.3 RT:1075:103501 ET:8 Rmac:00:00:5e:00:01:01
vrouter-bgw-11 192.168.0.11:3 10201 [48]:[00:3b:01:00:00:01] 2 172.16.133.3 ET:8 RT:64518:10201
vrouter-bgw-11 192.168.0.11:3 10201 [32]:[172.16.133.3] 3 172.16.133.3 ET:8 RT:64518:10201
vrouter-bgw-11 192.168.0.11:4 10403 [32]:[172.16.133.3] 3 172.16.133.3 ET:8 RT:64518:10403
vrouter-bgw-11 192.168.0.13:3 10404 [32]:[172.16.134.3] 3 172.16.134.3 RT:1075:10404 ET:8
vrouter-bgw-11 192.168.0.13:4 10201 [48]:[00:3d:01:00:00:01] 2 172.16.134.3 RT:1075:10201 ET:8
vrouter-bgw-11 192.168.0.13:4 10201 [32]:[172.16.134.3] 3 172.16.134.3 RT:1075:10201 ET:8
vrouter-bgw-11 192.168.0.14:3 10404 [32]:[172.16.134.3] 3 172.16.134.3 RT:1075:10404 ET:8
vrouter-bgw-11 192.168.0.14:4 10201 [48]:[00:3d:01:00:00:01] 2 172.16.134.3 RT:1075:10201 ET:8
vrouter-bgw-11 192.168.0.14:4 10201 [32]:[172.16.134.3] 3 172.16.134.3 RT:1075:10201 ET:8
Note that the route type is listed in a separate column. The EVPN route contents are shown in the network column (IP addresses for type 5, MAC addresses for type 2, etc.). Also note that EVPN Route Targets (RT) and Route Distinguishers (RD) are displayed. The RT field is used to transport the VNI values.
You can selectively display a single route type with these commands:
CLI (network-admin@bgw11*) > vrouter-evpn-bgp-routes-show route-type 2 format vrouter-name,rd,vni,network,route-type,next-hop,path,extended-community
vrouter-name rd vni network route-type next-hop path extended-community
-------------- -------------- ----- ------------------------ ---------- ------------ ---- -------------------
vrouter-bgw-11 192.168.0.11:3 10201 [48]:[00:3b:01:00:00:01] 2 172.16.133.3 ET:8 RT:64518:10201
vrouter-bgw-11 192.168.0.13:4 10201 [48]:[00:3d:01:00:00:01] 2 172.16.134.3 RT:1075:10201 ET:8
vrouter-bgw-11 192.168.0.14:4 10201 [48]:[00:3d:01:00:00:01] 2 172.16.134.3 RT:1075:10201 ET:8
CLI (network-admin@bgw11*) > vrouter-evpn-bgp-routes-show route-type 3 format vrouter-name,rd,vni,network,route-type,next-hop,path,extended-community
vrouter-name rd vni network route-type next-hop path extended-community
-------------- -------------- ----- ------------------- ---------- ------------ ---- -------------------
vrouter-bgw-11 192.168.0.11:3 10201 [32]:[172.16.133.3] 3 172.16.133.3 ET:8 RT:64518:10201
vrouter-bgw-11 192.168.0.11:4 10403 [32]:[172.16.133.3] 3 172.16.133.3 ET:8 RT:64518:10403
vrouter-bgw-11 192.168.0.13:3 10404 [32]:[172.16.134.3] 3 172.16.134.3 RT:1075:10404 ET:8
vrouter-bgw-11 192.168.0.13:4 10201 [32]:[172.16.134.3] 3 172.16.134.3 RT:1075:10201 ET:8
vrouter-bgw-11 192.168.0.14:3 10404 [32]:[172.16.134.3] 3 172.16.134.3 RT:1075:10404 ET:8
vrouter-bgw-11 192.168.0.14:4 10201 [32]:[172.16.134.3] 3 172.16.134.3 RT:1075:10201 ET:8
CLI (network-admin@bgw11*) > vrouter-evpn-bgp-routes-show route-type 5 format vrouter-name,rd,vni,network,route-type,next-hop,extended-community
vrouter-name rd vni network route-type next-hop extended-community
-------------- --------- ------ --------------- ---------- ------------ -------------------------------------------
vrouter-bgw-11 0.0.0.0:2 103501 [24]:[10.2.3.0] 5 172.16.133.3 ET:8 RT:64518:103501 Rmac:00:00:5e:00:01:06
vrouter-bgw-11 0.0.0.0:2 103501 [24]:[10.2.4.0] 5 172.16.134.3 RT:1075:103501 ET:8 Rmac:00:00:5e:00:01:01
vrouter-bgw-11 0.0.0.0:2 103501 [24]:[10.2.4.0] 5 172.16.134.3 RT:1075:103501 ET:8 Rmac:00:00:5e:00:01:01
Displaying VRFs
VRF information, including the associated L3 VNIs, can be displayed with the command:
CLI (network-admin@bgw11*) > vrf-show
name vnet scope anycast-mac vrf-gw vrf-gw2 vrf-gw-ip6 vrf-gw2-ip6 l3-vni active hw-router-mac hw-vrid flags enable description
---------- ---- ------ ----------------- ------ ------- ---------- ----------- ------ ------ ----------------- ------- ------ ------ -----------
vrf-pod1-1 0:0 fabric 64:0e:94:40:00:02 :: :: :: :: 103501 yes 66:0e:94:13:bd:c8 1 subnet yes
Subnet information, including the associated VRF, can be displayed with the command:
CLI (network-admin@bgw11*) > subnet-show
name scope vlan vxlan vrf network anycast-gw-ip packet-relay forward-proto state enable
-------- ------ ---- ----- ---------- ----------- ------------- ------------ ------------- ----- ------
vlan-403 fabric 403 10403 vrf-pod1-1 10.2.3.0/24 10.2.3.1 disable dhcp ok yes
Displaying Routing Information
To verify the RIB routes associated with EVPN, you can use either of the following commands:
CLI (network-admin@bgw11*) > vrouter-rib-routes-show vrid 1
vrid ip prelen number-of-nexthops nexthop flags vnet bd vlan intf-ip intf-id
---- -------- ------ ------------------ ------------ ------------------ ---- -- ---- -------- -------
1 10.2.4.0 24 1 172.16.134.3 in-hw,evpn 3501 :: 10
1 10.2.3.0 24 1 10.2.3.1 in-hw,local-subnet 403 10.2.3.1 11
CLI (network-admin@bgw11*) > vrouter-rib-routes-show vrid 1 flags evpn
vrid ip prelen number-of-nexthops nexthop flags vnet bd vlan intf-ip intf-id
---- -------- ------ ------------------ ------------ ---------- ---- -- ---- ------- -------
1 10.2.4.0 24 1 172.16.134.3 in-hw,evpn 3501 :: 10
On non-border gateway nodes (such as Leaf 1), you can verify the presence of an automatically created default route pointing to the border gateway(s) with either of the following commands:
CLI (network-admin@leaf1*) > vrouter-rib-routes-show vrid 1
vrid ip prelen number-of-nexthops nexthop flags vnet bd vlan intf-ip intf-id
---- -------- ------ ------------------ ------------ ------------------ ---- -- ---- -------- -------
1 0.0.0.0 0 1 172.16.133.3 in-hw,evpn 3501 :: 8
1 10.2.3.0 24 1 10.2.3.1 in-hw,local-subnet 403 10.2.3.1 9
CLI (network-admin@leaf1*) > vrouter-rib-routes-show vrid 1 flags evpn
vrid ip prelen number-of-nexthops nexthop flags vnet bd vlan intf-ip intf-id
---- ------- ------ ------------------ ------------ ---------- ---- -- ---- ------- -------
1 0.0.0.0 0 1 172.16.133.3 in-hw,evpn 3501 :: 8
In this case the default gateway points to a VIP address of a redundant border gateway pair, which can be displayed like so:
CLI (network-admin@bgw11*) > vrouter-interface-show is-vip true
vrouter-name nic ip mac vlan vlan-type nic-state is-vip vrrp-id vrrp-primary vrrp-state mtu priority-tag
-------------- ---------- --------------- ----------------- ---- --------- --------- ------ ------- ------------ ---------- ---- ------------
vrouter-bgw-11 eth21.3808 172.16.133.3/29 00:00:5e:00:01:06 3808 public up true 6 eth20.3808 master 9216 off
Displaying BGP Neighbor Information
To display BGP neighbors associated with EVPN, you can use the following command:
CLI (network-admin@bgw11*) > vrouter-evpn-bgp-neighbor-summary-show format all
vrouter-name neighbor ver as msg-rcvd msg-sent Tb/Ver InQ OutQ up/down state pfxrcd
-------------- ------------ --- ----- -------- -------- ------ --- ---- -------- ----------- ------
vrouter-bgw-11 172.16.134.1 4 65011 122 226 0 0 0 00:00:39 Connect 0
vrouter-bgw-11 172.16.134.2 4 65011 83 190 0 0 0 00:07:36 Established 4
Displaying vPort Information
On a non-border gateway node such as Leaf 1, you can verify the vPort database information associated with the EVPN control plane by using the following command:
CLI (network-admin@leaf1*) > vport-show state evpn
owner mac vxlan state vtep-ip
----- ----------------- ------ ----- ------------
leaf1 64:0e:94:40:00:02 10201 evpn 172.16.133.3
leaf1 64:0e:94:40:00:02 103501 evpn 172.16.133.3
Displaying Additional BGP Control Plane Information
EVPN uses BGP as control plane to exchange various kinds of information that are necessary for proper network functioning, such as VNI values, MAC/IP address pair entries (as found in ARP caches), and Layer 2 address information.
Note that the EVPN control plane in Netvisor ONE is divided into two layers. The higher layer comprises the familiar Netvisor ONE commands, which provide a holistic view on the network entities as described in the above sections. Instead, the lower layer (also known as FRR) provides a low-level 'raw' view on certain control plane functions.
For troubleshooting purposes, it can be useful to check that the upper layer and the lower layer information match.
For instance, it is possible to verify the list of active VNIs on a Border Gateway with the familiar 'high-level' vtep-vxlan-show command. For example, on bgw21 (limited to vtep-leaf2 for brevity's sake) the command output shows:
CLI (network-admin@bgw21*) > vtep-vxlan-show name vtep-leaf2 sort-asc vxlan,
name vxlan isolated
---------- ------ --------
vtep-leaf2 10110 no
vtep-leaf2 10112 no
vtep-leaf2 10120 no
vtep-leaf2 10121 no
vtep-leaf2 10122 no
vtep-leaf2 10130 no
vtep-leaf2 10131 no
vtep-leaf2 10140 no
vtep-leaf2 10145 no
vtep-leaf2 10146 no
vtep-leaf2 10147 no
vtep-leaf2 10148 no
vtep-leaf2 10149 no
vtep-leaf2 10150 no
The same VNI information can be checked also in the low-level 'raw' control plane view with the vrouter-evpn-bgp-vni-show command, which additionally shows that the VNIs are transported in the RT values. The command also shows that Netvisor ONE imports and exports all RTs from/to neighbor switches:
CLI (network-admin@bgw21*) > vrouter-evpn-bgp-vni-show sort-asc vni,
switch vrouter-name vni type rd import-rt export-rt
------ ------------ ------ ---- ---------- ------------ ------------
bgw21 bgw21-vr 10110 2 2.4.0.1:15 65012:10110 65012:10110
bgw21 bgw21-vr 10112 2 2.4.0.1:27 65012:10112 65012:10112
bgw21 bgw21-vr 10120 2 2.4.0.1:30 65012:10120 65012:10120
bgw21 bgw21-vr 10121 2 2.4.0.1:35 65012:10121 65012:10121
bgw21 bgw21-vr 10122 2 2.4.0.1:39 65012:10122 65012:10122
bgw21 bgw21-vr 10130 2 2.4.0.1:16 65012:10130 65012:10130
bgw21 bgw21-vr 10131 2 2.4.0.1:17 65012:10131 65012:10131
bgw21 bgw21-vr 10140 2 2.4.0.1:20 65012:10140 65012:10140
bgw21 bgw21-vr 10145 2 2.4.0.1:33 65012:10145 65012:10145
bgw21 bgw21-vr 10146 2 2.4.0.1:13 65012:10146 65012:10146
bgw21 bgw21-vr 10147 2 2.4.0.1:19 65012:10147 65012:10147
bgw21 bgw21-vr 10148 2 2.4.0.1:32 65012:10148 65012:10148
bgw21 bgw21-vr 10149 2 2.4.0.1:12 65012:10149 65012:10149
bgw21 bgw21-vr 10150 2 2.4.0.1:28 65012:10150 65012:10150
Similarly, it is possible to check the Layer 3 table's contents (MAC/IP address pairs) populated by EVPN and then also verify that the low-level entries match. For that you can use the following two commands (in this example, for the sake of brevity, only the entries associated with VLAN 211 and VNI 10211 are shown):
CLI (network-admin@bgw21*) > l3-table-show vlan 211
mac ip vlan vxlan state
----------------- ----------------- ---- ----- -----------
00:11:01:00:00:06 100.0.211.6 211 10211 active,evpn
00:11:01:00:00:02 2000::100:0:211:2 211 10211 active,evpn
00:11:01:00:00:0a 2000::100:0:211:a 211 10211 active,evpn
00:11:01:00:00:04 100.0.211.4 211 10211 active,evpn
00:11:01:00:00:03 2000::100:0:211:3 211 10211 active,evpn
00:11:01:00:00:07 100.0.211.7 211 10211 active,evpn
00:11:01:00:00:05 2000::100:0:211:5 211 10211 active,evpn
00:11:01:00:00:08 2000::100:0:211:8 211 10211 active,evpn
00:11:01:00:00:07 2000::100:0:211:7 211 10211 active,evpn
00:11:01:00:00:09 100.0.211.9 211 10211 active,evpn
00:11:01:00:00:05 100.0.211.5 211 10211 active,evpn
00:11:01:00:00:04 2000::100:0:211:4 211 10211 active,evpn
00:11:01:00:00:08 100.0.211.8 211 10211 active,evpn
00:11:01:00:00:02 100.0.211.2 211 10211 active,evpn
00:11:01:00:00:03 100.0.211.3 211 10211 active,evpn
00:11:01:00:00:0a 100.0.211.10 211 10211 active,evpn
00:11:01:00:00:06 2000::100:0:211:6 211 10211 active,evpn
00:11:01:00:00:09 2000::100:0:211:9 211 10211 active,evpn
The corresponding low-level command is:
CLI (network-admin@bgw21*) > vrouter-evpn-arp-cache-show vni 10211
name vni ip mac remote-vtep type state remote-seq local-seq
-------- ----- ----------------- ----------------- ----------- ------ ------ ---------- ---------
bgw21-vr 10211 2000::100:0:211:8 00:11:01:00:00:08 7.1.1.1 remote active 0 0
bgw21-vr 10211 100.0.211.7 00:11:01:00:00:07 7.1.1.1 remote active 0 0
bgw21-vr 10211 100.0.211.6 00:11:01:00:00:06 7.1.1.1 remote active 0 0
bgw21-vr 10211 2000::100:0:211:9 00:11:01:00:00:09 7.1.1.1 remote active 0 0
bgw21-vr 10211 2000::100:0:211:a 00:11:01:00:00:0a 7.1.1.1 remote active 0 0
bgw21-vr 10211 100.0.211.3 00:11:01:00:00:03 7.1.1.1 remote active 0 0
bgw21-vr 10211 2000::100:0:211:6 00:11:01:00:00:06 7.1.1.1 remote active 0 0
bgw21-vr 10211 100.0.211.2 00:11:01:00:00:02 7.1.1.1 remote active 0 0
bgw21-vr 10211 2000::100:0:211:7 00:11:01:00:00:07 7.1.1.1 remote active 0 0
bgw21-vr 10211 2000::100:0:211:4 00:11:01:00:00:04 7.1.1.1 remote active 0 0
bgw21-vr 10211 100.0.211.10 00:11:01:00:00:0a 7.1.1.1 remote active 0 0
bgw21-vr 10211 100.0.211.8 00:11:01:00:00:08 7.1.1.1 remote active 0 0
bgw21-vr 10211 2000::100:0:211:2 00:11:01:00:00:02 7.1.1.1 remote active 0 0
bgw21-vr 10211 100.0.211.4 00:11:01:00:00:04 7.1.1.1 remote active 0 0
bgw21-vr 10211 100.0.211.5 00:11:01:00:00:05 7.1.1.1 remote active 0 0
bgw21-vr 10211 100.0.211.1 00:11:01:00:00:01 7.1.1.1 remote active 0 0
bgw21-vr 10211 100.0.211.9 00:11:01:00:00:09 7.1.1.1 remote active 0 0
bgw21-vr 10211 2000::100:0:211:3 00:11:01:00:00:03 7.1.1.1 remote active 0 0
bgw21-vr 10211 2000::100:0:211:1 00:11:01:00:00:01 7.1.1.1 remote active 0 0
bgw21-vr 10211 2000::100:0:211:5 00:11:01:00:00:05 7.1.1.1 remote active 0 0
Furthermore, to check the Layer 2 table's contents (e.g., the MAC addresses learned locally and remotely via EVPN) and then also to verify that the low-level entries match, you can use the following two commands (in this example, for the sake of brevity, only the entries associated with VNI 10211 are displayed):
CLI (network-admin@bgw21*) > l2-table-show vlan 211 format mac,vlan,vxlan,ip,state,peer-state,peer-owner-state,status
mac vlan vxlan ip state peer-state peer-owner-state status
----------------- ---- ----- ------------------- ----------- ----------- ---------------- ------
00:11:01:00:00:01 211 10211 2000::100:0:211:1 tunnel,evpn evpn-wo-bgp
00:11:01:00:00:04 211 10211 2000::100:0:211:4 tunnel,evpn
00:12:01:00:00:08 211 10211 2000::100:0:211:107 tunnel tunnel active host
00:11:01:00:00:08 211 10211 2000::100:0:211:8 tunnel,evpn
00:11:01:00:00:07 211 10211 2000::100:0:211:7 tunnel,evpn
00:12:01:00:00:01 211 10211 2000::100:0:211:100 tunnel tunnel active host
00:11:01:00:00:05 211 10211 2000::100:0:211:5 tunnel,evpn
00:12:01:00:00:06 211 10211 2000::100:0:211:105 tunnel tunnel active host
00:11:01:00:00:03 211 10211 2000::100:0:211:3 tunnel,evpn
00:11:01:00:00:02 211 10211 2000::100:0:211:2 tunnel,evpn
00:12:01:00:00:02 211 10211 2000::100:0:211:101 tunnel tunnel active host
00:11:01:00:00:06 211 10211 2000::100:0:211:6 tunnel,evpn
00:12:01:00:00:07 211 10211 2000::100:0:211:106 tunnel tunnel active host
00:12:01:00:00:05 211 10211 2000::100:0:211:104 tunnel tunnel active host
00:12:01:00:00:0a 211 10211 2000::100:0:211:109 tunnel tunnel active host
00:11:01:00:00:0a 211 10211 100.0.211.10 tunnel,evpn
00:12:01:00:00:03 211 10211 2000::100:0:211:102 tunnel tunnel active host
00:12:01:00:00:04 211 10211 2000::100:0:211:103 tunnel tunnel active host
00:11:01:00:00:09 211 10211 2000::100:0:211:9 tunnel,evpn
00:12:01:00:00:09 211 10211 2000::100:0:211:108 tunnel tunnel active host
In the above command output it's possible to look at the state of a MAC address entry to check if it's learned from a local tunnel and/or from the evpn control plane. When the peer-state is evpn-wo-bgp, it means that the entry was learned (initially) directly from the data plane.
The corresponding low-level command to check the Layer 2 entries is:
CLI (network-admin@bgw21*) > vrouter-evpn-vni-mac-show vni 10211
vrouter-name vni mac type intf/remote-vtep vlan local-seq remote-seq
------------ ----- ----------------- ------ ---------------- ---- --------- ----------
bgw21-vr 10211 00:12:01:00:00:02 local 9000becvni10211 0 0 0
bgw21-vr 10211 00:12:01:00:00:0a local 9000becvni10211 0 0 0
bgw21-vr 10211 00:11:01:00:00:05 remote 7.1.1.1 0 0 0
bgw21-vr 10211 00:11:01:00:00:0a remote 7.1.1.1 0 0 0
bgw21-vr 10211 00:12:01:00:00:07 local 9000becvni10211 0 0 0
bgw21-vr 10211 00:11:01:00:00:02 remote 7.1.1.1 0 0 0
bgw21-vr 10211 00:12:01:00:00:09 local 9000becvni10211 0 0 0
bgw21-vr 10211 00:11:01:00:00:07 remote 7.1.1.1 0 0 0
bgw21-vr 10211 00:11:01:00:00:04 remote 7.1.1.1 0 0 0
bgw21-vr 10211 00:12:01:00:00:03 local 9000becvni10211 0 0 0
bgw21-vr 10211 00:11:01:00:00:08 remote 7.1.1.1 0 0 0
bgw21-vr 10211 00:12:01:00:00:08 local 9000becvni10211 0 0 0
bgw21-vr 10211 00:11:01:00:00:01 remote 7.1.1.1 0 0 0
bgw21-vr 10211 00:11:01:00:00:03 remote 7.1.1.1 0 0 0
bgw21-vr 10211 00:12:01:00:00:06 local 9000becvni10211 0 0 0
bgw21-vr 10211 00:12:01:00:00:01 local 9000becvni10211 0 0 0
bgw21-vr 10211 00:11:01:00:00:09 remote 7.1.1.1 0 0 0
bgw21-vr 10211 00:12:01:00:00:05 local 9000becvni10211 0 0 0
bgw21-vr 10211 00:12:01:00:00:04 local 9000becvni10211 0 0 0
bgw21-vr 10211 00:11:01:00:00:06 remote 7.1.1.1 0 0 0
Displaying EVPN Route History Information
Displaying EVPN route history information is useful to verify the inter-pod information exchanged over type 2, 3 and 5 messages and for forensic tracing.
To display the type 2 route history, you can use the vport-history-show command. For example, to display the injected (i.e., created/added) vPorts, you can use:
CLI (network-admin@bgw21) > vport-history-show caller inject, within-last 2d
time log-type reason owner mac vxlan state vtep-ip
-------------- --------- ------ ----- ----------------- ----- ----- ---------
01-18,06:33:40 l2-modify create bgw21 66:0e:94:3f:64:4a 40400 evpn 20.0.12.1
Or you can use this command to show all the latest actions, for example limited to a certain VNI:
CLI (network-admin@bgw21*) > vport-history-show within-last 2d vxlan 100200 state evpn-wo-bgp,
time log-type reason owner mac vlan vxlan state hostname status
-------------- --------- ------ ----- ----------------- ---- ------ ------------------ -------- -----------
01-18,06:23:37 l2-modify modify bgw21 20:04:0f:12:95:62 200 100200 tunnel,evpn-wo-bgp bgw21 PN-internal
01-18,06:23:52 l2-modify modify bgw21 50:9a:4c:d3:fd:f0 200 100200 tunnel,evpn-wo-bgp bgw21 PN-internal
01-18,06:29:34 l2-modify flush bgw21 20:04:0f:12:95:62 200 100200 evpn-wo-bgp bgw21 PN-internal
01-18,06:29:34 l2-modify flush bgw21 50:9a:4c:d3:fd:f0 200 100200 evpn-wo-bgp bgw21 PN-internal
You can display the type 5 route history with the following command, for example filtered for a specific address:
CLI (network-admin@bgw21) > vrouter-rib-history-show within-last 2d flags evpn, ip 200.1.1.0
time caller reason vrid ip prelen nexthop flags vlan intf-ip
-------------- ------ ------ ---- --------- ------ ---------- ---------- ---- --------
01-17,17:12:12 vrf add 1 200.1.1.0 24 190.12.1.1 in-hw,evpn 3434 33.3.3.3
01-17,20:17:32 vrf add 1 200.1.1.0 24 190.12.1.1 in-hw,evpn 3434 33.3.3.3
01-17,20:26:39 vrf add 1 200.1.1.0 24 190.12.1.1 in-hw,evpn 3434 33.3.3.3
01-17,20:56:09 vrf add 1 200.1.1.0 24 190.12.1.1 in-hw,evpn 3434 33.3.3.3
01-17,21:09:06 vrf add 1 200.1.1.0 24 190.12.1.1 in-hw,evpn 3434 33.3.3.3
01-17,21:23:45 vrf add 1 200.1.1.0 24 190.12.1.1 in-hw,evpn 3434 33.3.3.3
01-17,21:32:43 vrf add 1 200.1.1.0 24 190.12.1.1 in-hw,evpn 3434 33.3.3.3
01-17,21:36:37 vrf add 1 200.1.1.0 24 190.12.1.1 in-hw,evpn 3434 33.3.3.3
01-17,21:42:27 vrf add 1 200.1.1.0 24 190.12.1.1 in-hw,evpn 3434 33.3.3.3
01-17,22:50:22 vrf add 1 200.1.1.0 24 190.12.1.1 in-hw,evpn 3434 33.3.3.3
01-17,23:32:06 vrf add 1 200.1.1.0 24 190.12.1.1 in-hw,evpn 3434 33.3.3.3
01-17,23:32:07 vrf add 1 200.1.1.0 24 190.12.1.1 in-hw,evpn 3434 33.3.3.3
01-17,23:35:48 vrf add 1 200.1.1.0 24 190.12.1.1 in-hw,evpn 3434 33.3.3.3
Furthermore, to see the type 3 message-triggered VTEP changes you can filter the audit log with grep, for example like so:
CLI (network-admin@bgw21*) > log-audit-show name user_command format message, | grep evpn
message
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-----------------------------------------------------------------------
Command "vtep-create scope local vtep-source evpn name __evpn__190.12.1.1 location 16777214 ip 190.12.1.1" result success
Command "vtep-vxlan-add name __evpn__190.12.1.1 vxlan 10400" result success
<snip>