Configuring vRouter-based VRF (Virtual Routing and Forwarding)
Virtual Routing and Forwarding (VRF) is a technology used to partition the routing table into virtual instances (called VRF instances, or simply VRFs) that segregate routing entries in the control plane as well as in the data plane. Because the routing instances are independent, the same or overlapping IP addresses can be used without conflicting with each other.
Before NetVisor OS release 6.1.1, in order to support multiple VRF instances you needed to configure multiple vRouters to create isolated Layer 3 routing contexts in software. By using this strategy, you could create a ‘dumb’ 1:1 association between a vRouter and a VRF instance used to isolate Layer 3 domains in the data plane. Unicast and Multicast Fabric VRFs (see the Configuring VXLAN section for more details) were introduced as hardware entities for high-performance distributed routing and traffic segmentation. There was no corresponding VRF-aware vRouter entity associated to them and therefore no routing protocols could be run to exchange routing information (only static routing was supported).
NetVisor OS release 6.1.1 introduces ‘native’ multi-VRF support to vRouters by making use of new advanced control plane capabilities.
The new implementation is more scalable compared to previous releases as it is much less demanding in terms of memory usage and CPU load (note that the previous vRouter-per-VRF approach is basically limited by the maximum number of supported vRouters, which is switch model-dependent). In addition, since a vRouter can now run routing protocols that are VRF-aware, per VRF peering is supported with BGP.
The new implementation also allows the same interface IP address to be used across different VRF instances. (However, note that reusing the same VLAN number for two interfaces of the same vRouter is not supported.) From a multi-tenant management perspective, VRF instances can be allocated to different vNETs so that each vNET administrator can independently add or remove the VRFs from the vRouters.
The configuration of vRouter-based VRF instances leverages the existing vRouter CLI, which is augmented to include the vrf parameter in many of the existing commands (see the list below).
In addition, a key new command, vrouter-vrf-add, is added to associate VRF instances to vRouters. Corresponding vrouter-vrf-modify, vrouter-vrf-remove and vrouter-vrf-show commands are introduced too. (Note that vrouter-vrf-remove removes a VRF instance as well as all vRouter interfaces belonging to it.)
For example, you can create a vRouter and associate two VRFs to it by using the following commands:
CLI (network-admin@switch) > vrf-create name VRF1 scope fabric
CLI (network-admin@switch) > vrf-create name VRF2 scope fabric
Note: The maximum VRF name length is 15 characters in NetVisor OS.
CLI (network-admin@switch) > vrouter-create name vRouter1 vnet vNET1 router-type hardware bgp-as 100
CLI (network-admin@switch) > vrouter-vrf-add vrouter-name vRouter1 vrf VRF1
CLI (network-admin@switch) > vrouter-vrf-add vrouter-name vRouter1 vrf VRF2
Note : The same VRF cannot be added to more than one vRouter on the same node.
The next step is to configure Layer 3 interfaces and add them to the specific VRFs, for example:
CLI (network-admin@switch) > vrouter-interface-add vrouter-name vRouter1 vrf VRF1 ip 100.1.1.1/24 vlan 100
CLI (network-admin@switch) > vrouter-interface-add vrouter-name vRouter1 vrf VRF2 ip 100.1.1.1/24 vlan 101
Then you can specify the BGP neighbors on a per VRF basis with these commands:
CLI (network-admin@switch) > vrouter-bgp-add vrouter-name vRouter1 vrf VRF1 neighbor 100.1.1.2 remote-as 100
CLI (network-admin@switch) > vrouter-bgp-add vrouter-name vRouter1 vrf VRF2 neighbor 100.1.1.2 remote-as 101
Per-VRF BGP Parameters
Starting from NetVisor OS release 6.1.1 the addition of the vrf keyword makes the routing parameters VRF-aware. Some BGP parameters (such as bgp-as, bgp-redistribute, etc.) can now be specified during the vrouter-vrf-add configuration like so:
CLI (network-admin@switch) > vrouter-vrf-add vrouter-name <name> vrf <vrf-name> [bgp-as|bgp-redistribute|...]
(If a parameter is not specified, its value is inherited from the vRouter configuration.)
Subsequently, a parameter can be modified using the vrouter-vrf-modify command:
CLI (network-admin@switch) > vrouter-vrf-modify vrouter-name <name> vrf <vrf-name> [bgp-as|bgp-redistribute|...]
For example, it is possible to configure and modify a router ID on a per-VRF basis with the vrouter-vrf-add and vrouter-vrf-modify commands like so:
CLI (network-admin@switch) > vrouter-vrf-add vrouter-name vRouter1 vrf VRF1 router-id 11.1.1.1
CLI (network-admin@switch) > vrouter-vrf-show format vrf,hw-vrid,bgp-as,router-id,
vrouter-name vrf hw-vrid bgp-as router-id
------------ ---- ------- ------ ---------
vRouter1 VRF1 3 100 11.1.1.1
CLI (network-admin@switch) > vrouter-vrf-modify vrouter-name vRouter1 vrf VRF1 router-id 11.1.1.10
CLI (network-admin@switch) > vrouter-vrf-show format vrf,hw-vrid,bgp-as,router-id,
vrouter-name vrf hw-vrid bgp-as router-id
------------ ---- ------- ------ ---------
vRouter1 VRF1 3 100 11.1.1.10
BGP uses the per-VRF router-id parameter value when it is configured. If it’s not, the operational per-VRF router ID value is inherited from the corresponding vRouter’s (global) router ID.
This is the full list of supported per-VRF parameters:
|
bgp-as |
BGP Autonomous System number from 1 to 4294967295 |
|
router-id |
BGP router id |
|
bgp-redistribute |
BGP route redistribution |
|
bgp-redist-static-metric |
BGP route redistribution static metric |
|
bgp-redist-static-route-map |
Route map for BGP redistribution of static routes |
|
bgp-redist-connected-metric |
Metric for redistributing BGP connected routes |
|
bgp-redist-connected-route-map |
Route map for BGP redistribution of connected routes |
|
bgp-redist-rip-metric |
Metric for redistributing RIP connected routes |
|
bgp-redist-ospf-metric |
Metric for BGP to redistribute OSPF connected routes |
|
bgp-redist-ospf-route-map |
Route map for BGP redistribution of OSPF routes |
|
bgp-cluster-id |
IP address for BGP cluster ID |
|
no-bgp-dampening |
No dampening for BGP routes |
|
bgp-dampening |
dampening is active for BGP routes |
|
bgp-keepalive-interval |
BGP Keepalive interval (seconds) - default 60 |
|
bgp-holdtime |
BGP Holdtime (seconds) - default 180 |
|
bgp-distance-external |
BGP distance for routes external to AS |
|
bgp-distance-internal |
BGP distance for routes internal to AS |
|
bgp-distance-local |
BGP distance for local routes |
|
no-bgp-default-shutdown |
Disabled |
|
bgp-default-shutdown |
Enabled |
|
no-bgp-redist-static-route-map |
Remove BGP static redist route-map |
|
no-bgp-redist-connected-route-map |
Remove BGP connect redist route-map |
|
no-bgp-redist-ospf-route-map |
Remove BGP OSPF redist route-map |
|
no-bgp-graceful-shutdown |
BGP graceful shutdown RFC 8326 |
|
bgp-graceful-shutdown |
BGP graceful shutdown RFC 8326 |
The BGP parameters below, instead, are inherited from the vRouter (in other words, for them per VRF configuration is not supported):
bgp-delayed-startup
bgp-update-delayed-strict
bgp-max-paths
bgp-ibgp-multipath
bgp-bestpat-as-path
bgp-global-nh-preference
VRF-aware BFD Support
VRF support was added to BFD for both static routing and dynamic routing with BGP. The following vRouter commands are VRF aware. For static routing:
CLI (network-admin@switch) > vrouter-static-bfd-show
vrouter-name src-ip dst-ip type vrf
------------ --------- --------- ---------- ----
vRouter1 100.1.1.1 100.1.1.2 single-hop VRF1
vRouter2 100.1.1.2 100.1.1.1 single-hop VRF2
CLI (network-admin@switch) > vrouter-bfd-neighbor-show format out-addr,neighbor,holdown,multiplier,state,interface,vrf,flap-count,remote-router
vrouter-name out-addr neighbor holdown(ms) multiplier state interface vrf flap-count remote-router
------------ --------- --------- ----------- ---------- ----- --------- ---- ---------- -------------
vRouter1 100.1.1.1 100.1.1.2 2191 3 up eth0.100 VRF1 0 vr1
vRouter2 100.1.1.2 100.1.1.1 2230 3 up eth1.100 VRF2 0 vr2
For BGP
CLI (network-admin@switch) > vrouter-bgp-neighbor-show format neighbor,l3-port,nic,ver,remote-as,up/down,state/pfxrcd,remote-router,description,vrf
vrouter-name neighbor l3-port nic ver remote-as up/down state/pfxrcd remote-router description vrf
------------ --------- ------- --- --- --------- -------- ------------ ------------- ----------- ----
vRouter1 100.1.1.2 0 4 200 00:00:18 Established vr1 VRF1
vRouter2 100.1.1.1 0 4 100 00:00:19 Established vr2 VRF2
CLI (network-admin@switch) > vrouter-bfd-neighbor-show format out-addr,neighbor,holdown,multiplier,state,interface,vrf,flap-count,remote-router
vrouter-name out-addr neighbor holdown(ms) multiplier state interface vrf flap-count remote-router
------------ --------- --------- ----------- ---------- ----- --------- ---- ---------- -------------
vRouter1 100.1.1.1 100.1.1.2 2118 3 up eth0.100 VRF1 0 vr1
vRouter2 100.1.1.2 100.1.1.1 2024 3 up eth1.100 VRF2 0 vr2
VRF-aware Commands
The full list of vRouter commands with the new vrf keyword is:
- vrouter-bfd-neighbor-show
- vrouter-bgp-add
- vrouter-bgp-modify
- vrouter-bgp-neighbor-detail-show
- vrouter-bgp-neighbor-no-shutdown
- vrouter-bgp-neighbor-reset
- vrouter-bgp-neighbor-show
- vrouter-bgp-neighbor-shutdown
- vrouter-bgp-network-add
- vrouter-bgp-network-remove
- vrouter-bgp-network-show
- vrouter-bgp-remove
- vrouter-bgp-show
- vrouter-cached-routes-show
- vrouter-ecmp-cached-routes-show
- vrouter-ecmp-group-show
- vrouter-fib-arps-show
- vrouter-fib-routes-show
- vrouter-interface-add
- vrouter-interface-modify
- vrouter-interface-show
- vrouter-ping
- vrouter-routes-show
- vrouter-routes-stats-show
- vrouter-static-bfd-add
- vrouter-static-bfd-remove
- vrouter-static-bfd-show
- vrouter-static-route-add
- vrouter-static-route-remove
- vrouter-static-route-show
- vrouter-traceroute
Below are a few examples of VRF-aware commands:
CLI (network-admin@switch) > vrf-show format switch,name,scope,anycast-mac,active,hw-vrid,flags,enable
switch name scope anycast-mac active hw-vrid flags enable
------ ----- ------- ----------------- ------ ------- ------- ------
switch vrf1 cluster 64:0e:94:40:00:02 no 1 vrouter yes
switch vrf10 cluster 64:0e:94:40:00:02 no 10 vrouter yes
switch vrf11 cluster 64:0e:94:40:00:02 no 11 vrouter yes
switch vrf12 cluster 64:0e:94:40:00:02 no 12 vrouter yes
switch vrf13 cluster 64:0e:94:40:00:02 no 13 vrouter yes
switch vrf14 cluster 64:0e:94:40:00:02 no 14 vrouter yes
switch vrf15 cluster 64:0e:94:40:00:02 no 15 vrouter yes
switch vrf2 cluster 64:0e:94:40:00:02 no 2 vrouter yes
switch vrf3 cluster 64:0e:94:40:00:02 no 3 vrouter yes
switch vrf4 cluster 64:0e:94:40:00:02 no 4 vrouter yes
switch vrf5 cluster 64:0e:94:40:00:02 no 5 vrouter yes
switch vrf6 cluster 64:0e:94:40:00:02 no 6 vrouter yes
switch vrf7 cluster 64:0e:94:40:00:02 no 7 vrouter yes
switch vrf8 cluster 64:0e:94:40:00:02 no 8 vrouter yes
switch vrf9 cluster 64:0e:94:40:00:02 no 9 vrouter yes
CLI (network-admin@switch) > vrouter-vrf-show vrouter-name leaf1_vr1 format vrf,hw-vrid,bgp-as,bgp-max-paths,bgp-bestpath-as-path,bgp-ibgp-multipath
vrouter-name vrf hw-vrid bgp-as bgp-max-paths bgp-bestpath-as-path bgp-ibgp-multipath
------------ ----- ------- ------ ------------- -------------------- ------------------
leaf1_vr1 vrf1 1 12101 16 multipath-relax 16
leaf1_vr1 vrf2 2 12102 16 multipath-relax 16
leaf1_vr1 vrf3 3 12103 16 multipath-relax 16
leaf1_vr1 vrf4 4 12104 16 multipath-relax 16
leaf1_vr1 vrf5 5 12100 16 multipath-relax 16
leaf1_vr1 vrf6 6 12106 16 multipath-relax 16
leaf1_vr1 vrf7 7 12107 16 multipath-relax 16
leaf1_vr1 vrf8 8 12108 16 multipath-relax 16
leaf1_vr1 vrf9 9 12109 16 multipath-relax 16
leaf1_vr1 vrf10 10 12110 16 multipath-relax 16
leaf1_vr1 vrf11 11 12111 16 multipath-relax 16
leaf1_vr1 vrf12 12 12112 16 multipath-relax 16
leaf1_vr1 vrf13 13 12113 16 multipath-relax 16
leaf1_vr1 vrf14 14 12114 16 multipath-relax 16
leaf1_vr1 vrf15 15 12115 16 multipath-relax 16
CLI (network-admin@switch) > vrouter-interface-show vrouter-name leaf1_vr1 vrf vrf1 format nic,ip,mac,vlan,vlan-type,nic-state,vrf
vrouter-name nic ip mac vlan vlan-type nic-state vrf
------------ --------- ------------- ----------------- ---- --------- --------- -----
leaf1_vr1 eth0.110 11.1.0.2/24 66:0e:94:1b:c5:21 110 public up vrf1
leaf1_vr1 eth1.110 11.1.0.1/24 00:00:5e:00:01:79 110 public down vrf1
leaf1_vr1 eth0.111 11.1.1.2/24 66:0e:94:1b:c5:21 111 public up vrf1
leaf1_vr1 eth1.111 11.1.1.1/24 00:00:5e:00:01:79 111 public down vrf1
leaf1_vr1 eth0.112 11.1.2.2/24 66:0e:94:1b:c5:21 112 public up vrf1
leaf1_vr1 eth1.112 11.1.2.1/24 00:00:5e:00:01:79 112 public down vrf1
leaf1_vr1 eth0.113 11.1.3.2/24 66:0e:94:1b:c5:21 113 public up vrf1
leaf1_vr1 eth1.113 11.1.3.1/24 00:00:5e:00:01:79 113 public down vrf1
leaf1_vr1 eth0.114 11.1.4.2/24 66:0e:94:1b:c5:21 114 public up vrf1
leaf1_vr1 eth1.114 11.1.4.1/24 00:00:5e:00:01:79 114 public down vrf1
leaf1_vr1 eth2.3001 11.1.254.3/29 66:0e:94:1b:c5:21 3001 public up vrf1
CLI (network-admin@switch) > vrouter-bgp-neighbor-show vrouter-name leaf1_vr1 remote-router leaf2_vr1 vrf vrf1 format neighbor,remote-as,msg_rcvd,msg_sent,up/down,remote-router,vrf
vrouter-name neighbor remote-as msg_rcvd msg_sent up/down remote-router vrf
------------ ---------- --------- -------- -------- -------- ------------- -----
leaf1_vr1 11.1.254.4 12101 7133 7147 4d22h42m leaf2_vr1 vrf1
CLI (network-admin@switch) > vrouter-bfd-neighbor-show vrouter-name leaf1_vr1 remote-router leaf2_vr1 vrf vrf1 format out-addr,neighbor,state,interface,vrf,flap-count,remote-router
vrouter-name out-addr neighbor state interface vrf flap-count remote-router
------------ ---------- ---------- ----- --------- -------- ---------- -------------
leaf1_vr1 11.1.254.3 11.1.254.4 up eth2.3001 vrf1 0 leaf2_vr1