Configuring a Network using Pluribus ML2 Plugin



Note: As a network administrator, you must first provision and configure the Pluribus switch fabric. Use OpenStack to configure the Pluribus switch ports so that the VMs in the compute node sends east-west traffic through the Pluribus switch fabric.



Note: For details on the OpenStack configuration commands, see the OpenStack documentation from the Red Hat website or other vendor specific websites.


You can access the ML2 plugin and configure a VLAN network by using two methods: (1) using the OpenStack CLI commands and/or (2) using the Horizon Dashboard.


To configure a network:



Note: Enable web server on the Pluribus switches (the two seed nodes: primary and secondary)  specified in the ml2_conf.ini file. Use the admin-service-show command on the Pluribus switch to verify if the web server is enabled or not.


  1. Create the OpenStack network by using the OpenStack CLI command. For example, use the openstack network create <name>  --provider-network-type vlan <segment id>  command.


This command creates a VLAN network on the Pluribus switch and associates a VXLAN ID to the VLAN network. The VLAN ID is fetched from the OpenStack Neutron server and the corresponding VXLAN ID will be <11million + VLAN ID>. Then, the VXLAN network identifier (VNI) gets added to all the VXLAN Tunnel Endpoint (VTEPs), if configured on the switch fabric, or is added to all static tunnels. All the cluster ports also get added to the VLAN network.


  1. Launch the VM instances by using the OpenStack CLI command, openstack server create. For example, openstack server create --image cirros --flavor m1.tiny --network test-network MyInstance.


Table - 1  Mapping actions of Pluribus ML2 plugin OpenStack commands on the Pluribus switch fabric:


OpenStack Commands/Actions

Mapping action on Pluribus switches

openstack network create

Creates a VLAN network with scope cluster on the Pluribus switch fabric and associates the VXLAN network ID (VNI)  with the VLAN network, if configured.

openstack network delete

Deletes the previously configured VLAN network from the switch fabric.

openstack subnet create

The ML2 plugin receives a port create/update request along with the hostname of the controller. The plugin fetches all the controller facing ports on the fabric and adds them to VLAN network.


For example, to create a subnet 24, use the command:


[root@server-os-4 ~(keystone_admin)]# openstack subnet create --subnet-range 192.10.10.0/24 --network scorpius subnet-1.


Use the Netvisor ONE command, lldp-show  on the  switch to verify the created/updated port  details that got added to the VLAN network.


See the Related Netvisor ONE CLI commands section to view show output details.

openstack server create


Note: Ensure to create the subnets before you execute the server create command.


When a VM instance is launched, the Pluribus ML2 plugin receives port update request along with the binding host or the compute node details on which the VM is being spun. The ML2 plugin automatically detects all the host facing ports by using the LLDP information on the switch fabric and adds them to the VLAN network.

openstack server delete

When a VM instance is deleted, the ML2 plugin receives the port delete request and decides whether to remove switch ports from VLAN or not because multiple VM instances co-exist on the same compute node and in the same network. The host facing ports are removed from the VLAN only if it is the last instance that is being deleted.

Alternatively, use the following commands to manually add or delete ports later.

openstack port create


Note: To add a host facing port to the VLAN network,  use either of the two commands:  openstack server create or openstack port create.


When a port create/update request is received along with the binding host (compute node) information, the Pluribus ML2 plugin automatically detects all the host facing ports on the fabric and adds them to the VLAN received.


This is an optional command provided,  if you want to configure the VLAN network for a particular host.


For example, openstack port create test-port --host linux-host1 --network test-network.


This command adds the correct switch ports to previously created VLAN for the compute node linux-host1. The ML2 plugin queries all the ports connected to the host linux-host1 by using the LLDP details and adds them to the VLAN network.


openstack port delete

Removes all the host facing ports from the VLAN if that is the last port being deleted as explained in openstack server delete command description.


For example, the openstack port delete test-port command removes the port from the VLAN with which it was associated earlier.


Further to the OpenStack commands described in the table; below is an example of a sample configuration for creating a VLAN network on the OpenStack controller by using the ML2 plugin:


  • Create an OpenStack network, scorpius with type, vlan and ID, 259 on the controller by using the OpenStack commands. The plugin programs VLAN 259 (example here) on all the cluster pair switches in the fabric.


[root@server-os-4 ~(keystone_admin)]# openstack network create scorpius --provider-network-type vlan --provider-physical-network physnet1 --provider-segment 259


+---------------------------+--------------------------------------+

| Field                     | Value                                |

+---------------------------+--------------------------------------+

| admin_state_up            | UP                                   |

| availability_zone_hints   |                                      |

| availability_zones        |                                      |

| created_at                | 2020-03-02T06:12:34Z                 |

| description               |                                      |

| dns_domain                | None                                 |

| id                        | cd2ee636-50ce-4800-8e46-8f1d9625b54c |

| ipv4_address_scope        | None                                 |

| ipv6_address_scope        | None                                 |

| is_default                | False                                |

| is_vlan_transparent       | None                                 |

| mtu                       | 1500                                 |

| name                      | scorpius                             |

| port_security_enabled     | True                                 |

| project_id                | df11960a3ec14733849e4b8dd83cf484     |

| provider:network_type     | vlan                                 |

| provider:physical_network | physnet1                             |

| provider:segmentation_id  | 259                                  |

| qos_policy_id             | None                                 |

| revision_number           | 2                                    |

| router:external           | Internal                             |

| segments                  | None                                 |

| shared                    | False                                |

| status                    | ACTIVE                               |

| subnets                   |                                      |

| tags                      |                                      |

| updated_at                | 2020-03-02T06:12:34Z                 |

+---------------------------+--------------------------------------+

[root@server-os-4 ~(keystone_admin)]#


  • On the Pluribus switch, verify if a cluster scoped VLAN network is created and if  the network has all the cluster scope ports within the VLAN by using the Netvisor ONE CLI command:


CLI (network-admin@sw-1) > vlan-show id 259


switch id  type    vxlan  auto-vxlan  scope   description    active  ports   untagged-ports active-edge-ports

------ --- ------ ------- ---------- ------- --------------- ------ ----------- -------------- ---------------

sw-1  259 public 11000259   no      cluster  ostack-vlan-259  yes   0,27-30,272  none           none

sw1-1 259 public 11000259   no      cluster  ostack-vlan-259  yes   23-26,272    none           none

sw1-2 259 public 11000259   no      cluster  ostack-vlan-259  yes   23-26,272    none           none

sw-2  259 public 11000259   no      cluster  ostack-vlan-259  yes   0,27-30,272  none           none



Note: The VXLAN in the show output is of the range <11 million+VLAN ID>.


  • Assign a Subnet, 192.10.10.0/24 to the previously created network, Scorpius on the controller:


[root@server-os-4 ~(keystone_admin)]# openstack subnet create --subnet-range 192.10.10.0/24 --network scorpius subnet-1

+-------------------+--------------------------------------+

| Field             | Value                                |

+-------------------+--------------------------------------+

| allocation_pools  | 192.168.76.2-192.168.76.254          |

| cidr              | 192.168.76.0/24                      |

| created_at        | 2020-03-02T06:16:04Z                 |

| description       |                                      |

| dns_nameservers   |                                      |

| enable_dhcp       | True                                 |

| gateway_ip        | 192.168.76.1                         |

| host_routes       |                                      |

| id                | eb932f42-4184-4411-99f5-16c3f33cfbf0 |

| ip_version        | 4                                    |

| ipv6_address_mode | None                                 |

| ipv6_ra_mode      | None                                 |

| name              | subnet-1                             |

| network_id        | cd2ee636-50ce-4800-8e46-8f1d9625b54c |

| prefix_length     | None                                 |

| project_id        | df11960a3ec14733849e4b8dd83cf484     |

| revision_number   | 0                                    |

| segment_id        | None                                 |

| service_types     |                                      |

| subnetpool_id     | None                                 |

| tags              |                                      |

| updated_at        | 2020-03-02T06:16:04Z                 |

+-------------------+--------------------------------------+

[root@server-os-4 ~(keystone_admin)]


  • Verify the subnet list by using the command:


[root@server-os-4 ~(keystone_admin)]# openstack subnet list

+--------------------------------------+----------+--------------------------------------+-----------------+

| ID                                   | Name     | Network                              | Subnet          |

+--------------------------------------+----------+--------------------------------------+-----------------+

| eb932f42-4184-4411-99f5-16c3f33cfbf0 | subnet-1 | cd2ee636-50ce-4800-8e46-8f1d9625b54c | 192.168.76.0/24 |

+--------------------------------------+----------+--------------------------------------+-----------------+

[root@server-os-4 ~(keystone_admin)]#


  • Create a VM instance on server-os-9 compute node:


[root@server-os-4 ~(keystone_admin)]# openstack server create --image cirros --flavor m1.tiny --availability-zone nova:server-os-9.pluribusnetworks.com   --nic net-id=cd2ee636-50ce-4800-8e46-8f1d9625b54c vm-1

+-------------------------------------+-----------------------------------------------+

| Field                               | Value                                         |

+-------------------------------------+-----------------------------------------------+

| OS-DCF:diskConfig                   | MANUAL                                        |

| OS-EXT-AZ:availability_zone         | nova                                          |

| OS-EXT-SRV-ATTR:host                | None                                          |

| OS-EXT-SRV-ATTR:hypervisor_hostname | None                                          |

| OS-EXT-SRV-ATTR:instance_name       |                                               |

| OS-EXT-STS:power_state              | NOSTATE                                       |

| OS-EXT-STS:task_state               | scheduling                                    |

| OS-EXT-STS:vm_state                 | building                                      |

| OS-SRV-USG:launched_at              | None                                          |

| OS-SRV-USG:terminated_at            | None                                          |

| accessIPv4                          |                                               |

| accessIPv6                          |                                               |

| addresses                           |                                               |

| adminPass                           | yR7hki8eH9Di                                  |

| config_drive                        |                                               |

| created                             | 2020-03-10T22:44:11Z                          |

| flavor                              | m1.tiny (1)                                   |

| hostId                              |                                               |

| id                                  | 32b54fc5-b6bb-4778-8e19-48ed28e1e4a6          |

| image                               | cirros (ddac232d-8cc2-45f2-91b8-2628f5c17697) |

| key_name                            | None                                          |

| name                                | vm-1                                          |

| progress                            | 0                                             |

| project_id                          | df11960a3ec14733849e4b8dd83cf484              |

| properties                          |                                               |

| security_groups                     | name='default'                                |

| status                              | BUILD                                         |

| updated                             | 2020-03-10T22:44:11Z                          |

| user_id                             | 93b2ce819c0347a9aa6ecf7ab5c4ba28              |

| volumes_attached                    |                                               |

+-------------------------------------+-----------------------------------------------+

[root@server-os-4 ~(keystone_admin)]#


  • You can use the openstack network delete command to delete the previously created network or openstack port delete command to delete the ports.


After the above configurations are done, verify the details by  using the Netvisor ONE  command,  vlan-show id 259:


CLI (network-admin@aq-os-2) > vlan-show id 259


switch   id  type   vxlan   replicators scope   description    active stats ports              untagged-ports active-edge-ports

-------- --- ------ -------- ---------- ------  -------        ------- ---- -----------       --------------- --------------

aq-os-2  259 public 11000259  none      cluster ostack-vlan-259 yes    no    0-2,27-30,272-273    none           273

leo-os-1 259 public 11000259  none      cluster ostack-vlan-259 yes    no    23-26,272            none           none

leo-os-2 259 public 11000259  none      cluster ostack-vlan-259 yes    no    23-26,272            none           none

aq-os-1  259 public 11000259  none      cluster ostack-vlan-259 yes    no    0-2,11,27-30,272-273 none           273


Note that  in the above output, the VXLAN IDs corresponds to <11million+VLAN ID> and also the ports are added to the VLAN 259.


For more details on verifying the configurations using the corresponding Netvisor ONE CLI commands, see the Related Netvisor ONE CLI commands section.