docs/doc/source/node_management/kubernetes/node_interfaces/provisioning-sr-iov-interfaces-using-the-cli.rst

Provision SR-IOV Interfaces using the CLI

interfaces must be provisioned on at least one host before using network attachments in a container.

Note

For the purpose of enabling and using needVhostNet, needs to be configured on a worker host.

By default, a Kubernetes container is started with a default network virtual interface for cluster networking. For accelerated networking between containers or external networks, additional backed interfaces (network attachments) can be added to the container.

You can use the CLI to provision interfaces on a system.

The device plugin discovers and advertises network virtual functions (VFs) in a Kubernetes host. To enable the device plugin, all hosts on which accelerated networking pods will be enabled should have the sriovdp label applied.

The device plugin can also expose the /dev/vhost-net device to a container by using the needVhostNet parameter. To enable it, apply the sriovdp-vhostnet label.

For information about creating and using network attachments, see : Create Network Attachment Definitions <creating-network-attachment-definitions> and : Use Network Attachment Definitions in a Container <using-network-attachment-definitions-in-a-container>.

You must create data networks before you can provision the interfaces. See : Add Data Networks using the CLI <adding-data-networks-using-the-cli>.

1. Lock the host to which you will assign the label.

   Note

   hosts do not need to be locked to provision an interface and assign it to a data network.

   For example:

   ~(keystone_admin)$ system host-lock compute-0

2. Use the host-label-assign command to assign the sriovdp label to the node.

   For example, to set sriovdp on worker-0 you would do the following:

   ~(keystone_admin)$ system host-label-assign compute-0 sriovdp=enabled
   +-------------+--------------------------------------+
   | Property    | Value                                |
   +-------------+--------------------------------------+
   | uuid        | a2c5d21b-f91a-4b8b-8dbc-d40b2f3bdaa9 |
   | host_uuid   | 772df330-6a42-4b8e-9a18-ae9a9f3f2336 |
   | label_key   | sriovdp                              |
   | label_value | enabled                              |
   +-------------+--------------------------------------+

3. To expose the vhost-net device to a container, use the host-label-assign command to assign the sriovdp-vhostnet label to the node.

   For example, to set sriovdp-vhostnet on worker-0:

   [sysadmin@controller-0 ~(keystone_admin)]$ system host-label-assign compute-0 sriovdp-vhostnet=enabled
   +-------------+--------------------------------------+
   | Property    | Value                                |
   +-------------+--------------------------------------+
   | uuid        | d2eff38a-889d-46e2-ae4f-43acb1c957a9 |
   | host_uuid   | 8c28fe10-8849-4597-8a5d-89a97ea8febf |
   | label_key   | sriovdp-vhostnet                     |
   | label_value | enabled                              |
   +-------------+--------------------------------------+

4. Identify the interfaces to be configured.

   To list all interfaces, use the system host-if-list command and include the -a flag.

   For example:

   ~(keystone_admin)$ system host-if-list -a compute-0
   +---...+----------+----------+...+---------------+...+-------------------+
   | uuid | name     | class    |   | ports         |   | data networks     |
   +---...+----------+----------+...+---------------+...+-------------------+
   | 68...| ens787f3 | None     |   | [u'ens787f3'] |   | []                |
   | 79...| data0    | data     |   | [u'ens787f0'] |   | [u'group0-data0'] |
   | 78...| cluster0 | platform |   | []            |   | []                |
   | 89...| ens513f3 | None     |   | [u'ens513f3'] |   | []                |
   | 97...| ens803f1 | None     |   | [u'ens803f1'] |   | []                |
   | d6...| pxeboot0 | platform |   | [u'eno2']     |   | []                |
   | d6...| mgmt0    | platform |   | []            |   | []                |
   +---...+----------+----------+...+---------------+...+-------------------+

5. Use the system host-if-modify and interface-datanetwork-assign commands to configure the interfaces.

   ~(keystone_admin)$ system host-if-modify -m 1500 -n sriov1 -c pci-sriov -N <numvfs> --vf-driver=<drivername> compute-0 <ethname>
   ~(keystone_admin)$ system interface-datanetwork-assign compute-0 <interface> <networks>

   where the following options are available:

   hostname

   This is the host name or ID of the compute (worker) node.

   numvfs

   The number of virtual functions to enable on the device.

   drivername

   An optional virtual function driver to use. Valid choices are 'vfio' and 'netdevice'. The default value is netdevice, which will cause virtual function interfaces to appear as kernel network devices in the container. A value of 'vfio' will cause the device to be bound to the vfio-pci driver. Vfio based devices will not appear as kernel network interfaces, but may be used by based applications.

   Note

   • Applications backed by Mellanox should use the netdevice VF driver.
   • If a mix of netdevice and vfio based containers is required, a separate data network should be created for each type.

   ethname

   The name or UUID of the Ethernet interface to use.

   networks

   A list of data networks that are attached to the interface, delimited by quotes and separated by commas; for example, "data-net-a,data-net-b". To specify a single data network, omit the quotes.

   For example, to attach Ethernet interface ens787f3 to data network datanet-a configured with 16 virtual functions, do the following:

   ~(keystone_admin)$ system host-if-modify -m 1500 -n sriov1 -c pci-sriov -N 16 compute-0 ens787f3
   +----------------+--------------------------------------+
   | Property        | Value                                |
   +-----------------+--------------------------------------+
   | ifname          | sriov1                               |
   | iftype          | ethernet                             |
   | ports           | [u'ens787f3']                        |
   | imac            | 68:05:ca:3a:18:4b                    |
   | imtu            | 1500                                 |
   | ifclass         | pci-sriov                            |
   | networks        |                                      |
   | aemode          | None                                 |
   | schedpolicy     | None                                 |
   | txhashpolicy    | None                                 |
   | uuid            | 68544dbc-244c-4d24-a629-ca8e4543c6f8 |
   | ihost_uuid      | 54c28c7c-5b53-4191-97b5-9ddde3cbec81 |
   | vlan_id         | None                                 |
   | uses            | []                                   |
   | used_by         | []                                   |
   | created_at      | 2019-05-14T00:12:56.673418+00:00     |
   | updated_at      | 2019-05-14T00:16:56.864997+00:00     |
   | sriov_numvfs    | 16                                   |
   | sriov_vf_driver | None                                 |
   | accelerated     | [True]                               |
   +-----------------+--------------------------------------+
   
   ~(keystone_admin)$ system interface-datanetwork-assign compute-0 sriov1 datanet-a

   Note

   It is not possible to have multiple data networks assigned to the same interface. If you want to use multiple data network, it is possible to create a new interface on top of another and have different data networks assigned to each of them.

6. Unlock the host.

   Note

   AIO-SX hosts do not need to be locked to provision an interface and assign it to a data network.

   ~(keystone_admin)$ system host-unlock compute-0

When launching an -enabled Kubernetes deployment, pods will only be scheduled on hosts with the sriovdp label enabled.