=====================================
Host FPGA Configuration - Intel N3000
=====================================
This guide describes how to configure and integrate an Intel N3000 |FPGA|
|PAC| on StarlingX.
.. contents:: |minitoc|
:local:
:depth: 1
--------
Overview
--------
The `Intel FPGA PAC N3000 <https://www.intel.com/content/www/us/en/programmable/products/boards_and_kits/dev-kits/altera/intel-fpga-pac-n3000/overview.html>`_ contains two Intel
XL710 |NICs|, memory, and an Intel |FPGA|. The system discovers and inventories the
device as a |NIC|, with the XL710 ports available in the host port list and host
interface list.
--------------------
Update device images
--------------------
The Intel |FPGA| |PAC| N3000 as shipped from the factory is expected to have
production BMC and factory images. The following procedure describes how to
update the user image on a host.
Device image types:
* root-key: The root-key image sets the main authentication key on the
hardware.
* functional-key: The functional device image performs the desired work on
behalf of the application. If a rootkey device image has been written to the
hardware, then the functional image will only be accepted if it has been
signed by a |CSK| generated from the root key which has
not been revoked.
* key-revocation: The key-revocation device image will revoke a |CSK|. If a
root-key device image has been written to the hardware, then the
key-revocation device image will only be accepted if it has been signed by
the root key.
The following items are specific to the Intel |FPGA| |PAC| N3000:
* The root-key image is called the *root entry hash bitstream* and can only be
set once.
* The functional device image is known as the *user image*.
* The key-revocation device image is known as the *CSK ID cancellation
bitstream*.
* |CSKs| are revoked by specifying an integer |CSK| ID.
* 128 |CSK| ID cancellation slots exist.
For the Intel |FPGA| |PAC| N3000, a |CSK| is revoked by specifying an integer
ID, and all |CSKs| with that ID will be revoked. Writing the root-key device
image or a key-revocation device image is essentially permanent. Reverting to
factory status requires physical access to the card and specialized equipment.
#. Upload the device image.
To upload a root-key device image:
.. code-block:: none
~(keystone_admin)$ system device-image-upload imagefile root-key pci_vendor
pci_device --key-signature key_signature --name imagename --description
description --image-version version
To upload a functional device image:
.. code-block:: none
~(keystone_admin)$ system device-image-upload imagefile functional pci_vendor
pci_device --functional bitstream_id --name imagename --description description
-image-version version
To upload a revocation key device image:
.. code-block:: none
~(keystone_admin)$ system device-image-upload imagefile key-revocation
pci_vendor pci_device --revoke-key-id revoke_key_id --name imagename --
description description --image-version version
where:
.. code-block:: none
imagefile # The filepath of the binary device image file.
pci_vendor # The hexadecimal string identifying the PCI vendor ID of the device this image applies to.
pci_device # The hexadecimal string identifying the PCI device ID of the device this image applies to.
key_signature # A hexadecimal string identifying the root key device image.
revoke_key_id # A decimal key ID for the key revocation device image.
bitstream_id # A hexadecimal string of the functional device image.
name # The name of the device image (optional).
description # The description of the device image (optional).
image-version # The version of the device image (optional).
#. Assign a device label to the device.
Labels are key-value pairs that are assigned to host |PCI| devices and are
used to specify attributes of the devices. Labels can be used to identify
certain properties of the |PCI| devices where the same device image can be
used.
The command syntax is:
.. code-block:: none
system host-device-label-assign [--overwrite] hostname_or_id pci_name_or_address
name=value [name=value ...]
Overwrite the label using the ``--overwrite`` option. This option is not
allowed while the image update is in progress after running
``host-device-image-update``. Once assigned, a device label can be
referenced by multiple ``device-image-apply`` commands.
#. Apply the device image on one or all supported devices.
.. note::
A device firmware update in progress alarm is raised once the first
device image is applied.
The ``system device-image-state-list`` will show the status of the device as
pending.
* Apply a device image to all supported devices:
.. code-block:: none
~(keystone_admin)$ system device-image-apply image_uuid
* Alternatively, apply a device image to devices with a specified label:
.. code-block:: none
~(keystone_admin)$ system device-image-apply image_uuid key1=value1
#. Write pending device images on the host to hardware.
.. code-block:: none
~(keystone)admin)$ system host-device-image-update hostname
.. note::
This operation currently supports one pending device image at a time.
Any previously-attempted device image writes for this host that are in a
failed state will be reset to pending and retried.
Root and revocation key updates can be expected to take 1-2 minutes.
Functional image updates can take approximately 40 minutes for the Intel
|FPGA| |PAC| N3000.
* Once a device update is complete, ``system device-image-state-list``
will show the status as completed for that device/image.
* Once all pending device updates for the host are complete,
``system host-show hostname`` will again display an empty string for
``device_image_update``.
#. Lock and unlock the host.
#. (Optional) Upload, apply, and update any additional key-revocation device
images or functional device images as needed.
New device images can be uploaded as needed, and already-uploaded images
can be applied with new labels. Devices can also have new labels applied to
them, and any device images with matching labels will be automatically
applied.
--------------------------
Device management commands
--------------------------
This section lists the commands used to control the Intel |FPGA| |PAC| N3000.
.. code-block:: none
Listing uploaded device images
system device-image-list
Listing device labels
system host-device-label-list hostname_or_id devicename_or_address
Removing device labels
system host-device-label-remove hostname_or_id key
Remove a device image
To remove a device image from all devices
system device-image-remove image_uuid
To remove the device image from all devices with a matching label
system device-image-remove image_uuid key1=value1
Initiating a Device Image Update for a Host
system host-device-image-update hostname_or_host_ID
Displaying the status of device images
system device-image-state-list
-------------------------------
Enable Forward Error Correction
-------------------------------
The Intel |FPGA| |PAC| N3000 supports |FEC| capabilities, which are exposed as
a |PCI| device. The |PCI| device can be used by a `DPDK
<https://www.dpdk.org/>`_ enabled container application to perform accelerated
5G LDPC encoding and decoding operations.
After the |FPGA| device is programmed, the list of host devices shows the |FEC|
device with device ID 0xd8f, as shown below.
.. code-block:: none
system host-device-list <worker-node>
+------------------+--------------+----------+-----------+-----------+---------------------------+-------------------------+-------------------------------------+-----------+---------+
| name | address | class id | vendor id | device id | class name | vendor name | device name | numa_node | enabled |
+------------------+--------------+----------+-----------+-----------+---------------------------+-------------------------+-------------------------------------+-----------+---------+
| pci_0000_b7_00_0 | 0000:b7:00.0 | 120000 | 8086 | 0d8f | Processing accelerators | Intel Corporation | Device 0d8f | 1 | True |
+------------------+--------------+----------+-----------+-----------+---------------------------+-------------------------+-------------------------------------+-----------+---------+
...
To enable the |FEC| device for |SRIOV|, set the number of |VFs| and set the
appropriate userspace drivers for the |PF| and |VF|.
For example:
.. code-block:: none
system host-lock <worker>
system host-device-modify <worker> <name> --driver <vf driver> --vf-driver <vf driver> -N <number of vfs>
system host-unlock <worker>
The supported PF driver(s) are:
- igb_uio
The supported VF driver(s) are currently:
- igb_uio
- vfio
To pass the |FEC| device to a container, enter the following requests/limits
into the pod specification:
.. code-block:: none
intel.com/intel_fpga_fec: '<number of vfs>'
For example:
.. code-block:: none
apiVersion: v1
kind: Pod
metadata:
name: 5gnr
annotations:
k8s.v1.cni.cncf.io/networks: '[
{ "name": "sriov1" }
]'
spec:
restartPolicy: Never
containers:
- name: 5gnr
image: "5gnr-image"
volumeMounts:
- mountPath: /mnt/huge-1048576kB
name: hugepage
stdin: true
tty: true
resources:
requests:
memory: 4Gi
intel.com/intel_fpga_fec: '1'
intel.com/pci_sriov_net_datanetwork_a: '1'
limits:
hugepages-1Gi: 4Gi
memory: 4Gi
intel.com/intel_fpga_fec: '1'
intel.com/pci_sriov_net_datanetwork_a: '1'
volumes:
- name: hugepage
emptyDir:
medium: HugePages
-------------------------
Configure NICs for SR-IOV
-------------------------
You can configure the Intel XL710 NICs for |SRIOV| by first identifying the
|NICs| on the Intel |FPGA| |PAC| N3000 using the following command:
.. code-block:: none
system host-port-list <worker>
+--------------------------------------+------------+----------+--------------+--------+-----------+-------------+------------------------------------------------+
| uuid | name | type | pci address | device | processor | accelerated | device type |
+--------------------------------------+------------+----------+--------------+--------+-----------+-------------+------------------------------------------------+
| 6c79c0d0-0463-4551-a19a-24d52a9403c6 | enp177s0f0 | ethernet | 0000:b1:00.0 | 0 | 1 | False | Device [0d58] |
+--------------------------------------+------------+----------+--------------+--------+-----------+-------------+------------------------------------------------+
...
Next, set the number of |VFs| and set the appropriate
userspace drivers for the |VF|.
For example:
.. code-block:: none
system host-lock <worker>
system host-if-list -a <worker>
system host-if-modify <worker> <interface name or uuid> -c pci-sriov --vf-driver <vf driver> -N <number of vfs>
system interface-datanetwork-assign <worker> <interface> <datanetwork>
system host-unlock <worker>
The supported VF driver(s) are currently:
- vfio
- netdevice
.. note::
If ``--vf-driver`` is not specified in the ``system host-if-modify``
command, then ``netdevice`` (kernel driver) will be assigned.