a3feea0cdb
Add sections to configure gpu devices, qat devices and intel device plugin operator. Add section to uninstall device plugins. Delete old qat device section. Story: 2010604 Task: 48177 Change-Id: I0f40bb50abc50889adb1c63316e9857ca9a371bc Signed-off-by: Elisamara Aoki Goncalves <elisamaraaoki.goncalves@windriver.com>
250 lines
7.0 KiB
ReStructuredText
250 lines
7.0 KiB
ReStructuredText
.. WARNING: Add no lines of text between the label immediately following
|
|
.. and the title.
|
|
|
|
.. _qat-device-plugin-configuration-616551306371:
|
|
|
|
===============================
|
|
QAT Device Plugin Configuration
|
|
===============================
|
|
|
|
Intel® QuickAssist Technology (Intel® QAT) accelerates cryptographic workloads
|
|
by offloading the data to hardware that is capable of optimizing those
|
|
functions.
|
|
|
|
This section describes how to enable and consume the Intel |QAT| device plugin
|
|
in |prod|.
|
|
|
|
.. rubric:: |prereq|
|
|
|
|
- The host should have Intel |QAT| hardware. Supported |QAT| devices are 4940
|
|
and 4942. After |prod| is installed, do the following verification to ensure
|
|
|QAT| devices are configured.
|
|
|
|
- Verify |QAT| |SRIOV| physical functions are configured.
|
|
|
|
.. code-block::
|
|
|
|
$ for i in 4942 4940; do lspci -d** **8086:$i; done
|
|
|
|
- Verify |QAT| |SRIOV| virtual functions are configured.
|
|
|
|
.. code-block::
|
|
|
|
$ for i in 4943 4941; do lspci -d** **8086:$i; done
|
|
|
|
$ sudo /etc/init.d/qat_service status # Must list all the virtual functions
|
|
|
|
Checking status of all devices.
|
|
|
|
There is 34 QAT acceleration device(s) in the system:
|
|
|
|
qat_dev0 - type: 4xxx, inst_id: 0, node_id: 0, bsf: 0000:f3:00.0,
|
|
#accel: 1 #engines: 9 state: up
|
|
|
|
qat_dev1 - type: 4xxx, inst_id: 1, node_id: 0, bsf: 0000:f7:00.0,
|
|
#accel: 1 #engines: 9 state: up
|
|
|
|
qat_dev2 - type: 4xxxvf, inst_id: 0, node_id: 0, bsf: 0000:f3:00.1,
|
|
#accel: 1 #engines: 1 state: up
|
|
|
|
qat_dev3 - type: 4xxxvf, inst_id: 1, node_id: 0, bsf: 0000:f3:00.2,
|
|
#accel: 1 #engines: 1 state: up
|
|
|
|
- Verify the |QAT| driver ``vfio_pci`` is installed.
|
|
|
|
.. code-block::
|
|
|
|
$ lsmod | grep vfio_pci
|
|
|
|
vfio_pci 69632 0
|
|
vfio_virqfd 16384 1 vfio_pci
|
|
vfio 45056 4 intel_qat,vfio_mdev,vfio_iommu_type1,\ **vfio_pci**
|
|
irqbypass 16384 3 intel_qat,vfio_pci,kvm
|
|
|
|
- Node Feature Discovery application must be installed, using the following
|
|
commands.
|
|
|
|
.. code-block::
|
|
|
|
~(keystone_admin)]$ system application-upload /usr/local/share/applications/helm/node-feature-discovery*.tgz
|
|
~(keystone_admin)]$ system application-apply node-feature-discovery
|
|
|
|
|
|
Enable Intel QAT Device Plugin
|
|
------------------------------
|
|
|
|
The following steps should be performed to enable the Intel |QAT| device plugin
|
|
for discovering and advertising |QAT| VF (Virtual Functions) resources to
|
|
Kubernetes host.
|
|
|
|
#. Locate the application tarball in the ``/usr/local/share/applications/helm``
|
|
directory. For example:
|
|
|
|
``/usr/local/share/applications/helm/intel-device-plugins-operator-<version>.tgz``
|
|
|
|
#. Upload the application using the following command.
|
|
|
|
.. code-block::
|
|
|
|
~(keystone_admin**\ **)]$ system application-upload intel-device-plugins-operator-<version>.tgz
|
|
|
|
Replace ``<version>`` with the latest version number.
|
|
|
|
#. Verify that the application has been uploaded successfully.
|
|
|
|
.. code-block::
|
|
|
|
~(keystone_admin**\ **)]$ system application-list
|
|
|
|
#. Check the Hellm chart status.
|
|
|
|
.. code-block::
|
|
|
|
~(keystone_admin*)]$ system helm-override-list intel-device-plugins-operator -long**
|
|
|
|
#. Enable QAT helm chart.
|
|
|
|
.. code-block::
|
|
|
|
~(keystone_admin)]$ system helm-chart-attribute-modify --enabled true intel-device-plugins-operator intel-device-plugins-qat intel-device-plugins-operator
|
|
|
|
#. Apply the application.
|
|
|
|
.. code-block::
|
|
|
|
~(keystone_admin)]$ system application-apply intel-device-plugins-operator
|
|
|
|
#. Monitor the status of the application.
|
|
|
|
.. code-block::
|
|
|
|
~(keystone_admin*)]$ watch -n 5 system application-list
|
|
|
|
OR
|
|
|
|
.. code-block:: none
|
|
|
|
~(keystone_admin)]$ watch kubectl get pods -n intel-device-plugins-operator
|
|
|
|
#. Check the pods.
|
|
|
|
.. code-block::
|
|
|
|
$ kubectl get pods -n intel-device-plugins-operator
|
|
|
|
NAME READY STATUS RESTARTS AGE
|
|
|
|
intel-qat-plugin-qatdeviceplugin-sample-g8n45 1/1 Running 0 34s
|
|
inteldeviceplugins-controller-manager-74f4c 2/2 Running 0 64s
|
|
|
|
#. Verify |QAT| devices by checking the node's resource allocations. The |QAT|
|
|
4940 device and the |QAT| 4942 device each have 16 virtual functions. If
|
|
both devices are present, the following command will display a total of 32
|
|
virtual functions:
|
|
|
|
.. code-block::
|
|
|
|
$ kubectl describe node <node name> \| grep qat.intel.com/asym-dc
|
|
|
|
Capacity:
|
|
---
|
|
qat.intel.com/asym-dc: 32
|
|
---
|
|
Allocatable:
|
|
---
|
|
qat.intel.com/asym-dc: 32
|
|
---
|
|
|
|
Use Intel QAT Device Plugin
|
|
---------------------------
|
|
|
|
This section describes the steps for using |QAT| device plugin.
|
|
|
|
#. Deploy a pod using the following sample POD specification file. The pod
|
|
specification file can be modified for required resource request and limit.
|
|
|
|
The ``qat.intel.com/asym-dc: <number of devices>`` field is used to
|
|
configure the requested |QAT| virtual functions.
|
|
|
|
For a |DPDK|-based workload, you may need to add a hugepage request and
|
|
limit.
|
|
|
|
``qat-dpdk.yaml``
|
|
|
|
.. code-block:: yaml
|
|
|
|
kind: Pod
|
|
apiVersion: v1
|
|
metadata:
|
|
name: dpdk-test-crypto-perf
|
|
spec:
|
|
containers:
|
|
- name: crypto-perf
|
|
image: intel/crypto-perf:devel
|
|
imagePullPolicy: IfNotPresent
|
|
command: [ "/bin/bash", "-c", "--" ]
|
|
args: [ "while true; do sleep 300000; done;" ]
|
|
volumeMounts:
|
|
- mountPath: /dev/hugepages
|
|
name: hugepage
|
|
- mountPath: /var/run/dpdk
|
|
name: dpdk-runtime
|
|
resources:
|
|
requests:
|
|
cpu: "3"
|
|
memory: "128Mi"
|
|
qat.intel.com/asym-dc: '4'
|
|
hugepages-2Mi: "128Mi"
|
|
limits:
|
|
cpu: "3"
|
|
memory: "128Mi"
|
|
qat.intel.com/asym-dc: '4'
|
|
hugepages-2Mi: "128Mi"
|
|
securityContext:
|
|
readOnlyRootFilesystem: true
|
|
allowPrivilegeEscalation: false
|
|
capabilities:
|
|
add:
|
|
["IPC_LOCK"]
|
|
restartPolicy: Never
|
|
volumes:
|
|
- name: dpdk-runtime
|
|
emptyDir:
|
|
medium: Memory
|
|
- name: hugepage
|
|
emptyDir:
|
|
medium: HugePages
|
|
|
|
|
|
Apply the pod specification file to create ``dpdk-test-crypto-perf`` pod.
|
|
|
|
.. code-block::
|
|
|
|
$ kubectl apply -k qat-dpdk.yaml
|
|
|
|
#. Verify the pod status and the allocated |QAT| virtual functions.
|
|
|
|
.. code-block::
|
|
|
|
$ kubectl get pods
|
|
|
|
NAME READY STATUS RESTARTS AGE
|
|
dpdk-test-crypto-perf 1/1 Running 0 27m
|
|
|
|
$ kubectl describe pod dpdk-test-crypto-perf**
|
|
|
|
Requests:
|
|
---
|
|
qat.intel.com/asym-dc: 4
|
|
---
|
|
|
|
$ kubectl describe node <controller-name>
|
|
|
|
Allocated resources:
|
|
---
|
|
qat.intel.com/asym-dc: 4
|
|
---
|
|
|
|
For more information, see: `Demos and Testing
|
|
<https://github.com/intel/intel-device-plugins-for-kubernetes/blob/main/cmd/qat_plugin/README.md#demos-and-testing>`__.
|