starlingx/docs
Code Issues Proposed changes
docs/doc/source/node_management/kubernetes/hardware_acceleration_devices/set-up-pods-to-use-sriov.rst
Nidhi Shivashankara Belur 8562e0f2fb Document Updates for fec operator v2.9.0 
This commit includes following updates to the documentation.

- Configuration of Intel® vRAN Boost 2.0 (VRB2) using a new API
  `SriovVrbClusterConfig`.
- Re-arrange config specs.
- Replace ACC200 with VRB1 for documentation only, not in the config
  specs.
- Replace status output of SriovFecNodeConfig with latest results to
  include the pf_bb_config version used by the FEC Operator.

Story: 2011121
Task: 50460

Change-Id: I9cd93087636b3967c804f6bb6c4b24c3850f5aeb
Signed-off-by: Nidhi Shivashankara Belur <nidhi.shivashankara.belur@intel.com>
2024-08-02 09:52:59 -07:00

6.0 KiB
Raw Blame History

Set Up Pods to Use SRIOV to Access HW Accelerators

You can configure pods with access to a HW accelerator device by adding the appropriate 'resources' request in the pod specification.

The following procedure shows an example of launching a container image with 'resources' request for a to the ACC100 device.

  1. Request the resource name of the desired accelerator.

    ~(keystone_admin)$ kubectl get node controller-0 -o json | json_pp | grep -A 15 '"allocatable"'
"allocatable" : {
  "cpu" : "126",
  "ephemeral-storage" : "9417620260",
  "hugepages-1Gi" : "10Gi",
  "hugepages-2Mi" : "0",
  "intel.com/intel_fec_acc100" : "16",
  "memory" : "503822580Ki",
  "pods" : "110"
},
"capacity" : {
  "cpu" : "128",
  "ephemeral-storage" : "10218772Ki",
  "hugepages-1Gi" : "10Gi",
  "hugepages-2Mi" : "0",
  "intel.com/intel_fec_acc100" : "16",
  "memory" : "525674740Ki",

  2. Create a pod.yml file that requests 16 ACC100 VFs (i.e. intel.com/intel_fec_acc100: '16')

    ~(keystone_admin)$ cat >> pod0.yml << EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: pod0
  annotations:
  labels:
    app: pod0
spec:
  replicas: 1
  selector:
    matchLabels:
      app: pod0
  template:
    metadata:
      name: pod0
      labels:
        app: pod0
    spec:
      restartPolicy: Always
      containers:
      - name: pod0
        image: "windse/pktgen-testpmd-bbdev:d1911r4p1912"
        volumeMounts:
        - mountPath: /mnt/huge-1048576kB
          name: hugepage
        - mountPath: /sys/devices
          name: uio
        command: ["/bin/bash", "-ec", "sleep infinity"]
        securityContext:
          privileged: false
          capabilities:
            add:
              ["IPC_LOCK", "SYS_ADMIN"]
        resources:
          requests:
            memory: 4Gi
            intel.com/intel_fec_acc100: '16'
            windriver.com/isolcpus: 24
          limits:
            hugepages-1Gi: 2Gi
            memory: 4Gi
            intel.com/intel_fec_acc100: '16'
            windriver.com/isolcpus: 24
      volumes:
      - name: hugepage
        emptyDir:
          medium: HugePages
      - name: uio
        hostPath:
          path: /sys/devices
EOF

  3. Start the pod.

    ~(keystone_admin)$ kubectl create -f pod0.yml

  4. Wait for the pod to start and execute the following command:

    ~(keystone_admin)$ kubectl exec -it pod0 -- env | grep PCIDEVICE_INTEL_COM_INTEL_FEC_ACC100

    The following PCI addresses and "VFIO_TOKEN" corresponding to the are displayed:

    PCIDEVICE_INTEL_COM_INTEL_FEC_ACC100=0000:32:00.5,0000:32:00.1,0000:32:00.2,0000:32:00.4,0000:32:01.0,0000:32:01.1,0000:32:01.2,0000:32:01.3,0000:32:00.0,0000:32:00.6,0000:32:00.7,0000:32:01.4,0000:32:01.7,0000:32:00.3,0000:32:01.6,0000:32:01.5
PCIDEVICE_INTEL_COM_INTEL_FEC_ACC100_INFO={"0000:32:00.0":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:00.0"},"vfio":{"dev-mount":"/dev/vfio/315","mount":"/dev/vfio/vfio"}},"0000:32:00.1":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:00.1"},"vfio":{"dev-mount":"/dev/vfio/316","mount":"/dev/vfio/vfio"}},"0000:32:00.2":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:00.2"},"vfio":{"mount":"/dev/vfio/vfio"}},"0000:32:00.3":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:00.3"},"vfio":{"mount":"/dev/vfio/vfio"}},"0000:32:00.4":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:00.4"},"vfio":{"mount":"/dev/vfio/vfio"}},"0000:32:00.5":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:00.5"},"vfio":{"mount":"/dev/vfio/vfio"}},"0000:32:00.6":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:00.6"},"vfio":{"mount":"/dev/vfio/vfio"}},"0000:32:00.7":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:00.7"},"vfio":{"mount":"/dev/vfio/vfio"}},"0000:32:01.0":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:01.0"},"vfio":{"mount":"/dev/vfio/vfio"}},"0000:32:01.1":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:01.1"},"vfio":{"mount":"/dev/vfio/vfio"}},"0000:32:01.2":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:01.2"},"vfio":{"mount":"/dev/vfio/vfio"}},"0000:32:01.3":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:01.3"},"vfio":{"mount":"/dev/vfio/vfio"}},"0000:32:01.4":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:01.4"},"vfio":{"mount":"/dev/vfio/vfio"}},"0000:32:01.5":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:01.5"},"vfio":{"mount":"/dev/vfio/vfio"}},"0000:32:01.6":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:01.6"},"vfio":{"mount":"/dev/vfio/vfio"}},"0000:32:01.7":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:01.7"},"vfio":{"mount":"/dev/vfio/vfio"}}}