Mingyuan Qi d17e0951f1 Deploy edgeworker nodes

Add a page for edgeworker node deployment guide.

Story: 2008129
Task: 41784

Change-Id: I3d6deff17de9950e8f670e03144927c1cc4ee07e
Signed-off-by: Mingyuan Qi <mingyuan.qi@intel.com>

2021-05-26 23:52:17 +08:00

12 KiB

Raw Blame History

Deploy Edgeworker Nodes

Introduction

Edgeworker is a new personality of node introduced in stx.5.0¹. Ubuntu is supported for a node with edgeworker personality as its operating system². Edgeworkers are typically small systems running dedicated workloads that can now be managed by .

The edgeworker personality is targeted for nodes with below criteria:

A node is running a customized OS, or
A node is running a Type1 hypervisor, or
A node does not meet worker’s minimum requirements.

These nodes can be added to a running cluster in minutes. During the provisioning stage, Kubernetes components will be provisioned on the edgeworker nodes.

The controller manages edgeworker nodes as normal worker nodes including the following functionalities:

Container orchestration by Kubernetes
Distributed storage cluster management by Rook-ceph

Notes

Hardware Requirements

The minimum requirements for edgeworker host (bare metal or virtual machine) are:

Minimum Requirement	Edgeworker Node
Minimum processor class	Intel® Core™ Processor Family
Minimum memory	16GB
Primary disk	256GB SSD or NVMe
Minimum network ports	Mgmt/Cluster: 1 x 1000Base Data: 0 or more x 10GE

Prepare Edgeworker Nodes

Prior to provisioning edgeworker nodes, the setup must be prepared in the following condition:

AIO-Duplex or Standard setup deployed.
Edgeworker node cabled for power and powered on.
Edgeworker node cabled for networking and connected to mgmt switch.
Set primary disk as 1st boot device in BIOS setting of the edgeworker node.
Ubuntu 18.04(or above) has been installed in edgeworker node.
Openssh-server and python package installed.

The network connection of edgeworker nodes to the existing cluster is shown in below figure.

Provision Edgeworker Nodes

By connecting a node’s network interface to the management network switch, a node will be detected and will appear in the list of hosts managed by system inventory (with personality set to None).

[sysadmin@controller-0 ~(keystone_admin)]$ system host-list
+----+--------------+-------------+----------------+-------------+--------------+
| id | hostname     | personality | administrative | operational | availability |
+----+--------------+-------------+----------------+-------------+--------------+
| 1  | controller-0 | controller  | unlocked       | enabled     | available    |
| 2  | controller-1 | controller  | unlocked       | enabled     | available    |
| 3  | None         | None        | locked         | disabled    | offline      |
+----+--------------+-------------+----------------+-------------+--------------+

Update the node’s hostname and personality by system host-update.

[sysadmin@controller-0 ~(keystone_admin)]$ system host-update 3 hostname=edgeworker-0 personality=edgeworker
+-----------------------+--------------------------------------+
| Property              | Value                                |
+-----------------------+--------------------------------------+
| action                | none                                 |
| administrative        | locked                               |
| availability          | offline                              |
| bm_ip                 | None                                 |
| bm_type               | None                                 |
| bm_username           | None                                 |
| boot_device           | /dev/sda                             |
| capabilities          | {}                                   |
| clock_synchronization | ntp                                  |
| config_applied        | None                                 |
| config_status         | None                                 |
| config_target         | None                                 |
| console               | ttyS0,115200                         |
| created_at            | 2021-02-05T02:12:42.136121+00:00     |
| device_image_update   | None                                 |
| hostname              | edgeworker-0                         |
| id                    | 3                                    |
| install_output        | text                                 |
| install_state         | None                                 |
| install_state_info    | None                                 |
| inv_state             | None                                 |
| invprovision          | None                                 |
| location              | {}                                   |
| mgmt_ip               | 192.188.204.21                       |
| mgmt_mac              | 52:54:00:76:e6:cb                    |
| operational           | disabled                             |
| personality           | edgeworker                           |
| reboot_needed         | False                                |
| reserved              | False                                |
| rootfs_device         | /dev/sda                             |
| serialid              | None                                 |
| software_load         | 20.12                                |
| task                  | None                                 |
| tboot                 | false                                |
| ttys_dcd              | None                                 |
| updated_at            | None                                 |
| uptime                | 0                                    |
| uuid                  | a2c8f3ec-1dc4-4a0a-b471-6e5c01abc187 |
| vim_progress_status   | None                                 |
+-----------------------+--------------------------------------+

Alternatively, if the node is not shown in host list automatically, you can also add the host with system host-add.

[sysadmin@controller-0 ~(keystone_admin)]$ system host-add -n edgeworker-0 -p edgeworker -m <mgmt_mac>

Check pre-requisites of the edgeworker node:

Note

We suggest using the same username and password for all the edgeworker nodes.

EWUSER=<edgeworker username>
EWPASSWD=<edgeworker password>
SYSADMINPASSWD=<sysadmin password>
EWNODE=(edgeworker-0)
PYTHONBIN=()
for NODE in ${EWNODE[@]}; do
  EDGEWORKER=$(system host-show ${NODE} | awk '/mgmt_ip/{print$4}');
  PYTHONBIN+=($(sshpass -p ${EWPASSWD} ssh -o "StrictHostKeyChecking no" ${EWUSER}@${EDGEWORKER} which python));
done
ERR=0
for i in ${PYTHONBIN[@]}; do
  if [ -z "$i" ]; then
    ERR=1
    break
  fi
done
if [ $ERR == 1 ]; then
  echo "Error, python should be intalled on edgeworker nodes."
else
  echo "Pre-requisites check passed."
fi

If the following error occurrs, refresh the client for the mgmt interface on the edgeworker nodes to get the right ip address assigned.

ssh: connect to host XX.XX.XX.XX port 22: No route to host

Create an edgeworker inventory file with the variables from the last step.

tee ./edgeworker_inventory.yml << EOF
all:
  hosts:
    localhost:
      ansible_connection: local
  children:
    edgeworker:
      hosts:
EOF
for i in ${!EWNODE[@]}; do
  echo "        ${EWNODE[$i]}:"
  echo "          ansible_ssh_user: ${EWUSER}"
  echo "          ansible_ssh_pass: ${EWPASSWD}"
  echo "          ansible_become_pass: ${EWPASSWD}"
  echo "          ansible_python_interpreter: ${PYTHONBIN[$i]}"
done | tee -a edgeworker_inventory.yml
tee -a ./edgeworker_inventory.yml << EOF
  vars:
    ansible_ssh_user: sysadmin
    ansible_ssh_pass: ${SYSADMINPASSWD}
    ansible_become_pass: ${SYSADMINPASSWD}
EOF

Provision the edgeworker node using ansible playbook. You can provision edgeworker node one at a time or multiple nodes at once. Run the playbook with the underlying inventory

ansible-playbook -i ./edgeworker_inventory.yml /usr/share/ansible/stx-ansible/playbooks/provision_edgeworker.yml

After the provisioning, the edgeworker node will be Ready in Kubernetes cluster.

TASK [provision-edgeworker/prepare-edgeworker/kubernetes : set_fact] *********************************************************************************************************
ok: [edgeworker-0]

TASK [provision-edgeworker/prepare-edgeworker/kubernetes : Pull k8s gcr images from controller registry to edgeworker-0] *****************************************************
changed: [edgeworker-0]

RUNNING HANDLER [provision-edgeworker/prepare-edgeworker/kubernetes : Restart kubelet] ***************************************************************************************
changed: [edgeworker-0]

PLAY [localhost] *************************************************************************************************************************************************************

TASK [provision-edgeworker/prepare-controller/cleanup : Cleanup temporary files] *********************************************************************************************
changed: [localhost]

TASK [provision-edgeworker/prepare-controller/cleanup : Remove the provision flag] *******************************************************************************************
changed: [localhost]

PLAY RECAP *******************************************************************************************************************************************************************
edgeworker-0               : ok=54   changed=10   unreachable=0    failed=0
localhost                  : ok=31   changed=19   unreachable=0    failed=0

[sysadmin@controller-0 ~(keystone_admin)]$ kubectl get node
NAME           STATUS   ROLES    AGE   VERSION
controller-0   Ready    master   42h   v1.18.1
controller-1   Ready    master   42h   v1.18.1
edgeworker-0   Ready    <none>   21m   v1.18.1