diff --git a/doc/source/deploy_install_guides/index.rst b/doc/source/deploy_install_guides/index.rst index ec1c02a04..53b00e01c 100755 --- a/doc/source/deploy_install_guides/index.rst +++ b/doc/source/deploy_install_guides/index.rst @@ -8,28 +8,28 @@ Each guide provides instruction on a specific StarlingX configuration .. _latest_release: ------------------------ -Latest release (stable) ------------------------ +------------------------ +Supported release (R4.0) +------------------------ -StarlingX R3.0 is the latest officially released version of StarlingX. - -.. toctree:: - :maxdepth: 1 - - r3_release/index - ---------------------- -Upcoming R4.0 release ---------------------- - -StarlingX R4.0 is the forthcoming version of StarlingX under development. +StarlingX R4.0 is the most recent supported release of StarlingX. .. toctree:: :maxdepth: 1 r4_release/index +------------------------- +Upcoming release (latest) +------------------------- + +StarlingX R5.0 is under development. + +.. toctree:: + :maxdepth: 1 + + r5_release/index + ----------------- Archived releases @@ -38,6 +38,7 @@ Archived releases .. toctree:: :maxdepth: 1 + r3_release/index r2_release/index r1_release/index @@ -53,24 +54,25 @@ Archived releases .. Making a new release -.. 1. Archive the previous 'latest' release. - Move the toctree link from the Latest release section into the Archived +.. 1. Archive the previous 'supported' release. + Move the toctree link from the Supported release section into the Archived releases toctree. -.. 2. Make the previous 'upcoming' release the new 'latest.' - Move the toctree link from the Upcoming release section into the Latest - release. Update narrative text for the Latest release section to use the +.. 2. Make the previous 'upcoming' release the new 'supported'. + Move the toctree link from the Upcoming release section into the Supported + release. Update intro text for the Supported release section to use the latest version. -.. 3. Add new 'upcoming' release. - If the new upcoming release docs arent ready, remove toctree from Upcoming - section and just leave narrative text. Update text for the upcoming release - version. Once the new upcoming docs are ready, add them in the toctree here. +.. 3. Add new 'upcoming' release, aka 'Latest' on the version button. + If new upcoming release docs aren't ready, remove toctree from Upcoming + section and just leave intro text. Update text for the upcoming + release version. Once the new upcoming docs are ready, add them in the + toctree here. -.. Adding new release docs - .. 1. Make sure the most recent release versioned docs are complete for that - release. - .. 2. Make a copy of the most recent release folder e.g. 'r2_release.' Rename the - folder for the new release e.g. 'r3_release'. - .. 3. Search and replace all references to previous release number with the new - release number. For example replace all 'R2.0' with 'R3.0.' Also search and - replease any links that may have a specific release in the path. - .. 4. Link new version on this page (the index page). +.. Adding new release docs +.. 1. Make sure the most recent release versioned docs are complete for that + release. +.. 2. Make a copy of the most recent release folder e.g. 'r4_release.' Rename + the folder for the new release e.g. 'r5_release'. +.. 3. Search and replace all references to previous release number with the new + release number. For example replace all 'R4.0' with 'R5.0.' Also search + and replace any links that may have a specific release number in the path. +.. 4. Link new version on this page (the index page). diff --git a/doc/source/deploy_install_guides/r1_release/index.rst b/doc/source/deploy_install_guides/r1_release/index.rst index 9ae77d46f..f712ae237 100644 --- a/doc/source/deploy_install_guides/r1_release/index.rst +++ b/doc/source/deploy_install_guides/r1_release/index.rst @@ -8,11 +8,11 @@ StarlingX R1.0 Installation since the R1.0 release. Due to these changes, the R1.0 installation instructions may not work as described. - Installation of the the :ref:`latest_release` is recommended. + Installation of the current :ref:`latest_release` is recommended. -This is the installation guide for the StarlingX R1.0 release. If this is not the -installation guide you want to use, see the -:doc:`available installation guides `. +This is the installation guide for the StarlingX R1.0 release. If this is not +the installation guide you want to use, see the :doc:`available installation +guides `. ------------ Introduction diff --git a/doc/source/deploy_install_guides/r2_release/index.rst b/doc/source/deploy_install_guides/r2_release/index.rst index ea951a138..2dbf2c58e 100644 --- a/doc/source/deploy_install_guides/r2_release/index.rst +++ b/doc/source/deploy_install_guides/r2_release/index.rst @@ -8,11 +8,11 @@ StarlingX R2.0 Installation since the R2.0 release. Due to these changes, the R2.0 installation instructions may not work as described. - Installation of the the :ref:`latest_release` is recommended. + Installation of the current :ref:`latest_release` is recommended. -StarlingX provides a pre-defined set of standard -:doc:`deployment configurations `. Most deployment options may -be installed in a virtual environment or on bare metal. +StarlingX provides a pre-defined set of standard :doc:`deployment configurations +`. Most deployment options may be installed in a +virtual environment or on bare metal. ----------------------------------------------------- Install StarlingX Kubernetes in a virtual environment diff --git a/doc/source/deploy_install_guides/r5_release/ansible_bootstrap_configs.rst b/doc/source/deploy_install_guides/r5_release/ansible_bootstrap_configs.rst new file mode 100644 index 000000000..544991336 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/ansible_bootstrap_configs.rst @@ -0,0 +1,420 @@ +================================ +Ansible Bootstrap Configurations +================================ + +This section describes Ansible bootstrap configuration options. + +.. contents:: + :local: + :depth: 1 + + +.. _install-time-only-params-r5: + +---------------------------- +Install-time-only parameters +---------------------------- + +Some Ansible bootstrap parameters can not be changed or are very difficult to +change after installation is complete. + +Review the set of install-time-only parameters before installation and confirm +that your values for these parameters are correct for the desired installation. + +.. note:: + + If you notice an incorrect install-time-only parameter value *before you + unlock controller-0 for the first time*, you can re-run the Ansible bootstrap + playbook with updated override values and the updated values will take effect. + +**************************** +Install-time-only parameters +**************************** + +**System Properties** + +* ``system_mode`` +* ``distributed_cloud_role`` + +**Network Properties** + +* ``pxeboot_subnet`` +* ``pxeboot_start_address`` +* ``pxeboot_end_address`` +* ``management_subnet`` +* ``management_start_address`` +* ``management_end_address`` +* ``cluster_host_subnet`` +* ``cluster_host_start_address`` +* ``cluster_host_end_address`` +* ``cluster_pod_subnet`` +* ``cluster_pod_start_address`` +* ``cluster_pod_end_address`` +* ``cluster_service_subnet`` +* ``cluster_service_start_address`` +* ``cluster_service_end_address`` +* ``management_multicast_subnet`` +* ``management_multicast_start_address`` +* ``management_multicast_end_address`` + +**Docker Proxies** + +* ``docker_http_proxy`` +* ``docker_https_proxy`` +* ``docker_no_proxy`` + +**Docker Registry Overrides** + +* ``docker_registries`` + + * ``k8s.gcr.io`` + + * ``url`` + * ``username`` + * ``password`` + * ``secure`` + + * ``gcr.io`` + + * ``url`` + * ``username`` + * ``password`` + * ``secure`` + + * ``quay.io`` + + * ``url`` + * ``username`` + * ``password`` + * ``secure`` + + * ``docker.io`` + + * ``url`` + * ``username`` + * ``password`` + * ``secure`` + + * ``docker.elastic.co`` + + * ``url`` + * ``username`` + * ``password`` + * ``secure`` + + * ``defaults`` + + * ``url`` + * ``username`` + * ``password`` + * ``secure`` + +**Certificates** + +* ``k8s_root_ca_cert`` +* ``k8s_root_ca_key`` + +**Kubernetes Parameters** + +* ``apiserver_oidc`` + +---- +IPv6 +---- + +If you are using IPv6, provide IPv6 configuration overrides for the Ansible +bootstrap playbook. Note that all addressing, except pxeboot_subnet, should be +updated to IPv6 addressing. + +Example IPv6 override values are shown below: + +:: + + dns_servers: + ‐ 2001:4860:4860::8888 + ‐ 2001:4860:4860::8844 + pxeboot_subnet: 169.254.202.0/24 + management_subnet: 2001:db8:2::/64 + cluster_host_subnet: 2001:db8:3::/64 + cluster_pod_subnet: 2001:db8:4::/64 + cluster_service_subnet: 2001:db8:4::/112 + external_oam_subnet: 2001:db8:1::/64 + external_oam_gateway_address: 2001:db8::1 + external_oam_floating_address: 2001:db8::2 + external_oam_node_0_address: 2001:db8::3 + external_oam_node_1_address: 2001:db8::4 + management_multicast_subnet: ff08::1:1:0/124 + +.. note:: + + The `external_oam_node_0_address`, and `external_oam_node_1_address` parameters + are not required for the AIO‐SX installation. + +---------------- +Private registry +---------------- + +To bootstrap StarlingX you must pull container images for multiple system +services. By default these container images are pulled from public registries: +k8s.gcr.io, gcr.io, quay.io, and docker.io. + +It may be required (or desired) to copy the container images to a private +registry and pull the images from the private registry (instead of the public +registries) as part of the StarlingX bootstrap. For example, a private registry +would be required if a StarlingX system was deployed in an air-gapped network +environment. + +Use the `docker_registries` structure in the bootstrap overrides file to specify +alternate registry(s) for the public registries from which container images are +pulled. These alternate registries are used during the bootstrapping of +controller-0, and on :command:`system application-apply` of application packages. + +The `docker_registries` structure is a map of public registries and the +alternate registry values for each public registry. For each public registry the +key is a fully scoped registry name of a public registry (for example "k8s.gcr.io") +and the alternate registry URL and username/password (if authenticated). + +url + The fully scoped registry name (and optionally namespace/) for the alternate + registry location where the images associated with this public registry + should now be pulled from. + + Valid formats for the `url` value are: + + * Domain. For example: + + :: + + example.domain + + * Domain with port. For example: + + :: + + example.domain:5000 + + * IPv4 address. For example: + + :: + + 1.2.3.4 + + * IPv4 address with port. For example: + + :: + + 1.2.3.4:5000 + + * IPv6 address. For example: + + :: + + FD01::0100 + + * IPv6 address with port. For example: + + :: + + [FD01::0100]:5000 + +username + The username for logging into the alternate registry, if authenticated. + +password + The password for logging into the alternate registry, if authenticated. + + +Additional configuration options in the `docker_registries` structure are: + +defaults + A special public registry key which defines common values to be applied to + all overrideable public registries. If only the `defaults` registry + is defined, it will apply `url`, `username`, and `password` for all + registries. + + If values under specific registries are defined, they will override the + values defined in the defaults registry. + + .. note:: + + The `defaults` key was formerly called `unified`. It was renamed + in StarlingX R3.0 and updated semantics were applied. + + This change affects anyone with a StarlingX installation prior to R3.0 that + specifies alternate Docker registries using the `unified` key. + +secure + Specifies whether the registry(s) supports HTTPS (secure) or HTTP (not secure). + Applies to all alternate registries. A boolean value. The default value is + True (secure, HTTPS). + +.. note:: + + The ``secure`` parameter was formerly called ``is_secure_registry``. It was + renamed in StarlingX R3.0. + +If an alternate registry is specified to be secure (using HTTPS), the certificate +used by the registry may not be signed by a well-known Certificate Authority (CA). +This results in the :command:`docker pull` of images from this registry to fail. +Use the `ssl_ca_cert` override to specify the public certificate of the CA that +signed the alternate registry’s certificate. This will add the CA as a trusted +CA to the StarlingX system. + +ssl_ca_cert + The `ssl_ca_cert` value is the absolute path of the certificate file. The + certificate must be in PEM format and the file may contain a single CA + certificate or multiple CA certificates in a bundle. + +The following example will apply `url`, `username`, and `password` to all +registries. + +:: + + docker_registries: + defaults: + url: my.registry.io + username: myreguser + password: myregP@ssw0rd + +The next example applies `username` and `password` from the defaults registry +to all public registries. `url` is different for each public registry. It +additionally specifies an alternate CA certificate. + +:: + + docker_registries: + k8s.gcr.io: + url: my.k8sregistry.io + gcr.io: + url: my.gcrregistry.io + quay.io: + url: my.quayregistry.io + docker.io: + url: my.dockerregistry.io + defaults: + url: my.registry.io + username: myreguser + password: myregP@ssw0rd + + ssl_ca_cert: /path/to/ssl_ca_cert_file + +------------ +Docker proxy +------------ + +If the StarlingX OAM interface or network is behind a http/https proxy, relative +to the Docker registries used by StarlingX or applications running on StarlingX, +then Docker within StarlingX must be configured to use these http/https proxies. + +Use the following configuration overrides to configure your Docker proxy settings. + +docker_http_proxy + Specify the HTTP proxy URL to use. For example: + + :: + + docker_http_proxy: http://my.proxy.com:1080 + +docker_https_proxy + Specify the HTTPS proxy URL to use. For example: + + :: + + docker_https_proxy: https://my.proxy.com:1443 + +docker_no_proxy + A no-proxy address list can be provided for registries not on the other side + of the proxies. This list will be added to the default no-proxy list derived + from localhost, loopback, management, and OAM floating addresses at run time. + Each address in the no-proxy list must neither contain a wildcard nor have + subnet format. For example: + + :: + + docker_no_proxy: + - 1.2.3.4 + - 5.6.7.8 + +.. _k8s-root-ca-cert-key-r5: + +-------------------------------------- +Kubernetes root CA certificate and key +-------------------------------------- + +By default the Kubernetes Root CA Certificate and Key are auto-generated and +result in the use of self-signed certificates for the Kubernetes API server. In +the case where self-signed certificates are not acceptable, use the bootstrap +override values `k8s_root_ca_cert` and `k8s_root_ca_key` to specify the +certificate and key for the Kubernetes root CA. + +k8s_root_ca_cert + Specifies the certificate for the Kubernetes root CA. The `k8s_root_ca_cert` + value is the absolute path of the certificate file. The certificate must be + in PEM format and the value must be provided as part of a pair with + `k8s_root_ca_key`. The playbook will not proceed if only one value is provided. + +k8s_root_ca_key + Specifies the key for the Kubernetes root CA. The `k8s_root_ca_key` + value is the absolute path of the certificate file. The certificate must be + in PEM format and the value must be provided as part of a pair with + `k8s_root_ca_cert`. The playbook will not proceed if only one value is provided. + +.. important:: + + The default length for the generated Kubernetes root CA certificate is 10 + years. Replacing the root CA certificate is an involved process so the custom + certificate expiry should be as long as possible. We recommend ensuring root + CA certificate has an expiry of at least 5-10 years. + +The administrator can also provide values to add to the Kubernetes API server +certificate Subject Alternative Name list using the 'apiserver_cert_sans` +override parameter. + +apiserver_cert_sans + Specifies a list of Subject Alternative Name entries that will be added to the + Kubernetes API server certificate. Each entry in the list must be an IP address + or domain name. For example: + + :: + + apiserver_cert_sans: + - hostname.domain + - 198.51.100.75 + +StarlingX automatically updates this parameter to include IP records for the OAM +floating IP and both OAM unit IP addresses. + +---------------------------------------------------- +OpenID Connect authentication for Kubernetes cluster +---------------------------------------------------- + +The Kubernetes cluster can be configured to use an external OpenID Connect +:abbr:`IDP (identity provider)`, such as Azure Active Directory, Salesforce, or +Google, for Kubernetes API authentication. + +By default, OpenID Connect authentication is disabled. To enable OpenID Connect, +use the following configuration values in the Ansible bootstrap overrides file +to specify the IDP for OpenID Connect: + +:: + + apiserver_oidc: + client_id: + issuer_url: + username_claim: + +When the three required fields of the `apiserver_oidc` parameter are defined, +OpenID Connect is considered active. The values will be used to configure the +Kubernetes cluster to use the specified external OpenID Connect IDP for +Kubernetes API authentication. + +In addition, you will need to configure the external OpenID Connect IDP and any +required OpenID client application according to the specific IDP's documentation. + +If not configuring OpenID Connect, all values should be absent from the +configuration file. + +.. note:: + + Default authentication via service account tokens is always supported, + even when OpenID Connect authentication is configured. \ No newline at end of file diff --git a/doc/source/deploy_install_guides/r5_release/ansible_install_time_only.txt b/doc/source/deploy_install_guides/r5_release/ansible_install_time_only.txt new file mode 100644 index 000000000..a6cacd159 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/ansible_install_time_only.txt @@ -0,0 +1,7 @@ +.. important:: + + Some Ansible bootstrap parameters cannot be changed or are very difficult to change after installation is complete. + + Review the set of install-time-only parameters before installation and confirm that your values for these parameters are correct for the desired installation. + + Refer to :ref:`Ansible install-time-only parameters ` for details. \ No newline at end of file diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex.rst new file mode 100644 index 000000000..376decf6f --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex.rst @@ -0,0 +1,26 @@ +============================================== +Bare metal All-in-one Duplex Installation R5.0 +============================================== + +-------- +Overview +-------- + +.. include:: ../desc_aio_duplex.txt + +The bare metal AIO-DX deployment configuration may be extended with up to four +worker nodes (not shown in the diagram). Installation instructions for +these additional nodes are described in :doc:`aio_duplex_extend`. + +.. include:: ../ipv6_note.txt + +------------ +Installation +------------ + +.. toctree:: + :maxdepth: 1 + + aio_duplex_hardware + aio_duplex_install_kubernetes + aio_duplex_extend \ No newline at end of file diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex_extend.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex_extend.rst new file mode 100644 index 000000000..b7b7e7bb9 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex_extend.rst @@ -0,0 +1,192 @@ +================================= +Extend Capacity with Worker Nodes +================================= + +This section describes the steps to extend capacity with worker nodes on a +**StarlingX R5.0 bare metal All-in-one Duplex** deployment configuration. + +.. contents:: + :local: + :depth: 1 + +-------------------------------- +Install software on worker nodes +-------------------------------- + +#. Power on the worker node servers and force them to network boot with the + appropriate BIOS boot options for your particular server. + +#. As the worker nodes boot, a message appears on their console instructing + you to configure the personality of the node. + +#. On the console of controller-0, list hosts to see newly discovered worker node + hosts (hostname=None): + + :: + + system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | controller-0 | controller | unlocked | enabled | available | + | 3 | None | None | locked | disabled | offline | + | 4 | None | None | locked | disabled | offline | + +----+--------------+-------------+----------------+-------------+--------------+ + +#. Using the host id, set the personality of this host to 'worker': + + :: + + system host-update 3 personality=worker hostname=worker-0 + system host-update 4 personality=worker hostname=worker-1 + + This initiates the install of software on worker nodes. + This can take 5-10 minutes, depending on the performance of the host machine. + +#. Wait for the install of software on the worker nodes to complete, for the + worker nodes to reboot, and for both to show as locked/disabled/online in + 'system host-list'. + + :: + + system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | controller-1 | controller | unlocked | enabled | available | + | 3 | worker-0 | worker | locked | disabled | online | + | 4 | worker-1 | worker | locked | disabled | online | + +----+--------------+-------------+----------------+-------------+--------------+ + +---------------------- +Configure worker nodes +---------------------- + +#. Assign the cluster-host network to the MGMT interface for the worker nodes: + + (Note that the MGMT interfaces are partially set up automatically by the + network install procedure.) + + :: + + for NODE in worker-0 worker-1; do + system interface-network-assign $NODE mgmt0 cluster-host + done + +#. Configure data interfaces for worker nodes. Use the DATA port names, for + example eth0, that are applicable to your deployment environment. + + .. important:: + + This step is **required** for OpenStack. + + This step is optional for Kubernetes: Do this step if using SRIOV network + attachments in hosted application containers. + + For Kubernetes SRIOV network attachments: + + * Configure SRIOV device plug in: + + :: + + system host-label-assign controller-1 sriovdp=enabled + + * If planning on running DPDK in containers on this host, configure the number + of 1G Huge pages required on both NUMA nodes: + + :: + + system host-memory-modify controller-1 0 -1G 100 + system host-memory-modify controller-1 1 -1G 100 + + For both Kubernetes and OpenStack: + + :: + + DATA0IF= + DATA1IF= + PHYSNET0='physnet0' + PHYSNET1='physnet1' + SPL=/tmp/tmp-system-port-list + SPIL=/tmp/tmp-system-host-if-list + + # configure the datanetworks in sysinv, prior to referencing it + # in the ``system host-if-modify`` command'. + system datanetwork-add ${PHYSNET0} vlan + system datanetwork-add ${PHYSNET1} vlan + + for NODE in worker-0 worker-1; do + echo "Configuring interface for: $NODE" + set -ex + system host-port-list ${NODE} --nowrap > ${SPL} + system host-if-list -a ${NODE} --nowrap > ${SPIL} + DATA0PCIADDR=$(cat $SPL | grep $DATA0IF |awk '{print $8}') + DATA1PCIADDR=$(cat $SPL | grep $DATA1IF |awk '{print $8}') + DATA0PORTUUID=$(cat $SPL | grep ${DATA0PCIADDR} | awk '{print $2}') + DATA1PORTUUID=$(cat $SPL | grep ${DATA1PCIADDR} | awk '{print $2}') + DATA0PORTNAME=$(cat $SPL | grep ${DATA0PCIADDR} | awk '{print $4}') + DATA1PORTNAME=$(cat $SPL | grep ${DATA1PCIADDR} | awk '{print $4}') + DATA0IFUUID=$(cat $SPIL | awk -v DATA0PORTNAME=$DATA0PORTNAME '($12 ~ DATA0PORTNAME) {print $2}') + DATA1IFUUID=$(cat $SPIL | awk -v DATA1PORTNAME=$DATA1PORTNAME '($12 ~ DATA1PORTNAME) {print $2}') + system host-if-modify -m 1500 -n data0 -c data ${NODE} ${DATA0IFUUID} + system host-if-modify -m 1500 -n data1 -c data ${NODE} ${DATA1IFUUID} + system interface-datanetwork-assign ${NODE} ${DATA0IFUUID} ${PHYSNET0} + system interface-datanetwork-assign ${NODE} ${DATA1IFUUID} ${PHYSNET1} + set +ex + done + +************************************* +OpenStack-specific host configuration +************************************* + +.. important:: + + **This step is required only if the StarlingX OpenStack application + (stx-openstack) will be installed.** + +#. **For OpenStack only:** Assign OpenStack host labels to the worker nodes in + support of installing the stx-openstack manifest and helm-charts later. + + :: + + for NODE in worker-0 worker-1; do + system host-label-assign $NODE openstack-compute-node=enabled + system host-label-assign $NODE openvswitch=enabled + system host-label-assign $NODE sriov=enabled + done + +#. **For OpenStack only:** Setup disk partition for nova-local volume group, + needed for stx-openstack nova ephemeral disks. + + :: + + for NODE in worker-0 worker-1; do + echo "Configuring Nova local for: $NODE" + ROOT_DISK=$(system host-show ${NODE} | grep rootfs | awk '{print $4}') + ROOT_DISK_UUID=$(system host-disk-list ${NODE} --nowrap | grep ${ROOT_DISK} | awk '{print $2}') + PARTITION_SIZE=10 + NOVA_PARTITION=$(system host-disk-partition-add -t lvm_phys_vol ${NODE} ${ROOT_DISK_UUID} ${PARTITION_SIZE}) + NOVA_PARTITION_UUID=$(echo ${NOVA_PARTITION} | grep -ow "| uuid | [a-z0-9\-]* |" | awk '{print $4}') + system host-lvg-add ${NODE} nova-local + system host-pv-add ${NODE} nova-local ${NOVA_PARTITION_UUID} + done + + +------------------- +Unlock worker nodes +------------------- + +Unlock worker nodes in order to bring them into service: + +:: + + for NODE in worker-0 worker-1; do + system host-unlock $NODE + done + +The worker nodes will reboot to apply configuration changes and come into +service. This can take 5-10 minutes, depending on the performance of the host +machine. + diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex_hardware.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex_hardware.rst new file mode 100644 index 000000000..5b8f062d9 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex_hardware.rst @@ -0,0 +1,58 @@ +===================== +Hardware Requirements +===================== + +This section describes the hardware requirements and server preparation for a +**StarlingX R5.0 bare metal All-in-one Duplex** deployment configuration. + +.. contents:: + :local: + :depth: 1 + +----------------------------- +Minimum hardware requirements +----------------------------- + +The recommended minimum hardware requirements for bare metal servers for various +host types are: + ++-------------------------+-----------------------------------------------------------+ +| Minimum Requirement | All-in-one Controller Node | ++=========================+===========================================================+ +| Number of servers | 2 | ++-------------------------+-----------------------------------------------------------+ +| Minimum processor class | - Dual-CPU Intel® Xeon® E5 26xx family (SandyBridge) | +| | 8 cores/socket | +| | | +| | or | +| | | +| | - Single-CPU Intel® Xeon® D-15xx family, 8 cores | +| | (low-power/low-cost option) | ++-------------------------+-----------------------------------------------------------+ +| Minimum memory | 64 GB | ++-------------------------+-----------------------------------------------------------+ +| Primary disk | 500 GB SSD or NVMe (see :doc:`../../nvme_config`) | ++-------------------------+-----------------------------------------------------------+ +| Additional disks | - 1 or more 500 GB (min. 10K RPM) for Ceph OSD | +| | - Recommended, but not required: 1 or more SSDs or NVMe | +| | drives for Ceph journals (min. 1024 MiB per OSD journal)| +| | - For OpenStack, recommend 1 or more 500 GB (min. 10K RPM)| +| | for VM local ephemeral storage | ++-------------------------+-----------------------------------------------------------+ +| Minimum network ports | - Mgmt/Cluster: 1x10GE | +| | - OAM: 1x1GE | +| | - Data: 1 or more x 10GE | ++-------------------------+-----------------------------------------------------------+ +| BIOS settings | - Hyper-Threading technology enabled | +| | - Virtualization technology enabled | +| | - VT for directed I/O enabled | +| | - CPU power and performance policy set to performance | +| | - CPU C state control disabled | +| | - Plug & play BMC detection disabled | ++-------------------------+-----------------------------------------------------------+ + +-------------------------- +Prepare bare metal servers +-------------------------- + +.. include:: prep_servers.txt \ No newline at end of file diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex_install_kubernetes.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex_install_kubernetes.rst new file mode 100644 index 000000000..4b6a4fbf0 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex_install_kubernetes.rst @@ -0,0 +1,480 @@ +================================================= +Install StarlingX Kubernetes on Bare Metal AIO-DX +================================================= + +This section describes the steps to install the StarlingX Kubernetes platform +on a **StarlingX R5.0 bare metal All-in-one Duplex** deployment configuration. + +.. contents:: + :local: + :depth: 1 + +--------------------- +Create a bootable USB +--------------------- + +Refer to :doc:`/deploy_install_guides/bootable_usb` for instructions on how to +create a bootable USB with the StarlingX ISO on your system. + +-------------------------------- +Install software on controller-0 +-------------------------------- + +.. include:: aio_simplex_install_kubernetes.rst + :start-after: incl-install-software-controller-0-aio-simplex-start: + :end-before: incl-install-software-controller-0-aio-simplex-end: + +-------------------------------- +Bootstrap system on controller-0 +-------------------------------- + +#. Login using the username / password of "sysadmin" / "sysadmin". + When logging in for the first time, you will be forced to change the password. + + :: + + Login: sysadmin + Password: + Changing password for sysadmin. + (current) UNIX Password: sysadmin + New Password: + (repeat) New Password: + +#. Verify and/or configure IP connectivity. + + External connectivity is required to run the Ansible bootstrap playbook. The + StarlingX boot image will DHCP out all interfaces so the server may have + obtained an IP address and have external IP connectivity if a DHCP server is + present in your environment. Verify this using the :command:`ip addr` and + :command:`ping 8.8.8.8` commands. + + Otherwise, manually configure an IP address and default IP route. Use the + PORT, IP-ADDRESS/SUBNET-LENGTH and GATEWAY-IP-ADDRESS applicable to your + deployment environment. + + :: + + sudo ip address add / dev + sudo ip link set up dev + sudo ip route add default via dev + ping 8.8.8.8 + +#. Specify user configuration overrides for the Ansible bootstrap playbook. + + Ansible is used to bootstrap StarlingX on controller-0. Key files for Ansible + configuration are: + + ``/etc/ansible/hosts`` + The default Ansible inventory file. Contains a single host: localhost. + + ``/usr/share/ansible/stx-ansible/playbooks/bootstrap.yml`` + The Ansible bootstrap playbook. + + ``/usr/share/ansible/stx-ansible/playbooks/host_vars/bootstrap/default.yml`` + The default configuration values for the bootstrap playbook. + + ``sysadmin home directory ($HOME)`` + The default location where Ansible looks for and imports user + configuration override files for hosts. For example: ``$HOME/.yml``. + + .. include:: ../ansible_install_time_only.txt + + Specify the user configuration override file for the Ansible bootstrap + playbook using one of the following methods: + + #. Use a copy of the default.yml file listed above to provide your overrides. + + The default.yml file lists all available parameters for bootstrap + configuration with a brief description for each parameter in the file comments. + + To use this method, copy the default.yml file listed above to + ``$HOME/localhost.yml`` and edit the configurable values as desired. + + #. Create a minimal user configuration override file. + + To use this method, create your override file at ``$HOME/localhost.yml`` + and provide the minimum required parameters for the deployment configuration + as shown in the example below. Use the OAM IP SUBNET and IP ADDRESSing + applicable to your deployment environment. + + :: + + cd ~ + cat < localhost.yml + system_mode: duplex + + dns_servers: + - 8.8.8.8 + - 8.8.4.4 + + external_oam_subnet: / + external_oam_gateway_address: + external_oam_floating_address: + external_oam_node_0_address: + external_oam_node_1_address: + + admin_username: admin + admin_password: + ansible_become_pass: + + # Add these lines to configure Docker to use a proxy server + # docker_http_proxy: http://my.proxy.com:1080 + # docker_https_proxy: https://my.proxy.com:1443 + # docker_no_proxy: + # - 1.2.3.4 + + EOF + + Refer to :doc:`/deploy_install_guides/r5_release/ansible_bootstrap_configs` + for information on additional Ansible bootstrap configurations for advanced + Ansible bootstrap scenarios, such as Docker proxies when deploying behind a + firewall, etc. Refer to :doc:`/../../configuration/docker_proxy_config` for + details about Docker proxy settings. + +#. Run the Ansible bootstrap playbook: + + :: + + ansible-playbook /usr/share/ansible/stx-ansible/playbooks/bootstrap.yml + + Wait for Ansible bootstrap playbook to complete. + This can take 5-10 minutes, depending on the performance of the host machine. + +---------------------- +Configure controller-0 +---------------------- + +#. Acquire admin credentials: + + :: + + source /etc/platform/openrc + +#. Configure the OAM and MGMT interfaces of controller-0 and specify the + attached networks. Use the OAM and MGMT port names, for example eth0, that are + applicable to your deployment environment. + + :: + + OAM_IF= + MGMT_IF= + system host-if-modify controller-0 lo -c none + IFNET_UUIDS=$(system interface-network-list controller-0 | awk '{if ($6=="lo") print $4;}') + for UUID in $IFNET_UUIDS; do + system interface-network-remove ${UUID} + done + system host-if-modify controller-0 $OAM_IF -c platform + system interface-network-assign controller-0 $OAM_IF oam + system host-if-modify controller-0 $MGMT_IF -c platform + system interface-network-assign controller-0 $MGMT_IF mgmt + system interface-network-assign controller-0 $MGMT_IF cluster-host + +#. Configure NTP servers for network time synchronization: + + :: + + system ntp-modify ntpservers=0.pool.ntp.org,1.pool.ntp.org + +#. Configure Ceph storage backend + + .. important:: + + This step is required only if your application requires + persistent storage. + + **If you want to install the StarlingX Openstack application + (stx-openstack) this step is mandatory.** + + :: + + system storage-backend-add ceph --confirmed + +#. Configure data interfaces for controller-0. Use the DATA port names, for example + eth0, applicable to your deployment environment. + + .. important:: + + This step is **required** for OpenStack. + + This step is optional for Kubernetes: Do this step if using SRIOV network + attachments in hosted application containers. + + For Kubernetes SRIOV network attachments: + + * Configure the SRIOV device plugin + + :: + + system host-label-assign controller-0 sriovdp=enabled + + * If planning on running DPDK in containers on this host, configure the number + of 1G Huge pages required on both NUMA nodes. + + :: + + system host-memory-modify controller-0 0 -1G 100 + system host-memory-modify controller-0 1 -1G 100 + + For both Kubernetes and OpenStack: + + :: + + DATA0IF= + DATA1IF= + export NODE=controller-0 + PHYSNET0='physnet0' + PHYSNET1='physnet1' + SPL=/tmp/tmp-system-port-list + SPIL=/tmp/tmp-system-host-if-list + system host-port-list ${NODE} --nowrap > ${SPL} + system host-if-list -a ${NODE} --nowrap > ${SPIL} + DATA0PCIADDR=$(cat $SPL | grep $DATA0IF |awk '{print $8}') + DATA1PCIADDR=$(cat $SPL | grep $DATA1IF |awk '{print $8}') + DATA0PORTUUID=$(cat $SPL | grep ${DATA0PCIADDR} | awk '{print $2}') + DATA1PORTUUID=$(cat $SPL | grep ${DATA1PCIADDR} | awk '{print $2}') + DATA0PORTNAME=$(cat $SPL | grep ${DATA0PCIADDR} | awk '{print $4}') + DATA1PORTNAME=$(cat $SPL | grep ${DATA1PCIADDR} | awk '{print $4}') + DATA0IFUUID=$(cat $SPIL | awk -v DATA0PORTNAME=$DATA0PORTNAME '($12 ~ DATA0PORTNAME) {print $2}') + DATA1IFUUID=$(cat $SPIL | awk -v DATA1PORTNAME=$DATA1PORTNAME '($12 ~ DATA1PORTNAME) {print $2}') + + system datanetwork-add ${PHYSNET0} vlan + system datanetwork-add ${PHYSNET1} vlan + + system host-if-modify -m 1500 -n data0 -c data ${NODE} ${DATA0IFUUID} + system host-if-modify -m 1500 -n data1 -c data ${NODE} ${DATA1IFUUID} + system interface-datanetwork-assign ${NODE} ${DATA0IFUUID} ${PHYSNET0} + system interface-datanetwork-assign ${NODE} ${DATA1IFUUID} ${PHYSNET1} + +#. Add an OSD on controller-0 for Ceph. The following example adds an OSD + to the `sdb` disk: + + .. important:: + + This step requires a configured Ceph storage backend + + :: + + echo ">>> Add OSDs to primary tier" + system host-disk-list controller-0 + system host-disk-list controller-0 | awk '/\/dev\/sdb/{print $2}' | xargs -i system host-stor-add controller-0 {} + system host-stor-list controller-0 + +#. If required, and not already done as part of bootstrap, configure Docker to + use a proxy server. + + #. List Docker proxy parameters: + + :: + + system service-parameter-list platform docker + + #. Refer to :doc:`/../../configuration/docker_proxy_config` for + details about Docker proxy settings. + +************************************* +OpenStack-specific host configuration +************************************* + +.. include:: aio_simplex_install_kubernetes.rst + :start-after: incl-config-controller-0-openstack-specific-aio-simplex-start: + :end-before: incl-config-controller-0-openstack-specific-aio-simplex-end: + +------------------- +Unlock controller-0 +------------------- + +.. include:: aio_simplex_install_kubernetes.rst + :start-after: incl-unlock-controller-0-aio-simplex-start: + :end-before: incl-unlock-controller-0-aio-simplex-end: + +------------------------------------- +Install software on controller-1 node +------------------------------------- + +#. Power on the controller-1 server and force it to network boot with the + appropriate BIOS boot options for your particular server. + +#. As controller-1 boots, a message appears on its console instructing you to + configure the personality of the node. + +#. On the console of controller-0, list hosts to see newly discovered controller-1 + host (hostname=None): + + :: + + system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | None | None | locked | disabled | offline | + +----+--------------+-------------+----------------+-------------+--------------+ + +#. Using the host id, set the personality of this host to 'controller': + + :: + + system host-update 2 personality=controller + +#. Wait for the software installation on controller-1 to complete, for controller-1 to + reboot, and for controller-1 to show as locked/disabled/online in 'system host-list'. + + This can take 5-10 minutes, depending on the performance of the host machine. + + :: + + system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | controller-1 | controller | locked | disabled | online | + +----+--------------+-------------+----------------+-------------+--------------+ + +---------------------- +Configure controller-1 +---------------------- + +#. Configure the OAM and MGMT interfaces of controller-1 and specify the + attached networks. Use the OAM and MGMT port names, for example eth0, that are + applicable to your deployment environment: + + (Note that the MGMT interface is partially set up automatically by the network + install procedure.) + + :: + + OAM_IF= + MGMT_IF= + system host-if-modify controller-1 $OAM_IF -c platform + system interface-network-assign controller-1 $OAM_IF oam + system interface-network-assign controller-1 mgmt0 cluster-host + +#. Configure data interfaces for controller-1. Use the DATA port names, for example + eth0, applicable to your deployment environment. + + .. important:: + + This step is **required** for OpenStack. + + This step is optional for Kubernetes: Do this step if using SRIOV network + attachments in hosted application containers. + + For Kubernetes SRIOV network attachments: + + * Configure the SRIOV device plugin: + + :: + + system host-label-assign controller-1 sriovdp=enabled + + * If planning on running DPDK in containers on this host, configure the number + of 1G Huge pages required on both NUMA nodes: + + :: + + system host-memory-modify controller-1 0 -1G 100 + system host-memory-modify controller-1 1 -1G 100 + + + For both Kubernetes and OpenStack: + + :: + + DATA0IF= + DATA1IF= + export NODE=controller-1 + PHYSNET0='physnet0' + PHYSNET1='physnet1' + SPL=/tmp/tmp-system-port-list + SPIL=/tmp/tmp-system-host-if-list + system host-port-list ${NODE} --nowrap > ${SPL} + system host-if-list -a ${NODE} --nowrap > ${SPIL} + DATA0PCIADDR=$(cat $SPL | grep $DATA0IF |awk '{print $8}') + DATA1PCIADDR=$(cat $SPL | grep $DATA1IF |awk '{print $8}') + DATA0PORTUUID=$(cat $SPL | grep ${DATA0PCIADDR} | awk '{print $2}') + DATA1PORTUUID=$(cat $SPL | grep ${DATA1PCIADDR} | awk '{print $2}') + DATA0PORTNAME=$(cat $SPL | grep ${DATA0PCIADDR} | awk '{print $4}') + DATA1PORTNAME=$(cat $SPL | grep ${DATA1PCIADDR} | awk '{print $4}') + DATA0IFUUID=$(cat $SPIL | awk -v DATA0PORTNAME=$DATA0PORTNAME '($12 ~ DATA0PORTNAME) {print $2}') + DATA1IFUUID=$(cat $SPIL | awk -v DATA1PORTNAME=$DATA1PORTNAME '($12 ~ DATA1PORTNAME) {print $2}') + + system datanetwork-add ${PHYSNET0} vlan + system datanetwork-add ${PHYSNET1} vlan + + system host-if-modify -m 1500 -n data0 -c data ${NODE} ${DATA0IFUUID} + system host-if-modify -m 1500 -n data1 -c data ${NODE} ${DATA1IFUUID} + system interface-datanetwork-assign ${NODE} ${DATA0IFUUID} ${PHYSNET0} + system interface-datanetwork-assign ${NODE} ${DATA1IFUUID} ${PHYSNET1} + +#. Add an OSD on controller-1 for Ceph: + + .. important:: + + This step requires a configured Ceph storage backend + + :: + + echo ">>> Add OSDs to primary tier" + system host-disk-list controller-1 + system host-disk-list controller-1 | awk '/\/dev\/sdb/{print $2}' | xargs -i system host-stor-add controller-1 {} + system host-stor-list controller-1 + +************************************* +OpenStack-specific host configuration +************************************* + +.. important:: + + **This step is required only if the StarlingX OpenStack application + (stx-openstack) will be installed.** + +#. **For OpenStack only:** Assign OpenStack host labels to controller-1 in + support of installing the stx-openstack manifest and helm-charts later. + + :: + + system host-label-assign controller-1 openstack-control-plane=enabled + system host-label-assign controller-1 openstack-compute-node=enabled + system host-label-assign controller-1 openvswitch=enabled + system host-label-assign controller-1 sriov=enabled + +#. **For OpenStack only:** Set up disk partition for nova-local volume group, + which is needed for stx-openstack nova ephemeral disks. + + :: + + export NODE=controller-1 + + echo ">>> Getting root disk info" + ROOT_DISK=$(system host-show ${NODE} | grep rootfs | awk '{print $4}') + ROOT_DISK_UUID=$(system host-disk-list ${NODE} --nowrap | grep ${ROOT_DISK} | awk '{print $2}') + echo "Root disk: $ROOT_DISK, UUID: $ROOT_DISK_UUID" + + echo ">>>> Configuring nova-local" + NOVA_SIZE=34 + NOVA_PARTITION=$(system host-disk-partition-add -t lvm_phys_vol ${NODE} ${ROOT_DISK_UUID} ${NOVA_SIZE}) + NOVA_PARTITION_UUID=$(echo ${NOVA_PARTITION} | grep -ow "| uuid | [a-z0-9\-]* |" | awk '{print $4}') + system host-lvg-add ${NODE} nova-local + system host-pv-add ${NODE} nova-local ${NOVA_PARTITION_UUID} + sleep 2 + +------------------- +Unlock controller-1 +------------------- + +Unlock controller-1 in order to bring it into service: + +:: + + system host-unlock controller-1 + +Controller-1 will reboot in order to apply configuration changes and come into +service. This can take 5-10 minutes, depending on the performance of the host +machine. + +---------- +Next steps +---------- + +.. include:: ../kubernetes_install_next.txt diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/aio_simplex.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_simplex.rst new file mode 100644 index 000000000..b704e0316 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_simplex.rst @@ -0,0 +1,21 @@ +=============================================== +Bare metal All-in-one Simplex Installation R5.0 +=============================================== + +-------- +Overview +-------- + +.. include:: ../desc_aio_simplex.txt + +.. include:: ../ipv6_note.txt + +------------ +Installation +------------ + +.. toctree:: + :maxdepth: 1 + + aio_simplex_hardware + aio_simplex_install_kubernetes diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/aio_simplex_hardware.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_simplex_hardware.rst new file mode 100644 index 000000000..18b6cec54 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_simplex_hardware.rst @@ -0,0 +1,58 @@ +===================== +Hardware Requirements +===================== + +This section describes the hardware requirements and server preparation for a +**StarlingX R5.0 bare metal All-in-one Simplex** deployment configuration. + +.. contents:: + :local: + :depth: 1 + +----------------------------- +Minimum hardware requirements +----------------------------- + +The recommended minimum hardware requirements for bare metal servers for various +host types are: + ++-------------------------+-----------------------------------------------------------+ +| Minimum Requirement | All-in-one Controller Node | ++=========================+===========================================================+ +| Number of servers | 1 | ++-------------------------+-----------------------------------------------------------+ +| Minimum processor class | - Dual-CPU Intel® Xeon® E5 26xx family (SandyBridge) | +| | 8 cores/socket | +| | | +| | or | +| | | +| | - Single-CPU Intel® Xeon® D-15xx family, 8 cores | +| | (low-power/low-cost option) | ++-------------------------+-----------------------------------------------------------+ +| Minimum memory | 64 GB | ++-------------------------+-----------------------------------------------------------+ +| Primary disk | 500 GB SSD or NVMe (see :doc:`../../nvme_config`) | ++-------------------------+-----------------------------------------------------------+ +| Additional disks | - 1 or more 500 GB (min. 10K RPM) for Ceph OSD | +| | - Recommended, but not required: 1 or more SSDs or NVMe | +| | drives for Ceph journals (min. 1024 MiB per OSD | +| | journal) | +| | - For OpenStack, recommend 1 or more 500 GB (min. 10K | +| | RPM) for VM local ephemeral storage | ++-------------------------+-----------------------------------------------------------+ +| Minimum network ports | - OAM: 1x1GE | +| | - Data: 1 or more x 10GE | ++-------------------------+-----------------------------------------------------------+ +| BIOS settings | - Hyper-Threading technology enabled | +| | - Virtualization technology enabled | +| | - VT for directed I/O enabled | +| | - CPU power and performance policy set to performance | +| | - CPU C state control disabled | +| | - Plug & play BMC detection disabled | ++-------------------------+-----------------------------------------------------------+ + +-------------------------- +Prepare bare metal servers +-------------------------- + +.. include:: prep_servers.txt \ No newline at end of file diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/aio_simplex_install_kubernetes.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_simplex_install_kubernetes.rst new file mode 100644 index 000000000..d700f88b5 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_simplex_install_kubernetes.rst @@ -0,0 +1,413 @@ +================================================= +Install StarlingX Kubernetes on Bare Metal AIO-SX +================================================= + +This section describes the steps to install the StarlingX Kubernetes platform +on a **StarlingX R5.0 bare metal All-in-one Simplex** deployment configuration. + +.. contents:: + :local: + :depth: 1 + +--------------------- +Create a bootable USB +--------------------- + +Refer to :doc:`/deploy_install_guides/bootable_usb` for instructions on how to +create a bootable USB with the StarlingX ISO on your system. + +-------------------------------- +Install software on controller-0 +-------------------------------- + +.. incl-install-software-controller-0-aio-simplex-start: + +#. Insert the bootable USB into a bootable USB port on the host you are + configuring as controller-0. + +#. Power on the host. + +#. Attach to a console, ensure the host boots from the USB, and wait for the + StarlingX Installer Menus. + +#. Make the following menu selections in the installer: + + #. First menu: Select 'All-in-one Controller Configuration' + #. Second menu: Select 'Graphical Console' or 'Textual Console' depending on + your terminal access to the console port + +#. Wait for non-interactive install of software to complete and server to reboot. + This can take 5-10 minutes, depending on the performance of the server. + +.. incl-install-software-controller-0-aio-simplex-end: + +-------------------------------- +Bootstrap system on controller-0 +-------------------------------- + +#. Login using the username / password of "sysadmin" / "sysadmin". + When logging in for the first time, you will be forced to change the password. + + :: + + Login: sysadmin + Password: + Changing password for sysadmin. + (current) UNIX Password: sysadmin + New Password: + (repeat) New Password: + +#. Verify and/or configure IP connectivity. + + External connectivity is required to run the Ansible bootstrap playbook. The + StarlingX boot image will DHCP out all interfaces so the server may have + obtained an IP address and have external IP connectivity if a DHCP server is + present in your environment. Verify this using the :command:`ip addr` and + :command:`ping 8.8.8.8` commands. + + Otherwise, manually configure an IP address and default IP route. Use the + PORT, IP-ADDRESS/SUBNET-LENGTH and GATEWAY-IP-ADDRESS applicable to your + deployment environment. + + :: + + sudo ip address add / dev + sudo ip link set up dev + sudo ip route add default via dev + ping 8.8.8.8 + +#. Specify user configuration overrides for the Ansible bootstrap playbook. + + Ansible is used to bootstrap StarlingX on controller-0. Key files for Ansible + configuration are: + + ``/etc/ansible/hosts`` + The default Ansible inventory file. Contains a single host: localhost. + + ``/usr/share/ansible/stx-ansible/playbooks/bootstrap.yml`` + The Ansible bootstrap playbook. + + ``/usr/share/ansible/stx-ansible/playbooks/host_vars/bootstrap/default.yml`` + The default configuration values for the bootstrap playbook. + + ``sysadmin home directory ($HOME)`` + The default location where Ansible looks for and imports user + configuration override files for hosts. For example: ``$HOME/.yml``. + + .. include:: ../ansible_install_time_only.txt + + Specify the user configuration override file for the Ansible bootstrap + playbook using one of the following methods: + + #. Use a copy of the default.yml file listed above to provide your overrides. + + The default.yml file lists all available parameters for bootstrap + configuration with a brief description for each parameter in the file comments. + + To use this method, copy the default.yml file listed above to + ``$HOME/localhost.yml`` and edit the configurable values as desired. + + #. Create a minimal user configuration override file. + + To use this method, create your override file at ``$HOME/localhost.yml`` + and provide the minimum required parameters for the deployment configuration + as shown in the example below. Use the OAM IP SUBNET and IP ADDRESSing + applicable to your deployment environment. + + :: + + cd ~ + cat < localhost.yml + system_mode: simplex + + dns_servers: + - 8.8.8.8 + - 8.8.4.4 + + external_oam_subnet: / + external_oam_gateway_address: + external_oam_floating_address: + + admin_username: admin + admin_password: + ansible_become_pass: + + # Add these lines to configure Docker to use a proxy server + # docker_http_proxy: http://my.proxy.com:1080 + # docker_https_proxy: https://my.proxy.com:1443 + # docker_no_proxy: + # - 1.2.3.4 + + EOF + + Refer to :doc:`/deploy_install_guides/r5_release/ansible_bootstrap_configs` + for information on additional Ansible bootstrap configurations for advanced + Ansible bootstrap scenarios, such as Docker proxies when deploying behind a + firewall, etc. Refer to :doc:`/../../configuration/docker_proxy_config` for + details about Docker proxy settings. + +#. Run the Ansible bootstrap playbook: + + :: + + ansible-playbook /usr/share/ansible/stx-ansible/playbooks/bootstrap.yml + + Wait for Ansible bootstrap playbook to complete. + This can take 5-10 minutes, depending on the performance of the host machine. + +---------------------- +Configure controller-0 +---------------------- + +#. Acquire admin credentials: + + :: + + source /etc/platform/openrc + +#. Configure the OAM interface of controller-0 and specify the attached network + as "oam". Use the OAM port name that is applicable to your deployment + environment, for example eth0: + + :: + + OAM_IF= + system host-if-modify controller-0 $OAM_IF -c platform + system interface-network-assign controller-0 $OAM_IF oam + +#. Configure NTP servers for network time synchronization: + + :: + + system ntp-modify ntpservers=0.pool.ntp.org,1.pool.ntp.org + +#. Configure Ceph storage backend + + .. important:: + + This step required only if your application requires + persistent storage. + + **If you want to install the StarlingX Openstack application + (stx-openstack) this step is mandatory.** + + :: + + system storage-backend-add ceph --confirmed + +#. Configure data interfaces for controller-0. Use the DATA port names, for example + eth0, applicable to your deployment environment. + + .. important:: + + This step is **required** for OpenStack. + + This step is optional for Kubernetes: Do this step if using SRIOV network + attachments in hosted application containers. + + For Kubernetes SRIOV network attachments: + + * Configure the SRIOV device plugin + + :: + + system host-label-assign controller-0 sriovdp=enabled + + * If planning on running DPDK in containers on this host, configure the number + of 1G Huge pages required on both NUMA nodes. + + :: + + system host-memory-modify controller-0 0 -1G 100 + system host-memory-modify controller-0 1 -1G 100 + + For both Kubernetes and OpenStack: + + :: + + DATA0IF= + DATA1IF= + export NODE=controller-0 + PHYSNET0='physnet0' + PHYSNET1='physnet1' + SPL=/tmp/tmp-system-port-list + SPIL=/tmp/tmp-system-host-if-list + system host-port-list ${NODE} --nowrap > ${SPL} + system host-if-list -a ${NODE} --nowrap > ${SPIL} + DATA0PCIADDR=$(cat $SPL | grep $DATA0IF |awk '{print $8}') + DATA1PCIADDR=$(cat $SPL | grep $DATA1IF |awk '{print $8}') + DATA0PORTUUID=$(cat $SPL | grep ${DATA0PCIADDR} | awk '{print $2}') + DATA1PORTUUID=$(cat $SPL | grep ${DATA1PCIADDR} | awk '{print $2}') + DATA0PORTNAME=$(cat $SPL | grep ${DATA0PCIADDR} | awk '{print $4}') + DATA1PORTNAME=$(cat $SPL | grep ${DATA1PCIADDR} | awk '{print $4}') + DATA0IFUUID=$(cat $SPIL | awk -v DATA0PORTNAME=$DATA0PORTNAME '($12 ~ DATA0PORTNAME) {print $2}') + DATA1IFUUID=$(cat $SPIL | awk -v DATA1PORTNAME=$DATA1PORTNAME '($12 ~ DATA1PORTNAME) {print $2}') + + system datanetwork-add ${PHYSNET0} vlan + system datanetwork-add ${PHYSNET1} vlan + + system host-if-modify -m 1500 -n data0 -c data ${NODE} ${DATA0IFUUID} + system host-if-modify -m 1500 -n data1 -c data ${NODE} ${DATA1IFUUID} + system interface-datanetwork-assign ${NODE} ${DATA0IFUUID} ${PHYSNET0} + system interface-datanetwork-assign ${NODE} ${DATA1IFUUID} ${PHYSNET1} + +#. Add an OSD on controller-0 for Ceph. The following example adds an OSD + to the `sdb` disk: + + .. important:: + + This step requires a configured Ceph storage backend + + :: + + echo ">>> Add OSDs to primary tier" + system host-disk-list controller-0 + system host-disk-list controller-0 | awk '/\/dev\/sdb/{print $2}' | xargs -i system host-stor-add controller-0 {} + system host-stor-list controller-0 + +#. If required, and not already done as part of bootstrap, configure Docker to + use a proxy server. + + #. List Docker proxy parameters: + + :: + + system service-parameter-list platform docker + + #. Refer to :doc:`/../../configuration/docker_proxy_config` for + details about Docker proxy settings. + +************************************* +OpenStack-specific host configuration +************************************* + +.. incl-config-controller-0-openstack-specific-aio-simplex-start: + +.. important:: + + **This step is required only if the StarlingX OpenStack application + (stx-openstack) will be installed.** + +#. **For OpenStack only:** Assign OpenStack host labels to controller-0 in + support of installing the stx-openstack manifest and helm-charts later. + + :: + + system host-label-assign controller-0 openstack-control-plane=enabled + system host-label-assign controller-0 openstack-compute-node=enabled + system host-label-assign controller-0 openvswitch=enabled + system host-label-assign controller-0 sriov=enabled + +#. **For OpenStack only:** Configure the system setting for the vSwitch. + + StarlingX has OVS (kernel-based) vSwitch configured as default: + + * Runs in a container; defined within the helm charts of stx-openstack + manifest. + * Shares the core(s) assigned to the platform. + + If you require better performance, OVS-DPDK (OVS with the Data Plane + Development Kit, which is supported only on bare metal hardware) should be + used: + + * Runs directly on the host (it is not containerized). + * Requires that at least 1 core be assigned/dedicated to the vSwitch function. + + To deploy the default containerized OVS: + + :: + + system modify --vswitch_type none + + Do not run any vSwitch directly on the host, instead, use the containerized + OVS defined in the helm charts of stx-openstack manifest. + + To deploy OVS-DPDK, run the following command: + + :: + + system modify --vswitch_type ovs-dpdk + system host-cpu-modify -f vswitch -p0 1 controller-0 + + Once vswitch_type is set to OVS-DPDK, any subsequent nodes created will + default to automatically assigning 1 vSwitch core for AIO controllers and 2 + vSwitch cores for compute-labeled worker nodes. + + When using OVS-DPDK, configure vSwitch memory per NUMA node with the following + command: + + :: + + system host-memory-modify -f -1G <1G hugepages number> + + For example: + + :: + + system host-memory-modify -f vswitch -1G 1 worker-0 0 + + VMs created in an OVS-DPDK environment must be configured to use huge pages + to enable networking and must use a flavor with property: hw:mem_page_size=large + + Configure the huge pages for VMs in an OVS-DPDK environment with the command: + + :: + + system host-memory-modify -1G <1G hugepages number> + + For example: + + :: + + system host-memory-modify worker-0 0 -1G 10 + + .. note:: + + After controller-0 is unlocked, changing vswitch_type requires + locking and unlocking all compute-labeled worker nodes (and/or AIO + controllers) to apply the change. + +#. **For OpenStack only:** Set up disk partition for nova-local volume group, + which is needed for stx-openstack nova ephemeral disks. + + :: + + export NODE=controller-0 + + echo ">>> Getting root disk info" + ROOT_DISK=$(system host-show ${NODE} | grep rootfs | awk '{print $4}') + ROOT_DISK_UUID=$(system host-disk-list ${NODE} --nowrap | grep ${ROOT_DISK} | awk '{print $2}') + echo "Root disk: $ROOT_DISK, UUID: $ROOT_DISK_UUID" + + echo ">>>> Configuring nova-local" + NOVA_SIZE=34 + NOVA_PARTITION=$(system host-disk-partition-add -t lvm_phys_vol ${NODE} ${ROOT_DISK_UUID} ${NOVA_SIZE}) + NOVA_PARTITION_UUID=$(echo ${NOVA_PARTITION} | grep -ow "| uuid | [a-z0-9\-]* |" | awk '{print $4}') + system host-lvg-add ${NODE} nova-local + system host-pv-add ${NODE} nova-local ${NOVA_PARTITION_UUID} + sleep 2 + +.. incl-config-controller-0-openstack-specific-aio-simplex-end: + +------------------- +Unlock controller-0 +------------------- + +.. incl-unlock-controller-0-aio-simplex-start: + +Unlock controller-0 in order to bring it into service: + +:: + + system host-unlock controller-0 + +Controller-0 will reboot in order to apply configuration changes and come into +service. This can take 5-10 minutes, depending on the performance of the host machine. + +.. incl-unlock-controller-0-aio-simplex-end: + +---------- +Next steps +---------- + +.. include:: ../kubernetes_install_next.txt diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/controller_storage.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/controller_storage.rst new file mode 100644 index 000000000..8139e3e95 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/controller_storage.rst @@ -0,0 +1,22 @@ +============================================================= +Bare metal Standard with Controller Storage Installation R5.0 +============================================================= + +-------- +Overview +-------- + +.. include:: ../desc_controller_storage.txt + +.. include:: ../ipv6_note.txt + + +------------ +Installation +------------ + +.. toctree:: + :maxdepth: 1 + + controller_storage_hardware + controller_storage_install_kubernetes diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/controller_storage_hardware.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/controller_storage_hardware.rst new file mode 100644 index 000000000..c79456547 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/controller_storage_hardware.rst @@ -0,0 +1,56 @@ +===================== +Hardware Requirements +===================== + +This section describes the hardware requirements and server preparation for a +**StarlingX R5.0 bare metal Standard with Controller Storage** deployment +configuration. + +.. contents:: + :local: + :depth: 1 + +----------------------------- +Minimum hardware requirements +----------------------------- + +The recommended minimum hardware requirements for bare metal servers for various +host types are: + ++-------------------------+-----------------------------+-----------------------------+ +| Minimum Requirement | Controller Node | Worker Node | ++=========================+=============================+=============================+ +| Number of servers | 2 | 2-10 | ++-------------------------+-----------------------------+-----------------------------+ +| Minimum processor class | - Dual-CPU Intel® Xeon® E5 26xx family (SandyBridge) | +| | 8 cores/socket | ++-------------------------+-----------------------------+-----------------------------+ +| Minimum memory | 64 GB | 32 GB | ++-------------------------+-----------------------------+-----------------------------+ +| Primary disk | 500 GB SSD or NVMe (see | 120 GB (Minimum 10k RPM) | +| | :doc:`../../nvme_config`) | | ++-------------------------+-----------------------------+-----------------------------+ +| Additional disks | - 1 or more 500 GB (min. | - For OpenStack, recommend | +| | 10K RPM) for Ceph OSD | 1 or more 500 GB (min. | +| | - Recommended, but not | 10K RPM) for VM local | +| | required: 1 or more SSDs | ephemeral storage | +| | or NVMe drives for Ceph | | +| | journals (min. 1024 MiB | | +| | per OSD journal) | | ++-------------------------+-----------------------------+-----------------------------+ +| Minimum network ports | - Mgmt/Cluster: 1x10GE | - Mgmt/Cluster: 1x10GE | +| | - OAM: 1x1GE | - Data: 1 or more x 10GE | ++-------------------------+-----------------------------+-----------------------------+ +| BIOS settings | - Hyper-Threading technology enabled | +| | - Virtualization technology enabled | +| | - VT for directed I/O enabled | +| | - CPU power and performance policy set to performance | +| | - CPU C state control disabled | +| | - Plug & play BMC detection disabled | ++-------------------------+-----------------------------+-----------------------------+ + +-------------------------- +Prepare bare metal servers +-------------------------- + +.. include:: prep_servers.txt \ No newline at end of file diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/controller_storage_install_kubernetes.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/controller_storage_install_kubernetes.rst new file mode 100644 index 000000000..35b9cd7c9 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/controller_storage_install_kubernetes.rst @@ -0,0 +1,655 @@ +=========================================================================== +Install StarlingX Kubernetes on Bare Metal Standard with Controller Storage +=========================================================================== + +This section describes the steps to install the StarlingX Kubernetes platform +on a **StarlingX R5.0 bare metal Standard with Controller Storage** deployment +configuration. + +.. contents:: + :local: + :depth: 1 + +------------------- +Create bootable USB +------------------- + +Refer to :doc:`/deploy_install_guides/bootable_usb` for instructions on how to +create a bootable USB with the StarlingX ISO on your system. + +-------------------------------- +Install software on controller-0 +-------------------------------- + +.. incl-install-software-controller-0-standard-start: + +#. Insert the bootable USB into a bootable USB port on the host you are + configuring as controller-0. + +#. Power on the host. + +#. Attach to a console, ensure the host boots from the USB, and wait for the + StarlingX Installer Menus. + +#. Make the following menu selections in the installer: + + #. First menu: Select 'Standard Controller Configuration' + #. Second menu: Select 'Graphical Console' or 'Textual Console' depending on + your terminal access to the console port + +#. Wait for non-interactive install of software to complete and server to reboot. + This can take 5-10 minutes, depending on the performance of the server. + +.. incl-install-software-controller-0-standard-end: + +-------------------------------- +Bootstrap system on controller-0 +-------------------------------- + +.. incl-bootstrap-sys-controller-0-standard-start: + +#. Login using the username / password of "sysadmin" / "sysadmin". + + When logging in for the first time, you will be forced to change the password. + + :: + + Login: sysadmin + Password: + Changing password for sysadmin. + (current) UNIX Password: sysadmin + New Password: + (repeat) New Password: + +#. Verify and/or configure IP connectivity. + + External connectivity is required to run the Ansible bootstrap playbook. The + StarlingX boot image will DHCP out all interfaces so the server may have + obtained an IP address and have external IP connectivity if a DHCP server is + present in your environment. Verify this using the :command:`ip addr` and + :command:`ping 8.8.8.8` commands. + + Otherwise, manually configure an IP address and default IP route. Use the + PORT, IP-ADDRESS/SUBNET-LENGTH and GATEWAY-IP-ADDRESS applicable to your + deployment environment. + + :: + + sudo ip address add / dev + sudo ip link set up dev + sudo ip route add default via dev + ping 8.8.8.8 + +#. Specify user configuration overrides for the Ansible bootstrap playbook. + + Ansible is used to bootstrap StarlingX on controller-0. Key files for Ansible + configuration are: + + ``/etc/ansible/hosts`` + The default Ansible inventory file. Contains a single host: localhost. + + ``/usr/share/ansible/stx-ansible/playbooks/bootstrap.yml`` + The Ansible bootstrap playbook. + + ``/usr/share/ansible/stx-ansible/playbooks/host_vars/bootstrap/default.yml`` + The default configuration values for the bootstrap playbook. + + ``sysadmin home directory ($HOME)`` + The default location where Ansible looks for and imports user + configuration override files for hosts. For example: ``$HOME/.yml``. + + .. include:: ../ansible_install_time_only.txt + + Specify the user configuration override file for the Ansible bootstrap + playbook using one of the following methods: + + #. Use a copy of the default.yml file listed above to provide your overrides. + + The default.yml file lists all available parameters for bootstrap + configuration with a brief description for each parameter in the file comments. + + To use this method, copy the default.yml file listed above to + ``$HOME/localhost.yml`` and edit the configurable values as desired. + + #. Create a minimal user configuration override file. + + To use this method, create your override file at ``$HOME/localhost.yml`` + and provide the minimum required parameters for the deployment configuration + as shown in the example below. Use the OAM IP SUBNET and IP ADDRESSing + applicable to your deployment environment. + + :: + + cd ~ + cat < localhost.yml + system_mode: duplex + + dns_servers: + - 8.8.8.8 + - 8.8.4.4 + + external_oam_subnet: / + external_oam_gateway_address: + external_oam_floating_address: + external_oam_node_0_address: + external_oam_node_1_address: + + admin_username: admin + admin_password: + ansible_become_pass: + + # Add these lines to configure Docker to use a proxy server + # docker_http_proxy: http://my.proxy.com:1080 + # docker_https_proxy: https://my.proxy.com:1443 + # docker_no_proxy: + # - 1.2.3.4 + + EOF + + Refer to :doc:`/deploy_install_guides/r5_release/ansible_bootstrap_configs` + for information on additional Ansible bootstrap configurations for advanced + Ansible bootstrap scenarios, such as Docker proxies when deploying behind a + firewall, etc. Refer to :doc:`/../../configuration/docker_proxy_config` for + details about Docker proxy settings. + +#. Run the Ansible bootstrap playbook: + + :: + + ansible-playbook /usr/share/ansible/stx-ansible/playbooks/bootstrap.yml + + Wait for Ansible bootstrap playbook to complete. + This can take 5-10 minutes, depending on the performance of the host machine. + +.. incl-bootstrap-sys-controller-0-standard-end: + + +---------------------- +Configure controller-0 +---------------------- + +.. incl-config-controller-0-storage-start: + +#. Acquire admin credentials: + + :: + + source /etc/platform/openrc + +#. Configure the OAM and MGMT interfaces of controller-0 and specify the + attached networks. Use the OAM and MGMT port names, for example eth0, that are + applicable to your deployment environment. + + :: + + OAM_IF= + MGMT_IF= + system host-if-modify controller-0 lo -c none + IFNET_UUIDS=$(system interface-network-list controller-0 | awk '{if ($6=="lo") print $4;}') + for UUID in $IFNET_UUIDS; do + system interface-network-remove ${UUID} + done + system host-if-modify controller-0 $OAM_IF -c platform + system interface-network-assign controller-0 $OAM_IF oam + system host-if-modify controller-0 $MGMT_IF -c platform + system interface-network-assign controller-0 $MGMT_IF mgmt + system interface-network-assign controller-0 $MGMT_IF cluster-host + +#. Configure NTP servers for network time synchronization: + + :: + + system ntp-modify ntpservers=0.pool.ntp.org,1.pool.ntp.org + +#. Configure Ceph storage backend + + .. important:: + + This step required only if your application requires + persistent storage. + + **If you want to install the StarlingX Openstack application + (stx-openstack) this step is mandatory.** + + :: + + system storage-backend-add ceph --confirmed + +#. If required, and not already done as part of bootstrap, configure Docker to + use a proxy server. + + #. List Docker proxy parameters: + + :: + + system service-parameter-list platform docker + + #. Refer to :doc:`/../../configuration/docker_proxy_config` for + details about Docker proxy settings. + +************************************* +OpenStack-specific host configuration +************************************* + +.. important:: + + **This step is required only if the StarlingX OpenStack application + (stx-openstack) will be installed.** + +#. **For OpenStack only:** Assign OpenStack host labels to controller-0 in + support of installing the stx-openstack manifest and helm-charts later. + + :: + + system host-label-assign controller-0 openstack-control-plane=enabled + +#. **For OpenStack only:** Configure the system setting for the vSwitch. + + StarlingX has OVS (kernel-based) vSwitch configured as default: + + * Runs in a container; defined within the helm charts of stx-openstack + manifest. + * Shares the core(s) assigned to the platform. + + If you require better performance, OVS-DPDK (OVS with the Data Plane + Development Kit, which is supported only on bare metal hardware) should be + used: + + * Runs directly on the host (it is not containerized). + * Requires that at least 1 core be assigned/dedicated to the vSwitch function. + + To deploy the default containerized OVS: + + :: + + system modify --vswitch_type none + + Do not run any vSwitch directly on the host, instead, use the containerized + OVS defined in the helm charts of stx-openstack manifest. + + To deploy OVS-DPDK, run the following command: + + :: + + system modify --vswitch_type ovs-dpdk + system host-cpu-modify -f vswitch -p0 1 controller-0 + + Once vswitch_type is set to OVS-DPDK, any subsequent nodes created will + default to automatically assigning 1 vSwitch core for AIO controllers and 2 + vSwitch cores for compute-labeled worker nodes. + + When using OVS-DPDK, configure vSwitch memory per NUMA node with the following + command: + + :: + + system host-memory-modify -f -1G <1G hugepages number> + + For example: + + :: + + system host-memory-modify -f vswitch -1G 1 worker-0 0 + + VMs created in an OVS-DPDK environment must be configured to use huge pages + to enable networking and must use a flavor with property: hw:mem_page_size=large + + Configure the huge pages for VMs in an OVS-DPDK environment with the command: + + :: + + system host-memory-modify -1G <1G hugepages number> + + For example: + + :: + + system host-memory-modify worker-0 0 -1G 10 + + .. note:: + + After controller-0 is unlocked, changing vswitch_type requires + locking and unlocking all compute-labeled worker nodes (and/or AIO + controllers) to apply the change. + +.. incl-config-controller-0-storage-end: + +------------------- +Unlock controller-0 +------------------- + +Unlock controller-0 in order to bring it into service: + +:: + + system host-unlock controller-0 + +Controller-0 will reboot in order to apply configuration changes and come into +service. This can take 5-10 minutes, depending on the performance of the host machine. + +------------------------------------------------- +Install software on controller-1 and worker nodes +------------------------------------------------- + +#. Power on the controller-1 server and force it to network boot with the + appropriate BIOS boot options for your particular server. + +#. As controller-1 boots, a message appears on its console instructing you to + configure the personality of the node. + +#. On the console of controller-0, list hosts to see newly discovered controller-1 + host (hostname=None): + + :: + + system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | None | None | locked | disabled | offline | + +----+--------------+-------------+----------------+-------------+--------------+ + +#. Using the host id, set the personality of this host to 'controller': + + :: + + system host-update 2 personality=controller + + This initiates the install of software on controller-1. + This can take 5-10 minutes, depending on the performance of the host machine. + +#. While waiting for the previous step to complete, power on the worker nodes. + Set the personality to 'worker' and assign a unique hostname for each. + + For example, power on worker-0 and wait for the new host (hostname=None) to + be discovered by checking 'system host-list': + + :: + + system host-update 3 personality=worker hostname=worker-0 + + Repeat for worker-1. Power on worker-1 and wait for the new host (hostname=None) to + be discovered by checking 'system host-list': + + :: + + system host-update 4 personality=worker hostname=worker-1-1 + +#. Wait for the software installation on controller-1, worker-0, and worker-1 to + complete, for all servers to reboot, and for all to show as locked/disabled/online in + 'system host-list'. + + :: + + system host-list + + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | controller-1 | controller | locked | disabled | online | + | 3 | worker-0 | worker | locked | disabled | online | + | 4 | worker-1 | worker | locked | disabled | online | + +----+--------------+-------------+----------------+-------------+--------------+ + +---------------------- +Configure controller-1 +---------------------- + +.. incl-config-controller-1-start: + +Configure the OAM and MGMT interfaces of controller-0 and specify the attached +networks. Use the OAM and MGMT port names, for example eth0, that are applicable +to your deployment environment. + +(Note that the MGMT interface is partially set up automatically by the network +install procedure.) + +:: + + OAM_IF= + MGMT_IF= + system host-if-modify controller-1 $OAM_IF -c platform + system interface-network-assign controller-1 $OAM_IF oam + system interface-network-assign controller-1 $MGMT_IF cluster-host + +************************************* +OpenStack-specific host configuration +************************************* + +.. important:: + + **This step is required only if the StarlingX OpenStack application + (stx-openstack) will be installed.** + +**For OpenStack only:** Assign OpenStack host labels to controller-1 in support +of installing the stx-openstack manifest and helm-charts later. + +:: + + system host-label-assign controller-1 openstack-control-plane=enabled + +.. incl-config-controller-1-end: + +------------------- +Unlock controller-1 +------------------- + +.. incl-unlock-controller-1-start: + +Unlock controller-1 in order to bring it into service: + +:: + + system host-unlock controller-1 + +Controller-1 will reboot in order to apply configuration changes and come into +service. This can take 5-10 minutes, depending on the performance of the host +machine. + +.. incl-unlock-controller-1-end: + +---------------------- +Configure worker nodes +---------------------- + +#. Add the third Ceph monitor to a worker node: + + (The first two Ceph monitors are automatically assigned to controller-0 and + controller-1.) + + :: + + system ceph-mon-add worker-0 + +#. Wait for the worker node monitor to complete configuration: + + :: + + system ceph-mon-list + +--------------------------------------+-------+--------------+------------+------+ + | uuid | ceph_ | hostname | state | task | + | | mon_g | | | | + | | ib | | | | + +--------------------------------------+-------+--------------+------------+------+ + | 64176b6c-e284-4485-bb2a-115dee215279 | 20 | controller-1 | configured | None | + | a9ca151b-7f2c-4551-8167-035d49e2df8c | 20 | controller-0 | configured | None | + | f76bc385-190c-4d9a-aa0f-107346a9907b | 20 | worker-0 | configured | None | + +--------------------------------------+-------+--------------+------------+------+ + +#. Assign the cluster-host network to the MGMT interface for the worker nodes: + + (Note that the MGMT interfaces are partially set up automatically by the + network install procedure.) + + :: + + for NODE in worker-0 worker-1; do + system interface-network-assign $NODE mgmt0 cluster-host + done + +#. Configure data interfaces for worker nodes. Use the DATA port names, for + example eth0, that are applicable to your deployment environment. + + .. important:: + + This step is **required** for OpenStack. + + This step is optional for Kubernetes: Do this step if using SRIOV network + attachments in hosted application containers. + + For Kubernetes SRIOV network attachments: + + * Configure SRIOV device plug in: + + :: + + for NODE in worker-0 worker-1; do + system host-label-assign ${NODE} sriovdp=enabled + done + + * If planning on running DPDK in containers on this host, configure the number + of 1G Huge pages required on both NUMA nodes: + + :: + + for NODE in worker-0 worker-1; do + system host-memory-modify ${NODE} 0 -1G 100 + system host-memory-modify ${NODE} 1 -1G 100 + done + + For both Kubernetes and OpenStack: + + :: + + DATA0IF= + DATA1IF= + PHYSNET0='physnet0' + PHYSNET1='physnet1' + SPL=/tmp/tmp-system-port-list + SPIL=/tmp/tmp-system-host-if-list + + # configure the datanetworks in sysinv, prior to referencing it + # in the ``system host-if-modify`` command'. + system datanetwork-add ${PHYSNET0} vlan + system datanetwork-add ${PHYSNET1} vlan + + for NODE in worker-0 worker-1; do + echo "Configuring interface for: $NODE" + set -ex + system host-port-list ${NODE} --nowrap > ${SPL} + system host-if-list -a ${NODE} --nowrap > ${SPIL} + DATA0PCIADDR=$(cat $SPL | grep $DATA0IF |awk '{print $8}') + DATA1PCIADDR=$(cat $SPL | grep $DATA1IF |awk '{print $8}') + DATA0PORTUUID=$(cat $SPL | grep ${DATA0PCIADDR} | awk '{print $2}') + DATA1PORTUUID=$(cat $SPL | grep ${DATA1PCIADDR} | awk '{print $2}') + DATA0PORTNAME=$(cat $SPL | grep ${DATA0PCIADDR} | awk '{print $4}') + DATA1PORTNAME=$(cat $SPL | grep ${DATA1PCIADDR} | awk '{print $4}') + DATA0IFUUID=$(cat $SPIL | awk -v DATA0PORTNAME=$DATA0PORTNAME '($12 ~ DATA0PORTNAME) {print $2}') + DATA1IFUUID=$(cat $SPIL | awk -v DATA1PORTNAME=$DATA1PORTNAME '($12 ~ DATA1PORTNAME) {print $2}') + system host-if-modify -m 1500 -n data0 -c data ${NODE} ${DATA0IFUUID} + system host-if-modify -m 1500 -n data1 -c data ${NODE} ${DATA1IFUUID} + system interface-datanetwork-assign ${NODE} ${DATA0IFUUID} ${PHYSNET0} + system interface-datanetwork-assign ${NODE} ${DATA1IFUUID} ${PHYSNET1} + set +ex + done + +************************************* +OpenStack-specific host configuration +************************************* + +.. important:: + + **This step is required only if the StarlingX OpenStack application + (stx-openstack) will be installed.** + +#. **For OpenStack only:** Assign OpenStack host labels to the worker nodes in + support of installing the stx-openstack manifest and helm-charts later. + + :: + + for NODE in worker-0 worker-1; do + system host-label-assign $NODE openstack-compute-node=enabled + system host-label-assign $NODE openvswitch=enabled + system host-label-assign $NODE sriov=enabled + done + +#. **For OpenStack only:** Set up disk partition for nova-local volume group, + which is needed for stx-openstack nova ephemeral disks. + + :: + + for NODE in worker-0 worker-1; do + echo "Configuring Nova local for: $NODE" + ROOT_DISK=$(system host-show ${NODE} | grep rootfs | awk '{print $4}') + ROOT_DISK_UUID=$(system host-disk-list ${NODE} --nowrap | grep ${ROOT_DISK} | awk '{print $2}') + PARTITION_SIZE=10 + NOVA_PARTITION=$(system host-disk-partition-add -t lvm_phys_vol ${NODE} ${ROOT_DISK_UUID} ${PARTITION_SIZE}) + NOVA_PARTITION_UUID=$(echo ${NOVA_PARTITION} | grep -ow "| uuid | [a-z0-9\-]* |" | awk '{print $4}') + system host-lvg-add ${NODE} nova-local + system host-pv-add ${NODE} nova-local ${NOVA_PARTITION_UUID} + done + +-------------------- +Unlock worker nodes +-------------------- + +Unlock worker nodes in order to bring them into service: + +:: + + for NODE in worker-0 worker-1; do + system host-unlock $NODE + done + +The worker nodes will reboot in order to apply configuration changes and come into +service. This can take 5-10 minutes, depending on the performance of the host machine. + +---------------------------- +Add Ceph OSDs to controllers +---------------------------- + +#. Add OSDs to controller-0. The following example adds OSDs to the `sdb` disk: + + .. important:: + + This step requires a configured Ceph storage backend. + + :: + + HOST=controller-0 + DISKS=$(system host-disk-list ${HOST}) + TIERS=$(system storage-tier-list ceph_cluster) + OSDs="/dev/sdb" + for OSD in $OSDs; do + system host-stor-add ${HOST} $(echo "$DISKS" | grep "$OSD" | awk '{print $2}') --tier-uuid $(echo "$TIERS" | grep storage | awk '{print $2}') + while true; do system host-stor-list ${HOST} | grep ${OSD} | grep configuring; if [ $? -ne 0 ]; then break; fi; sleep 1; done + done + + system host-stor-list $HOST + +#. Add OSDs to controller-1. The following example adds OSDs to the `sdb` disk: + + .. important:: + + This step requires a configured Ceph storage backend. + + :: + + HOST=controller-1 + DISKS=$(system host-disk-list ${HOST}) + TIERS=$(system storage-tier-list ceph_cluster) + OSDs="/dev/sdb" + for OSD in $OSDs; do + system host-stor-add ${HOST} $(echo "$DISKS" | grep "$OSD" | awk '{print $2}') --tier-uuid $(echo "$TIERS" | grep storage | awk '{print $2}') + while true; do system host-stor-list ${HOST} | grep ${OSD} | grep configuring; if [ $? -ne 0 ]; then break; fi; sleep 1; done + done + + system host-stor-list $HOST + +---------- +Next steps +---------- + +.. include:: ../kubernetes_install_next.txt diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/dedicated_storage.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/dedicated_storage.rst new file mode 100644 index 000000000..11cb22c39 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/dedicated_storage.rst @@ -0,0 +1,22 @@ + +============================================================ +Bare metal Standard with Dedicated Storage Installation R5.0 +============================================================ + +-------- +Overview +-------- + +.. include:: ../desc_dedicated_storage.txt + +.. include:: ../ipv6_note.txt + +------------ +Installation +------------ + +.. toctree:: + :maxdepth: 1 + + dedicated_storage_hardware + dedicated_storage_install_kubernetes diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/dedicated_storage_hardware.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/dedicated_storage_hardware.rst new file mode 100644 index 000000000..6866f3b85 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/dedicated_storage_hardware.rst @@ -0,0 +1,61 @@ +===================== +Hardware Requirements +===================== + +This section describes the hardware requirements and server preparation for a +**StarlingX R5.0 bare metal Standard with Dedicated Storage** deployment +configuration. + +.. contents:: + :local: + :depth: 1 + +----------------------------- +Minimum hardware requirements +----------------------------- + +The recommended minimum hardware requirements for bare metal servers for various +host types are: + ++---------------------+---------------------------+-----------------------+-----------------------+ +| Minimum Requirement | Controller Node | Storage Node | Worker Node | ++=====================+===========================+=======================+=======================+ +| Number of servers | 2 | 2-9 | 2-100 | ++---------------------+---------------------------+-----------------------+-----------------------+ +| Minimum processor | Dual-CPU Intel® Xeon® E5 26xx family (SandyBridge) 8 cores/socket | +| class | | ++---------------------+---------------------------+-----------------------+-----------------------+ +| Minimum memory | 64 GB | 64 GB | 32 GB | ++---------------------+---------------------------+-----------------------+-----------------------+ +| Primary disk | 500 GB SSD or NVMe (see | 120 GB (min. 10k RPM) | 120 GB (min. 10k RPM) | +| | :doc:`../../nvme_config`) | | | ++---------------------+---------------------------+-----------------------+-----------------------+ +| Additional disks | None | - 1 or more 500 GB | - For OpenStack, | +| | | (min. 10K RPM) for | recommend 1 or more | +| | | Ceph OSD | 500 GB (min. 10K | +| | | - Recommended, but | RPM) for VM | +| | | not required: 1 or | ephemeral storage | +| | | more SSDs or NVMe | | +| | | drives for Ceph | | +| | | journals (min. 1024 | | +| | | MiB per OSD | | +| | | journal) | | ++---------------------+---------------------------+-----------------------+-----------------------+ +| Minimum network | - Mgmt/Cluster: | - Mgmt/Cluster: | - Mgmt/Cluster: | +| ports | 1x10GE | 1x10GE | 1x10GE | +| | - OAM: 1x1GE | | - Data: 1 or more | +| | | | x 10GE | ++---------------------+---------------------------+-----------------------+-----------------------+ +| BIOS settings | - Hyper-Threading technology enabled | +| | - Virtualization technology enabled | +| | - VT for directed I/O enabled | +| | - CPU power and performance policy set to performance | +| | - CPU C state control disabled | +| | - Plug & play BMC detection disabled | ++---------------------+---------------------------+-----------------------+-----------------------+ + +-------------------------- +Prepare bare metal servers +-------------------------- + +.. include:: prep_servers.txt \ No newline at end of file diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/dedicated_storage_install_kubernetes.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/dedicated_storage_install_kubernetes.rst new file mode 100644 index 000000000..28c79c8d3 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/dedicated_storage_install_kubernetes.rst @@ -0,0 +1,367 @@ +========================================================================== +Install StarlingX Kubernetes on Bare Metal Standard with Dedicated Storage +========================================================================== + +This section describes the steps to install the StarlingX Kubernetes platform +on a **StarlingX R5.0 bare metal Standard with Dedicated Storage** deployment +configuration. + +.. contents:: + :local: + :depth: 1 + +------------------- +Create bootable USB +------------------- + +Refer to :doc:`/deploy_install_guides/bootable_usb` for instructions on how to +create a bootable USB with the StarlingX ISO on your system. + +-------------------------------- +Install software on controller-0 +-------------------------------- + +.. include:: controller_storage_install_kubernetes.rst + :start-after: incl-install-software-controller-0-standard-start: + :end-before: incl-install-software-controller-0-standard-end: + +-------------------------------- +Bootstrap system on controller-0 +-------------------------------- + +.. include:: controller_storage_install_kubernetes.rst + :start-after: incl-bootstrap-sys-controller-0-standard-start: + :end-before: incl-bootstrap-sys-controller-0-standard-end: + +---------------------- +Configure controller-0 +---------------------- + +.. include:: controller_storage_install_kubernetes.rst + :start-after: incl-config-controller-0-storage-start: + :end-before: incl-config-controller-0-storage-end: + +------------------- +Unlock controller-0 +------------------- + +.. important:: + + Make sure the Ceph storage backend is configured. If it is + not configured, you will not be able to configure storage + nodes. + +Unlock controller-0 in order to bring it into service: + +:: + + system host-unlock controller-0 + +Controller-0 will reboot in order to apply configuration changes and come into +service. This can take 5-10 minutes, depending on the performance of the host machine. + +----------------------------------------------------------------- +Install software on controller-1, storage nodes, and worker nodes +----------------------------------------------------------------- + +#. Power on the controller-1 server and force it to network boot with the + appropriate BIOS boot options for your particular server. + +#. As controller-1 boots, a message appears on its console instructing you to + configure the personality of the node. + +#. On the console of controller-0, list hosts to see newly discovered controller-1 + host (hostname=None): + + :: + + system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | None | None | locked | disabled | offline | + +----+--------------+-------------+----------------+-------------+--------------+ + +#. Using the host id, set the personality of this host to 'controller': + + :: + + system host-update 2 personality=controller + + This initiates the install of software on controller-1. + This can take 5-10 minutes, depending on the performance of the host machine. + +#. While waiting for the previous step to complete, power on the storage-0 and + storage-1 servers. Set the personality to 'storage' and assign a unique + hostname for each. + + For example, power on storage-0 and wait for the new host (hostname=None) to + be discovered by checking 'system host-list': + + :: + + system host-update 3 personality=storage + + Repeat for storage-1. Power on storage-1 and wait for the new host + (hostname=None) to be discovered by checking 'system host-list': + + :: + + system host-update 4 personality=storage + + This initiates the software installation on storage-0 and storage-1. + This can take 5-10 minutes, depending on the performance of the host machine. + +#. While waiting for the previous step to complete, power on the worker nodes. + Set the personality to 'worker' and assign a unique hostname for each. + + For example, power on worker-0 and wait for the new host (hostname=None) to + be discovered by checking 'system host-list': + + :: + + system host-update 5 personality=worker hostname=worker-0 + + Repeat for worker-1. Power on worker-1 and wait for the new host + (hostname=None) to be discovered by checking 'system host-list': + + :: + + system host-update 6 personality=worker hostname=worker-1 + + This initiates the install of software on worker-0 and worker-1. + +#. Wait for the software installation on controller-1, storage-0, storage-1, + worker-0, and worker-1 to complete, for all servers to reboot, and for all to + show as locked/disabled/online in 'system host-list'. + + :: + + system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | controller-1 | controller | locked | disabled | online | + | 3 | storage-0 | storage | locked | disabled | online | + | 4 | storage-1 | storage | locked | disabled | online | + | 5 | worker-0 | worker | locked | disabled | online | + | 6 | worker-1 | worker | locked | disabled | online | + +----+--------------+-------------+----------------+-------------+--------------+ + +---------------------- +Configure controller-1 +---------------------- + +.. include:: controller_storage_install_kubernetes.rst + :start-after: incl-config-controller-1-start: + :end-before: incl-config-controller-1-end: + +------------------- +Unlock controller-1 +------------------- + +.. include:: controller_storage_install_kubernetes.rst + :start-after: incl-unlock-controller-1-start: + :end-before: incl-unlock-controller-1-end: + +----------------------- +Configure storage nodes +----------------------- + +#. Assign the cluster-host network to the MGMT interface for the storage nodes: + + (Note that the MGMT interfaces are partially set up automatically by the + network install procedure.) + + :: + + for NODE in storage-0 storage-1; do + system interface-network-assign $NODE mgmt0 cluster-host + done + +#. Add OSDs to storage-0. The following example adds OSDs to the `sdb` disk: + + :: + + HOST=storage-0 + DISKS=$(system host-disk-list ${HOST}) + TIERS=$(system storage-tier-list ceph_cluster) + OSDs="/dev/sdb" + for OSD in $OSDs; do + system host-stor-add ${HOST} $(echo "$DISKS" | grep "$OSD" | awk '{print $2}') --tier-uuid $(echo "$TIERS" | grep storage | awk '{print $2}') + while true; do system host-stor-list ${HOST} | grep ${OSD} | grep configuring; if [ $? -ne 0 ]; then break; fi; sleep 1; done + done + + system host-stor-list $HOST + +#. Add OSDs to storage-1. The following example adds OSDs to the `sdb` disk: + + :: + + HOST=storage-1 + DISKS=$(system host-disk-list ${HOST}) + TIERS=$(system storage-tier-list ceph_cluster) + OSDs="/dev/sdb" + for OSD in $OSDs; do + system host-stor-add ${HOST} $(echo "$DISKS" | grep "$OSD" | awk '{print $2}') --tier-uuid $(echo "$TIERS" | grep storage | awk '{print $2}') + while true; do system host-stor-list ${HOST} | grep ${OSD} | grep configuring; if [ $? -ne 0 ]; then break; fi; sleep 1; done + done + + system host-stor-list $HOST + +-------------------- +Unlock storage nodes +-------------------- + +Unlock storage nodes in order to bring them into service: + +:: + + for STORAGE in storage-0 storage-1; do + system host-unlock $STORAGE + done + +The storage nodes will reboot in order to apply configuration changes and come +into service. This can take 5-10 minutes, depending on the performance of the +host machine. + +---------------------- +Configure worker nodes +---------------------- + +#. Assign the cluster-host network to the MGMT interface for the worker nodes: + + (Note that the MGMT interfaces are partially set up automatically by the + network install procedure.) + + :: + + for NODE in worker-0 worker-1; do + system interface-network-assign $NODE mgmt0 cluster-host + done + +#. Configure data interfaces for worker nodes. Use the DATA port names, for + example eth0, that are applicable to your deployment environment. + + .. important:: + + This step is **required** for OpenStack. + + This step is optional for Kubernetes: Do this step if using SRIOV network + attachments in hosted application containers. + + For Kubernetes SRIOV network attachments: + + * Configure SRIOV device plug in: + + :: + + for NODE in worker-0 worker-1; do + system host-label-assign ${NODE} sriovdp=enabled + done + + * If planning on running DPDK in containers on this host, configure the number + of 1G Huge pages required on both NUMA nodes: + + :: + + for NODE in worker-0 worker-1; do + system host-memory-modify ${NODE} 0 -1G 100 + system host-memory-modify ${NODE} 1 -1G 100 + done + + For both Kubernetes and OpenStack: + + :: + + DATA0IF= + DATA1IF= + PHYSNET0='physnet0' + PHYSNET1='physnet1' + SPL=/tmp/tmp-system-port-list + SPIL=/tmp/tmp-system-host-if-list + + # configure the datanetworks in sysinv, prior to referencing it + # in the ``system host-if-modify`` command'. + system datanetwork-add ${PHYSNET0} vlan + system datanetwork-add ${PHYSNET1} vlan + + for NODE in worker-0 worker-1; do + echo "Configuring interface for: $NODE" + set -ex + system host-port-list ${NODE} --nowrap > ${SPL} + system host-if-list -a ${NODE} --nowrap > ${SPIL} + DATA0PCIADDR=$(cat $SPL | grep $DATA0IF |awk '{print $8}') + DATA1PCIADDR=$(cat $SPL | grep $DATA1IF |awk '{print $8}') + DATA0PORTUUID=$(cat $SPL | grep ${DATA0PCIADDR} | awk '{print $2}') + DATA1PORTUUID=$(cat $SPL | grep ${DATA1PCIADDR} | awk '{print $2}') + DATA0PORTNAME=$(cat $SPL | grep ${DATA0PCIADDR} | awk '{print $4}') + DATA1PORTNAME=$(cat $SPL | grep ${DATA1PCIADDR} | awk '{print $4}') + DATA0IFUUID=$(cat $SPIL | awk -v DATA0PORTNAME=$DATA0PORTNAME '($12 ~ DATA0PORTNAME) {print $2}') + DATA1IFUUID=$(cat $SPIL | awk -v DATA1PORTNAME=$DATA1PORTNAME '($12 ~ DATA1PORTNAME) {print $2}') + system host-if-modify -m 1500 -n data0 -c data ${NODE} ${DATA0IFUUID} + system host-if-modify -m 1500 -n data1 -c data ${NODE} ${DATA1IFUUID} + system interface-datanetwork-assign ${NODE} ${DATA0IFUUID} ${PHYSNET0} + system interface-datanetwork-assign ${NODE} ${DATA1IFUUID} ${PHYSNET1} + set +ex + done + +************************************* +OpenStack-specific host configuration +************************************* + +.. important:: + + **This step is required only if the StarlingX OpenStack application + (stx-openstack) will be installed.** + +#. **For OpenStack only:** Assign OpenStack host labels to the worker nodes in + support of installing the stx-openstack manifest and helm-charts later. + + :: + + for NODE in worker-0 worker-1; do + system host-label-assign $NODE openstack-compute-node=enabled + system host-label-assign $NODE openvswitch=enabled + system host-label-assign $NODE sriov=enabled + done + +#. **For OpenStack only:** Set up disk partition for nova-local volume group, + which is needed for stx-openstack nova ephemeral disks. + + :: + + for NODE in worker-0 worker-1; do + echo "Configuring Nova local for: $NODE" + ROOT_DISK=$(system host-show ${NODE} | grep rootfs | awk '{print $4}') + ROOT_DISK_UUID=$(system host-disk-list ${NODE} --nowrap | grep ${ROOT_DISK} | awk '{print $2}') + PARTITION_SIZE=10 + NOVA_PARTITION=$(system host-disk-partition-add -t lvm_phys_vol ${NODE} ${ROOT_DISK_UUID} ${PARTITION_SIZE}) + NOVA_PARTITION_UUID=$(echo ${NOVA_PARTITION} | grep -ow "| uuid | [a-z0-9\-]* |" | awk '{print $4}') + system host-lvg-add ${NODE} nova-local + system host-pv-add ${NODE} nova-local ${NOVA_PARTITION_UUID} + done + +------------------- +Unlock worker nodes +------------------- + +Unlock worker nodes in order to bring them into service: + +:: + + for NODE in worker-0 worker-1; do + system host-unlock $NODE + done + +The worker nodes will reboot in order to apply configuration changes and come +into service. This can take 5-10 minutes, depending on the performance of the +host machine. + +---------- +Next steps +---------- + +.. include:: ../kubernetes_install_next.txt diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/ironic.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/ironic.rst new file mode 100644 index 000000000..5a1bcf3b5 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/ironic.rst @@ -0,0 +1,72 @@ +==================================== +Bare metal Standard with Ironic R5.0 +==================================== + +-------- +Overview +-------- + +Ironic is an OpenStack project that provisions bare metal machines. For +information about the Ironic project, see +`Ironic Documentation `__. + +End user applications can be deployed on bare metal servers (instead of +virtual machines) by configuring OpenStack Ironic and deploying a pool of 1 or +more bare metal servers. + +.. note:: + + If you are behind a corporate firewall or proxy, you need to set proxy + settings. Refer to :doc:`/../../configuration/docker_proxy_config` for + details. + +.. figure:: ../figures/starlingx-deployment-options-ironic.png + :scale: 50% + :alt: Standard with Ironic deployment configuration + + *Figure 1: Standard with Ironic deployment configuration* + +Bare metal servers must be connected to: + +* IPMI for OpenStack Ironic control +* ironic-provisioning-net tenant network via their untagged physical interface, + which supports PXE booting + +As part of configuring OpenStack Ironic in StarlingX: + +* An ironic-provisioning-net tenant network must be identified as the boot + network for bare metal nodes. +* An additional untagged physical interface must be configured on controller + nodes and connected to the ironic-provisioning-net tenant network. The + OpenStack Ironic tftpboot server will PXE boot the bare metal servers over + this interface. + +.. note:: + + Bare metal servers are NOT: + + * Running any OpenStack / StarlingX software; they are running end user + applications (for example, Glance Images). + * To be connected to the internal management network. + +------------ +Installation +------------ + +StarlingX currently supports only a bare metal installation of Ironic with a +standard configuration, either: + +* :doc:`controller_storage` + +* :doc:`dedicated_storage` + + +This guide assumes that you have a standard deployment installed and configured +with 2x controllers and at least 1x compute-labeled worker node, with the +StarlingX OpenStack application (stx-openstack) applied. + +.. toctree:: + :maxdepth: 1 + + ironic_hardware + ironic_install diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/ironic_hardware.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/ironic_hardware.rst new file mode 100644 index 000000000..6a59232a6 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/ironic_hardware.rst @@ -0,0 +1,51 @@ +===================== +Hardware Requirements +===================== + +This section describes the hardware requirements and server preparation for a +**StarlingX R5.0 bare metal Ironic** deployment configuration. + +.. contents:: + :local: + :depth: 1 + +----------------------------- +Minimum hardware requirements +----------------------------- + +* One or more bare metal hosts as Ironic nodes as well as tenant instance node. + +* BMC support on bare metal host and controller node connectivity to the BMC IP + address of bare metal hosts. + +For controller nodes: + +* Additional NIC port on both controller nodes for connecting to the + ironic-provisioning-net. + +For worker nodes: + +* If using a flat data network for the Ironic provisioning network, an additional + NIC port on one of the worker nodes is required. + +* Alternatively, use a VLAN data network for the Ironic provisioning network and + simply add the new data network to an existing interface on the worker node. + +* Additional switch ports / configuration for new ports on controller, worker, + and Ironic nodes, for connectivity to the Ironic provisioning network. + +----------------------------------- +BMC configuration of Ironic node(s) +----------------------------------- + +Enable BMC and allocate a static IP, username, and password in the BIOS settings. +For example, set: + +IP address + 10.10.10.126 + +username + root + +password + test123 diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/ironic_install.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/ironic_install.rst new file mode 100644 index 000000000..8e2a64e23 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/ironic_install.rst @@ -0,0 +1,392 @@ +================================ +Install Ironic on StarlingX R5.0 +================================ + +This section describes the steps to install Ironic on a standard configuration, +either: + +* **StarlingX R5.0 bare metal Standard with Controller Storage** deployment + configuration + +* **StarlingX R5.0 bare metal Standard with Dedicated Storage** deployment + configuration + +.. contents:: + :local: + :depth: 1 + +--------------------- +Enable Ironic service +--------------------- + +This section describes the pre-configuration required to enable the Ironic service. +All the commands in this section are for the StarlingX platform. + +First acquire administrative privileges: + +:: + + source /etc/platform/openrc + +******************************** +Download Ironic deployment image +******************************** + +The Ironic service requires a deployment image (kernel and ramdisk) which is +used to clean Ironic nodes and install the end-user's image. The cleaning done +by the deployment image wipes the disks and tests connectivity to the Ironic +conductor on the controller nodes via the Ironic Python Agent (IPA). + +The latest Ironic deployment image (**Ironic-kernel** and **Ironic-ramdisk**) +can be found here: + +* `Ironic-kernel ipa-centos8-master.kernel + `__ +* `Ironic-ramdisk ipa-centos8.initramfs + `__ + + +******************************************************* +Configure Ironic network on deployed standard StarlingX +******************************************************* + +#. Add an address pool for the Ironic network. This example uses `ironic-pool`: + + :: + + system addrpool-add --ranges 10.10.20.1-10.10.20.100 ironic-pool 10.10.20.0 24 + +#. Add the Ironic platform network. This example uses `ironic-net`: + + :: + + system addrpool-list | grep ironic-pool | awk '{print$2}' | xargs system network-add ironic-net ironic false + +#. Add the Ironic tenant network. This example uses `ironic-data`: + + .. note:: + + The tenant network is not the same as the platform network described in + the previous step. You can specify any name for the tenant network other + than ‘ironic’. If the name 'ironic' is used, a user override must be + generated to indicate the tenant network name. + + Refer to section `Generate user Helm overrides`_ for details. + + :: + + system datanetwork-add ironic-data flat + +#. Configure the new interfaces (for Ironic) on controller nodes and assign + them to the platform network. Host must be locked. This example uses the + platform network `ironic-net` that was named in a previous step. + + These new interfaces to the controllers are used to connect to the Ironic + provisioning network: + + **controller-0** + + :: + + system interface-network-assign controller-0 enp2s0 ironic-net + system host-if-modify -n ironic -c platform \ + --ipv4-mode static --ipv4-pool ironic-pool controller-0 enp2s0 + + # Apply the OpenStack Ironic node labels + system host-label-assign controller-0 openstack-ironic=enabled + + # Unlock the node to apply changes + system host-unlock controller-0 + + + **controller-1** + + :: + + system interface-network-assign controller-1 enp2s0 ironic-net + system host-if-modify -n ironic -c platform \ + --ipv4-mode static --ipv4-pool ironic-pool controller-1 enp2s0 + + # Apply the OpenStack Ironic node labels + system host-label-assign controller-1 openstack-ironic=enabled + + # Unlock the node to apply changes + system host-unlock controller-1 + +#. Configure the new interface (for Ironic) on one of the compute-labeled worker + nodes and assign it to the Ironic data network. This example uses the data + network `ironic-data` that was named in a previous step. + + :: + + system interface-datanetwork-assign worker-0 eno1 ironic-data + system host-if-modify -n ironicdata -c data worker-0 eno1 + +**************************** +Generate user Helm overrides +**************************** + +Ironic Helm Charts are included in the stx-openstack application. By default, +Ironic is disabled. + +To enable Ironic, update the following Ironic Helm Chart attributes: + +:: + + system helm-override-update stx-openstack ironic openstack \ + --set network.pxe.neutron_subnet_alloc_start=10.10.20.10 \ + --set network.pxe.neutron_subnet_gateway=10.10.20.1 \ + --set network.pxe.neutron_provider_network=ironic-data + +:command:`network.pxe.neutron_subnet_alloc_start` sets the DHCP start IP to +Neutron for Ironic node provision, and reserves several IPs for the platform. + +If the data network name for Ironic is changed, modify +:command:`network.pxe.neutron_provider_network` to the command above: + +:: + + --set network.pxe.neutron_provider_network=ironic-data + +******************************* +Apply stx-openstack application +******************************* + +Re-apply the stx-openstack application to apply the changes to Ironic: + +:: + + system helm-chart-attribute-modify stx-openstack ironic openstack \ + --enabled true + + system application-apply stx-openstack + +-------------------- +Start an Ironic node +-------------------- + +All the commands in this section are for the OpenStack application with +administrative privileges. + +From a new shell as a root user, without sourcing ``/etc/platform/openrc``: + +:: + + mkdir -p /etc/openstack + + tee /etc/openstack/clouds.yaml << EOF + clouds: + openstack_helm: + region_name: RegionOne + identity_api_version: 3 + endpoint_type: internalURL + auth: + username: 'admin' + password: 'Li69nux*' + project_name: 'admin' + project_domain_name: 'default' + user_domain_name: 'default' + auth_url: 'http://keystone.openstack.svc.cluster.local/v3' + EOF + + export OS_CLOUD=openstack_helm + +******************** +Create Glance images +******************** + +#. Create the **ironic-kernel** image: + + :: + + openstack image create \ + --file ~/coreos_production_pxe-stable-stein.vmlinuz \ + --disk-format aki \ + --container-format aki \ + --public \ + ironic-kernel + +#. Create the **ironic-ramdisk** image: + + :: + + openstack image create \ + --file ~/coreos_production_pxe_image-oem-stable-stein.cpio.gz \ + --disk-format ari \ + --container-format ari \ + --public \ + ironic-ramdisk + +#. Create the end user application image (for example, CentOS): + + :: + + openstack image create \ + --file ~/CentOS-7-x86_64-GenericCloud-root.qcow2 \ + --public --disk-format \ + qcow2 --container-format bare centos + +********************* +Create an Ironic node +********************* + +#. Create a node: + + :: + + openstack baremetal node create --driver ipmi --name ironic-test0 + +#. Add IPMI information: + + :: + + openstack baremetal node set \ + --driver-info ipmi_address=10.10.10.126 \ + --driver-info ipmi_username=root \ + --driver-info ipmi_password=test123 \ + --driver-info ipmi_terminal_port=623 ironic-test0 + +#. Set `ironic-kernel` and `ironic-ramdisk` images driver information, + on this bare metal node: + + :: + + openstack baremetal node set \ + --driver-info deploy_kernel=$(openstack image list | grep ironic-kernel | awk '{print$2}') \ + --driver-info deploy_ramdisk=$(openstack image list | grep ironic-ramdisk | awk '{print$2}') \ + ironic-test0 + +#. Set resource properties on this bare metal node based on actual Ironic node + capacities: + + :: + + openstack baremetal node set \ + --property cpus=4 \ + --property cpu_arch=x86_64\ + --property capabilities="boot_option:local" \ + --property memory_mb=65536 \ + --property local_gb=400 \ + --resource-class bm ironic-test0 + +#. Add pxe_template location: + + :: + + openstack baremetal node set --driver-info \ + pxe_template='/var/lib/openstack/lib64/python2.7/site-packages/ironic/drivers/modules/ipxe_config.template' \ + ironic-test0 + +#. Create a port to identify the specific port used by the Ironic node. + Substitute **a4:bf:01:2b:3b:c8** with the MAC address for the Ironic node + port which connects to the Ironic network: + + :: + + openstack baremetal port create \ + --node $(openstack baremetal node list | grep ironic-test0 | awk '{print$2}') \ + --pxe-enabled true a4:bf:01:2b:3b:c8 + +#. Change node state to `manage`: + + :: + + openstack baremetal node manage ironic-test0 + +#. Make node available for deployment: + + :: + + openstack baremetal node provide ironic-test0 + +#. Wait for ironic-test0 provision-state: available: + + :: + + openstack baremetal node show ironic-test0 + +--------------------------------- +Deploy an instance on Ironic node +--------------------------------- + +All the commands in this section are for the OpenStack application, but this +time with *tenant* specific privileges. + +#. From a new shell as a root user, without sourcing ``/etc/platform/openrc``: + + :: + + mkdir -p /etc/openstack + + tee /etc/openstack/clouds.yaml << EOF + clouds: + openstack_helm: + region_name: RegionOne + identity_api_version: 3 + endpoint_type: internalURL + auth: + username: 'joeuser' + password: 'mypasswrd' + project_name: 'intel' + project_domain_name: 'default' + user_domain_name: 'default' + auth_url: 'http://keystone.openstack.svc.cluster.local/v3' + EOF + + export OS_CLOUD=openstack_helm + +#. Create flavor. + + Set resource CUSTOM_BM corresponding to **--resource-class bm**: + + :: + + openstack flavor create --ram 4096 --vcpus 4 --disk 400 \ + --property resources:CUSTOM_BM=1 \ + --property resources:VCPU=0 \ + --property resources:MEMORY_MB=0 \ + --property resources:DISK_GB=0 \ + --property capabilities:boot_option='local' \ + bm-flavor + + See `Adding scheduling information + `__ + and `Configure Nova flavors + `__ + for more information. + +#. Enable service + + List the compute services: + + :: + + openstack compute service list + + Set compute service properties: + + :: + + openstack compute service set --enable controller-0 nova-compute + +#. Create instance + + .. note:: + + The :command:`keypair create` command is optional. It is not required to + enable a bare metal instance. + + :: + + openstack keypair create --public-key ~/.ssh/id_rsa.pub mykey + + + Create 2 new servers, one bare metal and one virtual: + + :: + + openstack server create --image centos --flavor bm-flavor \ + --network baremetal --key-name mykey bm + + openstack server create --image centos --flavor m1.small \ + --network baremetal --key-name mykey vm diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/prep_servers.txt b/doc/source/deploy_install_guides/r5_release/bare_metal/prep_servers.txt new file mode 100644 index 000000000..61a686201 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/prep_servers.txt @@ -0,0 +1,17 @@ +Prior to starting the StarlingX installation, the bare metal servers must be in +the following condition: + +* Physically installed + +* Cabled for power + +* Cabled for networking + + * Far-end switch ports should be properly configured to realize the networking + shown in Figure 1. + +* All disks wiped + + * Ensures that servers will boot from either the network or USB storage (if present) + +* Powered off \ No newline at end of file diff --git a/doc/source/deploy_install_guides/r5_release/desc_aio_duplex.txt b/doc/source/deploy_install_guides/r5_release/desc_aio_duplex.txt new file mode 100644 index 000000000..ec00437cf --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/desc_aio_duplex.txt @@ -0,0 +1,29 @@ +The All-in-one Duplex (AIO-DX) deployment option provides a pair of high +availability (HA) servers with each server providing all three cloud functions +(controller, worker, and storage). + +An AIO-DX configuration provides the following benefits: + +* Only a small amount of cloud processing and storage power is required +* Application consolidation using multiple containers or virtual machines on a + single pair of physical servers +* High availability (HA) services run on the controller function across two + physical servers in either active/active or active/standby mode +* A storage back end solution using a two-node CEPH deployment across two servers +* Containers or virtual machines scheduled on both worker functions +* Protection against overall server hardware fault, where + + * All controller HA services go active on the remaining healthy server + * All virtual machines are recovered on the remaining healthy server + +.. note:: + + If you are behind a corporate firewall or proxy, you need to set proxy + settings. Refer to :doc:`/../../configuration/docker_proxy_config` for + details. + +.. figure:: ../figures/starlingx-deployment-options-duplex.png + :scale: 50% + :alt: All-in-one Duplex deployment configuration + + *Figure 1: All-in-one Duplex deployment configuration* \ No newline at end of file diff --git a/doc/source/deploy_install_guides/r5_release/desc_aio_simplex.txt b/doc/source/deploy_install_guides/r5_release/desc_aio_simplex.txt new file mode 100644 index 000000000..56b22e94b --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/desc_aio_simplex.txt @@ -0,0 +1,24 @@ +The All-in-one Simplex (AIO-SX) deployment option provides all three cloud +functions (controller, worker, and storage) on a single server with the +following benefits: + +* Requires only a small amount of cloud processing and storage power +* Application consolidation using multiple containers or virtual machines on a + single pair of physical servers +* A storage backend solution using a single-node CEPH deployment + +.. note:: + + If you are behind a corporate firewall or proxy, you need to set proxy + settings. Refer to :doc:`/../../configuration/docker_proxy_config` for + details. + +.. figure:: ../figures/starlingx-deployment-options-simplex.png + :scale: 50% + :alt: All-in-one Simplex deployment configuration + + *Figure 1: All-in-one Simplex deployment configuration* + +An AIO-SX deployment gives no protection against overall server hardware fault. +Hardware component protection can be enabled with, for example, a hardware RAID +or 2x Port LAG in the deployment. \ No newline at end of file diff --git a/doc/source/deploy_install_guides/r5_release/desc_controller_storage.txt b/doc/source/deploy_install_guides/r5_release/desc_controller_storage.txt new file mode 100644 index 000000000..3d606d532 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/desc_controller_storage.txt @@ -0,0 +1,28 @@ +The Standard with Controller Storage deployment option provides two high +availability (HA) controller nodes and a pool of up to 10 worker nodes. + +A Standard with Controller Storage configuration provides the following benefits: + +* A pool of up to 10 worker nodes +* High availability (HA) services run across the controller nodes in either + active/active or active/standby mode +* A storage back end solution using a two-node CEPH deployment across two + controller servers +* Protection against overall controller and worker node failure, where + + * On overall controller node failure, all controller HA services go active on + the remaining healthy controller node + * On overall worker node failure, virtual machines and containers are + recovered on the remaining healthy worker nodes + +.. note:: + + If you are behind a corporate firewall or proxy, you need to set proxy + settings. Refer to :doc:`/../../configuration/docker_proxy_config` for + details. + +.. figure:: ../figures/starlingx-deployment-options-controller-storage.png + :scale: 50% + :alt: Standard with Controller Storage deployment configuration + + *Figure 1: Standard with Controller Storage deployment configuration* \ No newline at end of file diff --git a/doc/source/deploy_install_guides/r5_release/desc_dedicated_storage.txt b/doc/source/deploy_install_guides/r5_release/desc_dedicated_storage.txt new file mode 100644 index 000000000..79371f446 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/desc_dedicated_storage.txt @@ -0,0 +1,24 @@ +The Standard with Dedicated Storage deployment option is a standard installation +with independent controller, worker, and storage nodes. + +A Standard with Dedicated Storage configuration provides the following benefits: + +* A pool of up to 100 worker nodes +* A 2x node high availability (HA) controller cluster with HA services running + across the controller nodes in either active/active or active/standby mode +* A storage back end solution using a two-to-9x node HA CEPH storage cluster + that supports a replication factor of two or three +* Up to four groups of 2x storage nodes, or up to three groups of 3x storage + nodes + +.. note:: + + If you are behind a corporate firewall or proxy, you need to set proxy + settings. Refer to :doc:`/../../configuration/docker_proxy_config` for + details. + +.. figure:: ../figures/starlingx-deployment-options-dedicated-storage.png + :scale: 50% + :alt: Standard with Dedicated Storage deployment configuration + + *Figure 1: Standard with Dedicated Storage deployment configuration* \ No newline at end of file diff --git a/doc/source/deploy_install_guides/r5_release/distributed_cloud/index.rst b/doc/source/deploy_install_guides/r5_release/distributed_cloud/index.rst new file mode 100644 index 000000000..5765f4653 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/distributed_cloud/index.rst @@ -0,0 +1,313 @@ +=================================== +Distributed Cloud Installation R5.0 +=================================== + +This section describes how to install and configure the StarlingX distributed +cloud deployment. + +.. contents:: + :local: + :depth: 1 + +-------- +Overview +-------- + +Distributed cloud configuration supports an edge computing solution by +providing central management and orchestration for a geographically +distributed network of StarlingX Kubernetes edge systems/clusters. + +The StarlingX distributed cloud implements the OpenStack Edge Computing +Groups's MVP `Edge Reference Architecture +`_, +specifically the "Distributed Control Plane" scenario. + +The StarlingX distributed cloud deployment is designed to meet the needs of +edge-based data centers with centralized orchestration and independent control +planes, and in which Network Function Cloudification (NFC) worker resources +are localized for maximum responsiveness. The architecture features: + +- Centralized orchestration of edge cloud control planes. +- Full synchronized control planes at edge clouds (that is, Kubernetes cluster + master and nodes), with greater benefits for local services, such as: + + - Reduced network latency. + - Operational availability, even if northbound connectivity + to the central cloud is lost. + +The system supports a scalable number of StarlingX Kubernetes edge +systems/clusters, which are centrally managed and synchronized over L3 +networks from a central cloud. Each edge system is also highly scalable, from +a single node StarlingX Kubernetes deployment to a full standard cloud +configuration with controller, worker and storage nodes. + +------------------------------ +Distributed cloud architecture +------------------------------ + +A distributed cloud system consists of a central cloud, and one or more +subclouds connected to the SystemController region central cloud over L3 +networks, as shown in Figure 1. + +- **Central cloud** + + The central cloud provides a *RegionOne* region for managing the physical + platform of the central cloud and the *SystemController* region for managing + and orchestrating over the subclouds. + + - **RegionOne** + + In the Horizon GUI, RegionOne is the name of the access mode, or region, + used to manage the nodes in the central cloud. + + - **SystemController** + + In the Horizon GUI, SystemController is the name of the access mode, or + region, used to manage the subclouds. + + You can use the SystemController to add subclouds, synchronize select + configuration data across all subclouds and monitor subcloud operations + and alarms. System software updates for the subclouds are also centrally + managed and applied from the SystemController. + + DNS, NTP, and other select configuration settings are centrally managed + at the SystemController and pushed to the subclouds in parallel to + maintain synchronization across the distributed cloud. + +- **Subclouds** + + The subclouds are StarlingX Kubernetes edge systems/clusters used to host + containerized applications. Any type of StarlingX Kubernetes configuration, + (including simplex, duplex, or standard with or without storage nodes), can + be used for a subcloud. The two edge clouds shown in Figure 1 are subclouds. + + Alarms raised at the subclouds are sent to the SystemController for + central reporting. + +.. figure:: ../figures/starlingx-deployment-options-distributed-cloud.png + :scale: 45% + :alt: Distributed cloud deployment configuration + + *Figure 1: Distributed cloud deployment configuration* + + +-------------------- +Network requirements +-------------------- + +Subclouds are connected to the SystemController through both the OAM and the +Management interfaces. Because each subcloud is on a separate L3 subnet, the +OAM, Management and PXE boot L2 networks are local to the subclouds. They are +not connected via L2 to the central cloud, they are only connected via L3 +routing. The settings required to connect a subcloud to the SystemController +are specified when a subcloud is defined. A gateway router is required to +complete the L3 connections, which will provide IP routing between the +subcloud Management and OAM IP subnet and the SystemController Management and +OAM IP subnet, respectively. The SystemController bootstraps the subclouds via +the OAM network, and manages them via the management network. For more +information, see the `Install a Subcloud`_ section later in this guide. + +.. note:: + + All messaging between SystemControllers and Subclouds uses the ``admin`` + REST API service endpoints which, in this distributed cloud environment, + are all configured for secure HTTPS. Certificates for these HTTPS + connections are managed internally by StarlingX. + +--------------------------------------- +Install and provision the central cloud +--------------------------------------- + +Installing the central cloud is similar to installing a standard +StarlingX Kubernetes system. The central cloud supports either an AIO-duplex +deployment configuration or a standard with dedicated storage nodes deployment +configuration. + +To configure controller-0 as a distributed cloud central controller, you must +set certain system parameters during the initial bootstrapping of +controller-0. Set the system parameter *distributed_cloud_role* to +*systemcontroller* in the Ansible bootstrap override file. Also, set the +management network IP address range to exclude IP addresses reserved for +gateway routers providing routing to the subclouds' management subnets. + +.. note:: Worker hosts and data networks are not used in the + central cloud. + +Procedure: + +- Follow the StarlingX R5.0 installation procedures with the extra step noted below: + + - AIO-duplex: + `Bare metal All-in-one Duplex Installation R5.0 `_ + + - Standard with dedicated storage nodes: + `Bare metal Standard with Dedicated Storage Installation R5.0 `_ + +- For the step "Bootstrap system on controller-0", add the following + parameters to the Ansible bootstrap override file. + + .. code:: yaml + + distributed_cloud_role: systemcontroller + management_start_address: + management_end_address: + +------------------ +Install a subcloud +------------------ + +At the subcloud location: + +1. Physically install and cable all subcloud servers. +2. Physically install the top of rack switch and configure it for the + required networks. +3. Physically install the gateway routers which will provide IP routing + between the subcloud OAM and Management subnets and the SystemController + OAM and management subnets. +4. On the server designated for controller-0, install the StarlingX + Kubernetes software from USB or a PXE Boot server. + +5. Establish an L3 connection to the SystemController by enabling the OAM + interface (with OAM IP/subnet) on the subcloud controller using the + ``config_management`` script. This step is for subcloud ansible bootstrap + preparation. + + .. note:: This step should **not** use an interface that uses the MGMT + IP/subnet because the MGMT IP subnet will get moved to the loopback + address by the Ansible bootstrap playbook during installation. + + Be prepared to provide the following information: + + - Subcloud OAM interface name (for example, enp0s3). + - Subcloud OAM interface address, in CIDR format (for example, 10.10.10.12/24). + + .. note:: This must match the *external_oam_floating_address* supplied in + the subcloud's ansible bootstrap override file. + + - Subcloud gateway address on the OAM network + (for example, 10.10.10.1). A default value is shown. + - System Controller OAM subnet (for example, 10,10.10.0/24). + + .. note:: To exit without completing the script, use ``CTRL+C``. Allow a few minutes for + the script to finish. + + .. note:: The `config_management` in the code snippet configures the OAM + interface/address/gateway. + + .. code:: sh + + $ sudo config_management + Enabling interfaces... DONE + Waiting 120 seconds for LLDP neighbor discovery... Retrieving neighbor details... DONE + Available interfaces: + local interface remote port + --------------- ---------- + enp0s3 08:00:27:c4:6c:7a + enp0s8 08:00:27:86:7a:13 + enp0s9 unknown + + Enter management interface name: enp0s3 + Enter management address CIDR: 10.10.10.12/24 + Enter management gateway address [10.10.10.1]: + Enter System Controller subnet: 10.10.10.0/24 + Disabling non-management interfaces... DONE + Configuring management interface... DONE + RTNETLINK answers: File exists + Adding route to System Controller... DONE + +At the System Controller: + +1. Create a ``bootstrap-values.yml`` override file for the subcloud. For + example: + + .. code:: yaml + + system_mode: duplex + name: "subcloud1" + description: "Ottawa Site" + location: "YOW" + + management_subnet: 192.168.101.0/24 + management_start_address: 192.168.101.2 + management_end_address: 192.168.101.50 + management_gateway_address: 192.168.101.1 + + external_oam_subnet: 10.10.10.0/24 + external_oam_gateway_address: 10.10.10.1 + external_oam_floating_address: 10.10.10.12 + + systemcontroller_gateway_address: 192.168.204.101 + + .. important:: The `management_*` entries in the override file are required + for all installation types, including AIO-Simplex. + + .. important:: The `management_subnet` must not overlap with any other subclouds. + + .. note:: The `systemcontroller_gateway_address` is the address of central + cloud management network gateway. + +2. Add the subcloud using the CLI command below: + + .. code:: sh + + dcmanager subcloud add --bootstrap-address + --bootstrap-values + + Where: + + - ** is the OAM interface address set earlier on the subcloud. + - ** is the Ansible override configuration file, ``bootstrap-values.yml``, + created earlier in step 1. + + You will be prompted for the Linux password of the subcloud. This command + will take 5- 10 minutes to complete. You can monitor the progress of the + subcloud bootstrap through logs: + + .. code:: sh + + tail –f /var/log/dcmanager/_bootstrap_