.. WARNING: Add no lines of text between the label immediately following .. and the title. .. _deploy-software-releases-using-the-cli-1ea02eb230e5: ====================================== Deploy Software Releases using the CLI ====================================== After a software release has been uploaded to the central repository, it can be deployed first on the System Controller and then on a specific subcloud or across a group of subclouds through different orchestration methods. Deploy a Software Release on the System Controller -------------------------------------------------- Follow the instructions at :ref:`orchestrated-deployment-host-software-deployment-d234754c7d20` to complete the software deployment. .. note:: The step to upload the software release can be skipped since it has already been done. It is recommended to upgrade Kubernetes following the deployment of a major release. See :ref:`the-kubernetes-update-orchestration-process`. The System Controller is now ready for new subcloud deployment. .. note:: Starting release |prod-ver|, low-latency configuration via remote install is no longer supported. For new subcloud deployments that involve remote install using Redfish, avoid using ``install_type`` of 4 and 5 in ``install-values.yaml``. For existing subclouds that were originally deployed with remote install, use :command:`dcmanager subcloud update --install-values` to update the ``install_type`` so they can be restored in the future. Orchestrate the Deployment of a Software Release Across the Subclouds --------------------------------------------------------------------- After the System Controller has been successfully updated with a new major or patch software release; distributed software deployment orchestration can be initiated to bring the managed subclouds to the same software level as the System Controller. The subclouds must, however, be prestaged with the target software before orchestrated deployment can take place. For more information on Prestaging Subcloud Orchestration see :ref:`orchestrate-subcloud-prestage-using-the-cli-eb516473582f`. Before orchestrating a major software release deployment across the subclouds, it is strongly recommended that a backup of each subcloud to be upgraded is taken. See :ref:`backup-a-subcloud-group-of-subclouds-using-dcmanager-cli-f12020a8fc42` for instructions. .. rubric:: |prereq| The following conditions must be met for the orchestrated software deployment across the subclouds to be successful. - All target subclouds are managed, online and healthy. The Kubernetes cluster on each subcloud is fully operational and there are no management affecting alarms. - Subcloud certificates are up-to-date or will not expire during the deployment of the new software release. - All target subclouds have been prestaged with the software release and the new container images required for that release. See :ref:`orchestrate-subcloud-prestage-using-the-cli-eb516473582f` for instructions. - For the deployment of major software release, ensure that the controller-0 is the active controller on each subcloud. - |Optional| **Only for Major Release** Upload the new software release to the central software release repository: .. code-block:: none ~(keystone_admin)]$ software --os-region-name SystemController upload .. rubric:: |proc| #. Review software sync status of the subclouds. After new software release has been deployed on the System Controller, wait for approximately 60 seconds for the ``software_sync_status`` of all subclouds to be updated. To identify which subclouds are deploy-current (in-sync), use the :command:`dcmanager subcloud list` command. For example: .. code-block:: none ~(keystone_admin)]$ dcmanager subcloud list +----+-----------+--------------+--------------------+-------------+ | id | name | management | availability | sync | +----+-----------+--------------+--------------------+-------------+ | 1 | subcloud-1| managed | online | out-of-sync | | 2 | subcloud-2| managed | online | out-of-sync | | 3 | subcloud-3| managed | online | out-of-sync | | 4 | subcloud-4| managed | online | out-of-sync | +----+-----------+--------------+--------------------+-------------+ To see synchronization details for a subcloud, use the following command: .. code-block:: none ~(keystone_admin)]$ dcmanager subcloud show subcloud1 +-----------------------------+----------------------------------+ | Field | Value | +-----------------------------+----------------------------------+ | id | 2 | | name | subcloud1-stx-latest | | description | Ottawa Site | | location | YOW | | software_version | 24.09 | | management | unmanaged | | availability | online | | deploy_status | complete | | management_subnet | 192.168.101.0/24 | | management_start_ip | 192.168.101.2 | | management_end_ip | 192.168.101.50 | | management_gateway_ip | 192.168.101.1 | | systemcontroller_gateway_ip | 192.168.204.101 | | group_id | 1 | | peer_group_id | None | | created_at | 2025-01-21 18:34:30.410350 | | updated_at | 2025-01-21 19:03:08.836619 | | backup_status | None | | backup_datetime | None | | prestage_status | None | | prestage_versions | None | | dc-cert_sync_status | in-sync | | firmware_sync_status | unknown | | identity_sync_status | unknown | | kubernetes_sync_status | unknown | | kube-rootca_sync_status | unknown | | load_sync_status | unknown | | patching_sync_status | unknown | | platform_sync_status | unknown | | software_sync_status | unknown | | region_name | 0aa48e9b72bf48bbaad9a624cdd0cacb | +-----------------------------+----------------------------------+ #. To create a sw-deploy strategy, use the :command:`dcmanager sw-deploy-strategy create ` command. The sw-deploy strategy for a |prod-dc| system controls how updates/upgrades are applied to subclouds. .. code-block:: none ~(keystone_admin)]$ dcmanager sw-deploy-strategy create \ [--subcloud-apply-type ] \ [–-max-parallel-subclouds ] \ [–-stop-on-failure ] \ [--group group] \ [] where: **subcloud-apply-type** **parallel** or **serial**— determines whether the subclouds will be processed in parallel or serially. If this is not specified using the CLI, the values for :command:`subcloud_update_type` defined for each subcloud group will be used by default. **max-parallel-subclouds** Sets the maximum number of subclouds that can be upgraded in parallel (default 2). If this is not specified using the CLI, the values for :command:`max_parallel_subclouds` defined for each subcloud group will be used by default. **stop-on-failure** **true**\(default) or **false**— determines whether upgrade orchestration failure for a subcloud prevents application to subsequent subclouds. **group** Optionally pass the name or ID of a subcloud group to the :command:`dcmanager sw-deploy-strategy create` command. This results in a strategy that is only applied to all subclouds in the specified group. The subcloud group values are used for subcloud apply type and max parallel subclouds parameters. For example: .. code-block:: none ~(keystone_admin)]$ dcmanager sw-deploy-strategy create +------------------------+----------------------------+ | Field | Value | +------------------------+----------------------------+ | strategy type | sw-deploy | | subcloud apply type | None | | max parallel subclouds | 2 | | stop on failure | False | | release_id | WRCP-24.09.200 | | state | initial | | created_at | 2025-01-21T20:00:31.141872 | | updated_at | None | +------------------------+----------------------------+ #. To show the settings for the ``sw-deploy-strategy``, use the :command:`dcmanager sw-deploy-strategy show` command. For example: .. code-block:: none ~(keystone_admin)]$ dcmanager sw-deploy-strategy show +------------------------+----------------------------+ | Field | Value | +------------------------+----------------------------+ | strategy type | sw-deploy | | subcloud apply type | parallel | | max parallel subclouds | 2 | | stop on failure | False | | state | initial | | created_at | 2020-02-02T14:42:13.822499 | | updated_at | None | +------------------------+----------------------------+ .. note:: The values for `subcloud apply type` and `max parallel subclouds` will be taken from the subcloud group if specified through the ``--group`` parameter. #. Review the sw-deploy strategy for the subclouds. To show the subclouds that will be processed when the sw-deploy strategy is applied, use the :command:`dcmanager strategy-step list` command. For example: .. code-block:: none ~(keystone_admin)]$ dcmanager strategy-step list +----------------------+-------+---------+---------+------------+-------------+ | cloud | stage | state | details | started_at | finished_at | +----------------------+-------+---------+---------+------------+-------------+ | subcloud1-stx-latest | 1 | initial | | None | None | +----------------------+-------+---------+---------+------------+-------------+ .. note:: All the subclouds that are included in the same stage will be deployed in parallel. #. To apply the software deploy strategy, use the :command:`dcmanager sw-deploy-strategy apply` command. .. code-block:: none ~(keystone_admin)]$ dcmanager sw-deploy-strategy apply +------------------------+----------------------------+ | Field | Value | +------------------------+----------------------------+ | subcloud apply type | parallel | | max parallel subclouds | 2 | | stop on failure | False | | state | applying | | created_at | 2020-02-02T14:42:13.822499 | | updated_at | 2020-02-02T14:42:19.376688 | +------------------------+----------------------------+ #. To show the step currently being performed on each of the subclouds, use the :command:`dcmanager strategy-step list` command. For example: .. code-block:: none ~(keystone_admin)]$ dcmanager strategy-step list +------------------+--------+-----------------------+-------------------+----------------------------+----------------------------+ | cloud | stage | state | details | started_at | finished_at | +------------------+--------+-----------------------+-------------------+----------------------------+----------------------------+ | subcloud-1 | 3 | complete | | 2024-11-06 12:57:20.342429 | 2024-11-06 12:57:41.054664 | | subcloud-4 | 2 | apply VIM sw-deploy | | 2024-11-06 12:57:20.342429 | None | | subcloud-5 | 1 | initial | | None | None | | subcloud-6 | 1 | initial | | None | None | +------------------+--------+-------------+-----------------------------+----------------------------+----------------------------+ #. To show the step currently being performed on a subcloud, use the :command:`dcmanager strategy-step show` command. .. code-block:: none ~(keystone_admin)]$ dcmanager strategy-step show #. When all the subclouds within the distributed software deploy orchestration indicate they have entered either the complete or failed state, delete the sw-deploy strategy, using the :command:`dcmanager sw-deploy-strategy delete` command. .. code-block:: none ~(keystone_admin)]$ dcmanager sw-deploy-strategy delete +------------------------+----------------------------+ | Field | Value | +------------------------+----------------------------+ | subcloud apply type | parallel | | max parallel subclouds | 2 | | stop on failure | False | | state | deleting | | created_at | 2020-03-23T20:04:50.992444 | | updated_at | 2020-03-23T20:05:14.157352 | +------------------------+----------------------------+ .. rubric:: |postreq| If the subclouds are upgraded with a major software release, it is recommended to also upversion Kubernetes right after the software deployment. See :ref:`kubernetes-upversion-orchestration`. Error Handling During an Orchestrated Subcloud Software Deployment ------------------------------------------------------------------ If a failure occurs, follow the general steps below: #. Allow the failed strategy to complete on its own. #. Check the output using :command:`dcmanager strategy-step list` command. #. To see the cause of failure for a subcloud, use the :command:`dcmanager subcloud errors ` command. #. Resolve the condition that caused the failure. #. Retry the orchestration by deleting the failed strategy and create a new one for the failed subcloud(s).