starlingx/docs
Code Issues Proposed changes
docs/doc/source/dist_cloud/kubernetes/distributed-software-deploy-orchestration-process-using-the-cli.rst
Elisamara Aoki Gonçalves 9e5b12cae9 Upgrade Support doc update (r10.1) 
Story: 2010676
Task: 51423

Change-Id: I2e15fd4645c7661ab838a4ed0e69f7167a1cce79
Signed-off-by: Elisamara Aoki Gonçalves <elisamaraaoki.goncalves@windriver.com>
2024-12-06 13:57:40 +00:00

339 lines
15 KiB
ReStructuredText
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

.. pek1594745988225
.. _distributed-software-deploy-orchestration-process-using-the-cli:
===============================================================
Distributed Software Deploy Orchestration Process using the CLI
===============================================================
Distributed software deploy orchestration can be initiated after the System
Controller has been successfully updated (patch) or deployed.
For more information on Prestaging Subcloud Orchestration see,
:ref:`prestage-subcloud-orchestration-eb516473582f`.
.. note::
This section is applicable to users that use the dcmanager software deploy
orchestration strategy to manage upgrades across subclouds.
Refer to |updates-doc| for more details.
.. rubric:: |context|
The user first creates a distributed software deploy orchestration strategy, or
plan, for the automated software deploy procedure. This customizes the software
deploy orchestration, using parameters to specify:
.. _distributed-upgrade-orchestration-process-using-the-cli-ul-eyw-fyr-31b:
- whether to stop on failure of a subcloud update/upgrade or continue with
the next subcloud
- whether to update/upgrade hosts serially or in parallel
Based on these parameters, and the state of the subclouds, the software deploy
orchestration creates a number of stages for the overall sw-deploy strategy.
All the subclouds that are included in the same stage will be updated/upgraded
in parallel.
.. rubric:: |prereq|
Distributed software deploy orchestration can only be done on a system that
meets the following conditions:
.. _distributed-upgrade-orchestration-process-using-the-cli-ul-blp-gcx-ry:
- All subclouds are clear of management-affecting alarms (with the exception
of the alarm upgrade in progress).
- All hosts of all subclouds must be unlocked, enabled, and available.
- All subclouds should have been prestaged (``--for-sw-deploy``).
.. code-block:: none
~(keystone_admin)]$ dcmanager prestage-strategy create --for-sw-deploy
- All the subclouds do not have any VIM strategy different from ``sw-deploy``
or in an invalid state.
- No distributed software deploy orchestration strategy exists, to verify use
the command :command:`dcmanager sw-deploy-strategy show`. An update/upgrade
cannot be orchestrated while software deploy orchestration is in progress.
- The size and format of the persistent filesystem, /opt/platform-backup, of
each subcloud must be 30GiB (or larger) and ext4 respectively. From the shell
on a subcloud, use the following command to view the details of its
persistent file system:
:command:`df -Th /opt/platform-backup`
The type must be ext4 and the size must be 30GiB. For example, on
controller-0, run the following command:
.. code-block:: none
~(keystone_admin)]$ df -Th /opt/platform-backup
Filesystem Type Size Used Avail Use% Mounted on
/dev/sda1 ext4 29G 6.9G 21G 26% /opt/platform-backup
- **If a previous upgrade has been done on the subcloud**, from the shell on
each subcloud, use the following command to remove the previous upgrade
data:
:command:`sudo rm /opt/platform-backup/upgrade_data\*`
You can configure a software deploy Distributed Cloud orchestration strategy
using the dcmanager CLI or the Horizon web interface. If you prefer to use
Horizon, see
:ref:`create-a-software-deploy-orchestration-using-horizon-9f8c6c2f3706`.
.. rubric:: |proc|
.. _distributed-upgrade-orchestration-process-using-the-cli-steps-vcm-pq4-3mb:
#. Review the software status for the subclouds.
After the System Controller upgrade/deploy is completed, wait for 40
seconds for the **software_sync_status** of all subclouds to be updated.
To identify which subclouds are deploy-current (in-sync), use the
:command:`subcloud list` command. For example:
.. code-block:: none
~(keystone_admin)]$ dcmanager subcloud list
+----+-----------+--------------+--------------------+-------------+
| id | name | management | availability | sync |
+----+-----------+--------------+--------------------+-------------+
| 1 | subcloud-1| managed | online | out-of-sync |
| 2 | subcloud-2| managed | online | out-of-sync |
| 3 | subcloud-3| managed | online | out-of-sync |
| 4 | subcloud-4| managed | online | out-of-sync |
+----+-----------+--------------+--------------------+-------------+
.. note::
The subclouds are out-of-sync because the software-sync-status is
out-of-sync. All of the above subclouds are not deploy-current and,
therefore, need to be upgraded/updated.
To see synchronization details for a subcloud, use the following command:
.. code-block:: none
~(keystone_admin)]$ dcmanager subcloud show subcloud1
+-----------------------------+------------------------------+
| Field | Value |
+-----------------------------+------------------------------+
| id | 1 |
| name | subcloud-1 |
| description | None |
| location | None |
| software_version | nn.nn |
| management | managed |
| availability | online |
| deploy_status | complete |
| management_subnet | fd01:82::0/64 |
| management_start_ip | fd01:82::2 |
| management_end_ip | fd01:82::11 |
| management_gateway_ip | fd01:82::1 |
| systemcontroller_gateway_ip | fd01:81::1 |
| group_id | 1 |
| created_at | 2021-06-07 21:05:16.224664 |
| updated_at | 2021-06-09 20:01:37.525012 |
| dc-cert_sync_status | in-sync |
| firmware_sync_status | in-sync |
| identity_sync_status | in-sync |
| kubernetes_sync_status | in-sync |
| load_sync_status | not-available |
| patching_sync_status | not-available |
| software_sync_status | out-of-sync |
| platform_sync_status | in-sync |
+-----------------------------+------------------------------+
#. To create a sw-deploy strategy, use the :command:`dcmanager sw-deploy-strategy create <release_id>`
command.
The sw-deploy strategy for a |prod-dc| system controls how updates/upgrades
are applied to subclouds.
.. code-block:: none
~(keystone_admin)]$ dcmanager sw-deploy-strategy create \
[--subcloud-apply-type <type>] \
[–-max-parallel-subclouds <i>] \
[–-stop-on-failure <level>] \
[--group group] \
[<subcloud>]
where:
**subcloud-apply-type**
**parallel** or **serial**— determines whether the subclouds are
upgraded in parallel, or serially.
If this is not specified using the CLI, the values for
:command:`subcloud_update_type` defined for each subcloud group will
be used by default.
**max-parallel-subclouds**
Sets the maximum number of subclouds that can be upgraded in parallel
(default 2).
If this is not specified using the CLI, the values for
:command:`max_parallel_subclouds` defined for each subcloud group
will be used by default.
**stop-on-failure**
**true**\(default) or **false**— determines whether upgrade
orchestration failure for a subcloud prevents application to subsequent
subclouds.
**group**
Optionally pass the name or ID of a subcloud group to the
:command:`dcmanager sw-deploy-strategy create` command. This results in a
strategy that is only applied to all subclouds in the specified group.
The subcloud group values are used for subcloud apply type and max
parallel subclouds parameters.
For example:
.. code-block:: none
~(keystone_admin)]$ dcmanager sw-deploy-strategy create
+------------------------+----------------------------+
| Field | Value |
+------------------------+----------------------------+
| strategy type | sw-deploy |
| subcloud apply type | parallel |
| max parallel subclouds | 2 |
| stop on failure | False |
| state | initial |
| created_at | 2024-11-06 12:56:17.111621 |
| updated_at | None |
+------------------------+----------------------------+
#. To show the settings for the ``sw-deploy-strategy``, use the
:command:`dcmanager sw-deploy-strategy show` command.
For example:
.. code-block:: none
~(keystone_admin)]$ dcmanager sw-deploy-strategy show
+------------------------+----------------------------+
| Field | Value |
+------------------------+----------------------------+
| strategy type | sw-deploy |
| subcloud apply type | parallel |
| max parallel subclouds | 2 |
| stop on failure | False |
| state | initial |
| created_at | 2020-02-02T14:42:13.822499 |
| updated_at | None |
+------------------------+----------------------------+
.. note::
The values for `subcloud apply type` and `max parallel subclouds` will
be taken from the subcloud group if specified through the ``--group``
parameter.
#. Review the sw-deploy strategy for the subclouds.
To show the subclouds that will be upgraded when the sw-deploy strategy is
applied, use the :command:`dcmanager strategy-step list` command. For
example:
.. code-block:: none
~(keystone_admin)]$ dcmanager strategy-step list
+------------------+--------+---------+---------+------------+-------------+
| cloud | stage | state | details | started_at | finished_at |
+------------------+--------+---------+---------+------------+-------------+
| subcloud-1 | 1 | initial | | None | None |
| subcloud-4 | 1 | initial | | None | None |
| subcloud-5 | 2 | initial | | None | None |
| subcloud-6 | 2 | initial | | None | None |
+------------------+--------+---------+---------+------------+-------------+
.. note::
All the subclouds that are included in the same stage will be deployed
in parallel.
#. To apply the software deploy strategy, use the :command:`dcmanager sw-deploy-strategy apply`
command.
.. code-block:: none
~(keystone_admin)]$ dcmanager sw-deploy-strategy apply
+------------------------+----------------------------+
| Field | Value |
+------------------------+----------------------------+
| subcloud apply type | parallel |
| max parallel subclouds | 2 |
| stop on failure | False |
| state | applying |
| created_at | 2020-02-02T14:42:13.822499 |
| updated_at | 2020-02-02T14:42:19.376688 |
+------------------------+----------------------------+
#. To show the step currently being performed on each of the subclouds, use
the :command:`dcmanager strategy-step list` command.
For example:
.. code-block:: none
~(keystone_admin)]$ dcmanager strategy-step list
+------------------+--------+-----------------------+-------------------+----------------------------+----------------------------+
| cloud | stage | state | details | started_at | finished_at |
+------------------+--------+-----------------------+-------------------+----------------------------+----------------------------+
| subcloud-1 | 3 | complete | | 2024-11-06 12:57:20.342429 | 2024-11-06 12:57:41.054664 |
| subcloud-4 | 2 | apply VIM sw-deploy | | 2024-11-06 12:57:20.342429 | None |
| subcloud-5 | 1 | initial | | None | None |
| subcloud-6 | 1 | initial | | None | None |
+------------------+--------+-------------+-----------------------------+----------------------------+----------------------------+
#. To show the step currently being performed on a subcloud, use the
:command:`dcmanager strategy-step show` <subcloud> command.
.. code-block:: none
~(keystone_admin)]$ dcmanager strategy-step show <subcloud>
#. When all the subclouds within the distributed software deploy orchestration indicate
they have entered the complete state, delete the sw-deploy strategy, using
the :command:`dcmanager sw-deploy-strategy delete` command.
.. code-block:: none
~(keystone_admin)]$ dcmanager sw-deploy-strategy delete
+------------------------+----------------------------+
| Field | Value |
+------------------------+----------------------------+
| subcloud apply type | parallel |
| max parallel subclouds | 2 |
| stop on failure | False |
| state | deleting |
| created_at | 2020-03-23T20:04:50.992444 |
| updated_at | 2020-03-23T20:05:14.157352 |
+------------------------+----------------------------+
.. note::
Before attempting to log in to the subclouds using the sysadmin account,
verify that the subcloud ``platform_sync_status`` is synced. This would
ensure that the sysadmin password is successfully resynced to the subclouds
and that login attempts do not fail.
.. only:: partner
.. include:: /_includes/distributed-upgrade-orchestration-process-using-the-cli.rest
:start-after: DMupgrade-begin
:end-before: DMupgrade-end
View Git Blame Copy Permalink