Merge "Platform backup and restore optimization"

doc/source/backup/kubernetes/index-backup-kub-699e0d16c076.rst

@@ -23,5 +23,7 @@ System and storage restore
    restoring-starlingx-system-data-and-storage
    running-restore-playbook-locally-on-the-controller
    system-backup-running-ansible-restore-playbook-remotely
+   node-replacement-for-aiominussx-using-optimized-backup-and-restore-6603c650c80d
+
 
 

doc/source/backup/kubernetes/node-replacement-for-aiominussx-using-optimized-backup-and-restore-6603c650c80d.rst

@@ -0,0 +1,32 @@
+
+.. _node-replacement-for-aiominussx-using-optimized-backup-and-restore-6603c650c80d:
+
+================================
+AIO-SX - Restore on new hardware
+================================
+
+For |AIO-SX| Node Replacement with configured ceph storage backend it is not
+possible to restore ceph data, since OSD disks will not be present to recover.
+The flag ``wipe_ceph_osds=true`` should be set in this case when running the
+optimized restore playbook.
+
+The improved backup and restore supports |AIO-SX| Node Replacement, by the
+following method.
+
+Restore Playbook Parameter to Specify Node Replacement
+------------------------------------------------------
+
+An optional parameter is provided to the restore playbook with the new
+management MAC address:
+
+.. code-block:: none
+
+    -e replacement_mgmt_mac <mgmt_mac>
+
+This parameter is accepted only for controller-0.
+
+For example:
+
+.. code-block:: none
+
+    ~(keystone_admin)]$ ansible-playbook /usr/share/ansible/stx-ansible/playbooks/restore_platform.yml -e "initial_backup_dir=/home/sysadmin" -e "ansible_become_pass=St8rlingX*" -e "admin_password=St8rlingX*" -e "backup_filename=localhost_platform_backup.tgz" -e "restore_mode=optimized" -e "restore_registry_filesystem=true" -e "replacement_mgmt_mac=a1:a2:a3:a4:a5:a6"

doc/source/backup/kubernetes/running-ansible-backup-playbook-locally-on-the-controller.rst

@@ -10,14 +10,26 @@ Run Ansible Backup Playbook Locally on the Controller
 
 In this method the Ansible Backup playbook is run on the active controller.
 
-Use one of the following commands to run the Ansible Backup playbook and back up the
-|prod| configuration, data, and user container images in registry.local data:
+Use the following command to run the Ansible Backup playbook and back up the
+|prod| configuration, data, and user container images in registry.local:
 
 .. code-block:: none
 
-    ~(keystone_admin)]$ ansible-playbook /usr/share/ansible/stx-ansible/playbooks/backup.yml -e "ansible_become_pass=<sysadmin password> admin_password=<sysadmin password>" -e "backup_user_local_registry=true"
+    ~(keystone_admin)]$ ansible-playbook /usr/share/ansible/stx-ansible/playbooks/backup.yml -e "ansible_become_pass=<sysadmin password> admin_password=<sysadmin password>" -e "backup_registry_filesystem=true"
     ~(keystone_admin)]$ ansible-playbook /usr/share/ansible/stx-ansible/playbooks/backup.yml --ask-vault-pass -e "override_files_dir=$HOME/override_dir"
 
+
+To exclude a directory and all the files in it like ``/var/home*`` you can use
+the optional parameter:
+
+:command:`-e "exclude_dirs=/var/home/**,/var/home"`
+
+.. note::
+
+    A 'glob' pattern is required to use ``-e "exclude_dirs=/var/home/**,/var/home"``,
+    in order to ensure there is sufficient free space in the required
+    directories in any event.
+
 The <admin_password> and <ansible_become_pass> need to be set  correctly
 using the ``-e`` option on the command line, with an override file secured with
 ansible-vault (recommended).
@@ -31,9 +43,26 @@ command if the file needs to be edited after it is created.
 
    ansible_become_pass: "<admin_password>"
    admin_password: "<admin_password>"
-   backup_user_local_registry: "true"
+   backup_registry_filesystem: "true"
+   exclude_dirs: /var/home/**,/var/home"
+   ...
+   EOF
 
-The output files will be named:
+The extra var ``backup_registry_filesystem`` is an optional parameter and it is
+used to backup all images on the registry backup, generating a file named
+``{inventory_hostname}_image_registry_backup_YYYY_MM_DD_HH_mm_ss.tgz``. When
+not specified, the restore will download images from the upstream docker
+registry.
+
+For example:
+
+.. code-block:: none
+
+    ~(keystone_admin)]$ ansible-playbook /usr/share/ansible/stx-ansible/playbooks/backup.yml -e "backup_registry_filesystem=true"
+
+
+A list of possible output files, files created depend on backup options and
+system configuration.
 
 .. _running-ansible-backup-playbook-locally-on-the-controller-ul-wj1-vxh-pmb:
 
@@ -41,10 +70,12 @@ The output files will be named:
 
 -   inventory_hostname_wr-openstack_backup_timestamp.tgz
 
--   inventory_hostname_docker_local_registry_backup_timestamp.tgz
+-   inventory_hostname_user_images_backup_timestamp.tgz
 
 -   inventory_hostname_dc_vault_backup_timestamp.tgz
 
+-   inventory_hostname_image_registry_backup_timestamp.tgz
+
 The output files' prefixes can be overridden with the following variables
 using the ``-e`` option on the command line or by using an override file.
 

doc/source/backup/kubernetes/running-ansible-backup-playbook-remotely.rst

@@ -77,11 +77,11 @@ and target it at controller-0.
             yes/no: 'yes'
             sysadmin*: 'sysadmin'
             (current) UNIX password: 'sysadmin'
-            New password: 'Li69nux*'
-            Retype new password: 'Li69nux*'
-        admin_password: Li69nux*
-        ansible_become_pass: Li69nux*
-        ansible_ssh_pass: Li69nux*
+            New password: 'St8rlingX*'
+            Retype new password: 'St8rlingX*'
+        admin_password: St8rlingX*
+        ansible_become_pass: St8rlingX*
+        ansible_ssh_pass: St8rlingX*
 
     Save your changes and quit the editor. If you need to make additional
     changes, you can use the command :command:`ansible-vault edit

doc/source/backup/kubernetes/running-restore-playbook-locally-on-the-controller.rst

@@ -16,7 +16,32 @@ following command to run the Ansible Restore playbook:
 
 .. code-block:: none
 
-    ~(keystone_admin)]$ ansible-playbook /usr/share/ansible/stx-ansible/playbooks/restore_platform.yml -e "initial_backup_dir=<location_of_tarball ansible_become_pass=<admin_password> admin_password=<admin_password backup_filename=<backup_filename> <optional-restore-mode>"
+    ~(keystone_admin)]$ ansible-playbook /usr/share/ansible/stx-ansible/playbooks/restore_platform.yml -e "initial_backup_dir=<location_of_tarball ansible_become_pass=<admin_password> admin_password=<admin_password backup_filename=<backup_filename> wipe_ceph_osds=<true/false>" -e "restore_registry_filesystem=true"
+
+
+Other ``-e`` command line options:
+
+``-e restore_mode=optimized``
+    Enable optimized restore mode
+
+``-e "initial_backup_dir=/home/sysadmin"``
+    Where the backup tgz files are located on box.
+
+``-e backup_filename=localhost_platform_backup.tgz``
+    The basename of the platform backup tgz.  The full path will be a
+    combination ``{initial_backup_dir}/{backup_filename}``
+
+``-e restore_registry_filesystem=true``
+    Restores the registry images created during backup when
+    ``backup_registry_filesystem`` was true.  If the registry filesystem is not
+    restored the images must be pulled from upstream or registry.central.
+
+``-e registry_backup_filename=custom_name_registry_filesystem_backup.tgz``
+    By default this override is not required.  When
+    ``restore_registry_filesystem`` is true and a custom name was used during
+    backup, ``registry_backup_filename`` needs to be set to match. The full
+    path will be a combination
+    ``{initial_backup_dir}/{registry_backup_filename}``
 
 
 .. _running-restore-playbook-locally-on-the-controller-steps-usl-2c3-pmb:
@@ -54,7 +79,7 @@ following command to run the Ansible Restore playbook:
 
         .. code-block:: none
 
-            ~(keystone_admin)]$ ansible-playbook /usr/share/ansible/stx-ansible/playbooks/restore_platform.yml -e "initial_backup_dir=/home/sysadmin ansible_become_pass=St0rlingX* admin_password=St0rlingX* backup_filename=localhost_platform_backup_2020_07_27_07_48_48.tgz wipe_ceph_osds=true"
+            ~(keystone_admin)]$ ansible-playbook /usr/share/ansible/stx-ansible/playbooks/restore_platform.yml -e "initial_backup_dir=/home/sysadmin ansible_become_pass=St8rlingX* admin_password=St8rlingX* backup_filename=localhost_platform_backup_2020_07_27_07_48_48.tgz wipe_ceph_osds=true"
 
         .. note::
 
@@ -62,6 +87,11 @@ following command to run the Ansible Restore playbook:
             the patches and prompt you to reboot the system. Then you will need
             to re-run Ansible Restore playbook.
 
+            The flag ``wipe_ceph_osds=true`` is required for a restore in a new
+            hardware, for more details see :ref:`AIO-SX - Restore on new
+            hardware
+            <node-replacement-for-aiominussx-using-optimized-backup-and-restore-6603c650c80d>`.
+
 
     -   The ``ssl_ca_certificate_file`` defines the ssl_ca certificate that will be
         installed during the restore. It will replace the ``ssl_ca`` certificate
@@ -105,4 +135,4 @@ For example:
 
 .. code-block:: none
 
-    ~(keystone_admin)]$  ansible-playbook /usr/share/ansible/stx-ansible/playbooks/restore_user_images.yml -e "initial_backup_dir=/home/sysadmin backup_filename=localhost_docker_local_registry_backup_2020_07_15_21_24_22.tgz ansible_become_pass=St0rlingX*"
+    ~(keystone_admin)]$ ansible-playbook /usr/share/ansible/stx-ansible/playbooks/restore_user_images.yml -e "initial_backup_dir=/home/sysadmin backup_filename=localhost_docker_local_registry_backup_2020_07_15_21_24_22.tgz ansible_become_pass=St8rlingX*"