Update STARLINGX Developer/Contributor documentation on building ISOs and Patches

Remove patch requirements install step.
Update command.

Story: 2010676
Task: 50548

Change-Id: I0a36369a47f6fea6b855ac941700f5024fe7a8f8
Signed-off-by: Elisamara Aoki Goncalves <elisamaraaoki.goncalves@windriver.com>

doc/source/developer_resources/starlingx_patching.rst

@@ -21,7 +21,7 @@ vulnerabilities, feature enhancements, etc.
 
 StarlingX patching supports two kinds of patches:
 
-*  *In-service* patches, which don’t require a reboot for those hosts to be
+*  *In-service* patches, which do not require a reboot for those hosts to be
    patched. Only affected processes need to be restarted.
 
 *  *Reboot-required* patches, which require a host reboot operation to make the
@@ -42,8 +42,13 @@ A StarlingX patch includes one or more rpms that contain some updates to the
 system. Before starting to generate a patch, you must identify rpms to be
 installed against the deployed system.
 
+Starting at StarlingX 10.0, the patches will now be built and applied using
+|USM|, which means the patch process will be simplified as there is no need to
+deliver ostree deltas anymore and just debian packages (.deb's). Legacy
+patching will no longer be supported.
+
 *****************************
-Identify rpms to be installed
+Identify debs to be Installed
 *****************************
 
 First, figure out the software version of the deployed system, using either
@@ -58,7 +63,7 @@ Use Horizon GUI
 
    Discover your OAM floating IP address with the :command:`system oam-show` command.
 
-#. Log in to Horizon with an admin/<sysadmin-password>.
+#. Log in to Horizon with an ``admin/<sysadmin-password>``.
 
 #. Identify the software version of the deployed system by clicking
    :menuselection:`Admin > Platform > System Configuration` and clicking the
@@ -72,7 +77,7 @@ Use StarlingX CLI
    node to show the software version. In the example below, the software version
    is "19.09".
 
-   ::
+   .. code-block::
 
 	controller-0:~$ . /etc/platform/openrc
 	[sysadmin@controller-0 ~(keystone_admin)]$ system show
@@ -99,121 +104,199 @@ Use StarlingX CLI
 	+----------------------+--------------------------------------+
 
 
-After you figure out the software version, check the latest StarlingX
+After you figure out the software version, check the latest StarlingX build to
-build to identify updated rpms against that version. Select one or
+identify updated debs against that version. Select one or more debs for patch
-more rpms for patch generation and continue to the next section.
+generation and continue to the next section.
 
 *************
-Build patches
+Build Patches
 *************
 
-After you determine the rpms to be installed, set up an environment for
+After you determine the debs to be installed, set up an environment for
 patch build. As a StarlingX developer, the easiest way to set up this type of
 environment is to use a StarlingX build container, with a few changes.
 
-#############
+.. rubric:: |prereq|
-Prerequisites
-#############
 
-*  Set up a StarlingX build container following `the
+#.	Set up a StarlingX build container following `the
    	build guide <https://docs.starlingx.io/developer_resources/build_guide.html>`_.
 
-*  Download StarlingX source code.
+#. 	Download StarlingX source code.
 
-*  Be sure that the rpms to be installed are already available in the build
+#. 	Populate your build using ``build-pkgs --reuse`` or ``build-pkgs --all``.
-   container filesystem.
+  	Using the reuse flag should be faster if you already have the running env.
 
-Set up the patch build environment using the steps below. Note that
+#.	Make your changes, usually by cherry picking some commit.
-this environment is intended for developers working on this feature
+
+#.	Apply the changes to the necessary files and than  build the packages
+  	related to these files with:
+
+	.. code-block:: none
+
+		$  build-pkgs -p <package name>
+
+#.	Build all necessary packages. To find out the package name, look for a
+  	debian ``meta_data.yaml`` file.
+
+.. rubric:: |proc|
+
+Set up the patch build environment using the procedure below.
+
+.. note::
+
+	This environment is intended for developers working on this feature
 	and is not intended to be used for a StarlingX-based product.
 
-#. Install two python packages ``crypto`` and ``pycrypto``, which are the
+.. note::
-   python dependencies of the ``cgcs-patch`` package.
 
-   ::
+	Patch requirements are automatically installed when containers are created.
 
-    $ sudo pip install crypto pycrypto
 
-#. Use the script ``$MY_REPO/stx/stx-update/extras/scripts/patch_build.sh``
+#.	Create your patch recipe. The recipe file is used as input and it has all
-   to build patches. This script sources the **PLATFORM_RELEASE**
+  	the patch metadata. Packages that will be included in the patch need to be
-   variable from the **release-info.inc** file, and sets up the PYTHONPATH to
+  	added to the `packages` object ("stx" are packages we build, "binary" are
-   point to cgcs-patch package in the repo. This avoids the need to install
+  	packages we download).
-   cgcs-patch or manually set the PLATFORM_RELEASE variable.
 
-   View the details of the patch build command using the command:
+	.. code-block:: none
-   ``$MY_REPO/stx/stx-update/extras/scripts/patch_build.sh -h``
 
-   ::
+		<patch_recipe>
+			<!-- Software Version -->
+			<sw_version>1.0.0</sw_version>
+			<!-- Component -->
+			<component>starlingx</component>
+			<!-- Summary: Short text to give a summary about the patch -->
+			<summary>sample patch test</summary>
+			<!-- Description: Patch description. Usually it has a list of fixes -->
+			<description>Sample description</description>
+			<!-- Install Instructions: Any instructions to be done before the patch installation -->
+			<install_instructions>Sample instructions</install_instructions>
+			<!-- Warnings: Any warnings that this patch can trigger -->
+			<warnings>Sample warning</warnings>
+			<!-- Reboot required: Y (Yes) or N (No) for in service patch -->
+			<reboot_required>Y</reboot_required>
+			<!-- Unremovable: Y (Yes)/ N (No), specifices if the patch can be removed -->
+			<unremovable>N</unremovable>
+			<!-- Patch Status: Supported values are DEV (development) and REL (released) -->
+			<status>DEV</status>
+			<!-- Requires: List of patches that are required by this patch -->
+			<requires>
+				<!--
+				<id>PATCH_XYZ_01</id>
+				<id>PATCH_XYZ_02</id>
+				-->
+			</requires>
+			<semantics></semantics>
+			<!--
+				Pre and Post install hook scripts that are executed before/after patch installation.
+				Leave if blank if no scripts are required.
+			-->
+			<pre_install>scripts/pre-install.sh</pre_install>
+			<post_install>scripts/post-install.sh</post_install>
+			<!-- List Packages to be included in the patch -->
+			<stx_packages>
+				<!-- Starlingx packages list -->
+				<package>sysvinv</package>
+				<package>linux</package>
+				<package>linux-rt</package>
+			</stx_packages>
+			<!-- Binary packages list to be included in the patch (Packages that we download from 3rd party sources) -->
+			<binary_packages>
+				<!-- 3rd party packages list -->
+				<package>curl</package>
+			</binary_packages>
+		</patch_recipe>
 
-	$ $MY_REPO/stx/stx-update/cgcs-patch/bin/patch_build --help
+**Patch build**
-	Usage: patch_build [ <args> ] ... <rpm list>
-	Options:
-		--id <id>                   Patch ID
-		--release <version>         Platform release version
-		--status <status>           Patch Status Code (ie. O, R, V)
-		--unremovable               Marks patch as unremovable
-		--reboot-required <Y|N>     Marks patch as reboot-required (default=Y)
-		--summary <summary>         Patch Summary
-		--desc <description>        Patch Description
-		--warn <warnings>           Patch Warnings
-		--inst <instructions>       Patch Install Instructions
-		--req <patch_id>            Required Patch
-		--controller <rpm>          New package for controller
-		--worker <rpm>              New package for worker node
-		--worker-lowlatency <rpm>   New package for worker-lowlatency node
-		--storage <rpm>             New package for storage node
-		--controller-worker <rpm>   New package for combined node
-		--controller-worker-lowlatency <rpm>   New package for lowlatency
-		                            combined node
-		--all-nodes <rpm>           New package for all node types
 
-   You must specify the patch id, whether a reboot is required or not, any
+#.	Create the patch.
-   dependent patches, the file path of rpms, etc. For new rpm packages, you
-   must specify which hosts require them using options, for example
-   ``--controller``.
 
-   When the command is finished, you get a patch file with the name
+	.. code-block:: none
-   ``<patch-id>.patch``.
 
-Let’s dive a little deeper into the generated patch.
+		$ patch-builder --recipe EXAMPLES/patch-recipe-sample.xml
+		 ...
+		2024-06-12 13:12:49,566 - patch_builder - DEBUG: Copying pre-install script: /localdisk/designer/USER/master-env/cgcs-root/build-tools/stx/patch/scripts/pre-install.sh
+		2024-06-12 13:12:49,567 - patch_builder - INFO: Renaming /localdisk/designer/USER/master-env/cgcs-root/build-tools/stx/patch/scripts/pre-install.sh to pre-install.sh
+		2024-06-12 13:12:49,567 - patch_builder - DEBUG: Copying post-install script: /localdisk/designer/USER/master-env/cgcs-root/build-tools/stx/patch/scripts/post-install.sh
+		2024-06-12 13:12:49,567 - patch_builder - INFO: Renaming /localdisk/designer/USER/master-env/cgcs-root/build-tools/stx/patch/scripts/post-install.sh to post-install.sh
+		2024-06-12 13:12:49,567 - patch_builder - DEBUG: Generating metadata file
+		2024-06-12 13:12:49,569 - patch_builder - DEBUG: Generating signature for patch files ['metadata.tar', 'software.tar', 'pre-install.sh', 'post-install.sh']
+		2024-06-12 13:12:49,888 - patch_builder - INFO: Formal signing status: False
+		2024-06-12 13:12:49,888 - patch_builder - INFO: Saving file software.tar
+		2024-06-12 13:12:51,441 - patch_builder - INFO: Saving file post-install.sh
+		2024-06-12 13:12:51,441 - patch_builder - INFO: Saving file metadata.tar
+		2024-06-12 13:12:51,442 - patch_builder - INFO: Saving file signature
+		2024-06-12 13:12:51,442 - patch_builder - INFO: Saving file signature.v2
+		2024-06-12 13:12:51,442 - patch_builder - INFO: Saving file pre-install.sh
+		2024-06-12 13:12:51,442 - patch_builder - INFO: Patch file created /localdisk/loadbuild/USER/master-env/patch_output/starlingx-1.0.0.patch
 
-#. The patch file is a gzip compressed package. Check it
+#.	Verify patch content.
-   using the :command:`file` command. Below is an example, including command output.
 
-   ::
+	.. code-block:: none
 
-	$ file 001.patch
+		$ tar -tvf /localdisk/loadbuild/USER/master-env/patch_output/starlingx-1.0.0.patch
-	001.patch: gzip compressed data, was "001.patch", last modified:
+				USER/cgts 74772480 2024-06-12 13:12 software.tar
-	Fri Aug 16 05:56:59 2019, max compression
+				USER/cgts      440 2024-06-12 13:12 post-install.sh
+				USER/cgts    10240 2024-06-12 13:12 metadata.tar
+				USER/cgts       32 2024-06-12 13:12 signature
+				USER/cgts      256 2024-06-12 13:12 signature.v2
+				USER/cgts      433 2024-06-12 13:12 pre-install.sh
 
-#. Unzip the patch file to discover four files, ``software.tar``,
+#.	Verify patch metadata.
-   ``metadata.tar``, ``signature``, and ``signature.v2``.
 
-   ::
+	.. code-block:: none
 
-	$ tar -xf 001.patch
+		$ tar xf /localdisk/loadbuild/USER/stx-master/patch_output/sample_patch.patch -O metadata.tar | tar x -O
-	$ tree
+		<?xml version="1.0" ?>
-	├── 001.patch
+		<patch>
-	├── metadata.tar
+		  <id>starlingx-1.0.0</id>
-	├── signature
+		  <sw_version>1.0.0</sw_version>
-	├── signature.v2
+		  <component>starlingx</component>
-	└── software.tar
+		  <summary>sample patch test</summary>
+		  <install_instructions>Sample instructions</install_instructions>
+		  <warnings>Sample warning</warnings>
+		  <status>DEV</status>
+		  <unremovable>N</unremovable>
+		  <reboot_required>Y</reboot_required>
+		  <semantics/>
+		  <requires/>
+		  <pre_install>pre-install.sh</pre_install>
+		  <post_install>post-install.sh</post_install>
+		  <packages>
+			<deb>curl_7.74.0-1.3+deb11u11_amd64.deb</deb>
+			<deb>linux-compiler-gcc-10-x86_5.10.205-1.stx.79_amd64.deb</deb>
+			<deb>linux-headers-5.10.0-6-amd64_5.10.205-1.stx.84_amd64.deb</deb>
+			<deb>linux-headers-5.10.0-6-common_5.10.205-1.stx.84_all.deb</deb>
+			<deb>linux-image-5.10.0-6-amd64-unsigned_5.10.205-1.stx.84_amd64.deb</deb>
+			<deb>linux-kbuild-5.10_5.10.205-1.stx.84_amd64.deb</deb>
+			<deb>linux-libc-dev_5.10.205-1.stx.84_amd64.deb</deb>
+			<deb>linux-perf-5.10_5.10.205-1.stx.84_amd64.deb</deb>
+			<deb>linux-rt-headers-5.10.0-6-rt-amd64_5.10.205-1.stx.79_amd64.deb</deb>
+			<deb>linux-rt-headers-5.10.0-6-rt-common_5.10.205-1.stx.79_all.deb</deb>
+			<deb>linux-rt-image-5.10.0-6-rt-amd64-unsigned_5.10.205-1.stx.79_amd64.deb</deb>
+			<deb>linux-rt-kbuild-5.10_5.10.205-1.stx.79_amd64.deb</deb>
+		  </packages>
+		</patch>
 
-   *  Unzip ``software.tar`` to find that it contains all rpms to be
-      installed. Note that all rpms have been signed during patch
-      build using the key
-      ``$MY_REPO/build-tools/signing/ima_signing_key.priv``.
 
-   *  ``metadata.tar`` contains only one file ``metadata.xml``, which
+#.	Verify patch content (``software.tar``).
-      includes all the information provided by the patch build command. It is
-      used by the StarlingX patching runtime system.
 
-   *  ``signature`` is a combination of the md5 values of ``software.tar`` and
+	.. code-block:: none
-      ``metadata.tar``.
+
+		$ tar xf /localdisk/loadbuild/USER/stx-master/patch_output/sample_patch.patch -O software.tar | tar t
+
+			   linux-kbuild-5.10_5.10.205-1.stx.84_amd64.deb
+				linux-compiler-gcc-10-x86_5.10.205-1.stx.79_amd64.deb
+				linux-rt-image-5.10.0-6-rt-amd64-unsigned_5.10.205-1.stx.79_amd64.deb
+				linux-rt-kbuild-5.10_5.10.205-1.stx.79_amd64.deb
+				linux-perf-5.10_5.10.205-1.stx.84_amd64.deb
+				linux-rt-headers-5.10.0-6-rt-common_5.10.205-1.stx.79_all.deb
+				linux-rt-headers-5.10.0-6-rt-amd64_5.10.205-1.stx.79_amd64.deb
+				linux-headers-5.10.0-6-amd64_5.10.205-1.stx.84_amd64.deb
+				linux-image-5.10.0-6-amd64-unsigned_5.10.205-1.stx.84_amd64.deb
+				linux-libc-dev_5.10.205-1.stx.84_amd64.deb
+				curl_7.74.0-1.3+deb11u11_amd64.deb
+				linux-headers-5.10.0-6-common_5.10.205-1.stx.84_all.deb
 
-   *  ``signature.v2`` is a signature file for ``software.tar`` and
-      ``metadata.tar``. In this case, it is generated using the key file
-      ``$MY_REPO/build-tools/signing/dev-private-key.pem``.
 
 -------------
 Apply a patch

doc/source/shared/abbrevs.txt

@@ -208,6 +208,7 @@
 .. |UDS| replace:: :abbr:`UDS (Unix Domain Socket)`
 .. |UEFI| replace:: :abbr:`UEFI (Unified Extensible Firmware Interface)`
 .. |UFT| replace:: :abbr:`UFT (Unified Flow Tool)`
+.. |USM| replace:: :abbr:`USM (Unified Software Management)`
 .. |UUID| replace:: :abbr:`UUID (Universally Unique Identifier)`
 .. |UUIDs| replace:: :abbr:`UUIDs (Universally Unique Identifiers)`
 .. |VF| replace:: :abbr:`VF (Virtual Function)`