starlingx/docs
Code Issues Proposed changes

Doc Update for Harbor Support

Browse Source 
Story: 2010721
Task: 50133

Change-Id: Id5adb9966bf8c081629116b46352a155a0e4dfd5
Signed-off-by: Elisamara Aoki Goncalves <elisamaraaoki.goncalves@windriver.com>
This commit is contained in:
Elisamara Aoki Goncalves 2024-05-21 12:22:25 +00:00
parent 56c38b03bb
commit 029d1af2d8
6 changed files with 604 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

BIN
doc/source/admintasks/kubernetes/figures/interrogation-services.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 34 KiB

BIN
doc/source/admintasks/kubernetes/figures/library-harbor.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 36 KiB

BIN
doc/source/admintasks/kubernetes/figures/new-replication-rule.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 44 KiB

590
doc/source/admintasks/kubernetes/harbor-as-system-app-1d1e3ec59823.rst Normal file
View File

@ -0,0 +1,590 @@
.. _harbor-as-system-app-1d1e3ec59823:
============================
Harbor as System Application
============================
.. rubric:: |context|
Harbor is an open-source registry that secures artifacts with policies and
role-based access control, ensures images are scanned and free from
vulnerabilities, and signs images as trusted. Harbor has been evolved to a
complete |OCI| compliant cloud-native artifact registry.
With Harbor V2.0, users can manage images, manifest lists, Helm charts,
|CNABs|, |OPAs| among others which all adhere to the |OCI| image specification.
It also allows for pulling, pushing, deleting, tagging, replicating, and
scanning such kinds of artifacts. Signing images and manifest list are also
possible now.
Harbor supports replication of images between registries, and offers advanced
security features such as user management, access control and activity
auditing.
See https://goharbor.io/docs/2.0.0/ for more details on Harbor.
-------------------
Harbor Installation
-------------------
.. rubric:: |prereq|
- CephFS backed |PVCs| are recommended for Harbor such that when configuring
multiple replicas, for |AIO-DX| or Standard configurations, both Harbor
replicas can read and write to the registry.
- Create a secret as described below:
- Generate certificates and create secret.
A |CA| cert and server cert creation procedure using cert-manager is
specified below:
Create the certificate for Harbor using Cert-Manager and using the
local |CA|, system-local-ca, as the issuer. Note that the certificate
should be created in the ``harbor-tls SECRET`` in the Harbor
``NAMESPACE``.
For example:
#. Create the following Harbor certificate yaml configuration file:
.. code-block:: none
~(keystone_admin)]$ cat <<EOF > harbor-certificate.yaml
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: harbor-certificate
namespace: harbor
spec:
secretName: harbor-tls
issuerRef:
name: system-local-ca
kind: ClusterIssuer
duration: 2160h # 90 days
renewBefore: 360h # 15 days
commonName: < oam floating IP Address or FQDN >
subject:
organizations:
- ABC-Company
organizationalUnits:
- StarlingX-harbor
ipAddresses:
- < oam floating IP address >
dnsNames:
- < harbor dns> # e.g. harbor.yourdomian.com
- < notary dns > # optional, required only if exposed on ingress e.g. notary.yourdomian.com
EOF
#. Apply the configuration:
.. code-block:: none
~(keystone_admin)]$ kubectl apply -f harbor-certificate.yaml
#. Verify the configuration:
.. code-block:: none
~(keystone_admin)]$ kubectl get certificate harbor-certificate -n harbor
After successful configuration, the certificate's Ready status
will be True.
- nodePort
#. Create Harbor using NodePort to expose the service
.. note::
The instructions below assume that the NodePorts 30102, 30103
and 30104 are available; i.e. not used by any other
applications.
#. Locate the Harbor system application tarball in
``/usr/local/share/applications/helm``.
For example:
.. code-block:: none
/usr/local/share/applications/helm/harbor-<version>.tgz
#. Upload the Harbor application.
.. code-block:: none
~(keystone_admin)]$ system application-upload /usr/local/share/applications/helm/harbor-<version>.tgz
#. Configure the Helm Overrides for Harbor.
Below values need to be configured for nodePort:
.. code-block:: none
expose:
type: nodePort # Type should be nodeport
tls:
enabled: true
certSource: secret
secret: # Certificate Source is secret
secretName: "harbor-tls" # A secret containing tls.crt and tls.key
notarySecretName: "harbor-tls" # A secret containing tls.crt and tls.key
nodePort:
# The name of NodePort service
name: harbor
ports:
http:
# The service port Harbor listens on when serving HTTP
port: 80
# The node port Harbor listens on when serving HTTP
nodePort: 30002
https:
# The service port Harbor listens on when serving HTTPS
port: 443
# The node port Harbor listens on when serving HTTPS
nodePort: 30003
# Only needed when notary.enabled is set to true
notary:
# The service port Notary listens on
port: 4443
# The node port Notary listens on
nodePort: 30004
externalURL: https://harbor.yourdomian.com:30003 # URL of harbor listing on 30003 port
For |AIO-DX| and standard setup, add below ``storageClass`` and
``accessModes`` override.
Underlying PVCs pre-requisistes: ``Harbor-Jobservice`` and
``Harbor-Registry`` microservice.
For example:
.. code-block:: none
persistence:
enabled: true
resourcePolicy: "keep"
persistentVolumeClaim:
registry:
existingClaim: ""
storageClass: "cephfs"
subPath: ""
accessMode: ReadWriteMany
size: 5Gi
annotations: {}
jobservice:
jobLog:
existingClaim: ""
storageClass: "cephfs"
subPath: ""
accessMode: ReadWriteMany
size: 1Gi
annotations: {}
#. Execute Helm overrides.
.. code-block:: none
~(keystone_admin)]$ system helm-override-update harbor harbor harbor --values values.yaml
#. Apply/Create the Harbor system application.
.. code-block:: none
~(keystone_admin)]$ system application-apply harbor
- Ingress
Create Harbor using Ingress to expose the service.
.. note::
The instructions below assume that the URL
``harbor.yourdomain.com`` has been configured in the |DNS| server
owning ``yourdomain.com`` as the ``OAM FLOATING IP Address`` of
|prod|.
#. Locate the Harbor system application tarball in
``/usr/local/share/applications/helm``.
For example:
.. code-block:: none
/usr/local/share/applications/helm/harbor-<version>.tgz
#. Upload the Harbor application.
.. code-block:: none
~(keystone_admin)]$ system application-upload /usr/local/share/applications/helm/harbor-<version>.tgz
#. Configure the Helm overrides for Harbor configuration.
The values below need to be configured for ingress in the
``values.yaml`` file.
.. code-block:: none
expose:
type: ingress. # Type should be ingress
tls:
enabled: true
certSource: secret
secret: # Certificate Source is secret
secretName: "harbor-tls" # Above created secret name
notarySecretName: "harbor-tls" # Above created secret name
ingress:
hosts:
core: harbor.yourdomian.com # Harbor Domain name
notary: notary.yourdomian.com # Notary Domain name
annotations:
kubernetes.io/ingress.class: nginx. # Add ingressclass name. It would be # "nginx" if you are using default ingress # controller.
nginx.org/client-max-body-size: "0". # Add this notation for nginx otherwise nginx # will reject the image pull & push
externalURL: https://harbor.yourdomian.com # URL of harbor
For |AIO-DX| and standard setup, add below ``storageClass`` and
``accessModes`` override for |PVC| used for ``Harbor-Jobservice``
and ``Harbor-Registry`` microservice.
For example:
.. code-block:: none
persistence:
enabled: true
resourcePolicy: "keep"
persistentVolumeClaim:
registry:
existingClaim: ""
storageClass: "cephfs"
subPath: ""
accessMode: ReadWriteMany
size: 5Gi
annotations: {}
jobservice:
jobLog:
existingClaim: ""
storageClass: "cephfs"
subPath: ""
accessMode: ReadWriteMany
size: 1Gi
annotations: {}
Update the Helm overrides.
.. code-block:: none
~(keystone_admin)]$ system helm-override-update harbor harbor harbor --values values.yaml
#. Apply/Create the Harbor system application.
.. code-block:: none
~(keystone_admin)]$ system application-apply harbor
-------------------------------------------------
Configure LDAP Authentication for Harbor Registry
-------------------------------------------------
To configure Harbor to use |prod| Local |LDAP| for authentication, follow the
instructions in `Configure LDAP/Active Directory Authentication
<https://goharbor.io/docs/2.8.0/administration/configure-authentication/ldap-auth/>`__
with the following values:
For |prod| local |LDAP|:
.. code-block:: none
LDP URL: ldap://controller
LDAP search DN: cn=ldapadmin,dc=cgcs,dc=local
LDAP Search Password: <Password of ldapadmin>
LDAP Base DN: dc=cgcs,dc=local
LDAP UID: cn
--------------------------------------
Push an Image to a <project> in Harbor
--------------------------------------
#. Run :command:`sudo su` before Docker login.
#. Docker Login.
.. code-block:: none
docker login <harbor_address> -u <username>
.. note::
Replace ``<harbor_address>`` with actual harborURL and replace
``<username>`` with your actual username.
#. Tag the image.
.. code-block:: none
docker tag redis:latest <harbor_address>/<project>/redis:latest
#. Push the image.
.. code-block:: none
docker push <harbor_address>/<project>/redis:latest
-------------------------
Pull an Image from Harbor
-------------------------
Use command to pull an image:
.. code-block:: none
docker pull <harbor_address>/<project>/ redis:latest
Where ``<harbor-address>`` is either:
- for ``'Ingress' expose: harbor.yourdomian.com``
- for ``'NodePort' expose: https:// <oam-floating-ip>:30103``
----------------------------------
Push a Helm Chart as an OCI Object
----------------------------------
.. code-block:: none
helm package <Chart_Path>
helm push <chart_package> oci://<harbor_address>/<project>/
-----------------
Pull a Helm Chart
-----------------
.. code-block:: none
helm pull oci://<chart_package/<project>/harbor
------------------
Project Management
------------------
A project in Harbor contains all the repositories (images) of a particular
application; e.g. the |prod| project in Harbor would contain all the
images from |prod|.
For more details on creating and configuring a project see: `Create Projects
<https://goharbor.io/docs/2.8.0/working-with-projects/create-projects/>`__ and
`Project Configuration
<https://goharbor.io/docs/2.8.0/working-with-projects/project-configuration/>`__.
---------------------------------
Use Images from a Remote Registry
---------------------------------
A project in a local Harbor registry can be used as a pull-through cache for a
remote registry. For example, a Harbor registry project on a |prod| Subcloud
can be used as a pull-through cache of a Harbor registry project on its |prod|
system controller. This can be achieved using a proxy project which acts as a
proxy to the remote registry, or using a replication of the remote registry.
.. rubric:: Add a proxy project
Proxy cache allows you to use Harbor to proxy and cache images from a target
public or private registry. It does not allow to push the image by the user.
Below are the steps to create a proxy project:
#. Create a registry endpoint. See `Creating Replication Endpoints
<https://goharbor.io/docs/2.8.0/administration/configuring-replication/create-replication-endpoints/>`__.
#. Create a new project using the above created registry endpoint. See
`Configure Proxy Cache
<https://goharbor.io/docs/2.8.0/administration/configure-proxy-cache/>`__.
.. rubric:: Add Replication Rule
Setting up pull based replication replicates the images from the remote
registry based on a trigger. Trigger can be Manual, scheduled, or event based
as shown below.
.. image:: figures/new-replication-rule.png
#. Create a replication endpoint. See `Creating Replication Endpoints
<https://goharbor.io/docs/2.8.0/administration/configuring-replication/create-replication-endpoints/>`__.
#. Create a new replication rule with replication mode as ``pull based`` as
describe in `Creating a Replication Rule
<https://goharbor.io/docs/2.8.0/administration/configuring-replication/create-replication-rules/>`__.
------------------------
Add Users to the Project
------------------------
You can add individual users to a project, for more details see `Assign Users
to a Project
<https://goharbor.io/docs/2.8.0/working-with-projects/create-projects/add-users/>`__.
- |LDAP|/AD users can be added to the project.
- |LDAP|/AD groups to projects and assign a role to the group.
- You can see the various roles and user permission associated with the roles
in `User Permissions By Role
<https://goharbor.io/docs/2.8.0/administration/managing-users/user-permissions-by-role/>`__.
-------------------------------
Configure Signed Images Support
-------------------------------
For more details see: `Implementing Content Trust
<https://goharbor.io/docs/2.8.0/working-with-projects/project-configuration/implementing-content-trust/>`__.
#. Select cosign or notary for content trust. Harbor will then
only allow verified images to be pulled from the project.
.. image:: figures/library-harbor.png
#. Enable content trust by setting the following environment variables on the
machine on which you run the Docker client.
.. code-block:: none
export DOCKER_CONTENT_TRUST=1
export DOCKER_CONTENT_TRUST_SERVER=https://<harbor_address>:<notary port[30004]>
If you push the image for the first time, you will be asked to enter the
root key passphrase.
------------------------------
Configure Trivy Scanner Plugin
------------------------------
Trivy is installed and configured as a default scanner.
.. image:: figures/interrogation-services.png
:width: 800
-----------------------------
Configure Size of Registry DB
-----------------------------
Registry DB size can be configured by setting following in ``values.yaml``
under:
.. code-block:: none
persistence:
registry:
size: 5Gi
jobservice:
jobLog:
size: 1Gi
Use :command:`system helm-override` command to set the value (Default set to
5Gi).
------------------------------------------------------
Enforcement of Image Security Policies Using Portieris
------------------------------------------------------
Portieris allows to configure trust policies for an individual namespace or
cluster-wide and checks the image against a signed image list on a specified
notary server to enforce the configured image policies. An Administrator can
enforce |prod| image security policies using the Portieris admission
controller.
It is required to pull from a registry using a ``docker-registry`` secret.
Enforcing trust for anonymous image pulls is not supported.
To use portieris, an administrator needs to follow below steps:
#. Install portieris as specified in :ref:`install-portieris`.
#. Create a ``docker-registry`` secret.
.. code-block:: none
kubectl create secret docker-registry \
-n harbor harbor-registry-secret \
--docker-server=<harbor-dns>:port \
--docker-username=admin \
--docker-password=Test@123
.. note::
If the pod creation with the above secret fails, the user should try
with new secret with ``--docker-server`` as ``<harbor-dns>``.
#. Configure image policy to allow images from Harbor registry + notary as
specified
:ref:`portieris-clusterimagepolicy-and-imagepolicy-configuration`. Below
example shows the policy allowing image from Harbor registry.
.. code-block:: none
apiVersion: portieris.cloud.ibm.com/v1
kind: ImagePolicy
metadata:
name: allow-custom
.. code-block:: none
namespace: harbor
spec:
repositories:
- name: "<harbor-dns>:30003/*"
policy:
trust:
enabled: true
trustServer: "https://<notary dns>:30004" # Optional, custom trust server for repository
#. Pull a signed image from Harbor registry in a pod using ``harbor-secret``
created above. Please note that image policy and pod should be created in
the same namespace.
.. code-block:: none
apiVersion: v1
kind: Pod
metadata:
name: test-pod-public
spec:
containers:
- command:
- sleep
- '3600'
image: <harbor-dns>:30003/public-demo/redis:latest
imagePullPolicy: Always
name: test-pod
tolerations:
- key: "node-role.kubernetes.io/master"
operator: "Exists"
effect: "NoSchedule"
imagePullSecrets:
- name: harbor-registry-secret
----------
Limitation
----------
Harbor application cannot be deployed during bootstrap due to the bootstrap
deployment dependencies such as early availability of storage class.

9
doc/source/admintasks/kubernetes/index-admintasks-kub-ebc55fefc368.rst
View File

@ -88,6 +88,15 @@ O-RAN O2 Interface
oran-o2-application-b50a0c899e66 oran-o2-application-b50a0c899e66
--------------------
Harbor as System App
--------------------
.. toctree::
:maxdepth: 1
harbor-as-system-app-1d1e3ec59823
-------------------------------------- --------------------------------------
Technology Preview - Istio Application Technology Preview - Istio Application
-------------------------------------- --------------------------------------

5
doc/source/shared/abbrevs.txt
View File

@ -27,6 +27,8 @@
.. |CDI| replace:: :abbr:`CDI (Containerized Data Importer)` .. |CDI| replace:: :abbr:`CDI (Containerized Data Importer)`
.. |CLI| replace:: :abbr:`CLI (Command Line Interface)` .. |CLI| replace:: :abbr:`CLI (Command Line Interface)`
.. |CLIs| replace:: :abbr:`CLIs (Command Line Interfaces)` .. |CLIs| replace:: :abbr:`CLIs (Command Line Interfaces)`
.. |CNAB| replace:: :abbr:`CNAB (Cloud Native Application Bundle)`
.. |CNABs| replace:: :abbr:`CNABs (Cloud Native Application Bundles)`
.. |CNI| replace:: :abbr:`CNI (Container Networking Interface)` .. |CNI| replace:: :abbr:`CNI (Container Networking Interface)`
.. |CNIs| replace:: :abbr:`CNIs (Container Networking Interfaces)` .. |CNIs| replace:: :abbr:`CNIs (Container Networking Interfaces)`
.. |CoW| replace:: :abbr:`CoW (Copy on Write)` .. |CoW| replace:: :abbr:`CoW (Copy on Write)`
@ -121,9 +123,12 @@
.. |OAM| replace:: :abbr:`OAM (Operations, administration and management)` .. |OAM| replace:: :abbr:`OAM (Operations, administration and management)`
.. |OEM| replace:: :abbr:`OEM (Original Equipment Manufacturer)` .. |OEM| replace:: :abbr:`OEM (Original Equipment Manufacturer)`
.. |OC| replace:: :abbr:`OC (Ordinary Clock)` .. |OC| replace:: :abbr:`OC (Ordinary Clock)`
.. |OCI| replace:: :abbr:`OCI (Open Container Initiative)`
.. |OID| replace:: :abbr:`OID (Object Identifier)` .. |OID| replace:: :abbr:`OID (Object Identifier)`
.. |OIDC| replace:: :abbr:`OIDC (OpenID Connect)` .. |OIDC| replace:: :abbr:`OIDC (OpenID Connect)`
.. |ONAP| replace:: :abbr:`ONAP (Open Network Automation Program)` .. |ONAP| replace:: :abbr:`ONAP (Open Network Automation Program)`
.. |OPA| replace:: :abbr:`OPA (Open Policy Agent)`
.. |OPAs| replace:: :abbr:`OPAs (Open Policy Agents)`
.. |OVS| replace:: :abbr:`OVS (Open Virtual Switch)` .. |OVS| replace:: :abbr:`OVS (Open Virtual Switch)`
.. |OSD| replace:: :abbr:`OSD (Object Storage Daemons)` .. |OSD| replace:: :abbr:`OSD (Object Storage Daemons)`
.. |OSDs| replace:: :abbr:`OSDs (Object Storage Daemons)` .. |OSDs| replace:: :abbr:`OSDs (Object Storage Daemons)`