Elisamara Aoki Goncalves 029d1af2d8 Doc Update for Harbor Support

Story: 2010721
Task: 50133

Change-Id: Id5adb9966bf8c081629116b46352a155a0e4dfd5
Signed-off-by: Elisamara Aoki Goncalves <elisamaraaoki.goncalves@windriver.com>

2024-07-24 14:24:14 +00:00

19 KiB

Raw Blame History

Harbor as System Application

Harbor is an open-source registry that secures artifacts with policies and role-based access control, ensures images are scanned and free from vulnerabilities, and signs images as trusted. Harbor has been evolved to a complete compliant cloud-native artifact registry.

With Harbor V2.0, users can manage images, manifest lists, Helm charts, , among others which all adhere to the image specification. It also allows for pulling, pushing, deleting, tagging, replicating, and scanning such kinds of artifacts. Signing images and manifest list are also possible now.

Harbor supports replication of images between registries, and offers advanced security features such as user management, access control and activity auditing.

See https://goharbor.io/docs/2.0.0/ for more details on Harbor.

Harbor Installation

CephFS backed are recommended for Harbor such that when configuring multiple replicas, for or Standard configurations, both Harbor replicas can read and write to the registry.

Create a secret as described below:

Generate certificates and create secret.

A cert and server cert creation procedure using cert-manager is specified below:

Create the certificate for Harbor using Cert-Manager and using the local , system-local-ca, as the issuer. Note that the certificate should be created in the harbor-tls SECRET in the Harbor NAMESPACE.

For example:

Create the following Harbor certificate yaml configuration file:

~(keystone_admin)]$ cat <<EOF > harbor-certificate.yaml
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: harbor-certificate
namespace: harbor
spec:
secretName: harbor-tls
issuerRef:
    name: system-local-ca
    kind: ClusterIssuer
duration: 2160h # 90 days
renewBefore: 360h # 15 days
commonName: < oam floating IP Address or FQDN >
subject:
    organizations:
        - ABC-Company
organizationalUnits:
        - StarlingX-harbor
ipAddresses:
        - < oam floating IP address >
dnsNames:
        - < harbor dns> # e.g. harbor.yourdomian.com
        - < notary dns > # optional, required only if exposed on ingress e.g. notary.yourdomian.com
EOF

Apply the configuration:

~(keystone_admin)]$ kubectl apply -f harbor-certificate.yaml

Verify the configuration:
```
~(keystone_admin)]$ kubectl get certificate harbor-certificate -n harbor
```
After successful configuration, the certificate's Ready status will be True.

nodePort

Create Harbor using NodePort to expose the service

Note

The instructions below assume that the NodePorts 30102, 30103 and 30104 are available; i.e. not used by any other applications.
Locate the Harbor system application tarball in /usr/local/share/applications/helm.

For example:
```
/usr/local/share/applications/helm/harbor-<version>.tgz
```

Upload the Harbor application.

~(keystone_admin)]$ system application-upload /usr/local/share/applications/helm/harbor-<version>.tgz

Configure the Helm Overrides for Harbor.

Below values need to be configured for nodePort:

expose:

     type: nodePort                             # Type should be nodeport
     tls:
        enabled: true
        certSource: secret
        secret:                                 # Certificate Source is secret
            secretName: "harbor-tls"            # A secret containing tls.crt and tls.key
            notarySecretName: "harbor-tls"      #  A secret containing tls.crt and tls.key

   nodePort:
    # The name of NodePort service
    name: harbor
    ports:
      http:
        # The service port Harbor listens on when serving HTTP
        port: 80
        # The node port Harbor listens on when serving HTTP
        nodePort: 30002
      https:
        # The service port Harbor listens on when serving HTTPS
        port: 443
        # The node port Harbor listens on when serving HTTPS
        nodePort: 30003
      # Only needed when notary.enabled is set to true
      notary:
        # The service port Notary listens on
        port: 4443
        # The node port Notary listens on
        nodePort: 30004


 externalURL: https://harbor.yourdomian.com:30003     # URL of harbor listing on 30003 port

For and standard setup, add below storageClass and accessModes override.

Underlying PVCs pre-requisistes: Harbor-Jobservice and Harbor-Registry microservice.

For example:

persistence:
  enabled: true
  resourcePolicy: "keep"
  persistentVolumeClaim:
    registry:
      existingClaim: ""
      storageClass: "cephfs"
      subPath: ""
      accessMode: ReadWriteMany
      size: 5Gi
      annotations: {}
    jobservice:
      jobLog:
        existingClaim: ""
        storageClass: "cephfs"
        subPath: ""
        accessMode: ReadWriteMany
        size: 1Gi
        annotations: {}

Execute Helm overrides.

~(keystone_admin)]$ system helm-override-update harbor harbor harbor  --values values.yaml

Apply/Create the Harbor system application.

~(keystone_admin)]$ system application-apply harbor

Ingress

Create Harbor using Ingress to expose the service.

Note

The instructions below assume that the URL harbor.yourdomain.com has been configured in the server owning yourdomain.com as the OAM FLOATING IP Address of .

Locate the Harbor system application tarball in /usr/local/share/applications/helm.

For example:
```
/usr/local/share/applications/helm/harbor-<version>.tgz
```

Upload the Harbor application.

~(keystone_admin)]$ system application-upload /usr/local/share/applications/helm/harbor-<version>.tgz

Configure the Helm overrides for Harbor configuration.

The values below need to be configured for ingress in the values.yaml file.

expose:
      type: ingress.                          # Type should be ingress
      tls:
      enabled: true
          certSource: secret
      secret:                                 # Certificate Source is secret
          secretName: "harbor-tls"            # Above created secret name
          notarySecretName: "harbor-tls"      # Above created secret name
      ingress:
        hosts:
          core: harbor.yourdomian.com         # Harbor Domain name
          notary: notary.yourdomian.com       # Notary Domain name
      annotations:
          kubernetes.io/ingress.class: nginx.     # Add ingressclass name. It would be                                # "nginx" if you are using default ingress                          # controller.
          nginx.org/client-max-body-size: "0".    # Add this notation for nginx otherwise nginx                       # will reject the image pull & push
  externalURL: https://harbor.yourdomian.com      # URL of harbor

For and standard setup, add below storageClass and accessModes override for used for Harbor-Jobservice and Harbor-Registry microservice.

For example:

persistence:
  enabled: true
  resourcePolicy: "keep"
  persistentVolumeClaim:
    registry:
      existingClaim: ""
      storageClass: "cephfs"
      subPath: ""
      accessMode: ReadWriteMany
      size: 5Gi
      annotations: {}
    jobservice:
      jobLog:
        existingClaim: ""
        storageClass: "cephfs"
        subPath: ""
        accessMode: ReadWriteMany
        size: 1Gi
        annotations: {}

Update the Helm overrides.

~(keystone_admin)]$ system helm-override-update harbor harbor harbor  --values values.yaml

Apply/Create the Harbor system application.

~(keystone_admin)]$ system application-apply harbor

Configure LDAP Authentication for Harbor Registry

To configure Harbor to use Local for authentication, follow the instructions in Configure LDAP/Active Directory Authentication with the following values:

For local :

LDP URL: ldap://controller

LDAP search DN: cn=ldapadmin,dc=cgcs,dc=local

LDAP Search Password: <Password of ldapadmin>

LDAP Base DN: dc=cgcs,dc=local

LDAP UID:  cn

Push an Image to a <project> in Harbor

Run sudo su before Docker login.
Docker Login.
```
docker login <harbor_address> -u <username>
```
Note

Replace <harbor_address> with actual harborURL and replace <username> with your actual username.

Tag the image.

docker tag redis:latest <harbor_address>/<project>/redis:latest

Push the image.

docker push <harbor_address>/<project>/redis:latest

Pull an Image from Harbor

Use command to pull an image:

docker pull <harbor_address>/<project>/ redis:latest

Where <harbor-address> is either:

for 'Ingress' expose: harbor.yourdomian.com
for 'NodePort' expose: https:// <oam-floating-ip>:30103

Push a Helm Chart as an OCI Object

helm package <Chart_Path>

helm push <chart_package> oci://<harbor_address>/<project>/

Pull a Helm Chart

helm pull oci://<chart_package/<project>/harbor

Project Management

A project in Harbor contains all the repositories (images) of a particular application; e.g. the project in Harbor would contain all the images from .

For more details on creating and configuring a project see: Create Projects and Project Configuration.

Use Images from a Remote Registry

A project in a local Harbor registry can be used as a pull-through cache for a remote registry. For example, a Harbor registry project on a Subcloud can be used as a pull-through cache of a Harbor registry project on its system controller. This can be achieved using a proxy project which acts as a proxy to the remote registry, or using a replication of the remote registry.

Add a proxy project

Proxy cache allows you to use Harbor to proxy and cache images from a target public or private registry. It does not allow to push the image by the user. Below are the steps to create a proxy project:

Create a registry endpoint. See Creating Replication Endpoints.
Create a new project using the above created registry endpoint. See Configure Proxy Cache.

Add Replication Rule

Setting up pull based replication replicates the images from the remote registry based on a trigger. Trigger can be Manual, scheduled, or event based as shown below.

Create a replication endpoint. See Creating Replication Endpoints.
Create a new replication rule with replication mode as pull based as describe in Creating a Replication Rule.

Add Users to the Project

You can add individual users to a project, for more details see Assign Users to a Project.

/AD users can be added to the project.
/AD groups to projects and assign a role to the group.
You can see the various roles and user permission associated with the roles in User Permissions By Role.

Configure Signed Images Support

For more details see: Implementing Content Trust.

Select cosign or notary for content trust. Harbor will then only allow verified images to be pulled from the project.
Enable content trust by setting the following environment variables on the machine on which you run the Docker client.
```
export DOCKER_CONTENT_TRUST=1

export DOCKER_CONTENT_TRUST_SERVER=https://<harbor_address>:<notary port[30004]>
```
If you push the image for the first time, you will be asked to enter the root key passphrase.

Configure Trivy Scanner Plugin

Trivy is installed and configured as a default scanner.

Configure Size of Registry DB

Registry DB size can be configured by setting following in values.yaml under:

persistence:
    registry:
      size: 5Gi
    jobservice:
      jobLog:
        size: 1Gi

Use system helm-override command to set the value (Default set to 5Gi).

Enforcement of Image Security Policies Using Portieris

Portieris allows to configure trust policies for an individual namespace or cluster-wide and checks the image against a signed image list on a specified notary server to enforce the configured image policies. An Administrator can enforce image security policies using the Portieris admission controller.

It is required to pull from a registry using a docker-registry secret. Enforcing trust for anonymous image pulls is not supported.

To use portieris, an administrator needs to follow below steps:

Install portieris as specified in install-portieris.

Create a docker-registry secret.

kubectl create secret docker-registry \
-n harbor harbor-registry-secret \
--docker-server=<harbor-dns>:port \
--docker-username=admin \
--docker-password=Test@123

Note

If the pod creation with the above secret fails, the user should try with new secret with --docker-server as <harbor-dns>.

Configure image policy to allow images from Harbor registry + notary as specified portieris-clusterimagepolicy-and-imagepolicy-configuration. Below example shows the policy allowing image from Harbor registry.

apiVersion: portieris.cloud.ibm.com/v1
kind: ImagePolicy
metadata:
 name: allow-custom

namespace: harbor
spec:
repositories:
  - name: "<harbor-dns>:30003/*"
  policy:
      trust:
      enabled: true
      trustServer: "https://<notary dns>:30004" # Optional, custom trust server for repository

Pull a signed image from Harbor registry in a pod using harbor-secret created above. Please note that image policy and pod should be created in the same namespace.

apiVersion: v1
kind: Pod
metadata:
name: test-pod-public
spec:
containers:
- command:
    - sleep
    - '3600'
    image: <harbor-dns>:30003/public-demo/redis:latest
    imagePullPolicy: Always
    name: test-pod
tolerations:
- key: "node-role.kubernetes.io/master"
    operator: "Exists"
    effect: "NoSchedule"
imagePullSecrets:
- name: harbor-registry-secret

Limitation

Harbor application cannot be deployed during bootstrap due to the bootstrap deployment dependencies such as early availability of storage class.

19 KiB Raw Blame History

Harbor as System Application

Harbor Installation

Configure LDAP Authentication for Harbor Registry

Push an Image to a <project> in Harbor

Pull an Image from Harbor

Push a Helm Chart as an OCI Object

Pull a Helm Chart

Project Management

Use Images from a Remote Registry

Add Users to the Project

Configure Signed Images Support

Configure Trivy Scanner Plugin

Configure Size of Registry DB

Enforcement of Image Security Policies Using Portieris

Limitation

19 KiB

Raw Blame History