root/build-tools/build-docker-images
Davlet Panech 775ad108af docker-images: better registry error handling
This commit enables better error detection when checking whether an
image/tag exists in a remote registry. Current implementation sometimes
falsely believes a remote tag is missing and attempts to (re-)push the
images, potentially overwriting them.

Examples:
- Registry is not reachable due to a temporary network outage
- With docker.io: we exceed the request rate limit. Original script
  looked for remote tags by enumerating all tags. This resulted in
  dozens of REST calls per image, occasionally exceeding Dockerhub's
  request limit.

Solution: add new script that exits on connectivity errors, rather than
returning false. Script requires an external tool, regctl:
  https://github.com/regclient/regclient

TESTS
====================================
- Test with missing/existing images in Harbor, DockerHub and
  AWS ECR registries, as well as various connectivity errors.
- Run retag-images.sh and make sure it still works

Closes-Bug: 2003898
Change-Id: Id9dd0c30580748c0c4c4bfbbd520d4d38bdd2ec6
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
2023-05-05 17:03:30 -04:00
..
2022-09-14 10:56:06 -04:00
2019-03-24 21:03:21 -04:00

## Example commands for building StarlingX images

PRIVATE_REGISTRY_USERID=myuser
PRIVATE_REGISTRY=xxx.xxx.xxx.xxx:9001
VERSION=2018.11.13
OS=centos
OS_VERSION=7.5.1804
BUILD_STREAM=stable
HOST_PORT=8088

## Step 1: Build stx-centos
time $MY_REPO/build-tools/build-docker-images/build-stx-base.sh \
    --os ${OS} \
    --os-version ${OS_VERSION} \
    --version ${VERSION} \
    --user ${PRIVATE_REGISTRY_USERID} \
    --registry ${PRIVATE_REGISTRY} \
    --push \
    --repo stx-local-build,http://${HOSTNAME}:${HOST_PORT}/${MY_WORKSPACE}/std/rpmbuild/RPMS \
    --repo stx-mirror-distro,http://${HOSTNAME}:${HOST_PORT}/${MY_REPO}/cgcs-root/cgcs-${OS}-repo/Binary \
    --clean


## Step 2: Build wheels (output as tarball)
time $MY_REPO/build-tools/build-wheels/build-wheel-tarball.sh \
    --os ${OS} \
    --os-version ${OS_VERSION} \
    --stream ${BUILD_STREAM}

## Step 3: Build images
time $MY_REPO/build-tools/build-docker-images/build-stx-images.sh \
    --os ${OS} \
    --version ${VERSION} \
    --stream ${BUILD_STREAM} \
    --base ${PRIVATE_REGISTRY}/${PRIVATE_REGISTRY_USERID}/stx-${OS}:${VERSION} \
    --wheels http://${HOSTNAME}:${HOST_PORT}/${MY_WORKSPACE}/std/build-wheels-${OS}-${BUILD_STREAM}/stx-${OS}-${BUILD_STREAM}-wheels.tar \
    --user ${PRIVATE_REGISTRY_USERID} \
    --registry ${PRIVATE_REGISTRY} \
    --push --latest \
    --clean


## Note: Verify that lighttpd is not bound to "localhost"
vi /etc/lighttpd/lighttpd.conf
# server.bind = "localhost"
systemctl restart lighttpd

## Note: You may need to add an iptables rule to allow the docker
## containers to access the http server on your host. For example:
iptables -I INPUT 6 -i docker0 -p tcp --dport ${HOST_PORT} -m state --state NEW,ESTABLISHED -j ACCEPT