This patch is a follow up for patch:
I3573c067f63a09f22377612f53c80c80252965e4
Comment explains the string matching lines for the pip freeze output.
Change-Id: Ifc5b2ae002859c1ca7405c8e1f64380b311e983d
This configuration is intended to mask the creds supplied by the
local docker configuration when the registry is used as a
pull-through proxy to upstream. But we need to set the creds
to the empty string, rather than null, which ends up as the string
"None" in the docker config.
Change-Id: Ic36a5c764dec8df4fa460f6735ff5668a1dd3b50
When copying images from the intermediate registry to the buildset
registry, use the new push endpoint of the dual-registry system.
Also, use the push endpoint after a docker build to push the
new image to the buildset registry.
Change-Id: I3a11036bb9fb7cb3457a3d744fa83647c1b1b085
So that the docker mirror configuration can be used without the
install-docker role (e.g., if another process is used to install
docker on a target host), split the part of the role which
configures mirrors into its own role and include it in the
install-docker role.
Change-Id: I7cd7f61e2d1281cd2949c1f69a081c7217d0d328
To accomodate running in a production-simulation environment,
make it safe to run this role on a host before docker is installed.
This also adds support for the new dual-registry configuration
that run-buildset-registry uses.
This removes the region-local proxy from the registry-mirrors
configuration. Because the buildset registry acts as a pull-through
proxy, the region-local proxy won't be used even if we did include it.
Instead, we should update the run-buildset-registry role to proxy
to the region-local proxy if present.
Change-Id: I21011a3708f17ee61afd0034d90d75e8dc885575
The docker registry daemon can either act as a private registry,
or as a pull-through proxy, but not both. Yet we need to be able
to serve private (speculative buildset) images as well as plain
upstream images. Our registry is used as a mirror and requires
authentication, therefore docker's normal behavior of falling back
on docker.io won't work because it will attempt to use our
credentials.
However, the registry daemon stores all of its state in the
filesystem, therefore we can run two instances of the registry
service, both pointing at the same data store. The first acts
as a pull-through proxy and will serve whatever files are already
in the local storage, or will fetch them from docker.io. The second
can be used to upload images into the local storage.
To make a long story short, whenever we push into the buildset
registry, we will use the second endpoint. Whenever the docker
daemon pulls from the buildset registry, it will use the first.
Change-Id: I296029068b5ef28ee56543741fe8c8deeefb5dfa
Docker has a curious behavior with respect to the registry-mirror
setting: it will only use the credentials for dockerhub itself.
Therefore, configure the buildset registry to be a pull-through
cache for dockerhub, but with no credentials, so that we can set
buildset registry credentials under the dockerhub url without
their actually being used for dockerhub.
Change-Id: I5daf93868d408d71cf2766fbe1bb17f2cfa8a3c7
So that this role may be used unconditionally in jobs which may
or may not actually build images, skip the tasks if the
docker_images variable is undefined.
Change-Id: I6ef0c80230de628f86f523878020c82ce81a1e60
This relies on the list merge behavior in https://review.openstack.org/638005
however, this will work with the current code in Zuul as long as only
one artifact is returned, so a Depends-On is not necessary.
Change-Id: Ie5d3a61c8cc1038f3775a3aa81e94b9b909f265a
It seems like new minikube changed the default minikube config
permission. This change ensures the minikube config is owned by
the ansible_user
Change-Id: Ic1af1541fb1f32253678f989d7b62d21074514d5
The intermediate registry pull role is designed to be used in
the same playbook as the run-buildset-registry role, which sets
the buildset_registry fact. However, that fact is set on the
host where the registry runs, not localhost. Theoretically we
should be able to delegate setting that fact to all hosts in
the inventory, plus localhost, however, that doesn't seem to
work in local testing.
Work around this by, once again, loading the buildset_registry
fact from the zuul_return file.
Change-Id: Ia16b3af8782c875e64ad5eeeeb5f107482a3e30a
* In the build-image role, push to the buildset registry if it is defined.
* In the intermediate registry push and pull roles, ensure that the
buildset registry TLS cert is in place. This is a self-signed cert,
and so needs to be written for each run. This happens inside
bubblewrap where we have permission to write to /etc, which is an
ephemeral volume.
Change-Id: I47781d8a7adb93817dfe9266e2f4ad5fd829385c
There is no policy file available, and we aren't concerned with
verifying image signatures at this point. Add this option to
tell skopeo to proceed regardless.
Change-Id: I15a4978ec0fb338bc05c974b0ec6a21f680c853e
The push-to-intermediate-registry role is meant to be used in the
post playbook of a job which most likely started the buildset
registry in its pre-playbook. The buildset_registry variable
is returned to zuul and therefore available to child jobs, but
not to further playbooks in this job. Therefore, load the variable
back in to memory here.
We may add this functionality directly to Zuul, in which case this
can be removed later.
Change-Id: I81899a9be92789fc1e5858f985eac41871b9d87a
The attribute zuul.artifacts is only present if there are artifacts.
Use the empty list as default.
The default for image.tags should be 'latest' to match the rest
of the docker roles.
Change-Id: Iff6863043e3a0311cb1c8c2ef4cd3d37ff79cce5
Renames the role to upload-forge because you can actually
run your own Forge server if you want.
This patch adds a custom module to the upload-forge
role that provides the "forge_upload" module.
This directly interacts with the a Forge API to upload the
module. The only dependency is that the python requests
module is installed.
Change-Id: I5749364bd2c29ad6df866c2bd5a3584c8419f709
If the stage_dir/logs directory was a symlink, Ansible would refuse
to perform the chmod. However, if we tell Ansible that it is a
directory, it's fine. This (perhaps curiously) does not tell
Ansible to turn the symlink into a directory. It just tells it
that everything is okay.
Change-Id: Idbd78b97b95ce50b23dda8f384e32359d31013e3
Allow custom docker_group which is needed on CentOS when
use_upstream_docker: false as is value is 'dockeroot'.
Assures that docker service is started, which is not always true
with all distributions.
Validates that we can talk with docker at the end.
Change-Id: I993c4ac35055a2a18b5b0fb32c8b230e04b7eb87
So that this can more easily work with files like '/var/log/syslog',
make it so that if the user specifies "logs_txt" then the file gets
a .txt extension before being placed in the logs directory.
The existing extensions_to_txt variable doesn't work for this because
it requires an existing extension.
Change-Id: I34fea0c44030c04a5540d6b62976557143289196
This release of requests-toolbelt expects pyOpenSSL to be installed for
urllib3, but does not have it in its default requirements. This is being
tracked in [0], so should be fixed post 0.9.0. For now, block the use of
the 0.9.0 version.
[0] https://github.com/requests/toolbelt/compare/0.8.0...0.9.0
Change-Id: I06445adaaeecf244f6482b6db26db58a09426489
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
When these were copied in from system-config, the paths to the
upload and promote playbooks weren't updated.
Change-Id: I9da5d7dfb980b2b7690652ee186723ee9ab64cd5
In certain build projects, multiple Dockerfiles exist (for example,
one per distro) to simplify reading. However, this role is hardwired
to use dockerfiles only named "Dockerfile".
This is a problem, as you can't override the filename neither per
image, or globally.
This should fix the problem, allowing certain images to be build
by providing the dockerfile argument in docker_images, but also
have a globally overridable flag if you are using a different
convention (for example Dockerfile.distro_minordistroversion).
Change-Id: I075c365bc9f4f85f9ada832d22d1f1e213e68e21
In pip 19.0 release [1] the
Editable Git installs without a remote now freeze as editable. (#4759) [2]
bug is fixed. Now the output of pip freeze for such package looks like:
# Editable Git install with no remote (neutron==10.0.8.dev66)
-e /home/zuul/src/git.openstack.org/openstack/neutron
instead of:
## !! Could not determine repository location
neutron==10.0.8.dev66
Since for listing the package names the module uses the lines that contains
'==' the new behaviour leaves some garbage in front of the package name. So
in case the string contains '(' then remove the first part of the string.
[1] https://pip.pypa.io/en/stable/news/#id15
[2] https://github.com/pypa/pip/issues/4759
Change-Id: I3573c067f63a09f22377612f53c80c80252965e4
When doing the local build, go ahead and apply the tags to the
local image, so that one can use the role for building local
images for testing that will eventually be published with the given
tag.
Change-Id: I0249ddc4f9a8a2e17466f96a5711672282ce025c
The static zuul driver returns only a public_ipv4 address for
nodes in use. The multinode bridge role accesses the node
private_ipv4 address. As such, when the private address is
empty/null the role fails.
This review defaults that private address so that the role
uses the public address when the private address is null.
Change-Id: I32fb17bae98f13f735da4d5b9a6a01e948f21678
