This fixes a byte vs str problem on python3 in the tox role's sibling
library.
As part of this fix add unit testing of the function that was failing
under python3 and functional testing of the role that covers siblings.
Change-Id: I2df4ddce91dfd53f5c4177a19288e97120bc3e3b
There are a number of issues with this. Firstly, it needs to copy the
parent directories to make a heirarchy in the .zuul-siblings
directory. The current "cp -r" was only copying the final directory.
Switch into the source directory and use "--parent" to do this.
Also, it should be copying into the context dir. Add the
{{ item.context }} to the path where appropriate.
Make new testing image that copies in files from the siblings.
Because COPY will fail if the sources aren't there, this is like an
assert that we copied it correctly.
Change-Id: I9f3b0a1f71d20cf7511f224648dd2fa51a039015
Add support for "siblings:" to the build-container-image role. This
is a port of change I079d823e7194e15b1b496aea0f53f70f6b563f02 from the
build-docker-image role.
Note that we found a couple of issues with this role, fixed with
I9f3b0a1f71d20cf7511f224648dd2fa51a039015. However,
build-container-image has merged before this fix, so thess fixes now
follow this change. To avoid squashing this into something confusing
this update incorporates those fixes. However testing is left to the
follow-on change because the extant build-docker-image role would fail
the corrected testing. They should be merged roughly together.
One minor thing is the removal of an additional duplicated "tags:"
entry from the variable documentation.
Change-Id: I71e2f2ab29446bd113644ce8c75378b8d3ef8ba6
In order to edit the V2 registries.conf file used by podman, we
need to be able to manipulate toml from ansible. There is no
standard library or Ansible support for that now, and we don't want
to install any python packages on the remote node. Therefore,
vendor the remarshal and pytoml code into this role.
This is done in a standalone commit for easier review and auditing.
The originating projects are:
https://github.com/dbohdan/remarshalhttps://github.com/avakar/pytoml
And both are MIT licensed. Appropriate headers are added where
necessary.
Note that pytoml has been concatenated into one file in order to
adhere to Ansible's requirements for python modules.
Change-Id: I679ea5eb5cb29591be09d2f1b712400c49158abd
When you build from a Dockerfile, it runs in a given "context"; that
is the directory the Dockerfile is in and the directories below it.
It can not access anything outside that context during the build.
When building a container for a project in the gate, you may wish to
install sibling projects that Zuul has checked-out into your container
(i.e. so that Depends-On works). As mentioned, because
/home/zuul/src/<project> is not in the context of the current project,
you will not be able to access this source code during the container
build.
So to help facilitate dependencies, add a siblings: tag which can copy
some or all of the required-projects already specified for the job
into a special sub-directory of the current source.
Because all the code is now in the same context, this will allow build
scripts to be written that look for directories in .zuul-siblings and
can install the source code from there. To further help the scripts,
the ZUUL_SIBLINGS arg is set for the docker build giving the copied
paths.
The test is updated with some paths to test the copy.
Change-Id: I079d823e7194e15b1b496aea0f53f70f6b563f02
This is a bit of a hack, but allows installation of openvswitch for
testing the multinode roles. Official CentOS 8 RDO builds are still a
work in progress, but this will let us get a head-start on any
multi-node testing.
Change-Id: I2eb1a16e9995a19d61e309aa59b232577184b527
We install pydistutils.cfg to catch legacy cases where people
are doing things with easy_install. That code path is not
desirable and increasingly broken, as support for things like
allow-hosts is dropped upstream.
Stop installing the file. This will mean people using easy_install
won't get the benefit of the mirrors, but they shouldn't be
using easy_install anyway.
Remove pydistutils.cfg if it's there
Make sure we don't have any pydistutils.cfg around.
Change-Id: I24a05f456f87bd4cb57ebf89d4245477bf675f97
This is required in order to perform port forwarding when running
rootless.
Again, unclear why this isn't required in the package.
Change-Id: I61964a23ce7c80b1af149cb7f98897367e970d45
This package is required in order for UID mapping in rootless
operation to work. It is unclear why it isn't specified as a
dependency.
Change-Id: I7dd22b683c2fed9921de39a474de3bd9f2c3b0d4
Nova has switched to Python 3 only but devstack has not caught up (see
[1]). Switch on Python 3 here; it will soon be the default and we can
shake out any errors before that.
[1] https://review.opendev.org/694891
Change-Id: Idf26a7af08219421c1a6cc381317e59fb6950b86
Show full version details of install. This can be useful when you're
trying to decide what features you may or may not have.
Change-Id: Iedc6aa1b73b0e7d7612c27d890273e84d4d04a22
The "Start the buildset registry" task, which uses the
docker_container Ansible module, was failing with:
"No module named 'requests'"
This is because the earlier package install task was installing
the packages into the wrong python version because the
ansible_python_interpreter value was 'auto' and thus never
included the 'python3' value.
Change-Id: I887fdf8d000c8b916fdab281c531b7c98bdd5ae2
This implements the production change
I98c80f657f38c5e1ed5f28e5d36988a3429ad1f8 in the test role. Review
comments should be left there; we can merge this and then parent a job
to base-test to test it.
Change-Id: Id91350ff1c531fd7266f3bf76681a8415941481f
By doing this, we're not constrained about where to run the uploader
while still providing some useful testing in dry-run mode.
Change-Id: Ie4888606a8ca4ffe2eb99ddbbcd9d5cee8ceec44
We don't need to be explicit, ssh-keygen will pick a good size for us.
Meanwhile, 1024 bit keys are generally seen as security problems and
thus ring bells and are refused by some servers/services.
Change-Id: Iaea82e0b394a5a6b1da3b59637fc4e0f541e1978
If we want to run a second registry on a different port, we'll need
a different name for the docker container.
Change-Id: I887d9015c0d21e6d7f95379e6fa7fb4211e58d3d
This will allow us to run the role twice on the same host. This
will be useful when we test changes to this role.
Change-Id: I97baeb3172298648bcfef26c5be635ad4be036f0
The proxy functionality is no longer needed so it is removed.
Change-Id: I29ff75d331b433ea4ad3b66ed723eee14a90b404
Depends-On: https://review.opendev.org/689829
Change new variable zuul_additional_subunit_dirs to
fetch_subunit_output_additional_dirs to name it after the role.
Change-Id: I7ac0b5d343701409a8741439334fe27f5374a258
Per settings mentioned at: https://minikube.sigs.k8s.io/docs/tasks/debug/
The current logging level is very minimal for tracking down any
potential k8s issues.
Change-Id: I4ebd694481de936f9df790a46e195b251a803c5f
In addition to the main subunit file from zuul_work_dir,
collect the subunit files from the elements (directories)
of the zuul_additional_subunit_dirs list.
The default behavior is unchanged.
While the documentation of this role states that zuul_work_dir
contains an absolute path, this is not always true.
So make sure to not make any assumption about zuul_work_dir
in order to not fail spectacularly as it happened with the
previous patch[0].
Add also some tests for the role: both the basic case
and with an additional test directory.
[0] https://review.opendev.org/673885
Change-Id: Iabf2e0cf6d86e36a174778367186bbd39a65c3dd
This keeps each platform's "/etc" template files in a separate
top-level directory.
Additionally, we add a distro major version match to the task import
and rename centos7 (in preparation for centos8).
Change-Id: If65d51a27e30311b1da20522afb6dbce7ee6cf35
We've discovered that rackspace swift seems to always want to gzip
encode files when clients request their contents. When our files are
deflate encoded this results in files that are first deflate encoded
then gzip encoded. Not all browers or layer 7 firewalls can handle this
(despite being perfectly valid according to the HTTP RFCs). We'll use
gzip to see if that causes rackspace to not double encode the files.
To do this with memory efficienty we vendor a tool from pypi called
gzip-stream which allows us to read chunks of the compressed data at a
time without loading the entire file into memory or writing multiple
gzip headers in a single file.
Change-Id: I9483cfdbd8e7d0683eeb24d28dd6d8b0c0e772fa
This creates a testing role for upload-logs-swift so that we can test
chagnes to this role before moving them into the production role.
Change-Id: If55b0e2809d9309bc98e6763da32219d996dbfa4
We continue to see the odd task rc of -13 when running iptables-save to
store persistent set of firewall rules. Switch to shell to give us a bit
more debugging freedom if necessary (and to rule out the command
module).
Change-Id: I0c0208101cad985d4113de6b636c3816613b778c
