SetupTools 58 dropped support for its old use_2to3 option, which has
started surfacing a number of ancient Python packages in need of
updates. In this case, the last full release of funcparserlib (which
is a transitive dependency by way of blockdiag by way of
sphinxcontrib-blockdiag) was in 2013, but luckily they have an alpha
release which we can pin explicitly and pull in as a temporary
workaround to get docs builds going again.
Change-Id: I6903eeac2c479e2da795c1dbd215cdee33d09fd7
Ubuntu Xenial reached end of standard support almost 6 months ago.
Its default python3 is 3.5, which new versions of many libraries are
dropping support for as that hit EOL earlier this month. We are now
faced with a situation where we cannot test the fetch-sphinx-tarball
role on Xenial because latest SetupTools breaks old funcparserlib
(indirectly needed for our blockdiag and seqdiag elements in
zuul-jobs' docker-image.rst), and the only funcparserlib release
which is installable has dropped Python 3.5 support.
Cease running Ubuntu Xenial platform and Python 3.5 tests for
changes to zuul-jobs now. This is a soft removal, since we're not
actually removing Xenial platform checks from playbooks or roles
yet, nor removing the tox-py35 job definition.
Change-Id: I46b9b887427133147481e92a1e7c523e6592fd2d
Our Google Cloud log upload role relies on google-cloud-storage,
which in turn needs protobuf. Unfortunately, protobuf dropped Python
2.7 and 3.5 support in its 3.18.0 release, so we use an environment
marker to pin it in our test requirements.
Change-Id: I89caf4e36850fc4a912b76d75f368144ddb5e15f
It seems like BuildKit is the next generation, but not likely to be
enabled by default soon (https://github.com/moby/moby/issues/40379).
Add a flag so people who want to use its features can easily opt-in.
Change-Id: I862819959c77a557199f64b4d42109bc7915959c
We have seen instances where type -p (s)testr seems to return with a
leading blank line which confuses ansible later when trying to use the
first line of output as the path to (s)testr. Address this by chomping
with grep -v ^$. Additionally use type -P instead of -p to ensure we
always get a path even when the command may be an alias or builtin.
Change-Id: Ibffe1e1499eca18ef5dc3904fe222a55242b827d
The official podman ubuntu install guide[1] tells to install podman from
Kubic project repo for ubuntu Bionic and Focal. And project atomic PPA
repo[2] is deprecated.
But Kubic repo only provides x86_64 deb packages for Bionic. For non x86_64
platforms use project atomic PPA repo on Bionic.
Also add a job zuul-jobs-test-ensure-podman-ubuntu-focal.
[1] https://podman.io/getting-started/installation
[2] https://launchpad.net/~projectatomic/+archive/ubuntu/ppa
Change-Id: I402adf1866e4bb8f3b388216bc48b9927e1388b1
This option is ignored by Zuul as of release 4.7.0
This change is safe to merge now, as it won't break older versions.
Change-Id: I29d943462524b44cd04943df42838944fd43535d
We've seen a case where we can still push and pull tags from dockerhub,
but the web UI and API seem out of sync with the actual registry. In
this case, we would like to continue, even though it will leave some
unused tags in the repo (they can be cleaned up later if they ever
show up).
Change-Id: If000163a321c869c46cfed4233c2ea42c3e8471b
So that tests which use ZooKeeper can issue the 4-letter-word
debug commands, make sure they are enabled in the zoo.cfg file.
Change-Id: Ib614e918e02306564c2ed6adb4ec350e40df9043
The case where this isn't set isn't exercised by the tests, so we missed
this. We need to supply an empty list of artifacts to iterate over if
there are no zuul artifacts.
Change-Id: I082e3546ddc0ff57386063a4f697ae6584db9f90
Now that the opendev buildset registry job is fixed by the parent
change, these jobs can be re-added.
Change-Id: I0b904f552e377a8135e028106aa0b863d5094c04
Because buildset registries may be used by jobs that finish before other
jobs are finished using the buildset registry we must be careful not to
expose the registry credentials in the jobs that finish sooner.
Otherwise logs for the earlier job runs could potentially be used to
poison the registry for later jobs.
This is likely currently incomplete. Other Zuulians should look over it
carefully to ensure we're covering all the bases here.
The cases I've identified so far are:
* Setting facts that include passwords
* Reading and writing to files that include passwords (as content may be
logged)
* Calling modules with passwords passed as arguments (the module
invocation is logged)
I've also set no_log on zuul_return that passes up credentials because
while the logging for zuul_return is minimal today, I don't want to
count on it remaining that way.
We also use the yet to be merged secret_data attribute on zuul_return to
ensure that zuul_return itself does not expose anything unwanted.
Finally it would be great if others could check over the use of
buildset_registry variables to make sure there aren't any that got
missed. One thing I'm not sure of is whether or not when conditionals
get logged and if we need to be careful about their use too.
Temporarily remove some buildset-regitry jobs which are in a catch-22.
Change-Id: I2dea683e27f00b99a7766bf830981bf91b925265
This adds new lines between tasks, to make it a little easier to read.
Change-Id: I78ac55027fec58eabd95f097ff9946fa6b2cff9d
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Fedora 32 is now EOL, we should test against the newly released
version of Fedora which is 34.
The podman job is made non-voting while we investigate issues running
as non-root.
Depends-On: https://review.opendev.org/c/openstack/project-config/+/795604
Change-Id: I962a123e1fbf68f064a197700d0fd0da239fca72
Technically dnf doesn't require dnf-plugins-core so it's possible "dnf
copr" may not work. Our Fedora 34 images aren't pre-installing it
(something we should probably fix) but this should be fine as a
generic saftey bootstrap anyway.
Change-Id: I8a645f582f5955c93b4e115ad8bed7c46def5c82
This change enables using fetch-translation role along with
the fetch-output role. By default the role still synchronizes
artifacts back to the executor.
Change-Id: I85c021706c1fa20f8d28b3a1f56c9435ac3836d5
The `helm init` command fails under releases of Helm prior to 2.17.0
due to the fact that the stable charts have been moved[1].
Helm 2 is EOL and ideally this should be bumped to Helm 3 at some point,
but that is a bigger exercise that will require notifying all users
so this minor bump should improve overall UX without affecting users.
[1]: https://helm.sh/blog/new-location-stable-incubator-charts/
Change-Id: Ica60f3225bd7bb3f9cce0af27b486604bfb9b2d5
Properties can be used to tag files.
When another system is using a property filter, being able to set
the properties for zuul artifacts is very convenient.
Change-Id: Ib16ca0f6b532649daa77aa26a8ffa29b78429b71
The bindep jobs have a file matcher on the general-roles-jobs.yaml
file, but that shouldn't be necessary since Zuul runs jobs if they
definitions change. Remove the extraneous match so that we don't
run lots of unecessary bindep jobs.
Change-Id: Ic8b1e7bf4cccf555b3c47a7ac9fcb9aee422714b
The stage-output role had two assumptions:
1) The zuul_copy_output variable would always be defined
2) The role would be able to sudo on the remote node
To make it easier for users who want to use this in a base job,
remove both of those assumptions.
1) We now supply a default empty dict if the variable is not defined
so that the role does not fail with an error.
2) We check to see if we can sudo on the remote node, and if we can,
we do when copying files; otherwise we don't, and assume that the
user will only specify files they have access to (if they don't then
the copy will fail).
Change-Id: I5428c44adfafac4872342b59a92de311f41687b6
Every time I open up the Zuul console for certain jobs it unrolls the
pre-playbook section in the console because it looks like this task
has a "FAILED" status.
pkill man page says that 0/1 are "process killed" and "no match"
respectively; set failed_when to >1 so these return codes don't cause
spurious failures.
Change-Id: I23112b1101c991c1714d69f7568f83c2dcd605dd
Role copied and modified from ensure-podman
As focal doesn't exist for project atomic ppa [1]
Install is performed from opensuse repository only
[1] http://ppa.launchpad.net/projectatomic/ppa/ubuntu/dists/
Change-Id: I72fc2e68768664b80c39bd47295330131337d8b5
Debian Buster is the current stable, and Bullseye is the impending
stable (likely within the month). The parent of this change aims to
fix security mirror configuration for Bullseye, but to do that we
need to add testing for it. While we're at it, drop the Stretch
(oldoldstable) jobs to keep the job count from exploding too much.
Change-Id: I278246eff347dd4bdb3471a8db95d30fcba0f17a
Starting with Debian 11 (bullseye), security packages are in
bullseye-security as opposed to older releases like buster/updates.
List the last several stable releases in hopes nobody is trying to
use this role to configure platforms older than Debian 8 (jessie,
the current "oldoldstable").
A followup change demonstrates this works in the test-base-roles
job, but because the job matrices have to be updated in one fell
swoop, and many of those jobs won't work without this change already
merged (due to protected use in our base job), it's not tested
directly within this change.
Change-Id: I2d7712cbfd037a65b9025980a6c0cccd917f8947
Fixes several rules with test-playbooks and remove path exclusions.
Since we introduced the progressive mode, exclusions are no longer
needed and not haivng them will assure newly added code follows
the same guidelines.
Change-Id: I0a72d34aff8cf23172e26c7f44e0f61571ec74b6
On Centos8, during the docker-ce installation, the docker.socket service
is start with a bogus state:
docker.socket: Socket unit configuration has changed while unit has been running, no open socket file descriptor left. The socket unit is not functional until restarted.
Later, when the `Assure docker service is running` task tries to start
the service, it fails with the following error:
dockerd[29743]: failed to load listeners: no sockets found via socket activation: make sure the service was started by systemd
Example:
https://0c7366f2ce9149f2de0c-399b55a396b5093070500a70ecbf09b9.ssl.cf1.rackcdn.com/410/c233496b96c70cfc6204e75d10116a96b08d4663/check/ansible-test-sanity-docker/787388f/ara-report/index.html
Another example: https://github.com/kata-containers/tests/issues/3103
Also: Remove use of kubectl --generator=run-pod/v1
This has been deprecated since 1.17 and removed since 1.20. run-pod wound
up being the only generator that did anything, so this parameter became a
no-op. This has to be squashed into this commit to unbreak the gate.
Change-Id: I666046fe2a3aa079643092c71573803851a67be2
After a docker failure on a recent job I noticed a single file called
"Got" in the container logs directory; turns out this had tried to
parse the error message "Got permission denied while trying to connect
to the Docker daemon socket ..." as containers.
We don't want this to fail the job hence the ignores here. However,
we shouldn't try to collect any logs if listing the containers fails.
Move creating the directory first as that is safe.
Change-Id: I41db956964f695cfcc15e30cab8cd4f8c31d3706
ARM64 nodes are currently unavailable, so temporarily remove these
jobs. Unfortunately due to the auto-generated job listings, we need
to completely remove them, not just comment them out.
To be clear, we should add these back (but possibly in a dedicated
secondary pipeline) when we're able.
Change-Id: I462596dfb47ed88e2ba19e314c075227f6f912ac
