Zuul switched to a new base image, and it seems the new socat puts out
a warning (something like
... socat[489590] W ioctl(5, IOCTL_VM_SOCKETS_GET_LOCAL_CID, ...): Inappropriate ioctl for device
for reference).
Grep the output so we only get the line about what port it is
listening on.
Change-Id: I74fb86a9158b45e6601ee1fbc199ba80cd4991fe
The below error is coming on mol-centos7 jobs while
installing sibling for python2 packeges on centos7
Error: TypeError: initial_value must be unicode or None, not str
This patch handled the TypeError while installing sibling
for python packeges for centos7.
Closes-Bug: #1946641
Change-Id: Ie8058cca92d099e50af19b95b4c417c5a665da0d
build-container-image may ends up pushing large files. Network is not
always as reliable as we want. This commit ensures we retry several
times in case of failure.
Change-Id: Ieaf92a8c3531e24ab5e41783b540de1df806c02f
I think the secondary "rustup default" was intended to make sure the
Zuul user sets up to use the installed rust toolchain
(I32f9b285904a7036f9a80ada8a49fa9cf31b5163) but actually results in a
re-download of components and another local installation. This isn't
really the intention, and also doubles the time spent installing.
From the linked comment, it seems like we're not doing our global
install correctly; even putting it in /usr doesn't avoid the need for
RUST_HOME to be set. Take it's suggestion and install out-of-the-way
in /opt, use a small /usr/local/bin wrapper to call with correct env
vars set and then setup the installed global binary names to be called
via that.
Change-Id: I28ef747b809a17664305bfd9754022251390647b
With the original verbose output support in change
Iafeb88eaf9a596603ad4d2134a4574345d5189ab we looked for lines from
tox --showconfig output starting with an opening '[' but verbosity
also causes output from pip install activity to be included if a
tox.requires entry causes it to install packages before continuing
and these lines are prefixed by a process ID number wrapped in
brackets with the command string after that, which prematurely
triggered our search for the start of the INI content. Add a
stipulation that the first INI output line also end in ']' in order
to skip over those additional prepended lines.
Change-Id: If29e5a9abe3b92a145d87f5efc1b93350ea3908a
Update the alternate tox.ini files in test-playbooks/python to
trigger configuration we saw expose a bug in the siblings lib's
showconfig parser when a tox.requires value resulted in pip
activity. Update its minversion to the earliest tox which provides
the tox.requires configuration option, for clarity.
While we're there, switch to using tox_config_file instead of
overriding zuul_work_dir since that's now an option for this role
and was previously not exercised in the test job.
Change-Id: I521580da978903cd116c872447b1c028c470e63d
This reverts commit 8b1cc73ee30428517f9524624b9eb309e9b14c1f.
Apparently tox.requires adds lines like this to verbose showconfig
output:
using tox-3.24.4 from /.../tox/__init__.py (pid 2919)
Which is confusing the current parser in the siblings module. Roll
this back while we work on a fix and regression test.
Change-Id: If3b1d48b36a5d32fddfdabd9c0ec1b81dd6453f2
Update the deprecation policy to indicate that zuul-jobs is no
longer tested with EOL platforms. Also explicitly switch the minimum
Python 3 documented to 3.6, and add a note to the tox-py34 and
tox-py35 jobs mentioning that they're no longer directly tested.
Move those jobs to the deprecated jobs list as well, to help
reinforce the point that their continued use is not recommended.
Change-Id: I2edbf8ea010caf7a7641e0d88f360965fc0b96ab
In change Iafeb88eaf9a596603ad4d2134a4574345d5189ab we fixed a bug
with handling of `tox --showconfig -vv` in the siblings tasks for
the tox role. This arose because of the desire to add tox_extra_args
to all tox invocations in Ibfe77f67e43135ae5af7588d6859b8b3dbd4c3ca
because we currently set a default of '-vv' for the tox_extra_args
rolevar. This fix could easily regress in the future if we were to
ever change the default value, so explicitly test with it in the
zuul-jobs-test-tox job by adding it to the python/tox test playbook.
Change-Id: Ib6ddf7d188904193a5ecd520acac1d676e5d78ae
When the tox role was introduced, a tox_extra_args rolevar was
included allowing the tox command line to be extended with arbitrary
options. When siblings functionality was added, tox_extra_args did
not get included in its separate tox invocations. If a project has a
particular situation where some aspect of tox's functionality must
be overridden in order to work, doing so through tox_extra_args
needs to apply to every tox invocation, including siblings installs.
Change-Id: Ibfe77f67e43135ae5af7588d6859b8b3dbd4c3ca
Unfortunately, when tox combines --showconfig with verbosity options
like -vv, some non-config output gets streamed to stdout before the
configuration is emitted. Filter this preamble in
tox_install_sibling_packages by discarding any initial lines of
output before the first section heading.
Also extend get_envlist() to deal with the fact that additional
verbosity adds a [tox] section in the --showconfig output, which it
was previously relying on to determine whether the config had been
filtered to a subset of env sections. Instead also check the
tox.args string to determine whether a -e option was passed on the
command line.
Change-Id: Iafeb88eaf9a596603ad4d2134a4574345d5189ab
In some situations, projects may not keep a tox.ini in the root
directory of their repository, or may even have multiple tox
configuration files. Allow the location and name of the config to be
overridden explicitly through the use of a new rolevar.
Change-Id: I1927142e6d9fa75e96902ae001c8ca98d69c7443
SetupTools 58 dropped support for its old use_2to3 option, which has
started surfacing a number of ancient Python packages in need of
updates. In this case, the last full release of funcparserlib (which
is a transitive dependency by way of blockdiag by way of
sphinxcontrib-blockdiag) was in 2013, but luckily they have an alpha
release which we can pin explicitly and pull in as a temporary
workaround to get docs builds going again.
Change-Id: I6903eeac2c479e2da795c1dbd215cdee33d09fd7
Ubuntu Xenial reached end of standard support almost 6 months ago.
Its default python3 is 3.5, which new versions of many libraries are
dropping support for as that hit EOL earlier this month. We are now
faced with a situation where we cannot test the fetch-sphinx-tarball
role on Xenial because latest SetupTools breaks old funcparserlib
(indirectly needed for our blockdiag and seqdiag elements in
zuul-jobs' docker-image.rst), and the only funcparserlib release
which is installable has dropped Python 3.5 support.
Cease running Ubuntu Xenial platform and Python 3.5 tests for
changes to zuul-jobs now. This is a soft removal, since we're not
actually removing Xenial platform checks from playbooks or roles
yet, nor removing the tox-py35 job definition.
Change-Id: I46b9b887427133147481e92a1e7c523e6592fd2d
Our Google Cloud log upload role relies on google-cloud-storage,
which in turn needs protobuf. Unfortunately, protobuf dropped Python
2.7 and 3.5 support in its 3.18.0 release, so we use an environment
marker to pin it in our test requirements.
Change-Id: I89caf4e36850fc4a912b76d75f368144ddb5e15f
Work around lack of SNI support in old distutils versions shipped
with Python on platforms like CentOS 7 and Ubuntu 16.04 LTS by
installing PBR first so that distutils won't be compelled to do so.
Warehouse (PyPI) ceased supporting clients without SNI support in
March of this year.
Change-Id: Ic741d9a87c2ca3a5249cd03c3cbd38e2ad1f46a1
Avoid false-positive CI testing with tox where misconfigured tox
projects end-up skipping tests or running with different python
version than the required one.
While use of this option on development machine may be ok, when
executed in CI context, we never want to be relaxed about what
we test and which versions of python we use.
I seen projects running with wrong version of python for months
before someone discovered that a different version of python
was used on CI.
Change-Id: I5be9bce86833db11afd7072e477ccaf42658bf99
It seems like BuildKit is the next generation, but not likely to be
enabled by default soon (https://github.com/moby/moby/issues/40379).
Add a flag so people who want to use its features can easily opt-in.
Change-Id: I862819959c77a557199f64b4d42109bc7915959c
We have seen instances where type -p (s)testr seems to return with a
leading blank line which confuses ansible later when trying to use the
first line of output as the path to (s)testr. Address this by chomping
with grep -v ^$. Additionally use type -P instead of -p to ensure we
always get a path even when the command may be an alias or builtin.
Change-Id: Ibffe1e1499eca18ef5dc3904fe222a55242b827d
The official podman ubuntu install guide[1] tells to install podman from
Kubic project repo for ubuntu Bionic and Focal. And project atomic PPA
repo[2] is deprecated.
But Kubic repo only provides x86_64 deb packages for Bionic. For non x86_64
platforms use project atomic PPA repo on Bionic.
Also add a job zuul-jobs-test-ensure-podman-ubuntu-focal.
[1] https://podman.io/getting-started/installation
[2] https://launchpad.net/~projectatomic/+archive/ubuntu/ppa
Change-Id: I402adf1866e4bb8f3b388216bc48b9927e1388b1
This option is ignored by Zuul as of release 4.7.0
This change is safe to merge now, as it won't break older versions.
Change-Id: I29d943462524b44cd04943df42838944fd43535d
We've seen a case where we can still push and pull tags from dockerhub,
but the web UI and API seem out of sync with the actual registry. In
this case, we would like to continue, even though it will leave some
unused tags in the repo (they can be cleaned up later if they ever
show up).
Change-Id: If000163a321c869c46cfed4233c2ea42c3e8471b
So that tests which use ZooKeeper can issue the 4-letter-word
debug commands, make sure they are enabled in the zoo.cfg file.
Change-Id: Ib614e918e02306564c2ed6adb4ec350e40df9043
The case where this isn't set isn't exercised by the tests, so we missed
this. We need to supply an empty list of artifacts to iterate over if
there are no zuul artifacts.
Change-Id: I082e3546ddc0ff57386063a4f697ae6584db9f90
Now that the opendev buildset registry job is fixed by the parent
change, these jobs can be re-added.
Change-Id: I0b904f552e377a8135e028106aa0b863d5094c04
Because buildset registries may be used by jobs that finish before other
jobs are finished using the buildset registry we must be careful not to
expose the registry credentials in the jobs that finish sooner.
Otherwise logs for the earlier job runs could potentially be used to
poison the registry for later jobs.
This is likely currently incomplete. Other Zuulians should look over it
carefully to ensure we're covering all the bases here.
The cases I've identified so far are:
* Setting facts that include passwords
* Reading and writing to files that include passwords (as content may be
logged)
* Calling modules with passwords passed as arguments (the module
invocation is logged)
I've also set no_log on zuul_return that passes up credentials because
while the logging for zuul_return is minimal today, I don't want to
count on it remaining that way.
We also use the yet to be merged secret_data attribute on zuul_return to
ensure that zuul_return itself does not expose anything unwanted.
Finally it would be great if others could check over the use of
buildset_registry variables to make sure there aren't any that got
missed. One thing I'm not sure of is whether or not when conditionals
get logged and if we need to be careful about their use too.
Temporarily remove some buildset-regitry jobs which are in a catch-22.
Change-Id: I2dea683e27f00b99a7766bf830981bf91b925265
This adds new lines between tasks, to make it a little easier to read.
Change-Id: I78ac55027fec58eabd95f097ff9946fa6b2cff9d
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
