1929 Commits

Author SHA1 Message Date
Pierre-Louis Bonicoli
222056a26f
Avoid to use 'length' filter with null value
This error occurs when variables are null and
DEFAULT_JINJA2_NATIVE is enabled:

    object of type 'NoneType' has no len()

Change-Id: I7c69be60b846c03e30a0a6e8893bf2f3a3d5e5b8
2020-08-03 03:31:29 +02:00
Xing Zhang
1242a98137
Fix add repositories in RedHat and Suse
These tasks should be used with become like Debian.yaml, otherwise
error will occurred like this:
[Errno 13] Permission denied: '/etc/yum.repos.d/test.repo'

Change-Id: I9c6f0132536500d36ba0098b635b1858e05d7ccf
2020-07-21 10:51:28 +08:00
Zuul
9e20909819 Merge "Enable tls-proxy in ensure-devstack" 2020-07-20 20:49:12 +00:00
Zuul
15b4903a32 Merge "add-build-sshkey: Ensure .ssh exists, enable admin authorized_keys" 2020-07-20 15:57:17 +00:00
Albin Vass
45ea044d53 Remove copy paste from upload-logs-swift
Change-Id: Ia365e46bf58ebd6d17ec064e3a0114dd64278c77
2020-07-19 13:27:30 +02:00
Clark Boylan
7f5e3ef831 Enable tls-proxy in ensure-devstack
We do this because it better approximates a production environment and
it seems that devstack is better tested when tls-proxy is enabled.

Change-Id: If607caf301211181b4f37a2c7012f875de3d285c
2020-07-18 15:05:33 -07:00
Zuul
eb9eb8bc12 Merge "Add a test that exercises the GCS Credentials class" 2020-07-16 21:08:41 +00:00
James E. Blair
a488ed2cdc Add a test that exercises the GCS Credentials class
We subclass the GCS Credentials class.  We can't completely test
it without either hitting the live Google cloud or substantial
mocking, but we should be able to exercise most of the functionality
we typically use.  Do that by asking it to load a token from disk
and assert that it adds that token to a fake request headers
dictionary.

This also corrects a "problem" detected by the test.  The current
super() call uses the python3 form, which is fine in that all current
uses of this code are using python3, but we still run python27 tests
on this repo, so we'll use the python2/python3 compat syntax.

Change-Id: Ifa4209617f4be52008b6294ebd10f0deb9bd6a51
2020-07-16 13:30:13 -07:00
Zuul
fdd7867ee6 Merge "Remove default tox_envlist: venv" 2020-07-16 20:17:13 +00:00
Albin Vass
c388e61160 Remove default tox_envlist: venv
Setting tox_envlist to venv by default is unintuitive for
many users. Remove this behaviour and let default tox
behaviour be the same as running tox on the commandline.

Change-Id: I1b6d59ee4ebb7f6b3adcf4bd35d7148e83389008
2020-07-16 21:59:57 +02:00
James E. Blair
c0abf5cffc Fix GCS log upload
The google-auth Credential class added some new variables in the
constructor; make sure that we call it in order to avoid later
reference errors.

Change-Id: I338a3368b90a6612801ba587982d3053bbd3ba78
2020-07-16 12:32:02 -07:00
Zuul
8e4b52617c Merge "upload-afs-synchronize: expand documentation" 2020-07-15 16:07:39 +00:00
Zuul
2b5777e82a Merge "write-inventory: add per-host variables" 2020-07-14 21:26:48 +00:00
Ian Wienand
c19ea2e6f9 upload-afs-synchronize: expand documentation
The upload-afs-synchronize role was added with
I493d8829e3fd98e84f1b8f0e776e7ba41abf16c2 and used as part of the
artifact promote process for tarballs and other uploads.  However the
role is useful as a generic way to upload results to AFS and avoid
some commonly hit caveats such as getting permissions flags wrong.

Expand the README to explain things a little more.

Change-Id: Ia643b02b503560d1057697fc5a7d143fd728b177
2020-07-15 06:48:44 +10:00
Sean McGinnis
09b4be329e
Strip path from default ensure_pip_from_upstream_interpreters
The ensure-pip role has an option to provide multiple interpreters to
use. If not specified, we default to ansible_python.executable. This
default includes the full path to the executable. Those manually
providing which interpreters to use will not know what the full path is,
only the versions they would like to use. To make things consistent,
this strips off the path so we just have the version (python, python3,
python3.8, etc).

Change-Id: I339afc08393e9c6b1d26a05cf13b6fdc151f46d5
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-07-10 11:24:27 -05:00
Zuul
2f9ef55117 Merge "Install venv for all platforms in ensure-pip" 2020-07-10 14:30:20 +00:00
Sean McGinnis
0aca372814
Install venv for all platforms in ensure-pip
The ensure-pip tasks will install python3-venv, but multiple
interpreters may be specified. This adds handling to make sure the venv
package is installed for all requested interpreters.

Change-Id: I670d5815bfc902f9c50a98df715ef60b61bab594
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-07-09 16:02:45 -05:00
Albin Vass
a7faab5819 add-build-sshkey: Ensure .ssh exists, enable admin authorized_keys
Change-Id: I54608635711a4936ce01c3859706236627d80af9
2020-07-09 21:05:12 +02:00
Zuul
a0c29f2843 Merge "emit-job-header: add inventory hostname" 2020-07-09 13:02:03 +00:00
Albin Vass
dd59a158e8 Enforce BatchMode when setting up ssh for windows
Change-Id: I62742dc43df5276f51805e259229405ba9b19362
2020-07-09 14:09:27 +02:00
Zuul
25af9d4014 Merge "fetch-coverage-output: direct link to coverage data" 2020-07-08 07:59:01 +00:00
Ian Wienand
184152f07e write-inventory: add per-host variables
This allows us to write out host-specific variables into the inventory
from external jobs.

Change-Id: I0c3d08f6446174cb38d5eebaeae1db9dceb956b2
2020-07-08 17:20:21 +10:00
Zuul
3b8470b4a7 Merge "Allow deleting workspace after running terraform destroy" 2020-07-07 16:29:38 +00:00
Zuul
c900f1c4df Merge "Drop support for ansible 2.6" 2020-07-07 11:32:11 +00:00
Luigi Toscano
82828cea50 fetch-coverage-output: direct link to coverage data
Make it easier to find them.

Change-Id: I0e8ee7e3c9f6e55461168b97525b26625eb018f0
2020-07-06 01:03:00 +02:00
Alex Schultz
d8c3a4a808 Make persist-iptables more robust
We're seeing when the shell command fails (for whatever reason) and since
it's failed_when: false, the subsequent tasks actually fail.  We can
check if stdout is not defined as a failure condition because that means
the module itself failed and we should not continue to process the
next tasks since they depend on stdout being defined.. Additionally we
can add some retries to try again if possible.

Change-Id: Iabc1619009220c6f1c91b84f81d5262c4d9d8430
Related-Bug: #1885697
2020-07-02 09:42:58 -06:00
Ian Wienand
1842495fcb emit-job-header: add inventory hostname
When you're running a multi-node job, it would be nice to see the
inventory hostname in this list, so if you want to say, monitor one of
the nodes, you don't have to guess which is which.

Change-Id: Ibf822d51711c74d963a7287eaeaa9a40ff074e79
2020-07-02 15:59:15 +10:00
Zuul
5f6c714208 Merge "Handle multi-arch docker manifests in promote" 2020-07-01 22:53:08 +00:00
James E. Blair
b2b5ba1f27 Handle multi-arch docker manifests in promote
A multi-arch manifist is a manifest list which has a different content-type
than a regular manifest.  In order to re-tag the image correctly, tell
docker hub that we can accept both kinds of manifests, and re-upload
the one that it gives us.  This will be a manifest list if it exists, or
a regular manifest if it doesn't.

Change-Id: I7863b0c824c0b3cb20f94ba67399e823a216092b
2020-07-01 15:37:58 -07:00
Zuul
89cb77bf46 Merge "ensure-pip debian: update package lists" 2020-07-01 18:39:53 +00:00
Zuul
a0a44d034e Merge "Test multiarch release builds and use temp registry with buildx" 2020-07-01 18:19:12 +00:00
Albin Vass
06e35b2125 Test multiarch release builds and use temp registry with buildx
When using docker buildx to build a container image, use a temporary
registry to receive the built image instead of requiring a buildset
registry.

A multi-arch test is also added with a publication registry
using the same task list to reduce duplication.

Change-Id: Ib20d1c97f6cb63e0ff9d8888ea792d1941cd8690
Co-Authored-By: James E. Blair <jeblair@redhat.com>
2020-07-01 10:29:31 -07:00
Albin Vass
7ef4d9f3f5 Allow deleting workspace after running terraform destroy
Also fixes a bug where the workspace was created from the wrong directory.

Change-Id: I28b54b9aa57a3f449254574956701ee663618e8b
2020-07-01 13:25:39 +02:00
James E. Blair
8501cc23fc Ignore ansible lint E106
E106 rejects roles with '-' in their names, which is important
for collections, but these aren't collections.

https://docs.ansible.com/ansible/devel/dev_guide/developing_collections.html#roles-directory

Change-Id: I48f837e8d5bd7d0d9d708d728d93627deb4093cc
2020-06-30 16:18:59 -07:00
Zuul
62fd612dc5 Merge "Return upload_results in upload-logs-swift role" 2020-06-30 15:01:43 +00:00
Zuul
fccd019606 Merge "prepare-workspace: Add Role Variable in README.rst" 2020-06-30 14:45:25 +00:00
Thierry Carrez
a6fab9c48f upload-git-mirror: use retries to avoid races
To avoid errors in case of mirror race conditions, we currently
(try to) ignore mirror errors, and check for presence of the
reference after the mirror try. This requires having to play
tricks to avoid errors when the job is run in the check pipeline,
due to missing zuul.newrev. And for some reason, it does not seem
to work[1].

Instead of ignoring errors and running an additional check, let's
just retry the mirror operation three times.

The race is rare-enough that it's very unlikely that we would
hit it three times in a row.

[1] https://zuul.openstack.org/build/4966cd5617624d348fa0048de14f1f96

Change-Id: Ifa4f25ad8c961a1b2dbd9f07a3a1e4652b790c9c
2020-06-26 13:15:41 +02:00
Zuul
66e22d3b2e Merge "upload-git-mirror: check after mirror operation" 2020-06-25 14:32:51 +00:00
Thierry Carrez
e53fe850c8 upload-git-mirror: check after mirror operation
The upload-git-mirror job currently reports failure when multiple
jobs run concurrently for the same repository (one of them succeeds,
the other fails because the state of the mirror was changed by the
successful one).

However, since the job generally runs a few minutes after the commit
that triggered it merged, the successful job usually mirrored the two
needed commits, so the failed mirror operation does not mean the
mirror is out-of-sync. This results in false negatives.

If zuul.newrev is provided, rather than report failure if the mirror
operation fails, we should report failure if, after the mirror operation,
the commit that triggered the job is still not present on the mirror.

Change-Id: Iadae26473f4c937384c8cea74329e389c563aeed
2020-06-24 18:01:10 +02:00
Zuul
9d0e54f55b Merge "Simplify twine invocation for PyPI uploads" 2020-06-23 19:01:49 +00:00
Albin Vass
ed97235bf8 ensure-pip debian: update package lists
The 'package' module doesn't update any package lists before installing
packages which can cause problems if the package lists haven't been
updated in a while. So run 'apt-get update' before
installing pip.

Change-Id: I836f7faa25dd8c06d0158828e21226fbe93fbb40
2020-06-23 15:58:12 +02:00
Felix Edel
a108e6440f
Return upload_results in upload-logs-swift role
We are facing some issues where the log upload to swift fails, but the
role is always succeeding. To get some more information about the
upload failures, we let the upload() method return those to the Ansible
module and provide them in the module's JSON result.

The equivalent change in the test-upload-logs-swift [1] role is
validated in [2].

[1] https://review.opendev.org/#/c/735503/1
[2] https://review.opendev.org/#/c/737441/

Change-Id: Ie0d4ea2f3365600eae0e572e4c0790b131d3b13e
2020-06-23 09:01:52 +02:00
Guillaume Chauvel
21169062e6 prepare-workspace: Add Role Variable in README.rst
Change-Id: I9bd85cbbf334a8d8c57b4adabf633e7046c273fd
2020-06-22 21:17:38 +02:00
Zuul
ca56608a63 Merge "Fix and test multiarch docker builds in a release pipeline" 2020-06-22 15:55:32 +00:00
Zuul
05303ec7b2 Merge "Add tests for upload-docker-image" 2020-06-22 15:55:31 +00:00
James E. Blair
ce000177f8 Fix and test multiarch docker builds in a release pipeline
This corrects the following error when upload-docker-image is
used in a release pipeline:

  The task includes an option with an undefined variable.
  The error was: 'dict object' has no attribute 'change'

Change-Id: I70cd566ddd0b931d635b20107d652f3591a133b6
2020-06-22 06:55:26 -07:00
Albin Vass
0f55195335 Add tests for upload-docker-image
This also adds support for using a self hosted registry
with the docker roles.

Change-Id: I43caf46cfcc5d335ce14d72ad2abbef732b24645
2020-06-22 06:55:11 -07:00
Zuul
0da413c174 Merge "Fix zuul-jobs-test-ensure-python-pyenv files" 2020-06-21 19:12:43 +00:00
Mohammed Naser
72617b3cd5 ensure-chart-testing: install pip
In order for things to work properly, we need to make sure we at least
have pip installed.

Change-Id: I46e0593f0fa9042d866bfd53e5292251414693b0
2020-06-20 12:31:08 -04:00
Andreas Jaeger
5189873aad Fix zuul-jobs-test-ensure-python-pyenv files
The job  zuul-jobs-test-ensure-python-pyenv tests the ensure-python role
but - as shown in https://review.opendev.org/#/c/737060/ - changes to
that role do not run the testsuite. Update the files list.

Change-Id: I8c60f55aef8389ee893010f9cf92eb8cd109f74b
2020-06-19 21:31:03 +02:00