1484 Commits

Author SHA1 Message Date
Sorin Sbarnea
3f961ce202 Ensure wheel exists for build-release-python
We need it to exist for building wheels. It should be
installed in a pre-playbook.

Change-Id: Ieaec3d21911decf11283e82fed97dd26b3a93ffa
2020-06-17 16:59:24 -05:00
Zuul
e766c6cc9b Merge "Revert "Make sure wheel is installed for python releases"" 2020-06-17 13:10:46 +00:00
Sorin Sbarnea (zbr)
548d2b387a Revert "Make sure wheel is installed for python releases"
This broke multiple zuul deployments. Please do not make changes without having tests on them.

This reverts commit fe4ce563f22b6f95b6df3e5404c4167685c1626a.

Change-Id: I0eac14d9c9c4e717883965e37fa6e3bbaf37f35a
2020-06-17 12:25:17 +00:00
Zuul
db4b57026a Merge "Record artifact checksums and signatures to stdout" 2020-06-17 12:15:27 +00:00
Ian Wienand
0089a2f905 ensure-pip: remove Xenial venv workaround
This was introduced with Id8347b6b09735659a7ed9bbe7f9d2798fbec9620,
but is no longer necessary as the OpenDev images don't ship with
pip-and-virtualenv.  Remove to avoid any ongoing confusion.

Change-Id: I87c4d949c9d9602dfa39023b337fa593f8fafde0
2020-06-17 10:53:06 +10:00
Ian Wienand
ee2fe36585 ensure-virtualenv: call ensure-pip for Xenial
As described inline, we need a fresher pip to make virtualenv work on
Xenial.

Change-Id: I2187d18fec938c4b94e16f1d6631ea743d9325f9
2020-06-17 10:53:06 +10:00
Ian Wienand
9926c5ac80 ensure-pip: install from EPEL for Python 2
On CentOS 7, the python-pip package comes from EPEL.  CentOS 8 and
Fedora don't have python-pip so it's safe to switch this to "yum" and
then we can use enablerepo.

Change-Id: I267f082b2db4e501299226f0033b5d940752d931
2020-06-17 10:53:06 +10:00
Monty Taylor
fe4ce563f2 Make sure wheel is installed for python releases
We need to rework this whole thing, becuase we need to install
wheel into a virtualenv so that we can do it without root. But
this should get us by until we can do that.

Change-Id: I2816a9c76a049f8986839c17c641d6cad541833d
2020-06-16 18:08:40 -05:00
Jeremy Stanley
7c64b4bdb3 Record artifact checksums and signatures to stdout
In case of later upload failures, record the SHA2-256 checksum and
ASCII-armored OpenPGP signature of each signed artifact to the job's
output stream so they can later be used for manual uploading.

Change-Id: Ifd136b95357d499e088c5509fa57daf76a246cf4
2020-06-16 14:39:57 +00:00
Monty Taylor
4651548004 Add ensure-pip and ensure-virtualenv to build-sphinx-docs
We assume they're installed and use them, but we need to make sure
they're installed.

Change-Id: I962d8f432a8beac3acc7545f2cff6d0d99b30cbe
2020-06-16 08:20:58 -05:00
Zuul
08420e0827 Merge "Terraform roles and jobs." 2020-06-16 09:48:11 +00:00
Zuul
f69a3d2c5a Merge "Return upload_results in test-upload-logs-swift role" 2020-06-16 07:11:22 +00:00
okozachenko
9baf749f6d Fix namespace speicification in collect-kubernetes-logs role
Change-Id: Icff4ac0d0c09b8c314bcd96289db4b1710e8ecb3
2020-06-16 04:17:39 +03:00
Zuul
5c37f3fd89 Merge "Add namespace in the collect-k8s-logs role" 2020-06-15 15:08:25 +00:00
Felix Edel
2f8439148a
Return upload_results in test-upload-logs-swift role
This will only change the test role and can be used for validation.

We are facing some issues where the log upload to swift fails, but the
role is always succeeding. To get some more information about the
upload failures, we let the upload() method return those to the Ansible
module and provide them in the module's JSON result.

Change-Id: Ic54b877cd5ea58031e21e514eb3a7c50ea735795
2020-06-15 08:19:11 +02:00
James E. Blair
d46b41857c Fix upload-docker-image when in release pipeline
The zuul.change variable isn't defined when run in a tag-based
release pipeline.  Even though it's wrapped in a ternary operator,
it is still dereferenced even if it's not used.  Make sure we
dereference it safely.

Change-Id: I215a003493603de99296035d77eee64cc66e83f1
2020-06-12 13:57:09 -07:00
James E. Blair
c713ed7a55 Add test-build-container-image-release job
Add a job that tests the build-container-image role as it would be
used in a tag-based release pipeline (as opposed to check or
gate+promote).

Also, correct an issue in the role where we assumed zuul.change
existed.

Change-Id: If2566764a52726ce45fff9b5e96ce9a42d513d8d
2020-06-12 08:41:48 -07:00
Albin Vass
5bcf93c37d Terraform roles and jobs.
Adds terraform roles to install and execute terraform.

Supports adding an override.tf file to override configuration in CI
which is useful to let zuul handle module reposity authentication
instead of setting up credentials on the remote during the job.

Also returns the execution plan back as a comment for 'terraform plan'
to make it easy for reviewers.

Change-Id: I3b4f2bac7f055a0c0f9cb7888b4146ac9c007d25
2020-06-11 17:26:50 +02:00
James E. Blair
bf43f5bb8c Allow upload-docker-image role to be used outside of promote
This adds an option to the upload-docker-image role so that a job
can be written to run build-docker-image and upload-docker-image
in a release pipeline.  This lets users build tagged docker images
from release tags rather than using the promote pipeline (which
may have built the image before a release git tag was applied).

Change-Id: Id28d0a85e02e56640911c44ddbfff7b38547b0b4
2020-06-10 09:56:40 -07:00
Ian Wienand
3ce8adccf2 download-artifact: fix missed part of revert
This was missed in I9bc55d771ec1828d374684d0ffe5ec1d1494773e because
I9650a869d39f5a09851784b6ac50962b77e7f59e was merged as a requested
cleanup.  This recursive variable check should have been removed with
the revert.

Change-Id: I982b4f85b9ae151e9227743c2b0d13b20a1eda66
2020-06-05 15:23:58 +10:00
Ian Wienand
f946763d0a Revert "download-artifact : support recursive download"
This reverts commit 7101fe7d1c13250415f5c6f6392c2a22720bbe43.

This unfortunately has a number of problems.

Firstly, the "Fail if no wget" fails when download_artifact_recurse
isn't set, because we didn't check for wget.

Also, the download doesn't work with some providers.  wget asks for
gzip downloads with it's accept headers (which can't be turned off)
but the recursive download doesn't understand the gzipped index.html
file and thus doesn't find anything to walk.  The "--compression=auto"
flag is available to overcome this, but is not widely supported (and
not supported on the executor).  https://review.opendev.org/733728
attempted to work-around this but the problems with this approach seem
too much for now.

Change-Id: I9bc55d771ec1828d374684d0ffe5ec1d1494773e
2020-06-05 13:34:23 +10:00
Zuul
2c0eaed81d Merge "download-artifact: cleanup long when statement" 2020-06-05 00:00:44 +00:00
Zuul
eed60ad916 Merge "download-artifact : support recursive download" 2020-06-04 23:59:32 +00:00
Ian Wienand
ec8b46318c download-artifact: cleanup long when statement
Reformat for readability

Change-Id: I9650a869d39f5a09851784b6ac50962b77e7f59e
2020-06-05 08:35:25 +10:00
Ian Wienand
7101fe7d1c download-artifact : support recursive download
This adds a flag to download-artifact to download the artifact
directory recursively.

Change-Id: I9e07f2567d64518a7a159bc3f67e904636c9b610
2020-06-05 08:35:20 +10:00
Zuul
1d4ef3e866 Merge "Document twine_executable" 2020-06-04 05:43:06 +00:00
Zuul
40e6f47ac1 Merge "Add the namespace in wait-for-pods role" 2020-06-03 13:10:45 +00:00
Zuul
48a9b973ff Merge "Add addons args in ensure-kubernetes role" 2020-06-02 22:11:29 +00:00
okozachenko
e664a0215e Add addons args in ensure-kubernetes role
Minikube provides addons and we can enable specific addons by passing the --addons arg when using minikube cli

Change-Id: I8d10e3a3dde1070b5c79406285035dacb59dddc8
2020-06-02 22:49:00 +03:00
Dr. Jens Harbott
a0927f70c8 Fix download-logs template
- Do not insert an extra slash into the download URL, this breaks on
  some storage providers.
- Do not try to uncompress *.xz files, libcurl cannot handle that
  compression, also these are mostly journal files that are compressed
  for a good reason.
- Mention the download directory again at the end, avoids needing to
  scroll through umpteen log messages in order to find out the name of
  the tmpdir generated at the start of the script.

Change-Id: I8eb674121a9808dfc8edfc14b8c3525a77b518ab
2020-06-01 11:59:14 +00:00
Andreas Jaeger
e6396cca48 Document twine_executable
Change Ia06218a830b5d44f04e0f10e0444be2cf9ad682e missed to document
the variable, add it.

Change-Id: Ibbe4c40f8a4ea26a16e0a25a788636621e062902
2020-06-01 13:11:52 +02:00
Guillaume Chauvel
76eda39210 ensure-twine: Fix missing default value twine_executable
Change-Id: I7a9221ecc8fc40a152040b9ae857253a8f7775ff
2020-06-01 09:57:20 +02:00
Guillaume Chauvel
d370c590b3 ensure-twine: Check executable presence using shell+bin/bash
While making test with a node:
- with tox installed using "pip install --user" as a non-root user
- PATH modified using ~/.ssh/environment
- declared in nodepool to run as this non-root user

twine executable is not found:

using "command": twine is not found
using "shell" without /bin/bash: twine is not found
using "shell" with /bin/bash: twine is found!

Change-Id: Ia06218a830b5d44f04e0f10e0444be2cf9ad682e
2020-05-30 22:58:06 +02:00
Guillaume Chauvel
54645685a8 ensure-tox: fix tox_executable fact when tox is preinstalled
While making test with a node:
- with tox installed using "pip install --user" as a non-root user
- PATH modified using ~/.ssh/environment
- declared in nodepool to run as this non-root user

tox checks produce an error:
"Check if tox is installed" evalues correctly (shell, bash)
"Output tox version" cannot find tox (command, no sh, no etc ...)

This change sets fact using the output of the first install check

Change-Id: I5886858bcd57afc8ea5fd121ad44bd82afb1a671
2020-05-30 00:26:30 +02:00
okozachenko
81b584d5a1 Add namespace in the collect-k8s-logs role
Specify namespace to collect logs, so no need to change the context

Change-Id: If26bb9aa517bbd6462fbb93eb6cca99929e7e6aa
2020-05-29 18:11:20 +03:00
okozachenko
4b390f2d40 Add the namespace in wait-for-pods role
Specify the namespace for waiting, so no need to change the context

Change-Id: I4b961bdc8c740e20d6b72200f0161d2f5c5c2c19
2020-05-29 17:40:31 +03:00
Zuul
4090d65dab Merge "ensure-bazel: replace ignore_errors with failed_when" 2020-05-29 14:21:44 +00:00
Zuul
429049da4a Merge "upload-artifactory: fix variable misspelling" 2020-05-29 14:15:39 +00:00
Zuul
f0640a9fff Merge "upload-artifactory: documentation fix" 2020-05-28 15:50:46 +00:00
Albin Vass
506db4d538 upload-artifactory: documentation fix
Change-Id: I1457ddff87dec3c3e2acb0b2efe6098b1a820b65
2020-05-28 17:28:34 +02:00
Albin Vass
2477bb6d4d upload-artifactory: fix variable misspelling
Also adds a check that makes sure the instance the artifact
is to be uploaded to is actually defined.

Change-Id: Ie80fa9869e49566dc39e815c10146d45724f5744
2020-05-28 16:26:40 +02:00
Tristan Cacqueray
b8e57ffc60 shake-build: add shake build system job
This change adds a new job to run the shake build system.

Change-Id: Iffaf4e0675d91033b1b5d66c52d501f6bdc8acaf
2020-05-28 12:48:16 +00:00
Zuul
3d64b59f00 Merge "tox: envlist bugfixes" 2020-05-27 16:20:12 +00:00
Zuul
9f16010e05 Merge "ensure-haskell-stack: Make sure dnf copr is installed" 2020-05-27 15:50:59 +00:00
Zuul
fd711509a4 Merge "fetch-tox-output: empty envlist should behave like tox -e ALL" 2020-05-27 15:46:32 +00:00
Zuul
4fc5ff16c7 Merge "Fix py27 tox" 2020-05-27 15:28:51 +00:00
Albin Vass
a38becca99 Fix py27 tox
siblings: python2.7 ConfigParser has no __getitem__
Constrain soupsieve to <2 for python2.7
Add python2-dev to bindep for subprocess32

Change-Id: If9d6a0ae1a62a94dcec11f6bf637ffee7f0f4fc9
2020-05-27 17:12:32 +02:00
Zuul
8a35f60eb3 Merge "Remove unused tox_envlist in fetch-subunit-output" 2020-05-27 14:58:05 +00:00
Albin Vass
58b291b5cf tox: envlist bugfixes
Change-Id: I17e807aa544f6fb45e82604f35c027fe1f86c2f9
2020-05-27 12:32:07 +00:00
Albin Vass
cb90fcc033 ensure-haskell-stack: Make sure dnf copr is installed
Change-Id: I544e23a6445e7daea92a12d692a962a56051dcf2
2020-05-27 13:46:03 +02:00