1128 Commits

Author SHA1 Message Date
Jens Harbott
dd40f5ed9f Fix handling of dangling symlink on manifest generation
There may be broken symlinks within the log directories, those fail with
an error when os.stat() is executed on them. Let's just skip those.

Change-Id: I3e6982c53a08f57ac0e592b8a0041bbb39812d1f
2019-08-26 13:57:03 +00:00
Andreas Jaeger
c74b03649a Switch PDF fetching to known files
Some repos produce intermediate PDF files, for example from svg files,
and thus we have multiple PDF files.

Change the PDF fetching to a list of pdf_files which can be supplied,
the default is the name of the repos with ".pdf" attached, like
"octavia.pdf".

https://review.opendev.org/#/c/667249/3 illustrates the problem.

Change-Id: I5d3b6d6c2884ba66bdd35639b023d6d76f64ac01
2019-08-24 12:24:07 +02:00
Andreas Jaeger
601711dfa3 Handle sphinx PDF files
This enhances fetch-sphinx-tarball to fetch PDF files as well if
they are published

For now, it collects a single PDF file since sphinx only generates one.
The name of the PDF file is not known in advance.

The publishing is done with https://review.opendev.org/678142.
This will then be used by https://review.opendev.org/664555.

Change-Id: I872ab1f696495dc40cb845e0eee5f6f0ffe892d8
2019-08-23 08:30:05 +02:00
Ian Wienand
3e758a8790 upload-logs-swift: fix keystoneauth1 exceptions
Import keystoneauth1.exceptions to access the exceptions.  HttpError
also lives under "exceptions.http.", so update that reference.

Change-Id: I4afe8c9fc8239a31d62a2a1d09794211b5066472
2019-08-21 09:55:16 +10:00
Zuul
f075ededc4 Merge "doc: note that mirror_info isn't actually implemented" 2019-08-20 18:45:35 +00:00
Zuul
0a1de4786a Merge "Update documentation for Debian/Ubuntu mirror_info" 2019-08-20 18:25:43 +00:00
Tristan Cacqueray
56189ab972 doc: note that mirror_info isn't actually implemented
Change-Id: Icd24da50147802657229eebbf9c2667969c9cd40
2019-08-20 18:03:50 +00:00
Zuul
60e7542875 Merge "Remove sphinx_output_dir" 2019-08-16 17:17:05 +00:00
Zuul
fef6fa4321 Merge "install-openshift: bump version to 3.11.0" 2019-08-16 14:32:51 +00:00
Andreas Jaeger
56b9c68ba2 Remove sphinx_output_dir
this is a partial revert of Ifd270cbc2d4bd42418b89a3b3061748ce537ae42,
we do not need sphinx_output_dir anymore, the single use that needed it
has been changed.

Change-Id: I95543b1a56c04dbfb8591612e4c48be0d8631633
2019-08-16 15:39:59 +02:00
Tristan Cacqueray
7e7243fe15 install-openshift: bump version to 3.11.0
This change install the latest 3.x version of openshift.

Change-Id: I941f8831a9ecdb930b7a0edd85f7f15a3b8d57e6
2019-08-15 16:54:26 +00:00
Zuul
7f4b3a6486 Merge "Retry more operations" 2019-08-15 14:44:11 +00:00
Zuul
eb1568c9e4 Merge "Log errors better in case of unknown REST errors" 2019-08-15 14:39:13 +00:00
James E. Blair
8595026f46 Retry more operations
This adds a generic retry handler that takes a callable, and uses
that not only for the exsting retries around the POST call, but
also the actions when creating the container.

Change-Id: I910b8e886f107d4fe38a9334ba836f010f92557c
2019-08-14 14:42:38 -07:00
Monty Taylor
e5897bdd44 Log errors better in case of unknown REST errors
If we get an unexpected exception, it's helpful to know the
cloud and region involved. It's also nice to return that information
for maybe better readability.

Change-Id: I1c589744103512d981e64e1a3f9506d40e1bf4cf
2019-08-14 14:22:01 -07:00
Andreas Jaeger
22a17fa19b Revert "Partial sync fetch-sphinx-tarball with fetch-sphinx-output"
We don't need this here - let's change openstack-manuals instead.

This reverts commit 75ae975235305b7a832dd0dc7c8d4ce540df4794.

Change-Id: I9611950a391823efd2f7d2187da0992f00e0e454
2019-08-14 20:13:23 +00:00
Andreas Jaeger
75ae975235 Partial sync fetch-sphinx-tarball with fetch-sphinx-output
Add and use variable sphinx_output_dir as it's used in
fetch-sphinx-output.

Change-Id: I3fa7b1982463816d16ca6515edffb9e73b8cdbe5
2019-08-14 19:43:36 +02:00
James E. Blair
bf8442a895 Return preview artifact in fetch-sphinx-output
This adds an extra variable to the role which can be used to
better control the URL which is returned as the artifact.

Change-Id: Ifd270cbc2d4bd42418b89a3b3061748ce537ae42
2019-08-14 07:58:38 -07:00
James E. Blair
6e865fa04a Collect container logs from test-registry job
To help with debugging job errors.

Change-Id: I31bb65ee059f14603761d0fec0f8d3d5dc3344a1
2019-08-12 08:00:21 -07:00
James E. Blair
ea00b6aa5a Use human-readable names for artifact returns
This will cause links to appear on the build page with text like
"Docs preview site".

Change-Id: I65f8990b65607e72190a871409c3e31014ce86df
2019-08-12 07:13:27 -07:00
James E. Blair
ed0c36250c Remove download_artifact_name parameter
This parameter was deprecated.  Now it should be removed so that
the name field can be freed up to be a human-readable name.

Change-Id: I0ace733e037ebeffcaf516332e7c5b48b3c7a1a3
2019-08-12 07:10:52 -07:00
Jeremy Stanley
319ae65b93 Update documentation for Debian/Ubuntu mirror_info
Extend the Debian/Ubuntu mirror_info design to accommodate the
parameters required for sources.list entries, and more closely match
the scheme and terminology for them to what is described in the
sources.list(5) manpage.

Change-Id: I0e3cdfeaff838484a9dd0a8cace134c9f8e94650
2019-08-09 21:43:03 +00:00
Zuul
634fa42920 Merge "Don't install centos repos on RHEL" 2019-08-09 08:31:22 +00:00
Jeremy Stanley
2711d10186 Be consistent about spaces before and after vars
With the arrival of ansible-lint 4, Jinja2 variable expansions must
include spaces before and after the variable name inside the
brackets.

Adjust the new violations accordingly and remove the rule
206 exclusion.

Change-Id: Ib3ff7b0233a5d5cf99772f9c2adc81861cf34ffa
2019-08-07 20:17:41 +01:00
Sagi Shnaidman
cecf9e902c Don't install centos repos on RHEL
For running job on RHEL use openvswitch from RHEL8 deps repo,
because centos queens openvswitch causes conflicts with RHEL8
version.

Change-Id: Ia09227cb28ec4a4f9bdbb664b9c7caf714043e31
2019-08-07 10:36:44 +00:00
James E. Blair
4a1ecf50a5 Fix CORS in rackspace in upload-logs-swift
The CDN setup that rackspace has requires that the
Access-Control-Allow-Origin header be added to each object uploaded.

Change-Id: I0f6af613e2ebcf9cfe835dc8018a73922f9f0ed5
2019-08-02 13:40:07 -07:00
Zuul
41d06378b0 Merge "Add CORS support to upload-logs-swift" 2019-08-02 16:59:15 +00:00
Zuul
a0b390936f Merge "Support Rackspace in upload-logs-swift" 2019-08-02 16:18:20 +00:00
James E. Blair
c5e7174c2f Add CORS support to upload-logs-swift
When creating containers, add a CORS header which allows access
from any host, so that the zuul web app can fetch.

Change-Id: I013265643a8fcb4cff001375136c2c37958fd97a
2019-08-02 09:07:28 -07:00
Zuul
4e5b005133 Merge "Add test-bindep job" 2019-08-02 15:22:43 +00:00
Zuul
2ceabc5cc8 Merge "Don't compare to literal True/False" 2019-08-02 15:22:43 +00:00
James E. Blair
ccfdcd1e84 Support Rackspace in upload-logs-swift
Rackspace only provides unauthenticated access to object storage
via CDN in a non-openstack-standard way, so we need to do some
extra work to support that.

This also adds a helper script for testing which deletes a container
(since in order to do so, you may need to delete all the contents
first).

Some commented-out debug configuration lines are added for the
convenience of future developers.

Change-Id: I3d1fce824fb40136048f0988939d22f755236a59
2019-08-01 16:10:01 -07:00
Andreas Jaeger
04f4cecaec Revert "fetch-subunit-output: collect additional subunit files"
This reverts commit 8624201e0cc8b8860b4fe01994739a6828ec9615.

This breaks builds like in https://logs.opendev.org/01/673401/2/check/openstacksdk-tox-py36-tips/0e0bed9/job-output.txt.gz#_2019-08-01_17_04_29_549629

Change-Id: Ie4c8f966cfe72828a6ff9a7512948d6c55c62fde
2019-08-01 17:55:58 +00:00
Jeremy Stanley
b6d2616275 Don't compare to literal True/False
With the arrival of ansible-lint 4, comparisons to literal boolean
values are now forbidden. Adjust the new violations accordingly and
remove the rule 601 exclusion.

Change-Id: I18ba2d7d41fabaff35d10d520037188c7d9d1249
2019-08-01 19:01:37 +02:00
Luigi Toscano
8624201e0c fetch-subunit-output: collect additional subunit files
In addition to the main subunit file from zuul_work_dir,
collect the subunit files from the elements (directories)
of the zuul_additional_subunit_dirs list.

The default behavior is unchanged.

Change-Id: Ic3bc76aa104951bcc0c186347196067342e28751
2019-08-01 17:49:02 +02:00
James E. Blair
7b75da65a5 Add test-bindep job
Change-Id: I781500ae2cabef504ba32ba087f80bc9d2a0a417
2019-08-01 08:46:13 -07:00
Zuul
b194dc4e1d Merge "Add auth config to kubelet user for buildset registries" 2019-07-31 17:01:44 +00:00
Zuul
1d0584b012 Merge "Add mirror_info documentation" 2019-07-31 15:11:45 +00:00
Jeff Liu
2c33db2674 Add auth config to kubelet user for buildset registries
Change-Id: I3c6b434382aab2f7ef565808954873e9aa6b9b42
2019-07-30 16:48:27 -04:00
Zuul
1a955414f1 Merge "Fix typo in download-artifact" 2019-07-30 16:01:35 +00:00
Zuul
0cf9df50dc Merge "Update testing section" 2019-07-30 15:51:27 +00:00
James E. Blair
d940ecc30a Fix typo in download-artifact
Change-Id: Id7ebc574c03d936573c96f1d907f1238f55b344b
2019-07-30 08:46:05 -07:00
Zuul
c4aae4ffa5 Merge "Download-artifact: use the artifact type rather than name" 2019-07-29 16:14:47 +00:00
Monty Taylor
10cea0b22d Don't try to push images when the build failed
When an image build fails we're still trying to push the image to
the registry. This isn't going to work, as there is no image to
push since the build failed.

Add a check for the job being in a success state for if we're
pushing images. This way we can properly report FAILURE instead of
POST_FAILURE.

Change-Id: Idba5c8997111c6e84b44bc970394b58387519666
2019-07-28 14:07:47 -04:00
Clark Boylan
d480a536f2 Add note to clear-firewall docs
Change-Id: Ia5d4c777d71adfd9c75e7f2c836c5b6d8bf8c857
2019-07-25 14:29:52 -07:00
James E. Blair
97e8392fd8 Update testing section
This includes the new information described at:

http://lists.zuul-ci.org/pipermail/zuul-discuss/2019-July/000973.html

Change-Id: I346c03a55749cd6153e7fd52773039003d038d89
2019-07-25 14:17:41 -07:00
Monty Taylor
4a9d8b32e2 Add clear-firewall role
Some jobs need to start with a clean slate of firewall rules so that
they can manage their own rules. Add a simple role that clears out
everything.

Change-Id: I92d3b02a6bd4f19460294ca5293dfbbd67bfd295
2019-07-25 14:12:53 -07:00
Zuul
a796a2cf48 Merge "Skip test-setup.sh in pep8 jobs" 2019-07-25 15:57:39 +00:00
James E. Blair
87b391c84a Download-artifact: use the artifact type rather than name
So that the artifact "name" may be used for human consumption,
switch the download-artifact role to use the metadata.type
attribute of the artifact.

This deprecates the download_artifact_name role variable and
adds _type as a replacement.

Change-Id: I1c4a4be7d1599b233b56fe73117ff097b8ad381a
2019-07-24 09:08:55 -07:00
James E. Blair
02d9a29562 Fix typo in generate-zuul-manifest role
Change-Id: I763a7a6e95f08ba07b38ed6b1701fe16e2548471
2019-07-23 11:58:01 -07:00