120 Commits

Author SHA1 Message Date
Zuul
5b194d277e Merge "shake-build: add shake build system job" 2020-05-28 13:28:21 +00:00
Tristan Cacqueray
b8e57ffc60 shake-build: add shake build system job
This change adds a new job to run the shake build system.

Change-Id: Iffaf4e0675d91033b1b5d66c52d501f6bdc8acaf
2020-05-28 12:48:16 +00:00
Andreas Jaeger
8d90e3d4cb Fix typo: Deprecrated
Fix typo: Deprecrated -> Deprecated

Change-Id: I57f08a85f3dd0effe270c35d36c95b1b889de6ff
2020-05-28 08:09:49 +02:00
vass
ef2f2eacb1 Remove install-* roles
The install-* roles has been renamed to ensure-* but have been kept
to give users time to update their repositories.

Tuesday the 26th of may is four weeks after we announced the deprecation
of the install-* roles and the date we should remove them completely.

Depends-On: https://review.opendev.org/719323
Depends-On: https://review.opendev.org/719324
Depends-On: https://review.opendev.org/719325
Depends-On: https://review.opendev.org/719326
Depends-On: https://review.opendev.org/719333
Depends-On: https://review.opendev.org/719334
Depends-On: https://review.opendev.org/719401
Depends-On: https://review.opendev.org/719402
Depends-On: https://review.opendev.org/719404

Change-Id: I1bc87cf9ef544c06e2f1daf3177bba9d577e332e
2020-05-26 09:34:25 +02:00
Tristan de Cacqueray
1e0af6779b Add remove-zuul-sshkey
This change enables a kubectl connection job to just remove the
zuul sshkey, without using the add-build-sshkey role which doesn't
work on pod. To do that, this change moves the sshagent_remove_key
task to a new role and makes add-build-sshkey use the new role.

Change-Id: I5e7288592cad303df919220259f5a360bf522f64
2020-05-15 12:24:33 +00:00
Zuul
f05687fd78 Merge "Add upload-artifactory role" 2020-05-14 17:26:10 +00:00
Zuul
37d6544969 Merge "Policy rule for ownership between remote and executor" 2020-05-14 16:04:03 +00:00
Monty Taylor
886cdcc21f Combine javascript deployment and deployment-tarball jobs
There's no reason to not just build the tarball when we build the
content.

Set the default value for node_version in the base job.

Change-Id: Ifb3e5138e9ae19ec3de9250f1828fc07230ef739
2020-05-13 16:48:29 +00:00
Albin Vass
38053b0826 Policy rule for ownership between remote and executor
Change-Id: Ia0b6bcf221ee053a0c2a2073796cba35fa22efd0
2020-05-13 09:41:38 +02:00
Albin Vass
664b016fa5 Add upload-artifactory role
Adds a role to let users define a manifest of artifacts located
in zuul-output/artifacts that should be uploaded to pre defined
artifactory instances.

Change-Id: I00dc0302e85ce59b3808f6e62e2bcdadf2e41fde
2020-05-12 20:02:12 +02:00
Monty Taylor
0caa85cc48 Split browser and non-browser targetting js test jobs
Some javascript operations, like running tests for web-apps, need
browsers. Javascript packages that are not targetting browsers
do not.

Make a -browser version that installs the browsers and xvfb, and
a non-browser version that does not.

Change-Id: I33c12cb0d9516bdffef7d8d04af4dbcb03ed8355
2020-05-12 11:31:42 -05:00
Monty Taylor
5973c9b7b1 Extract ensure-javascript-build-tool role
The pre-playbook for the javascript jobs got complex. Extract it
to a role so we can better document it.

Also - stop installing javascript depends in pre - the depends
declared in a patch should be tested as part of the patch.

Change-Id: I50a483f223620cd3f9ecd82887062cfc9ac64b7f
2020-05-12 07:53:14 +02:00
Monty Taylor
202cce830e Add new non-npm specific javascript jobs
We have a bunch of jobs that are built around the npm role, but
for projects using yarn, that can lead to ignoring yarn.lock.

For projects with a yarn.lock, we can assume the user wants to
use yarn. Make a new js-package-manager role that can detect
if that's the case and otherwise use npm. Make an js_build_tool
parameter that allows the user to override that auto-detection.

Make a whole new suite of jobs that do this behavior, do not have
npm in their name, and default to the latest node LTS, version 14.

Don't install yarn if we're not going to use yarn. Also allow people
who want to use yarn but don't have a yarn.lock to override
js_build_tool everywhere we do that logic.

Mark the old jobs deprecated.

Shift the npm and yarn roles to use the new js-package-manager role
with defaults set.

Change-Id: I8013228ca05607a69f390a9bb75991fc6543f865
2020-05-12 07:50:19 +02:00
Tristan Cacqueray
4903ecd30b dhall-diff: add new job
This change adds a new dhall-diff job to verify generated config
is idempotent.

Change-Id: I96a335dc78c4fa74564b854997433e5be0b5e633
2020-05-03 19:06:37 +00:00
Zuul
096b1218c3 Merge "haskell-stack-test: add haskell tool stack test" 2020-05-01 16:09:58 +00:00
Zuul
43ad1fb400 Merge "Add Bazel build and ensure roles" 2020-05-01 15:39:41 +00:00
Tristan Cacqueray
cf64b73f78 haskell-stack-test: add haskell tool stack test
This change adds a new job to test stack based haskell project.

Change-Id: Ie3492097b351eec743d98e9ca8bc693c00c11a78
2020-05-01 15:35:33 +00:00
Mohammed Naser
e4345c217e ensure-package-repositories: Add role
This role allows users to supply a common variable and it will
contain all the logic to configure the correct repository based
on their operating system.

Co-Authored-By: Andreas Jaeger <aj@suse.com>
Change-Id: I4a3ab5f099c2e4671b78c751d8f0e8ccf2304afa
2020-05-01 15:49:21 +02:00
Paul Albertella
ad7297199d Add Bazel build and ensure roles
* Build role requires build target to be specified via a variable
* Ensure role checks for a specific version of Bazel and downloads
  and installs if missing; defaults to v3.1.0 and downloading from
  bazelbuild Github, but these can be overrriden using variables
* Also installs dependencies for Bazel; only supports Ubuntu/Debian
  at present, but includes scope to extend for other platforms
* Includes test playbook and job

Change-Id: I83f198aaf20c2b3664bea6fc05edd3b4fca13a4f
2020-04-30 18:05:41 +01:00
Zuul
d775467078 Merge "Add ensure-virtualenv" 2020-04-28 20:51:51 +00:00
Ian Wienand
08bbd16c52 Add ensure-virtualenv
This is part of the efforts to remove pip-and-virtualenv from our base
images [1].

There are some users who specifically require the virtualenv command
(perhaps, like dib, they have some code that uses the activate_this.py
mechanisms it provides wich venv does not).

This installs the virtualenv package for the currently running
distribution.

One of the main maintenance issues of pip-and-virtualenv is that tried
to ensure that "virtualenv" created a Python 2 environment always by
default.  Now that we have Python 3 only distributions like current
Fedora, this is not something we can continue to do (even if we wanted
to, which we don't).

What owns virtualenv and what it produces in our heterogeneous
environment is messy, and I think the best we can do is document it as
done here.

[1] https://docs.opendev.org/opendev/infra-specs/latest/specs/cleanup-test-node-python.html

Change-Id: I97d8bfb970ed2b5aaa02a0813899014c94994066
2020-04-28 06:09:54 +00:00
Tristan Cacqueray
2e82d88589 hlint: add haskell source code suggestions job
This change adds a new job to run hlint on haskell source files.

Change-Id: Ibf76c5552acecd68dfc56c4f31d8045ca5b233f1
2020-04-26 12:05:12 +00:00
Tristan Cacqueray
d92b39cae2 cabal-test: add initial haskell job
This change adds a new job to test cabal based haskell project.

Change-Id: Idb24eb5f8f9bd1409bae07bf4da415f1d05568e4
2020-04-24 13:12:13 +00:00
Ian Wienand
9e92459592 Document output variables
Several roles are setting facts that are expected to be consumed by
roles following them; generally things like the path to installed
tools, etc.

Add a section to policy about this behaviour in general, noting it
should use the cachable flags to persist the values.

Add specific documentation notes in an "Output Variable" section for
roles currently implementing this behaviour.

Change-Id: I05657fec198176c7d7345e84293f4902c81fa30c
2020-04-17 10:45:15 +02:00
Zuul
21928eab52 Merge "Adds roles to install and run hashicorp packer" 2020-04-15 16:01:18 +00:00
Ian Wienand
5068744eff ensure-pip: Add role
This role is intended to ensure that the `pip:` module, and jobs that
may wish to use `pip` from the shell, have the necessary requirements.

It is intended as a partial replacement for the pip-and-virtualenv
element in diskimage-builder, which currently pre-installs pip on our
infra CI images during image build.

We wish to remain broady compatible with this element, but not
replicate some of the more problematic areas of its implementation.

By default, this installs the system packages for pip and setuptools
(the latter being a requirement of the Ansible pip module, which
imports it directly, despite pip itself not requiring it).

In this role, we ensure the libraries for the currently running
ansible_python_interpreter version are installed.

There is provision to provide a flag to install the packages directly
from upstream via get-pip.py, although this is not recommended.

Story: #2007386
Task: #39309

Change-Id: Iac2d518a66caf1b801273225f75a0a748412903c
2020-04-14 10:20:33 +10:00
vass
32a3ce3f5a Adds roles to install and run hashicorp packer
Change-Id: I82274ef59eda00a735579082bbd37b64c0599c7d
2020-04-13 16:44:32 +02:00
vass
6dabffe402 Rename install-go to ensure-go for consistency
The old role will be kept and include ensure-go for backwards-compatability.

Change-Id: I4e6345563817057a6c47c124b9ec8b0c62b057d6
2020-04-06 21:10:11 +02:00
vass
c0b358fedb Rename install-if-python to ensure-if-python for consistency
The old role will be kept and include ensure-if-python for backwards-compatability.

Change-Id: Ief86799bcd91358d3ce8b6badda272a26ee04002
2020-04-04 15:41:29 -04:00
vass
16fe7bbd3f Rename install-pdk-dependencies to ensure-pdk-dependencies for consistency
The old role will be kept and include ensure-pdk-dependencies for backwards-compatability.

Change-Id: Ia269db36655685fe3f4dc54c60ae4bc09a913f52
2020-04-04 15:41:29 -04:00
vass
0b4ed70366 Rename install-yarn to ensure-yarn for consistency
The old role will be kept and include ensure-yarn for backwards-compatability.

Change-Id: I7631c7e588bc953d6e3b22759abe66a73ffca728
2020-04-04 15:41:29 -04:00
vass
ac72b44911 Rename install-nodejs to ensure-nodejs for consistency
The old role will be kept and include ensure-nodejs for backwards-compatability.

Change-Id: Ib4cf74d6c1e8774f6fc37d6a30f690eb8f9f3d08
2020-04-04 15:41:29 -04:00
vass
ff44b6cd4e Rename install-javascript-packages to ensure-javascript-packages for consistency
The old role will be kept and include ensure-javascript-packages for backwards-compatability.

Change-Id: Ibb896edf0652a19be13018b4abddb4a81c51f07b
2020-04-04 15:41:29 -04:00
vass
84c06a37c4 Rename install-devstack to ensure-devstack for consistency
The old role will be kept and include ensure-devstack for backwards-compatability.

Change-Id: Ia5fe33649a66a24ce6bb9de3cccbd5ef612dfb2f
2020-04-04 15:41:29 -04:00
vass
2cb9f249a8 Rename install-podman to ensure-podman for consistency
The old role will be kept and include ensure-podman for backwards-compatability.

Change-Id: I21cfdfef4a88f66f51d139abed04f70b3228a22d
2020-04-04 15:41:29 -04:00
vass
276a5e919a Rename install-openshift to ensure-openshift for consistency
The old role will be kept and include ensure-openshift for backwards-compatability.

Change-Id: I6ff81fbd04f25207c87834b36e4d636eb9d44a93
2020-04-04 15:41:29 -04:00
vass
fcf3202ba4 Rename install-kubernetes to ensure-kubernetes for consistency
The old role will be kept and include ensure-kubernetes for backwards compatability.

Change-Id: Ic0c18a833f4a07e123f8313673bd9c864d7d2919
2020-04-04 15:41:29 -04:00
vass
c80b593fb6 Rename install-docker to ensure-docker for consistency
The old role will be kept and include ensure-docker for backwards compatability.

Change-Id: Icb76a954b04631c405b09eb54343fd852a511b83
2020-04-04 15:41:29 -04:00
Zuul
b6b07fb2d8 Merge "Add phoronix-test-suite job" 2020-04-03 19:46:12 +00:00
Tristan Cacqueray
59c5820fa6 Add phoronix-test-suite job
This change adds new jobs to run and combine phoronix test suite results.

Co-Authored-By: Nicolas Hicher <nhicher@redhat.com>
Change-Id: If94cdbd91c8d80393a71cd5c36690563de9edcd6
2020-04-03 13:03:07 +00:00
Zuul
3e38894cc5 Merge "golangci-lint: add job" 2020-04-01 23:56:18 +00:00
Mohammed Naser
23236c12fa golangci-lint: add job
This patch adds a roles and jobs to run golangci-lint against a
a Golang project.  It's a very popular tool for linting go code.

It also adds a simple framework which allows us to create dynamic
tests for file comments by defining a simple YAML file.

Change-Id: Ic8358541adaf7c3279383f0279cd3da7b446a6e0
2020-04-01 17:30:55 -04:00
Ian Wienand
ca2ee69e60 local-log-download : role with script to download all log files
This is an alternative to I98c80f657f38c5e1ed5f28e5d36988a3429ad1f8
which does not modify the upload script, but rather queries the API
and manifest for what to download.

The script is a hybrid of python and bash to not implement json API
parsing badly in bash, but not replicate curl badly in python either.
The script sanity checks for dependencies, which are considered pretty
standard for any developer who would be interested in downloading logs
like this.

The role writes out the script with the correct build values coded
into it, so a potential user just has to run it without any arguments
or modification.

Change-Id: Ic33732adbfd3210191bf4976c3ee316cfc50568e
2020-04-01 07:41:26 +11:00
olesandr kozachenko
c20150566a add role for collecting the kubernetes pod&kubelet logs
Change-Id: I73a0cf471b9e7a8b924655e22e1f7dde069df5c7
2020-03-23 22:28:02 +02:00
Andreas Jaeger
78e91ef60c Keep doc/source/roles.rst sorted
The entries were sorted initially, resort them and add a comment for it.

Change-Id: I3f5c126360c52d6d2bb39e27fb1c600e69cd5524
2020-03-15 09:29:42 +01:00
Mohammed Naser
44608e16e7 wait-for-pods: Wait for all pods to become Ready
This refactors the code inside helm-template which waits for all
pods to become Ready to move inside it's own seperate role instead
which can be used for other tasks.

Change-Id: Ibb234c46c49fe656cf918a7a4af115e0d26f23f0
2020-03-14 14:14:43 -04:00
Zuul
59561f5a51 Merge "Remove deprecated upload-afs role" 2020-02-24 18:27:11 +00:00
Monty Taylor
2e33459b3e Add pause-buildset-registry role
This is a simple utility that provides the zuul_return functionality
for repos that need it encapsulating the existence of zuul_return.

For instance, system-config wanted to start doing this in a playbook
but linting the playbooks became super hard because of the new need
to have zuul installed for the zuul_return library. Just keep it here
in the family.

Change-Id: I6035009455f6434c2d6cfd976c7c6f414bdc2ae7
2020-02-24 17:17:12 +00:00
James E. Blair
bfb3507c5b Add ensure-java role
Change-Id: Ibaf45bd09a47d55ce21af34763563c20f9396cd2
2020-02-21 06:55:47 -06:00
James E. Blair
d0591ad98f Add ensure-bazelisk role
This adds a role which installs bazelisk if it is not already
present.  It also adds a test job which exercises the role; the
job will be run on any changes to the new role.

Change-Id: I69b28b58338b46ec3fea696f9d97be4271ed6ae4
2020-02-21 06:55:44 -06:00