106 Commits

Author SHA1 Message Date
Zuul
096b1218c3 Merge "haskell-stack-test: add haskell tool stack test" 2020-05-01 16:09:58 +00:00
Zuul
43ad1fb400 Merge "Add Bazel build and ensure roles" 2020-05-01 15:39:41 +00:00
Tristan Cacqueray
cf64b73f78 haskell-stack-test: add haskell tool stack test
This change adds a new job to test stack based haskell project.

Change-Id: Ie3492097b351eec743d98e9ca8bc693c00c11a78
2020-05-01 15:35:33 +00:00
Mohammed Naser
e4345c217e ensure-package-repositories: Add role
This role allows users to supply a common variable and it will
contain all the logic to configure the correct repository based
on their operating system.

Co-Authored-By: Andreas Jaeger <aj@suse.com>
Change-Id: I4a3ab5f099c2e4671b78c751d8f0e8ccf2304afa
2020-05-01 15:49:21 +02:00
Paul Albertella
ad7297199d Add Bazel build and ensure roles
* Build role requires build target to be specified via a variable
* Ensure role checks for a specific version of Bazel and downloads
  and installs if missing; defaults to v3.1.0 and downloading from
  bazelbuild Github, but these can be overrriden using variables
* Also installs dependencies for Bazel; only supports Ubuntu/Debian
  at present, but includes scope to extend for other platforms
* Includes test playbook and job

Change-Id: I83f198aaf20c2b3664bea6fc05edd3b4fca13a4f
2020-04-30 18:05:41 +01:00
Zuul
d775467078 Merge "Add ensure-virtualenv" 2020-04-28 20:51:51 +00:00
Ian Wienand
08bbd16c52 Add ensure-virtualenv
This is part of the efforts to remove pip-and-virtualenv from our base
images [1].

There are some users who specifically require the virtualenv command
(perhaps, like dib, they have some code that uses the activate_this.py
mechanisms it provides wich venv does not).

This installs the virtualenv package for the currently running
distribution.

One of the main maintenance issues of pip-and-virtualenv is that tried
to ensure that "virtualenv" created a Python 2 environment always by
default.  Now that we have Python 3 only distributions like current
Fedora, this is not something we can continue to do (even if we wanted
to, which we don't).

What owns virtualenv and what it produces in our heterogeneous
environment is messy, and I think the best we can do is document it as
done here.

[1] https://docs.opendev.org/opendev/infra-specs/latest/specs/cleanup-test-node-python.html

Change-Id: I97d8bfb970ed2b5aaa02a0813899014c94994066
2020-04-28 06:09:54 +00:00
Tristan Cacqueray
2e82d88589 hlint: add haskell source code suggestions job
This change adds a new job to run hlint on haskell source files.

Change-Id: Ibf76c5552acecd68dfc56c4f31d8045ca5b233f1
2020-04-26 12:05:12 +00:00
Tristan Cacqueray
d92b39cae2 cabal-test: add initial haskell job
This change adds a new job to test cabal based haskell project.

Change-Id: Idb24eb5f8f9bd1409bae07bf4da415f1d05568e4
2020-04-24 13:12:13 +00:00
Ian Wienand
9e92459592 Document output variables
Several roles are setting facts that are expected to be consumed by
roles following them; generally things like the path to installed
tools, etc.

Add a section to policy about this behaviour in general, noting it
should use the cachable flags to persist the values.

Add specific documentation notes in an "Output Variable" section for
roles currently implementing this behaviour.

Change-Id: I05657fec198176c7d7345e84293f4902c81fa30c
2020-04-17 10:45:15 +02:00
Zuul
21928eab52 Merge "Adds roles to install and run hashicorp packer" 2020-04-15 16:01:18 +00:00
Ian Wienand
5068744eff ensure-pip: Add role
This role is intended to ensure that the `pip:` module, and jobs that
may wish to use `pip` from the shell, have the necessary requirements.

It is intended as a partial replacement for the pip-and-virtualenv
element in diskimage-builder, which currently pre-installs pip on our
infra CI images during image build.

We wish to remain broady compatible with this element, but not
replicate some of the more problematic areas of its implementation.

By default, this installs the system packages for pip and setuptools
(the latter being a requirement of the Ansible pip module, which
imports it directly, despite pip itself not requiring it).

In this role, we ensure the libraries for the currently running
ansible_python_interpreter version are installed.

There is provision to provide a flag to install the packages directly
from upstream via get-pip.py, although this is not recommended.

Story: #2007386
Task: #39309

Change-Id: Iac2d518a66caf1b801273225f75a0a748412903c
2020-04-14 10:20:33 +10:00
vass
32a3ce3f5a Adds roles to install and run hashicorp packer
Change-Id: I82274ef59eda00a735579082bbd37b64c0599c7d
2020-04-13 16:44:32 +02:00
vass
6dabffe402 Rename install-go to ensure-go for consistency
The old role will be kept and include ensure-go for backwards-compatability.

Change-Id: I4e6345563817057a6c47c124b9ec8b0c62b057d6
2020-04-06 21:10:11 +02:00
vass
c0b358fedb Rename install-if-python to ensure-if-python for consistency
The old role will be kept and include ensure-if-python for backwards-compatability.

Change-Id: Ief86799bcd91358d3ce8b6badda272a26ee04002
2020-04-04 15:41:29 -04:00
vass
16fe7bbd3f Rename install-pdk-dependencies to ensure-pdk-dependencies for consistency
The old role will be kept and include ensure-pdk-dependencies for backwards-compatability.

Change-Id: Ia269db36655685fe3f4dc54c60ae4bc09a913f52
2020-04-04 15:41:29 -04:00
vass
0b4ed70366 Rename install-yarn to ensure-yarn for consistency
The old role will be kept and include ensure-yarn for backwards-compatability.

Change-Id: I7631c7e588bc953d6e3b22759abe66a73ffca728
2020-04-04 15:41:29 -04:00
vass
ac72b44911 Rename install-nodejs to ensure-nodejs for consistency
The old role will be kept and include ensure-nodejs for backwards-compatability.

Change-Id: Ib4cf74d6c1e8774f6fc37d6a30f690eb8f9f3d08
2020-04-04 15:41:29 -04:00
vass
ff44b6cd4e Rename install-javascript-packages to ensure-javascript-packages for consistency
The old role will be kept and include ensure-javascript-packages for backwards-compatability.

Change-Id: Ibb896edf0652a19be13018b4abddb4a81c51f07b
2020-04-04 15:41:29 -04:00
vass
84c06a37c4 Rename install-devstack to ensure-devstack for consistency
The old role will be kept and include ensure-devstack for backwards-compatability.

Change-Id: Ia5fe33649a66a24ce6bb9de3cccbd5ef612dfb2f
2020-04-04 15:41:29 -04:00
vass
2cb9f249a8 Rename install-podman to ensure-podman for consistency
The old role will be kept and include ensure-podman for backwards-compatability.

Change-Id: I21cfdfef4a88f66f51d139abed04f70b3228a22d
2020-04-04 15:41:29 -04:00
vass
276a5e919a Rename install-openshift to ensure-openshift for consistency
The old role will be kept and include ensure-openshift for backwards-compatability.

Change-Id: I6ff81fbd04f25207c87834b36e4d636eb9d44a93
2020-04-04 15:41:29 -04:00
vass
fcf3202ba4 Rename install-kubernetes to ensure-kubernetes for consistency
The old role will be kept and include ensure-kubernetes for backwards compatability.

Change-Id: Ic0c18a833f4a07e123f8313673bd9c864d7d2919
2020-04-04 15:41:29 -04:00
vass
c80b593fb6 Rename install-docker to ensure-docker for consistency
The old role will be kept and include ensure-docker for backwards compatability.

Change-Id: Icb76a954b04631c405b09eb54343fd852a511b83
2020-04-04 15:41:29 -04:00
Zuul
b6b07fb2d8 Merge "Add phoronix-test-suite job" 2020-04-03 19:46:12 +00:00
Tristan Cacqueray
59c5820fa6 Add phoronix-test-suite job
This change adds new jobs to run and combine phoronix test suite results.

Co-Authored-By: Nicolas Hicher <nhicher@redhat.com>
Change-Id: If94cdbd91c8d80393a71cd5c36690563de9edcd6
2020-04-03 13:03:07 +00:00
Zuul
3e38894cc5 Merge "golangci-lint: add job" 2020-04-01 23:56:18 +00:00
Mohammed Naser
23236c12fa golangci-lint: add job
This patch adds a roles and jobs to run golangci-lint against a
a Golang project.  It's a very popular tool for linting go code.

It also adds a simple framework which allows us to create dynamic
tests for file comments by defining a simple YAML file.

Change-Id: Ic8358541adaf7c3279383f0279cd3da7b446a6e0
2020-04-01 17:30:55 -04:00
Ian Wienand
ca2ee69e60 local-log-download : role with script to download all log files
This is an alternative to I98c80f657f38c5e1ed5f28e5d36988a3429ad1f8
which does not modify the upload script, but rather queries the API
and manifest for what to download.

The script is a hybrid of python and bash to not implement json API
parsing badly in bash, but not replicate curl badly in python either.
The script sanity checks for dependencies, which are considered pretty
standard for any developer who would be interested in downloading logs
like this.

The role writes out the script with the correct build values coded
into it, so a potential user just has to run it without any arguments
or modification.

Change-Id: Ic33732adbfd3210191bf4976c3ee316cfc50568e
2020-04-01 07:41:26 +11:00
olesandr kozachenko
c20150566a add role for collecting the kubernetes pod&kubelet logs
Change-Id: I73a0cf471b9e7a8b924655e22e1f7dde069df5c7
2020-03-23 22:28:02 +02:00
Andreas Jaeger
78e91ef60c Keep doc/source/roles.rst sorted
The entries were sorted initially, resort them and add a comment for it.

Change-Id: I3f5c126360c52d6d2bb39e27fb1c600e69cd5524
2020-03-15 09:29:42 +01:00
Mohammed Naser
44608e16e7 wait-for-pods: Wait for all pods to become Ready
This refactors the code inside helm-template which waits for all
pods to become Ready to move inside it's own seperate role instead
which can be used for other tasks.

Change-Id: Ibb234c46c49fe656cf918a7a4af115e0d26f23f0
2020-03-14 14:14:43 -04:00
Zuul
59561f5a51 Merge "Remove deprecated upload-afs role" 2020-02-24 18:27:11 +00:00
Monty Taylor
2e33459b3e Add pause-buildset-registry role
This is a simple utility that provides the zuul_return functionality
for repos that need it encapsulating the existence of zuul_return.

For instance, system-config wanted to start doing this in a playbook
but linting the playbooks became super hard because of the new need
to have zuul installed for the zuul_return library. Just keep it here
in the family.

Change-Id: I6035009455f6434c2d6cfd976c7c6f414bdc2ae7
2020-02-24 17:17:12 +00:00
James E. Blair
bfb3507c5b Add ensure-java role
Change-Id: Ibaf45bd09a47d55ce21af34763563c20f9396cd2
2020-02-21 06:55:47 -06:00
James E. Blair
d0591ad98f Add ensure-bazelisk role
This adds a role which installs bazelisk if it is not already
present.  It also adds a test job which exercises the role; the
job will be run on any changes to the new role.

Change-Id: I69b28b58338b46ec3fea696f9d97be4271ed6ae4
2020-02-21 06:55:44 -06:00
Zuul
cc4552bb04 Merge "fetch-output-openshift: initial role" 2020-02-07 16:01:35 +00:00
Zuul
965f68c5a1 Merge "Add upload-logs-gcs role" 2020-02-05 15:34:22 +00:00
Zuul
f2207abd50 Merge "Use unique loop vars to avoid conflicts" 2020-02-05 05:43:16 +00:00
Clark Boylan
4ed66807a0 Use unique loop vars to avoid conflicts
We have to be careful about avoiding outer loop loop_var conflicts in
ansible. Because the zuul-jobs roles are meant to be reconsumed
elsewhere we should not use 'item' loopvars and instead set them to
something a bit more unique.

We use a zj_ prefix to try and be unique to this repo and document this
convention.

Change-Id: I20b9327a914890e9eafcb2b36dc8c23fb472bc8f
2020-02-04 12:23:36 -08:00
James E. Blair
b8ce30f3cb Add upload-logs-gcs role
This uploads logs to Google Cloud Storage.  It is based on
upload-logs-swift.

Change-Id: I4d83c3f4eb7f186271302cbb6ebae6b1f280785b
2020-02-03 16:02:15 -08:00
Ian Wienand
8c44ef9385 Remove deprecated upload-afs role
See I493d8829e3fd98e84f1b8f0e776e7ba41abf16c2

Change-Id: I445b15a42e6c1f37edd12d7f43cfd5a1d055b450
Depends-On: https://review.opendev.org/705370
Depends-On: https://review.opendev.org/705371
Depends-On: https://review.opendev.org/705372
2020-02-04 07:41:59 +11:00
Ian Wienand
cd258e5a1d upload-afs: rename to upload-afs-roots; add afs-upload-synchronize
The upload-afs role is actually fairly specialised for uploading docs
with its root-marker scheme.  As we're moving more parts of our
infrastructure to publish via AFS I think it's worth making this quite
explicit by renaming the upload-afs role to upload-afs-roots.  This
should be transparent as we call the renamed role, and once callers
have been migrated we can use the usual deprecation process.

Add a partner role, upload-afs-synchronize, which is more of a
straight copy with options to synchronize.  This will be suitable for
uploading tarball and release artifacts.  This is similar in concept
to the promotion job used by zuul [1], but in this case will be
focused on the extant publishing jobs in project-config which don't
currently use the artifact-reporting/promote process.

[1] https://opendev.org/opendev/base-jobs/src/branch/master/playbooks/artifacts/promote.yaml#L39

Change-Id: I493d8829e3fd98e84f1b8f0e776e7ba41abf16c2
Story: #2006598
Task: #38600
2020-02-04 07:41:00 +11:00
Zuul
a7c7d4fa16 Merge "Add a markdownlint job and role" 2020-01-31 17:05:16 +00:00
Tristan Cacqueray
67e2abda5b fetch-output-openshift: initial role
This change adds a new fetch-output-openshift role to be able to
fetch-output from pod.

Change-Id: I45d5115a10254432805a02a77e0ebb35d7fd2dd4
2020-01-30 20:00:14 +00:00
Clint Byrum
33580f22b0 Add a markdownlint job and role
This tool will ensure that your markdown files follow some best
practices.

Change-Id: I7bcc70443dbe5fa31e3cc1139d608834c00851b9
2020-01-27 12:23:24 +00:00
Zuul
5ea9bac2df Merge "Add prepare-workspace-openshift role" 2020-01-23 16:19:37 +00:00
Mohammed Naser
5833a6f8e3 apply-helm-charts: Job to apply Helm charts
This job deploys a Kubernetes cluster and then runs all the
Helm charts provided.

Change-Id: If6c4a6d5bce912f55549d16175554154c86dd25a
2020-01-10 17:31:38 +00:00
Mohammed Naser
8ae49163f3 helm-template: Add role to run 'helm template'
This role runs Helm template of a specific chart and then makes
sure that all resources become ready afterwards.

Change-Id: I62e160eac673d13bd1018b58fd36cae54e1482b4
2020-01-10 17:31:05 +00:00
Mohammed Naser
228033e78a collect-container-logs: add role
We use this in a few different places and it's really useful
to collect all the logs of all containers.

Change-Id: Idc46a47f444bf48cd040f4f9724f3a6ee8bc8f8e
2020-01-09 19:52:01 -05:00