16788 Commits

Author SHA1 Message Date
Zuul
4822993fec Merge "Put new borg hosts in cacti" 2021-02-21 22:43:31 +00:00
Zuul
1b2435c349 Merge "backups: remove all bup" 2021-02-21 22:41:41 +00:00
James E. Blair
b6cbb52447 Add pull tasks for nodepool/zuul
So we can stop/pull/start, move the pull tasks to their own files
and add a playbook that invokes them.

Change-Id: I4f351c1d28e5e4606e0a778e545a3a805525ac71
2021-02-19 15:42:40 -08:00
Clark Boylan
8fd2857a8e Build some new python base images
This change adds comments to the python-base and python-builder
dockerfiles to force a rebuild of these images. We do this periodically
to pull in updates.

Change-Id: I109a46603a74a376cc36fdfbd35734f6dc835abe
2021-02-19 11:10:17 -08:00
Zuul
464bb363e9 Merge "grafana: update to 7.4.2" 2021-02-19 06:03:21 +00:00
Zuul
9db55a55f3 Merge "borg-backup: send explicit email on backup failure" 2021-02-19 05:20:01 +00:00
Jeremy Stanley
4ec78fe2e6 Put new borg hosts in cacti
Add the two new borg hosts to cacti. Also remove the old bup server
which was still lurking there.

Change-Id: I2bf9e401f93b59ecef162db7020f97ba1498e027
2021-02-18 23:59:19 +00:00
Ian Wienand
7577439ff8 grafana: update to 7.4.2
This includes a fix for I216528a76307189d8d87bd2fcfeff95c6ceb53cc.
Now it's released we can be a bit more explicit about why we added the
workaround.

Change-Id: Ibaf1850549b5e7ec3622418b650bc5e59a289ab6
2021-02-19 09:54:31 +11:00
Ian Wienand
5a1b8ac179 grafana: take some screenshots during testing
Take some simple screenshots for basic validation of any new releases.

Change-Id: I52770032a6cc91d76da23194f58474f5ceeaed38
2021-02-17 10:43:26 +11:00
Zuul
7c30c3d668 Merge "grafana-container: update to latest tag" 2021-02-16 20:15:28 +00:00
Ian Wienand
39ffc685d6 backups: remove all bup
All hosts are now running thier backups via borg to servers in
vexxhost and rax.ord.

For reference, the servers being backed up at this time are:

 borg-ask01
 borg-ethercalc02
 borg-etherpad01
 borg-gitea01
 borg-lists
 borg-review-dev01
 borg-review01
 borg-storyboard01
 borg-translate01
 borg-wiki-update-test
 borg-zuul01

This removes the old bup backup hosts, the no-longer used ansible
roles for the bup backup server and client roles, and any remaining
bup related configuration.

For simplicity, we will remove any remaining bup cron jobs on the
above servers manually after this merges.

Change-Id: I32554ca857a81ae8a250ce082421a7ede460ea3c
2021-02-16 16:00:28 +11:00
Zuul
7a5041140b Merge "Cleanup refstack job dependencies" 2021-02-16 04:27:53 +00:00
Zuul
8360a7ceab Merge "Run gerrit 3.2 and 3.3 functional tests" 2021-02-16 04:27:46 +00:00
Ian Wienand
5ca69113fd borg-backup: send explicit email on backup failure
This sets a global BORG_UNDER_CRON=1 environment variable for
production hosts and makes the borg-backup script send an email if any
part of the backup job appears to fail (this avoids spamming ourselves
if we're testing backups, etc).

We should ideally never get this email, but if we do it's something we
want to investigate quickly.  There's nothing worse than thinking
backups are working when they aren't.

Change-Id: Ibb63f19817782c25a5929781b0f6342fe4c82cf0
2021-02-16 14:49:38 +11:00
Zuul
94fe3610e5 Merge "borg-backup-server: make sure to append verification logs" 2021-02-16 03:14:30 +00:00
Zuul
ebdd2144bf Merge "Trigger promote for gerrit 3.3 image" 2021-02-16 02:39:30 +00:00
Zuul
e04a13b3ba Merge "Correct path in mk-archives-index cronjob on lists" 2021-02-16 01:16:03 +00:00
Zuul
06a863c91c Merge "main site: add note on service-incident list" 2021-02-16 00:56:56 +00:00
Zuul
3202f9a19c Merge "Add mirror-update.opendev.org to Cacti" 2021-02-15 22:21:19 +00:00
Zuul
60b5f789ad Merge "Clean up ethercalc server replacement transition" 2021-02-15 22:20:10 +00:00
Zuul
4a6afe927f Merge "Adjust the example Etherpad API delete command" 2021-02-15 22:12:44 +00:00
Zuul
494579cf71 Merge "Polish the main opendev.org page" 2021-02-15 21:24:34 +00:00
Zuul
c3aeb3a1bb Merge "grafana: ensure snapshots api returns a 403" 2021-02-15 15:42:46 +00:00
Jeremy Stanley
f62ffae973 main site: add note on service-incident list
This is a private list to contact administrators that is suitable
for raising security concerns. I've stolen the wording from Ian's
similar I886f67d875abd09753511f6c33312cfc5eb62933 docs change.

Change-Id: I7eb094d9fc75494ab33e0b9133fb451724f96dad
Co-Authored-By: Ian Wienand <iwienand@redhat.com>
2021-02-15 14:56:33 +00:00
Ian Wienand
f4209757c2 docs: add note on service-incident list
This is a private list to contact administrators that is suitable for
raising security concerns.

Change-Id: I886f67d875abd09753511f6c33312cfc5eb62933
2021-02-15 06:26:18 +00:00
Ian Wienand
c7de005738 grafana: ensure snapshots api returns a 403
Change-Id: I216528a76307189d8d87bd2fcfeff95c6ceb53cc
2021-02-15 17:01:15 +11:00
Ian Wienand
ece90fb7f7 borg-backup-server: make sure to append verification logs
We don't want to overwrite every run, but rather append to the log
file.

Change-Id: I304caedecbf6a9552f314636ca82a543ef16a8b6
2021-02-15 14:45:03 +11:00
Ian Wienand
f92b6711b2 grafana-container: update to latest tag
Change-Id: I8b83e9c1b0189c28289821e2c599cd398f56bdd7
2021-02-15 13:17:37 +11:00
Jeremy Stanley
6d0c4b0b3b Update AFS group vars filenames
Ifa5f251fdfb8de737ad2ed96491d45294ce23a0c renamed the afs and afsdb
groups to afs-file-server and afs-db-server, but didn't update the
group files.

Previously the firewall rules were duplicated in the afs/afsdb group;
but now all afs servers are in the afs-server-common group.  Rename
afs.yaml->afs-server-common.yaml and remove the now unnecessary
afsdb.yaml.

Remove one of the old group vars files and rename the other to
afs-server-common so we can restore the udp ports they open in our
firewall rules.

Change-Id: I17dd0596660addf061ade31b4450bf040c01ffe8
2021-02-12 18:23:45 +11:00
Zuul
036ac31060 Merge "Refactor AFS groups" 2021-02-11 22:46:00 +00:00
Zuul
a326daed61 Merge "borg-backup: fix backup script failure match" 2021-02-11 22:06:31 +00:00
Ian Wienand
87ad89c3fe Trigger promote for gerrit 3.3 image
Ib7e7d7313e0827a40009df840119444611d74ca2 did not match the promote
job file filter, so the image wasn't promoted.  Bump to create a new
image build.

Change-Id: I31e8368ec309e62a9cf3607acfd5157aa5b6c61e
2021-02-12 09:04:21 +11:00
Zuul
486073f14d Merge "refstack: use external https for API" 2021-02-11 21:52:44 +00:00
Ian Wienand
ad1992955a borg-backup: fix backup script failure match
Fix a typo in the failure match, and log the error code in that case.

Change-Id: Ie17042237986d0bed58e95c271f868c735c724d2
2021-02-12 08:16:20 +11:00
Zuul
d9f59d8728 Merge "Build Gerrit 3.3 images" 2021-02-11 14:55:32 +00:00
Zuul
03f5e8e0de Merge "borg-backup-server: run a weekly backup verification" 2021-02-11 05:53:16 +00:00
Zuul
d8779a42da Merge "PTGBot is now openinfraptg on #openinfra-events" 2021-02-11 03:44:29 +00:00
Ian Wienand
312b9bec24 Refactor AFS groups
Both the filesevers and db servers have common key material deployed
by the openafs-server-config role.  Put both types of server in a new
group "afs-server-common" so we can define this key material in just
one group file on bridge.

Then separate out the two into afs-<file|db>-server groups for
consistent naming.

Rename afs-admin for consistent naming.

The service file is updated to reflect the new groups.

Change-Id: Ifa5f251fdfb8de737ad2ed96491d45294ce23a0c
2021-02-11 13:35:16 +11:00
Zuul
faf041540c Merge "Revert "Install older setuptools in puppet apply jobs"" 2021-02-11 01:33:50 +00:00
Zuul
f3eb16601b Merge "refstack: capture container logs to disk" 2021-02-11 01:33:29 +00:00
Zuul
0546700459 Merge "refstack: fix typo in role matcher" 2021-02-11 01:31:05 +00:00
Ian Wienand
32b48c81a2 refstack: use external https for API
Currently this variable is setting several URL's used in the config to
internal http links (port 8000).  This bubbles through to the UI which
then can't talk to the API.  Emperically, changing these values in the
container config and restarting it makes things work.  Update this
variable to make it talk to external https.

Change-Id: If61ec1e0383b98d34d092c55ca0095588487902a
2021-02-11 11:44:39 +11:00
Ian Wienand
0d01d941b1 borg-backup-server: run a weekly backup verification
This checks the backup archives and alerts us if anything seems wrong.
This will take a few hours, so we run once a week.

Change-Id: I832c0d29a37df94d4bf2704c59bb3f8d855c3cc8
2021-02-11 00:43:16 +00:00
Zuul
9578f393a2 Merge "openafs-<db|file>-server: fix role name" 2021-02-10 23:59:56 +00:00
Ian Wienand
56a31cb114 refstack: capture container logs to disk
We have setup rsyslogd/logrotate to handle anything with docker- tags
to be persisted to disk in /var/log/containers.  Set this up here so
we keep track of the mariadb and refstack logs.

Change-Id: I760cfeb7226f79986fbf9d7dbc5f899fc87a0cd1
2021-02-11 10:51:30 +11:00
Ian Wienand
533e6d43fa refstack: fix typo in role matcher
Change-Id: I61929708be87a28669606ac38abf478afd70fc51
2021-02-11 10:37:31 +11:00
Clark Boylan
2bb3dd797b Cleanup refstack job dependencies
We need to depend on the buildset registry as we are building this image
in a separate job. We also don't need to depend on the build job in
gate, we only need the upload job.

Change-Id: Ie7c2ed29c028f8c23d67ad38edbe04b12e22d026
2021-02-10 15:11:54 -08:00
Clark Boylan
9b90e192b1 Run gerrit 3.2 and 3.3 functional tests
This change splits our existing system-config-run-review job into two
jobs, one for gerrit 3.2 and another for 3.3. The biggest change is that
we use a var called zuul_test_gerrit_version to select which version we
want and that ends up in the fake group file written out by Zuul for the
nested ansible run. The nested ansible run will then populate the
docker-compose file with the appropriate version for us.

Change-Id: I00b52c0f4aa8df3ecface964007fcf5724887e5e
2021-02-10 15:10:46 -08:00
Clark Boylan
7320c8e6ed Build Gerrit 3.3 images
Gerrit 3.3 has released. Lets start building images for it so that we
can do testing when ready to start that.

We also add testing files to the list of things that trigger the 3.3
builds. Strictly this isn't necessary since the test will continue to
use 3.2 images until we upgrade to 3.3, but this helps us avoid
forgetting to do this when we do upgrade. Little extra jobs run today to
ensure we continue to run the right jobs tomorrow.

Change-Id: Ib7e7d7313e0827a40009df840119444611d74ca2
2021-02-10 15:07:19 -08:00
Thierry Carrez
75e15d765d PTGBot is now openinfraptg on #openinfra-events
Channel for PTG discussion was moved to #openinfra-events, and
the bot was renames "openinfraptg" to match the extended scope
of the event.

Change-Id: I52718358ddb4a199d24fc6da6e71f81c646da1f2
2021-02-10 14:33:55 +01:00