f0b77485ec
Zuul is publishing lovely container images, so we should go ahead and start using them. We can't use containers for zuul-executor because of the docker->bubblewrap->AFS issue, so install from pip there. Don't start any of the containers by default, which should let us safely roll this out and then do a rolling restart. For things (like web or mergers) where it's safe to do so, a followup change will swap the flag. Change-Id: I37dcce3a67477ad3b2c36f2fd3657af18bc25c40
28 lines
1.2 KiB
YAML
28 lines
1.2 KiB
YAML
# Minutes, hours, days, etc are not specified here because we are
|
|
# interested in running this *every minute*.
|
|
# This is a mean of backing up status.json periodically in order to provide
|
|
# a mean of restoring lost scheduler queues if need be.
|
|
# If the status.json is unavailable for download, no new files are created.
|
|
- name: Install cron for status backup
|
|
cron:
|
|
name: 'zuul-scheduler-status-{{ tenant }}'
|
|
state: present
|
|
user: root
|
|
job: |
|
|
timeout -k 5 10 curl https://zuul.opendev.org/api/tenant/{{ tenant }}/status -o /var/lib/zuul/backup/{{ tenant }}_status_$(date +\\%s).json 2>/dev/null
|
|
|
|
# Rotate backups and keep no more than 120 files -- or 2 hours worth of
|
|
# backup if Zuul has 100% uptime.
|
|
# We're not basing the rotation on time because the scheduler/web service
|
|
# could be down for an extended period of time.
|
|
# This is run hourly so technically up to ~3 hours worth of backups will
|
|
# be kept.
|
|
- name: Clean up old status backups
|
|
cron:
|
|
name: 'zuul-scheduler-status-prune-{{ tenant }}'
|
|
state: present
|
|
user: root
|
|
minute: 0
|
|
job: |
|
|
flock -n /var/run/{{ tenant }}_status_prune.lock ls -dt -1 /var/lib/zuul/backup/{{ tenant }}_* |sed -e '1,120d' |xargs rm -f
|