We previously auto updated nodepool builders but not launchers when new
container images were present. This created confusion over what versions
of nodepool opendev is running. Use the same behavior for both services
now and auto restart them both.
There is a small chance that we can pull in an update that breaks things
so we run serially to avoid the most egregious instances of this
scenario.
Change-Id: Ifc3ca375553527f9a72e4bb1bdb617523a3f269e
This is a new config option for Gerrit 3.5. While it defaults to true we
set it explicitly to true to avoid any changes in behavior should that
default change eventually with newer Gerrit. They note this is expensive
to calculate, but our users rely on it and it hasn't caused us problems
yet. We can always explicitly disable it in the future if that becomes
necessary.
Change-Id: Idc002810de2d848af043978894ef9dc194ac5b6a
Add released Fedora 36 to the mirror. Traditionally we have kept two
releases (prior and current) around; but depending on what is broken
often we drop the prior release earlier if it is not worth fixing;
this is what happened with F34. Ergo this is adding 36 and leaving
35, for now.
Change-Id: I9864666be0a6e32edc730b736f81d8883411bcb2
This updates the gerrit configuration to deploy 3.5 in production.
For details of the upgrade process see:
https://etherpad.opendev.org/p/gerrit-upgrade-3.5
Change-Id: I50c9c444ef9f798c97e5ba3dd426cc4d1f9446c1
As part of the Gerrit 3.5 upgrade we are also upgrading the reviewdb
to the latest mariadb LTS. This should be merged after the update
process; see
https://etherpad.opendev.org/p/gerrit-upgrade-3.5
Change-Id: Ie30c84eeb003ee86a7a66e0c1c5fd7f95ddf3f5f
Previously the merger docker-compose restart value was set to always.
This caused the merger to immediately restart after asking it to
gracefully stop and our check for the merger stopping:
docker-compose ps -q | xargs docker wait
never saw it as being stopped.
Make the mergers match executors and restart only on failure. This
should allow us to gracefully stop the mergers with intention and detect
they are stopped for maintenance purposes.
Change-Id: Ia8d12fbf6a45e4ca85174ccafd18b5d2351c26c1
This serves two purposes. The first is that not all packages are updated
by unattended-upgrades beacuse it may not be safe to upgrade packages
while services are running. We should be safe in this situation because
we've gracefully stopped services and can proceed with package updates.
The other is unattended upgrades runs daily which means we could end up
almost 24 hours out of date prior to rebooting. This ensures we have the
latest and greatest packages installed just prior to rebooting.
Change-Id: Id351b5478e925ed1b4fbb6b3e27f2c0b6af8b897
This handles rolling the mergers and executors, but not yet
the schedulers.
Also, it does the executors in complete batches of 6, but could be
improved to stop 6 and then do each of the next as the first ones
complete.
Change-Id: I2dca104194c2f129b68dcef7721d7d08cb987c46
3.4.5 is a fairly minor update. Some bugs are fixed and jgit is updated.
3.4.5 release notes:
https://www.gerritcodereview.com/3.4.html#345
3.5.2 is a bigger update and important adds support for being able to
upgrade to 3.6.0 later. There is a new copy-approvals command that must
be run offline on 3.5.2 before upgrading to 3.6.0. This copies approvals
in the notedb in a way that 3.6.0 can handle them apparently. The
release notes indicate this may take some time to run. We don't need to
run it now though and instead need to make note of it when we prepare
for the 3.6.0 upgrade.
3.5.2 release notes:
https://www.gerritcodereview.com/3.5.html#352
For now don't overthink things and instead just get up to date with our
images.
Change-Id: I837c2cbb09e9a4ff934973f6fc115142d459ae0f
This appears to be a straightforward bug fix release according to the
release notes:
https://github.com/go-gitea/gitea/blob/v1.16.8/CHANGELOG.md
No template change between v1.16.7 and v1.16.8 according to git.
Change-Id: I0b9bb2f15beb7d3b1541c02e6e96601d25449e33
In our onging quest to find a stable upstream source, switch this to
the facebook mirror which seems to have been working well enough for
centos.
While we're here, reduce the frequency a bit as there's no need to
sync every two hours; it's a bit unfriendly to the remote end.
Change-Id: I50f38b58b3f0c0557fb1e2e0667a17a4f1d1dec6
It appears our opensuse mirror upstream has added an extra opensuse/ in
the path to the opensuse packages that we mirror.
Currently we fail with:
rsync: [sender] change_dir "/distribution/leap/15.2/repo" (in opensuse) failed: No such file or directory (2)
Browsing the repo via https we see the content exists at:
https://mirror.clarkson.edu/opensuse/opensuse/distribution/leap/15.2/repo/
which includes an extra opensuse/ path segment. Now it is possible that
https and rsync have different root dirs, but considering the current
error trying this seems reasonable.
Change-Id: I37aa701af409a58d228c22d664ad0f49a23a1fc5
If borg sees an ansible tmp file just before it is removed by ansible
then we can get errors of this form when running borg:
stat: [Errno 2] No such file or directory: '/root/.ansible/tmp/ansible-tmp-$IDENTIFIER'
This causes the backup run to report failure. Address this by adding
/root/.ansible/tmp/* to our borg backup excludes list.
Change-Id: Ie2c7081a4510959f6514b1cb7eb8facc4ac129fb
Apparently upstream stopped updating docker hub images semi recently. We
should pull from the quay.io location instead. As far as I can tell
docker.io/jboss/keycloak and quay.io/keycloak/keycloak orginate from
https://github.com/keycloak/keycloak-containers so this switch is in
theory fine. But I guess we'll find out.
Change-Id: I42c4a62468a5960332126fec90fcf6307c5b6674
Now that we're retiring the third-party-ci-announce mailing list,
which we never really used consistently anyway, just tell
third-party CI operators to make sure the E-mail address on their
account is current and reachable.
Change-Id: I6186149de25b06f2982702143a807de8bb01be73
This is the latest bug fix release. Upgrade to it to pull in those bug
fixes. You can see the full list of changes in the release notes:
https://github.com/go-gitea/gitea/blob/v1.16.7/CHANGELOG.md#1167---2022-05-02
I've checked template diffs between v1.16.6 and v1.16.7 and they are
empty.
Change-Id: I5c8cb5bf02ab5ca8f64da25f1384291921ba8bba
According to the docs [0] this shouldn't be necessary as performance
logging only happens if a performance tracing plugin is installed.
However according to this repo discuss thread [1] there is always a
dummy performanceLogging instance installed. This same thread identifies
this as a likely source for large increase in memory utilization by
Gerrit when upgrading to 3.5.
Let's explicitly disable this tracing due to the memory overhead in prep
for our 3.5 upgrade. We can always flip the setting if we install a
performance tracing plugin in our Gerrit.
[0] https://gerrit-review.googlesource.com/Documentation/config-gerrit.html#tracing
[1] https://groups.google.com/g/repo-discuss/c/QUD7_LsEVks/m/kBDEeam4AgAJ
Change-Id: Iff438695aa6488fb5886120121946494b1edf003
Because we proxy to Gerrit and set listenUrl with a proxy-http:// prefix
httpd.requestLog is disabled by default. We choose to explicitly enable
it here to add more logging to the Gerrit system even if this logging is
slightly less useful when behind a proxy. In particular this logging
will track memory utilization per request which we can use to benchmark
change query memory cost between 3.4 and 3.5.
Change-Id: Ia3ccf820ee0e5ca7d68bcc37da7004dea2ad7128
In preparation for retiring a number of mailing lists from
lists.openstack.org which have had no activity for over three years,
remove their configuration so our deployment automation won't
recreate them once they're gone. Also remove references to the
third-part-announce list in our documentation, since that's one of
the unused lists we're removing. See the announcement at
http://lists.openstack.org/pipermail/openstack-discuss/2022-February/027404.html
for details.
Change-Id: Ieedd8613363039d19d3ae47f1a83a38747419bdc
The status.openstack.org server is offline now that it no longer
hosts any working services. Remove all configuration for it in
preparation for retiring related Git repositories.
Also roll some related cleanup into this for the already retired
puppet-kibana module.
Change-Id: I3cfcc129983e3641dfbe55d5ecc208c554e97de4
