This migrates the afsmon script from puppet deploying on
mirror-update.openstack.org to ansible deploying on
mirror-update.opendev.org.
There is nothing particularly special and this just a straight install
with some minor dependencies. Since we have log publishing running on
the opendev.org server, we publish the update logs alongside the
others.
Change-Id: Ifa3b4d59f8d0fc23a4492e50348bab30766d5779
Fix the >> 2>&1 order from the cron job in
I62ae941e70c7d58e00bc663a50d52e79dfa5a684 so the logs gets captured.
Change-Id: I6414e95766c7a99d09cadfc853e50d5cb45dda34
This is a migration of the current periodic "vos release" script to
mirror-update.opendev.org.
The current script is deployed by puppet and run by a cron job on
afsdb01.dfw.openstack.org.
My initial motivation for this was wanting to better track our release
of these various volumes. With tarballs and releases moving to AFS
publishing, we are going to want to track the release process more
carefully.
Initially, I wanted to send timing statistics to graphite so we could
build a dashboard and track the release times of all volumes. Because
this requires an additional libraries and since we are deprecating
puppet, further development there is unappealing and it would better
live in ansible.
Since I6c96f89c6f113362e6085febca70d58176f678e7 we have the ability to
call "vos release" with "-localauth" permissions via ssh on
mirror-update; this avoids various timeout issues (see the changelog
comment there for more details). So we do not need to run this script
directly on the afsdb server.
We are alreadying publishing mirror update logs from mirror-update,
and it would be good to also publish these release logs so anyone can
see if there are problems.
All this points to mirror-update.opendev.org being a good future home
for this script.
The script has been refactored some to
- have a no-op mode
- send timing stats for each volume release
- call "vos release" via the ssh mecahnism we created
- use an advisory lock to avoid running over itself
It runs from a virtualenv and it's logs are published via the same
mechanism as the mirror logs (slightly misnamed now).
Note this script is currently a no-op to test the deployment, running
and log publishing. A follow-up will disable the old job and make
this active.
Change-Id: I62ae941e70c7d58e00bc663a50d52e79dfa5a684
We remove cloud-init but we don't remove the stuff it sucks in.
Run autoremove to take care of that.
Change-Id: I6530d7444197ec763d3695020200c411aed545b4
This adds a mirror to the new airship citycloud region. Add the host to
the inventory and add necessary host vars for LE setup.
Depends-On: https://review.opendev.org/706573
Change-Id: I33cefe914911b4f5ce5e09e0329ba48e039ede64
A bunch of the sites we've started managing LE certs for are not
getting their expirations checked, so fix that. In particular, sites
recently moved off the multi-domain SAN cert for the old
static.openstack.org server (omitted the logs site as it's
deprecated), and many of the rebuilt CI mirrors (with the exception
of mirror01.gra1.ovh.opendev.org which is presently in a SHUTOFF
state for unknown reasons). Also add graphite which was previously
missed, and review-dev because we can now that it's no longer
sporting snakeoil.
When this merges, we're also going to start getting alerts for an
expired cert on mirror.gra1.ovh.opendev.org, unless someone gets a
chance to look into it first.
Change-Id: I98a98e0d2ff081c51c33d980274f3ee8c0266802
Due to persistent, unresolved network issues between the London and US
cloud (that don't appear to happen the other way), we have decided on
a hard Brexit for nb03.o.o and started a new server in the US cloud :)
Change-Id: I6557a9f272351578216bc525b6ddaffcf625f9f3
We have been running out of disk recently with some indexes requiring
more than 400GB of space per index replica. Actual disk space
requirements are double that as we run with a replica. On top of that
the idea is that 5 of 6 elasticsearch nodes have enough space for all
our data so that we are resilient to losing a node.
Napkin math:
400 * 10 * 2 = ~8TB of disk
400 * 7 * 2 = ~5.6TB of disk
Each of the six ES nodes has 1TB of disk allocated to ES so 5.6TB should
get us just under the limit. Then for handling a node outage weekends
tend to not have as many records so our actual usage should be a little
lower.
Change-Id: Ie677bd47a9886870bc83876d2407742133299861
Add these hosts to static.opendev.org, serving from AFS. Note that
tarballs.openstack.org just redirects to static.opendev.org/openstack.
This should have no effect currently, it will only become live when we
switch DNS.
For more details see the thread at:
http://lists.openstack.org/pipermail/openstack-infra/2020-January/006584.html
Change-Id: Ie56fac17ffaa91ee55be986de636485a58125a02
Add a new review-dev server on the opendev domain with LE support
enabled.
Depends-On: https://review.opendev.org/705661
Change-Id: Ie32124cd617e9986602301f230e83bb138524fdf
This is a new cloud provided via citycloud that will add resources
capable of running Airship jobs. The goal is to use this as a stepping
stone to having Airship jobs run on our generic CI resources. This cloud
will provide both generic and larger resources to support this.
Change-Id: I63fd9023bc11f1382424c8906dc306cee5b3f58d
This appears to be the same thing we saw in Fedora with
Id24196791f80cd99fe8a330fb2c7c6d893fc9995, somehow upstream
directories have started acquiring a setgid bit, which breaks AFS
mirroring
rsync: failed to set permissions on "/afs/.openstack.org/mirror/centos/8/AppStream/aarch64/os": Permission denied (13)
and when we look
chmod("AppStream/aarch64/os/Packages", 02755) = -1 EACCES (Permission denied)
Drop the "-p" so we don't try and replicate these permissions.
Change-Id: Ib5db052cdd23e39aecbeead15cf08d4bd7fcab38
wiki, status, and single node ci should all run on xenial now. Switch
their testing to xenial from trusty.
Change-Id: I3a0c2faa47f2ec17809e3845c7226173188def63
yui-compressor is not happy with flot's jquery.flot.js file. These files
are actually pretty small especially when compared to our input json
data. Lets just serve them as is.
Depends-On: https://review.opendev.org/704716
Change-Id: Ibfd081bb73a6c352798a7822ab781c972ace4bc3
Our control plane servers generally have large ephemeral storage
attached at /opt; for many uses this is enough space that we don't
need to add extra cinder volumes for a reasonable cache (as we usually
do on mirror nodes; but there we create large caches for both openafs
and httpd reverse proxy whose needs exceed even what we get from
ephemeral storage).
Add an option to set the cache location, and use /opt for our new
static01.opendev.org server.
Change-Id: I16eed1734a0a7e855e27105931a131ce4dbd0793
Add this host for serving content from AFS.
The
_acme-challenge.governance.openstack.org
_acme-challenge.security.openstack.org
CNAMES should be in place for creating the certificates (added with
Ie1b92f06b71aa6069fe831b26ba1cc272ce4562c).
Also add a cert for the base server (static.opendev.org) since we
added the DNS entries for it.
Change-Id: I55e0ac7487b02f9a816ac486ed01b73f82b391a5
Story: #2006598
Task: #37757
Depends-On: https://review.opendev.org/704469
Flot sources moved from the repo top level dir into the source dir.
Accomodate this when we minimize and copy those js files.
Change-Id: I3522271361fc43550ac1c6dc2a690c5cc5ce9c64
Rax APIs don't support newer identity v3 or volume v2/v3. Set identity
to v2 so that catalogs can be listed and volume to v1 so that volumes
can be listed.
Change-Id: I6dddf93fb2c7b1a73315629e4a983a2d5a0142cc
Zuul is updating the default auth type for gerrit connections to better
reflect current Gerrit's expectations. We need to force digest instead
of basic auth in order to accomodate our older gerrit install.
Change-Id: I6ec64f0625abe0c9e3871a5d1942a35e1a58177c
All our AFS release roles use "kinit" for authentication. The only
scripts using k5start are the mirror scripts, but since that doesn't
run on CentOS we don't need it there.
This avoids us having to use EPEL or, on 8, an unsupported build.
Anything needing to be portable should use kinit from now on.
Change-Id: I6323cb835cedf9974cf8d96faa7eb55b8aaafd9a
There was an issue in pip that prevented correctly caching locally
built wheels [1]. This has been fixed in recent pip versions so
upgrade pip in both images so image caching works correctly. This is
needed to unbreak nodepool images that fail to install the locally
built netifaces package.
[1] https://github.com/pypa/pip/issues/6852
Change-Id: Ibbe12bcc53253a80d0bafa3d09a20c49a3a2b784
As a follow-on to Ie37abb4fd3eb3342b66ade52ab65024c420d7264 remove the
linaro credentials that were related to the (now removed) linaro-cn1
cloud.
Change-Id: Ia1e8dd3732164708c2e9fd82509e350829c438ba
We're retiring ubuntu-trusty and thus do not need instructions on
uploading these images anymore, remove the openstackci-images section.
Change-Id: I2b1491836f29fa72bc6eda62e427084ac43b5e1a
