15238 Commits

Author SHA1 Message Date
Monty Taylor
7c86c4d067 Update pip3 role to install from get-pip.py
Our base images have pip pre-installed from get-pip. This means
that the installation of pip and virtualenv from distro packages
in the ansible is misleading.

Update the role to match reality.

Change-Id: I500b14f9f9df00b6e0c4f152f8b4c7faa1bb94d4
2019-12-17 08:13:34 -05:00
Monty Taylor
5de73f6c36 Use explicit image paths
To make it clear that docker hub is but one of many possible registries,
update our usage of FROM and image: lines to include docker.io in the
path.

There are a few other FROM lines for the gitea images which are handled
in a separate stack.

Change-Id: I6fafd5f659ad19de6951574afc9a6b6a4cf184df
2019-12-17 08:13:34 -05:00
Monty Taylor
c6c8ed75be Update bazel to version 1.2.0
Upstream has updated.

Change-Id: I07af5f101270b6ec735b6c7e29599d5dae0e2a86
2019-12-17 08:13:33 -05:00
Zuul
29019411eb Merge "Run a gerrit container on review-dev01" 2019-12-15 19:00:21 +00:00
Zuul
18ee97749c Merge "mirror: remove debug output of apache config" 2019-12-09 23:45:07 +00:00
Zuul
f360e4dc94 Merge "mirror jobs: copy acme.sh output" 2019-12-09 22:54:49 +00:00
Ian Wienand
cad5d6482d mirror: remove debug output of apache config
This isn't necessary in production and creates a lot of noise.

Change-Id: I38c9353554beca234e1da6dc74ba4a1f6e9742e7
2019-12-10 09:37:20 +11:00
Zuul
e494efcf90 Merge "Update gitea docs" 2019-12-05 19:23:50 +00:00
Zuul
5497d9365c Merge "Add some details on where to find things to homepage" 2019-12-05 09:54:53 +00:00
Zuul
34ee6621e0 Merge "bridge.o.o: update to latest Ansible" 2019-12-05 09:30:13 +00:00
Zuul
99a0fed381 Merge "Split bindep and wheel invocations" 2019-12-04 17:41:22 +00:00
Zuul
3e7b9e9736 Merge "python-builder: install sibling packages" 2019-12-04 17:34:01 +00:00
Zuul
93067cb679 Merge "Update gitea to v1.10.0" 2019-12-04 16:39:46 +00:00
Monty Taylor
b3e5f0cab0 Update gitea to v1.10.0
1.10 introduces a PASSWORD_COMPLEXITY setting with a default value
of lower,upper,digit,spec - which requires passwords to have an
upper, lower, digit and special character. Our example password does
not have this, so set the PASSWORD_COMPLEXITY setting. We could
alternately leave it at the default and ensure that our passwords
meet the spec.

The sshd_config file is templated now, so we can set the listen port
via env var.

Change-Id: I6e4b595eabb9c6885d78fff1109ea9f602e89ef7
2019-12-04 07:54:49 -06:00
Monty Taylor
5e12438e0d Split bindep and wheel invocations
We need to run bindep before installing git, because otherwise if
a project needs git in its bindep, it won't show up because it'll
be on the build host.

Split the function in two and call them before and after the git
installation.

Change-Id: I316b1bc643eb9293500b31e676361eec7060701d
2019-12-03 11:13:56 -06:00
Ian Wienand
bd66a7cb1b python-builder: install sibling packages
In the dependent change, the docker roles will add sibling packages to
the .zuul-siblings directory of the checked-out source.

Refactor the "assemble" script to handle this.  Essentially we build
the wheel for "." and then iterate over ZUUL_SIBLINGS subdirectories
(set in a --build-arg by the role in dependent change) to also build
the sibling packages.  Note we concatenate the bindep.txt files, so
that we end up with the complete package list required by the main
code and its dependencies.

"install-from-bindep" now installs all the wheels, using --force to
make sure we re-install the speculatively built packages.

This means that a single Dockerfile works under Zuul when
ZUUL_SIBLINGS is set, pointing to Zuul's checkouts; but it also works
stand-alone -- in this case ZUUL_SIBLINGS is empty and we just install
from upstream as usual.

Depends-On: https://review.opendev.org/696987
Change-Id: I4943ae723b06b0ad808e7c7f20788109e21aa8bf
2019-12-03 20:51:32 +11:00
Zuul
5cabb8ca07 Merge "Retire elections-committee, openstack-content MLs" 2019-12-02 19:21:20 +00:00
Zuul
7975a8648b Merge "Retire the Women-of-OpenStack list" 2019-12-02 19:19:42 +00:00
Ian Wienand
ce15e3bed9 mirror jobs: copy acme.sh output
We have seen failures issuing keys, but can't see the output of the
letsencrypt wrapper without capturing this logfile.  Add it.

Also, when we updated the mirror to "mirror01.openafs." (because we
have WIP for non-openafs kafs mirrors too) we didn't update the
host-vars match for the apache logs either.  Fix this.

Change-Id: I810a02d309f473e8c4aa0ce1612088aba7868c33
2019-11-27 10:46:34 +11:00
Clark Boylan
4ab6673092 Add necessary ansible vars for inap mirror LE
This was missed when adding the new inap mirror host to our inventory
and groups.

Change-Id: I02d7088ce1722f0a55fe6b17192fd462028aae5c
2019-11-26 13:52:51 -08:00
Clark Boylan
43515ef2a6 Replace inap mirror with bigger instance
We are replacing the inap mirror with an bigger instance. The reason for
this is our cinder volume throughput hasn't been quick enough and mgagne
says that we'll get the best performance via local disk. In order to
host the caches we have on local disk we need a bigger root device which
means a bigger flavor.

Change-Id: Id7e641e3b62400f4e1181ef6763a51b9d1e9068c
2019-11-26 10:08:47 -08:00
Ian Wienand
98ab53ee42 rsync mirrors: use localauth vos release
We switched Fedora to do vos release via ssh with localauth in
I56ecdb2511597197deeeadf51f50da7e02f56954 and it has been working.

Switch the rest of the update scripts.  There is an increasing amount
of common code, start a common functions.sh script where we can put
this.

Change-Id: I4ba6d64a84bb66e8686901b16010352de942f303
2019-11-26 13:40:15 +00:00
Andreas Jaeger
e474f5ce73 Switch openSUSE mirror
The former mirror has outdated content, use a mirror from
http://download.opensuse.org/update/leap/15.1/oss/repodata/repomd.xml.mirrorlist
that is in the US with prio 100 (highest) and current content.

Change-Id: I5649584d01f0370fb09217b00038992a69cf964c
2019-11-25 09:07:25 +01:00
Ian Wienand
3f9c737483 Fedora mirror update: use localauth release
Use the new vos_release user on the remote host to release the volume
via localauth, to avoid any timeouts.

Change-Id: I56ecdb2511597197deeeadf51f50da7e02f56954
Depends-On: https://review.opendev.org/#/c/695554/
2019-11-22 01:13:50 +00:00
Zuul
e1ab2fb9ff Merge "Add missing newline in vos_release.sudo" 2019-11-21 21:19:35 +00:00
Zuul
869600a547 Merge "Use a wildcard regex for storyboard-dev OAuth/CORS" 2019-11-21 19:43:52 +00:00
Zuul
02ef233529 Merge "gitea: Use 1.9.6" 2019-11-21 19:43:50 +00:00
Jeremy Stanley
f2cf00dc89 Add missing newline in vos_release.sudo
The sudoers parser really, really, *really* doesn't like it when the
last line of data in your file lacks a trailing newline. Add one so
sudo will work again on these servers.

Change-Id: I40fbb535faf5b41cc56c56f09f248eea398df4e0
2019-11-21 19:08:30 +00:00
Ian Wienand
a6a4e5dfe5 vos-release: fix sudo
If you read the man page

 # This will cause sudo to read and parse any files in the /etc/sudoers.d
 # directory that do not end in '~' or contain a '.' character.

I don't know why sudo doesn't like files with a ".", but remove it

Fix the syntax in this file which has too many spaces

The theory that specifying a command means you can have nologin as
shell is debunked; change the shell to /bin/bash

 root@mirror-update01:~# ssh -i ~/.ssh/id_vos_release vos_release@afs01.dfw.openstack.org vos
 This account is currently not available.

Don't use shortcuts for positional parameters, suggested by jaltmann
in If70b27cb974eb8c1bafec2b7ef86d4f5cba3c4c5.

After hand applying these fixes, I can log in and run the script as
expected.

Change-Id: I058aadaa5ca5c7b8e94b275c4b8d26e1e0688ce8
2019-11-21 17:51:18 +11:00
Ian Wienand
f57154f91b vos-release: have separate user
I was trying to simplify things by having a restricted shell script
run by root.  However, our base-setup called my bluff as we also need
to setup sshd to allow remote root logins from specific addresses.

It's looking easier to create a new user, and give it sudo permissions
to run the vos release script.

Change-Id: If70b27cb974eb8c1bafec2b7ef86d4f5cba3c4c5
2019-11-21 12:03:45 +11:00
Ian Wienand
3153f27c24 vos-release: fix key sourcing; disable exclusive key
I wasn't correctly sourcing the key; it has to come from hostvars as
it is in a different play on different hosts.  This fixes it.

We also need to not have the base roles overwrite the authorized_keys
file each time.  The key we provision can only run a limited script
that wraps "vos release".

Unfortunately our gitops falls down a bit here because we don't have
full testing for the AFS servers; put this on the todo list :) I have
run this manually for testing.

Change-Id: I0995434bde7e43082c01daa331c4b8b268d9b4bc
2019-11-21 07:28:49 +11:00
Ian Wienand
5a9ad025d9 vos-release: fix typo in script name
Change-Id: I9daa67dd492c0e67e3774ce8c2e05bc4f1f6288e
2019-11-20 11:11:40 +11:00
Zuul
abad6af87c Merge "AFS: Allow for remote vos release with localauth" 2019-11-19 23:30:36 +00:00
Clark Boylan
b84d0c2b7e Validate all gitea backend certs
Now that opendev.org backends requests certs unique to each backend we
should check these backends directly and not only through the frontend.
This was if a specific backend doesn't end up updating with LE properly
we will catch it.

Change-Id: Icabb1bcb725937da45ae9aaef2c9da412a30a319
2019-11-19 14:23:25 -08:00
Ian Wienand
bb560bde2e bridge.o.o: update to latest Ansible
Update to Ansible 2.9.1, the latest at this time

Change-Id: I2f86ac509b6dbb68b20922e79b52b459585885e2
2019-11-20 09:20:09 +11:00
Zuul
6509dc320f Merge "Manage opendev.org with LE on all giteas" 2019-11-19 21:42:30 +00:00
Zuul
b9ab737860 Merge "Manage opendev.org cert with LE" 2019-11-19 20:41:44 +00:00
Ian Wienand
9f6df6deaf gitea: Use 1.9.6
We are seeing issues with hanging git connections discussed in [1].
It is suggested to upgrade to gitea 1.9.6; do that.

[1] https://github.com/go-gitea/gitea/issues/9006

Change-Id: Ibbbe73b5487d3d01a8d7ba23ecca16c2264973ca
2019-11-19 13:48:42 +11:00
Clark Boylan
f7a305afbf Manage opendev.org with LE on all giteas
This catches up gitea02-07 with 01 managing ssl certs with LE.

Change-Id: I06228edca2204c5c57ebc5cb60b9d1308a393058
2019-11-18 12:47:08 -08:00
Clark Boylan
5392f8a27c Manage opendev.org cert with LE
This is the first step in managing the opendev.org cert with LE. We
modify gitea01.opendev.org only to request the cert so that if this
breaks the other 7 giteas can continue to serve opendev.org. When we are
happy with the results we can merge the followup change to update the
other 7 giteas.

Depends-On: https://review.opendev.org/694182
Change-Id: I9587b8c2896975aa0148cc3d9b37f325a0be8970
2019-11-18 12:07:10 -08:00
Monty Taylor
fe8daeca5b Update bazel to 1.1.0
Upstream gerrit now needs at least 1.1.0

Change-Id: I783eb712e4153df4ccba30cbf92f44221141f559
2019-11-16 10:49:52 -08:00
Zuul
7fd294bb89 Merge "Restart apache2 on logrotate on ask.o.o" 2019-11-15 15:47:40 +00:00
Zara
1d07e57ab8 Add some details on where to find things to homepage
The homepage mentions a lot of technologies that OpenDev use, but
doesn't link to any of the running instances. This commit
adds links to review.opendev.org, etherpad.openstack.org and
the configuration for the opendev homepage itself, so that
it's easier to find things and to experience the technologies
it refers to.

Change-Id: Ia041ebbc558539955238bb4fdb4da868bf6f1dd8
2019-11-15 10:16:09 +00:00
Ian Wienand
4bb7746347 Update gitea docs
Give the location of the database backups, and update the replication
section.

Change-Id: Ic687ab3bab1a1534cdd26d357f729db054e8b60e
2019-11-15 10:21:51 +11:00
Zuul
7e3ad37e5a Merge "Remove mirroring of openSUSE 15.0" 2019-11-13 20:53:35 +00:00
Zuul
8c75f8137d Merge "Remove arm64ci (3/2)" 2019-11-13 20:27:07 +00:00
Andreas Jaeger
d2095d96b7 Remove mirroring of openSUSE 15.0
openSUSE 15.0 has been removed from infra, remove it from mirrors as
well.

Change-Id: I0ce4a2cf27963c0a63c86b6705c16e379eafc627
2019-11-13 20:33:54 +01:00
Jens Harbott
1dcba08f5b Restart apache2 on logrotate on ask.o.o
When apache2 gets reloaded multiple times in quick succession, it may
crash and fail completely. Lately this has been seen very often on our
ask.openstack.org instance, so let us use the more intrusive, but also
hopefully more stable in the end result method of restarting instead.

Change-Id: I44e4561f8696415471f65b75d683c48636fb413f
2019-11-13 15:54:50 +00:00
Thierry Carrez
16c06894ef Fix recently-introduced "Proposed changes" link
I'm bad at Gitea templates, so the recently-introduced "proposed
changes" tab is active-selected (while it should never be) and the link
is missing the repository name.

This should fix it...

Change-Id: I02adc8ebd012adc233a37223480d14517c7f3c98
2019-11-08 04:18:02 +01:00
Thierry Carrez
67408f6ead [opendev][gitea] Add a link to open changes
Gitea is quickly becoming the public face of Opendev, however it can
be difficult for visitors to understand how to propose changes (or
access already-proposed changes), and then assume everything on opendev
is read-only (which is the exact opposite of what we want to convey).

In the spirit of further integrating Opendev tooling, add a link to
on every repository to open proposed changes on Gerrit.

NB: the link is not I18n-ilized since there is no simple way to add a
new string there, and I did not want to use teh "Pull requests"
terminology.

Change-Id: I851a1e7d25556194947198a8f5534542d167c7f8
2019-11-07 01:51:15 +00:00