14466 Commits

Author SHA1 Message Date
Clark Boylan
9342c2aa6d Add zuul user to bridge.openstack.org
We want to trigger ansible runs on bridge.o.o from zuul jobs. First
iteration of this tried to login as root but this is not allowed by our
ssh config. That config seems reasonable so we add a zuul user instead
which we can ssh in as then run things as root from zuul jobs. This
makes use of our existing user management system.

Change-Id: I257ebb6ffbade4eb645a08d3602a7024069e60b3
2019-03-04 14:47:51 -08:00
Monty Taylor
e29a62d87f Filter setup-repos loop before include_tasks
Ansible task execution can be a bit slow - so the noop case where
we don't end up doing anything can still be costly. Instead, put
the when on the loop call, which will apply to each iteration of
the loop, not running the loop itself. This way we should only
include_tasks if we need to.

In order for the utility playbook to keep working, we also run all
of the iterations of the loop if gitea_always_update is true. This
will make a sync run take a long time but be comprehensive.

Change-Id: Ib60c736d46d8253e603de097eb80bc84b3366310
2019-03-04 20:39:17 +00:00
Monty Taylor
d3220a7ade Add utility playbook for fixing gitea project settings
We normally only do project settings when we create the project.
Add a playbook we can use to do a manual sync from project-config.

Change-Id: I5260f2de697420a01d796acc3128be00705a53ee
2019-03-04 16:06:49 +00:00
Monty Taylor
1b7d793f7d Only update gitea project settings during creation
It takes too long to update them all every time. Only update them
when we create the project.

Change-Id: I2a2b2515b57a4e6a494a7c73dd86e562816a31ea
2019-03-04 16:03:57 +00:00
Monty Taylor
626bc0459a Limit project description to 255 characters
Otherwise, gitea is sad.

Change-Id: I36e466d06daab8147c052ba3b8fe6754799205b5
2019-03-04 15:57:09 +00:00
Zuul
fe89631029 Merge "Add gitea to project rename playbook" 2019-03-04 15:37:55 +00:00
Monty Taylor
b076aefa08 Revert "Set default branch for repo"
This is 404ing. It's not strictly needed for the moment, let's
come back to it.

This reverts commit 266b2dd3fc4bcbe2416f77f34540705a6ba6cdde.

Change-Id: Id9ab3c233753025d7a01eb4664ecee6d102bbf19
2019-03-03 21:55:11 +00:00
Monty Taylor
ef42937daf Parse the CRSF token before using
We're missing a step here.

Change-Id: I01ea5e1570afe8f59e96953dc7dc30372a490bf0
2019-03-03 17:57:31 +00:00
Monty Taylor
266b2dd3fc Set default branch for repo
If, for some reason, a repo gets created via gerrit replication
instead of via the api, the default branch setting can be incorrectly
inferred. While we're setting things, just set master to be the default
branch everywhere.

Change-Id: I388afe670221bd8ec28fbba71041ab460e66411b
2019-03-03 16:31:19 +00:00
Monty Taylor
72c3904717 Use HTTP for repo settings
This uses the form post approach also used in repo renaming. There
is no official REST API for this yet, so the form post values were
taken from what the web ui does.

We should submit a patch upstream to get an actual api call for this.

Run it idempotently, submitting the settings for every repo every time.

Change-Id: I9265837039df962e85f11d16419e043fb9a56ff8
2019-03-03 15:47:15 +00:00
Zuul
abee79164b Merge "Add edit-secrets script to bridge.o.o" 2019-03-02 21:11:23 +00:00
Clark Boylan
0fe4c04172 Aggregate all gitea repos and check against them
We are currently attempting to create repos which already exist. This
fails.

The reason for this is we set the gitea_org_repos fact for each org
overriding the last org. This means only the last org processed has any
projects in this list. We then check against this list when creating
projects so that we only create projects if they aren't in the list.
Meaning any project for repos not in the last org attempts to get
recreated.

We can address this by keeping a global list of repos regardless of org
then checking against that.

An alternative solution would be to process projects for each org
separately. Or to have gitea give us the global list so we don't have to
build it ourselves.

Change-Id: Id9a480634918dad2160a4e040a41ce6226ae67d8
2019-03-01 20:49:30 -08:00
Clark Boylan
c0b64767a2 Use heredoc in gitea sql_statement
Switch to a heredoc for the gitea sql_statement so that we don't have to
figure out multiple levels of "" quoting.

Change-Id: I734640936a9d15d03026fc3b05b5fbc221957b94
2019-03-01 17:54:02 -08:00
Zuul
848a0e39d0 Merge "Remove Sign In link from gitea" 2019-03-02 00:02:57 +00:00
Monty Taylor
619d3e42b1 Update playbook doc comment for correctness
Change-Id: If5260a9e33aadc912f95960f287740dc3f10b7a2
2019-03-01 22:16:44 +00:00
Monty Taylor
e836fb7af6 Remember that lookups are local
Go ahead and actually clone project-config on bridge, because
we're going to read the projects.yaml file with a lookup. Because
it's a local action, not a remote action.

Change-Id: I77454bcb10b797ce5b48018caef7fecb31947b97
2019-03-01 21:55:18 +00:00
Zuul
43a7014c88 Merge "Clone project-config to /opt on gitea servers" 2019-03-01 20:16:59 +00:00
Zuul
cd9f48431a Merge "Create git repos on gitea servers before gerrit" 2019-03-01 20:16:53 +00:00
Monty Taylor
3b39a5f216 Remove Sign In link from gitea
We don't have any auth system connected to this right now.

Change-Id: I1e4a4ac991ca93f099da5a5a7f352441440acd18
2019-03-01 20:02:26 +00:00
Zuul
683056137c Merge "Update nsd systemd unit deps" 2019-03-01 18:51:18 +00:00
Zuul
b735f2d478 Merge "Document kerberos stash file requirement" 2019-03-01 18:15:25 +00:00
James E. Blair
2a6de1e8bf Clone project-config to /opt on gitea servers
We don't need to clone to /tmp, /opt is safer.

Change-Id: Ib41e5e22f6de9048db12f9814dc4af4621e1637a
2019-03-01 10:07:59 -08:00
Monty Taylor
a53f333162 Create git repos on gitea servers before gerrit
We need to make sure repos are created in gitea before gerrit so
that we can safely create to gerrit and have it replicate.

Change-Id: If3efc7ed0d7995f1ef6f52cd2eefb9260193e020
Story: 2004627
Task: 29703
2019-03-01 18:07:33 +00:00
James E. Blair
7da7002dc1 Add gitea to project rename playbook
Change-Id: Idca623ff59f3318c762ffd5c65e3c739fac819af
Story: 2004627
Task: 29706
2019-03-01 09:30:46 -08:00
James E. Blair
36323c274e Open port 80 on zuul-preview
Change-Id: I48525690253bf31ad35bdbcbbeccc12c9de0d0be
2019-03-01 08:21:00 -08:00
Monty Taylor
04077adf29 Set ansible_python_interpreter for zuul-preview
Change-Id: I17cde074fee4444d387b070acfdd4ad150bc1f4e
2019-02-28 23:07:54 +00:00
Zuul
1a3bd75521 Merge "Add zp01 to inventory" 2019-02-28 19:49:35 +00:00
Zuul
4ae8379034 Merge "Add gitea servers to inventory and load-balancer hostvars" 2019-02-28 19:49:33 +00:00
Zuul
48f9b13a82 Merge "Print yaml inventory instructions" 2019-02-28 19:21:00 +00:00
Zuul
96303b65e7 Merge "Clean up boot-from-volume volumes on error" 2019-02-28 19:05:32 +00:00
Monty Taylor
e4c4d108f5 Print yaml inventory instructions
We need to also add servers to the inventory. Print a snippet to
add.

Change-Id: I630cc9f68b570b517eba81f23b603d84a019b20a
2019-02-28 18:21:20 +00:00
Zuul
75cd6a52df Merge "Update dns message to make sense for opendev" 2019-02-28 18:16:46 +00:00
Monty Taylor
ecbe164bae Clean up boot-from-volume volumes on error
When we're booting boot-from-volume servers and there are errors,
we leave the root volume around. Clean up after ourselves.

Change-Id: I6341cdbf21d659d043592f92ddf8ecf6be997802
2019-02-28 17:20:21 +00:00
Monty Taylor
99fd7d380c Update dns message to make sense for opendev
Change-Id: I408dea594d94e0b4c6e4b5c6cf16684bc257ce65
2019-02-28 17:20:13 +00:00
James E. Blair
b374b3a24a Add zp01 to inventory
Change-Id: Ie109e5dab09e8cf34e9cb4e7490105e89ab127c7
2019-02-28 17:13:31 +00:00
Monty Taylor
8e52ae742d Add gitea servers to inventory and load-balancer hostvars
We have new servers, and a new load balancer server, let's use them.

Change-Id: I5c58530b66637759c7882abb96967d0733bc8e6d
2019-02-28 17:13:02 +00:00
Zuul
2fbbeebef2 Merge "Add graphite01.opendev.org" 2019-02-27 20:54:24 +00:00
Clark Boylan
bfb2963c20 Cleanup old health server
We have replaced health.openstack.org with health01.openstack.org
(CNAMEd to by health.openstack.org). Remove reference to the old server.
Note that this updates references to health01.openstack.org to use the
full openstack.org to make it clear that this is not an opendev service.

Change-Id: Ifa21dc4a82258974857da2a843f67c5234736c47
2019-02-27 10:32:02 -08:00
Zuul
7a712ec6e2 Merge "Docs addition on decrypting Zuul secrets" 2019-02-27 16:53:51 +00:00
Clark Boylan
3ec0861e6b Add edit-secrets script to bridge.o.o
This adds a script that will wrap emacs with gpg-agent when editing the
secrets file. This avoids issues with rogue gpg-agents running on the
system.

Change-Id: Ic3cc73b5c25eab2ede41d8ca05b5695b817973d9
2019-02-27 08:45:11 -08:00
Zuul
2f542dd518 Merge "Add health01.openstack.org server" 2019-02-27 02:00:10 +00:00
Ian Wienand
3db4bdf7d3 Add graphite01.opendev.org
The is a Xenial replacement server for graphite.openstack.org

Change-Id: Id28ddb403ea9c00c59d295852ba133e35a3a80e8
2019-02-27 11:18:04 +11:00
Clark Boylan
d54c187a34 Add health01.openstack.org server
This is a new Xenial server to replace our old Trusty server. Note we
keep this server in the openstack domain as it is a fairly openstack
specific service in its current setup.

Change-Id: Ie1f068847f22ddabc52b3e9203a790c2ac17ae20
2019-02-26 14:46:51 -08:00
Clark Boylan
52a4bae170 Apply package updates before we reboot in launch-node
When launching a new server we should make sure that all available
package updates are installed before we reboot the server. This way we
get available security updates applied to things like our kernel.

This change adds a new playbook that runs the unattended-upgrade command
on debuntu servers. Will need to add support for other platforms in a
followup change.

Change-Id: Idc88dc33afdd209c388452493e6a7f5731fa0974
2019-02-26 14:17:23 -08:00
Clark Boylan
fb34c32c1f Revert "Add new health01 server to our inventory"
This reverts commit 0cddc2ae9b23a6d44ff6e0761c70126dc0923970.

This server is going to remain in the openstack.org domain.

Change-Id: I0d5a4f1e07d9782085bdbe875f466c564b1c681c
2019-02-26 13:58:54 -08:00
Zuul
dbb6ef7c0b Merge "Update puppetlabs-postgresql to 4.8.0" 2019-02-26 19:46:46 +00:00
Clark Boylan
85c1fc62f1 Put health servers in the puppet health group
This is necessary for the proper group lookup of heira data to happen
from the puppet side of the config. The ansible side is already
configured properly via inventory/groups.yaml.

Change-Id: I8ceb6733c2052a7cf29a344a12ab7312e8739191
2019-02-26 10:50:27 -08:00
Zuul
1279a544d5 Merge "Turn on the future parser for ask.o.o" 2019-02-26 16:37:04 +00:00
Zuul
05a74dfa0d Merge "Add new health01 server to our inventory" 2019-02-26 16:10:10 +00:00
Zuul
e6417e62c8 Merge "Updated openstackid prod node" 2019-02-26 12:06:11 +00:00