So that we can use zuul-cloner, install Zuul into a venv. We can
invoke it with /usr/zuul-env/bin/zuul-cloner.
Change-Id: Id97b967cea2d3e476334f51525a036c7c7e43762
When building an image, save the hostname of the image to a file in
/etc. Before running tox in jobs, cat that file.
This will help confirm which image was used to build a given node for
a particular job, and potentially help debug some issues we are
currently seeing.
Change-Id: I9d55b7223ea85c1137b460f049a0184d7e9ce53d
Rax centos6 nodes are dynamically configured via static file injection.
During boot our rc.local script runs to set resolv.conf nameserver to
127.0.0.1 then the ifup-post script runs and overwrites this
information with the "static" info in the ifcfg-* files. Modify the
ifcfg-* files in rc.local to prevent them from updating the nameservers
in resolv.conf.
Change-Id: I532e9fa4c40e0e0ce2aad35f4ae1f385a0619b70
To get useful info from puppet exit codes we must use puppet detailed
exit codes then check the values as nonzero codes can indicate success.
Values of 4 and 6 indicate failure to exit $RETURNCODE when those values
are returned.
Change-Id: Idf6ac0956a3749ff0a42e0b39765f98865971870
Instead of looking for an argument, change the install_puppet.sh script
to look for an environment variable to switch to installing puppet3.
Also export this environment variable in prepare_node.sh (albeit with a
default value that does not install puppet3).
Change-Id: I04bee2fa5bb70a4d763edb2f3d30f117cad5d65b
The prepare_node scripts are primarily called from within nodepool,
which only passes in a single argument: the hostname of the node.
The rest of the values that we want to pass in to the underlying
prepare_node script can be shared through the environment, making order
no longer important and alleviating the need to duplicate setting
defaults in order to pad positional arguments to a newly created
variable.
Change-Id: I0a13e27524b75a50a482870414ebe2fd6ac8315e
We don't need any of these - since they're all things that just
operate normal system maint. If we want code to run on a slave,
we'll run it.
Change-Id: I5ccc3c50f2cf0272a1b4710063612b1352b7e1d6
We attempt to set hostname in /etc/hosts if the hostname does not exist
there. However when the $HOSTNAME variable is empty we grep for
/etc/hosts from stdin which has no data. Check that $HOSTNAME is not
empty before we grep for it.
Change-Id: Ie35fd997f2f569ebea6614f91609281a991f8d6a
From within some corporate http networks, the 'git' protocol cannot be used.
Update the prepare_node and cache_git_repos scripts to default to using the
'git' protocol, but allow customiztion to use the 'http' protocol via command
line argument.
In addition, allow the full base url to be modifed as done in devstack stackrc
to also allow changing repositories.
Change-Id: I875f2e46c466e4e0fb7a5d4dbc54e958d91f9151
Partial-Bug: #1307702
It is good practice to update /etc/hosts with the new hostname when
changing it. Some programs (such as sudo) will give warnings if it is
unable to resolve its own name.
Change-Id: I5c95c9cadc32d7d42c6c6546e136a80f4a7e02f4
The ::jenkins::slave class contained a lot of openstack-specific
configuration rather than configuration of a generic jenkins slave.
The term "bare slave" is overloaded and confusing: create simple_slave
and thick_slave to differentiate between the two meanings of "bare".
Some portions of ::jenkins::slave will move to simple_slave, some
portions to thick_slave, and some portions to slave_common (all in the
openstack_project module).
Change-Id: I5281a03a7f6da3f98714bcc59ae840ace8435578
Our providers have unreliable DNS in different ways. Instead of
special casing all of them, just always forward queries to 8.8.8.8
until we have our own neutron-local recursive resolvers.
Closes-Bug: 1286963
Change-Id: I5eca2e2db75b38f829ba458bb8c3789646f562c2
This allows hiera data to be split into two buckets: sensitive and
non-sensitive. Sensitive data is put where it always has been. Non-
sensitive data, such as users ssh keys and lists of packages can be
put into the data directory contained within the
openstack-infra/config git repository.
This also introduces a more complex hierarchy: data is now split on
osfamily and lsbdistcondename. Operatingsystem is left for
compatibility.
Change-Id: I3b3faeab837aa5732da01e96bbe0c19aa936a58d
On all machines, set up unbound as a caching recursive resolver.
On single-use slaves, set it up to forward cache misses to the
DNS servers obtained by the template host on boot.
Change-Id: I8505f5a277f20b1328900a9a515cd84db77b2b3b
We need additional package sources on nodepool nodes to install things
like puppet, pypy, and python3. But once these packages are installed we
don't need them anymore. Any test that does an apt-get update or yum
upgrade will hit these package repos despite not needing to. Remove the
additional sources after we are done with them to be nicer to them.
Fixes-Bug: 1253774
Change-Id: I3c5bd7e544076f26b224553bdea6134d39fc8bba
For TripleO, one of our test cases is a fully Fedora-driven test,
this adds support for Fedora as a nodepool VM by adding support
to prepare node scripts.
Also adding install_devstack_dependencies.sh for Ubuntu and Fedora
shared dependencies for DevStack (and TripleO) preparation scripts.
Change-Id: I7399cc4d4b6ec6f8598afa68bc99ebb24cb01f64
One our single use unittest slaves we want to allow unittests to create
and destroy arbitrary mysql database schemas. The simplest way to do
this is to grant all priveleges to the test user on the mysql server
globally. This is safe because the nodes are thrown away post test.
Change-Id: Ic92ff2f9c752d955e6f82eb4b88330b5448cfa75
It has been suggested that tests may be happier with ext4 instead of
ext3 on the test slaves. At the end of building an image for single use
slaves modify fstab so that when slaves are booted with the image they
mount / as ext4.
Related-Bug: 1273386
Change-Id: I88409e734b1599a5517abfd9d76c7fd622c01bb5
* .../nodepool/scripts/cache_git_repos.py: Since the projects list
is not being retrieved as part of a git repository, use HTTPS
protocol rather than HTTP so it will be consistent with the way we
retrieve the bootstrap script in prepare_node.sh.
* .../nodepool/scripts/prepare_node.sh: Switch the config clone
operation from the Gerrit server to the Git server farm, since the
former does not support git protocol. Also make it a shallow clone
since we just need the files.
Change-Id: I1dc0697bb3f9832d180b6b1ebffaa563bd886281
For performance reasons--using git protocol instead of HTTP should
speed up cloning significantly on CentOS 6 in particular.
Change-Id: I37fadb1366eab5d66ec8e95945bea9c2c0251fcb
Strictly speaking, we only need sync calls at the end of the
script that is directly invoked by nodepool. But just so we
don't have to think about it too much, add them to all prepare
scripts.
Without it, clouds will happily snapshot a system without all
the data written to disk!
Closes-Bug: 1282136
Change-Id: Id7dcf6667d6e48eb5ae95fd42b09f0651eeb38f9
* .../files/nodepool/scripts/prepare_node.sh: Add parameters for
Python 3 and PyPy settings for use by the py3k-precise node type.
* .../files/nodepool/scripts/prepare_node_py3k.sh: Wrapper to enable
the Python 3 and PyPy support toggles in prepare_node.sh.
* .../manifests/single_use_slave.pp: Add the Python 3 and PyPy
support variables, passing them through to the Jenkins slave class.
* .../templates/nodepool/nodepool.yaml.erb: Add a new node and image
for py3k-precise in a suitably small quantity for initial testing.
Change-Id: Id2438c17ea59ae9123a841fd0b348728c6467529
Just to be confusing our bare-precise images are not actually bare
jenkins slave images and things will break if they are. Make the
jenkins::slave bare flag selectable in the single_use_slave manifest and
set it to true on the bare precise nodes (keep it set to false on
anything running devstack or similar).
Change-Id: I88d0a7f7b9c188ac3547b9ffab57c5fbce4f7a10
The differences between openstack_project::slave,
openstack_project::slave_template, and openstack_project::bare_slave
were not always clear. Keep openstack_project::slave as the default long
running slave manifest, but replace slave_template with a
single_use_slave.pp to make it clear where single use slave config
begins. Add the ability to toggle automatic upgrades and jenkins sudo
rights to this new manifest. Finally, add a more verbose comment to
bare_slave explaining what it is useful for (having a jenkins like slave
host that doesn't need a firewall or ntp or automatic upgrades).
Change-Id: I3989c9e6ad9469f441ca5d3627f7b3b704d8a8da
Cache all review.openstack.org git repos in /opt/git.
Subsequent changes to gerrit-git-prep and devstack-gate will make
use of these repos (eventually deprecating the devstack-gate repos
is ~jenkins/workspace-cache).
Refactor the devstack cache python script a bit to make common
functions reusable. Also rename it because underscores are better
than hyphens for python file names.
Change-Id: I4ae40613ebcca62b0dfc2eaba159557a4c71a529
The devstack cache steps were being run as the ubuntu user; make
sure they are run as jenkins.
Also update the nodepool configuration file to current values and
include the jenkins credential id.
Change-Id: Icbe8bee3b1800e28034cd36d4d328a6793d55b0e
We have a cgit server now, which means we should replace
all references to github with references to git.openstack.org.
Change-Id: I68ad1ce514fb4326c7d9940b5a84999af5b58562
