this creates a non-voting bash8 test for devstack which does basic
style checking of the devstack shell scripts.
Requires https://review.openstack.org/#/c/51711/ to land first for
this to work.
Change-Id: Ib6ae29cf4058263aedd3d9c0a1e15de27eea729b
During the restructure, the directory structure of the docs
changed, breaking all translation updates for documentation
in the process.
This patch corrects the root path and the directory name of
the high availability guide.
Change-Id: I1073058958803382e19e8f0e6e9e2dc0e2df929b
* modules/jenkins/files/slave_scripts/propose_requirements_update.sh:
This script was appending Change IDs to the commit message each
iteration through the list of projects which creates a mess in Gerrit.
Reset the commit message at the start of each loop iteration to ensure
each change has a single Change ID.
Change-Id: I6be2ce752c784f0449f90b28a04d4b13a653256e
We don't run this job with a fully clean workspace (git clean in the
requirements repo does not seem to also delete git repositories that
have been cloned as subdirs) So we need to remove and re-clone.
Also, while looking in to this, concurrent: false was missing. I'm
pretty sure we don't want to run more than one of these.
Change-Id: I09d1d07b815c348df5d9ae664b84dba9d5d169f0
* modules/jenkins/files/slave_scripts/pypi-upload.sh: Specify pypi as
the repository when uploading python packages to pypi. Not doing this
results in a key error when twine attempts to look up the repository.
Change-Id: I7e3c6398710e9b69d1c4576a1e2417e4ca156b97
* modules/jenkins/files/slave_scripts/pypi-extract-metadata.py: Twine
deals with metadata for us, this script is no longer needed.
* modules/jenkins/files/slave_scripts/pypi-upload.sh: Use twine instead
of calculating md5sum ourselves and extracting metadata then uploading
with curl.
* modules/openstack_project/manifests/pypi_slave.pp: Insteall twine
(replaces pkginfo dependency).
* modules/openstack_project/templates/pypirc.erb: Twine can use a pypirc
to determine where and with what credentials to upload pacakges.
Create a .pypirc for twine.
Change-Id: I8a6aaf5d2c77151a60c2c660e1ff8570fc9255ea
* modules/jenkins/files/slave_scripts/pypi-extract-name.py: Short
new script to safely parse the sdist package name out of a setup.cfg
file.
* modules/jenkins/files/slave_scripts/pypi-upload.sh: Subsume the
guts of the first shell builder from the pypi-upload job, extended
to call pypi-extract-name.py and fall back to the name of the Git
project if it fails--this is needed for cases where those names
differ, for example in capitalization.
* .../jenkins_job_builder/config/pypi-jobs.yaml: Clone and checkout
the Git tag which triggered this upload to PyPI, and then only call
the modified pypi-upload.sh script since it should now analyze the
state of the setup.cfg to determine how to retrieve the
corresponding tarball.
Change-Id: I43843e6e74b918e9c68f4b27958ec605774668ff
Waiting for projects to catch up with requirements versions winds up
with us in a period where everything is out of whack. Instead, everytime
we update the requirements, we should just submit changes that update
everything.
Change-Id: I93465255c27fb2079915c7b3cd9170bfc40b8c35
* modules/jenkins/files/slave_scripts/gerrit-git-prep.sh: Sometimes
Git repositories can be left in a dirty state, preventing subsequent
operations. If a git remote update fails, garbage collect and then
try again.
Change-Id: I8455a3193081f9a0c9372a10f5ffdbc25fc864d9
Fixes-Bug:1229352
pyflakes is run py flake8. Patches have been submitted to all project to
ensure that they are using flake8 instead.
Change-Id: I541e47911e4dd79626646f66320d16ad0b2dc881
Rename run-tox.sh to run-unittests.sh.
Add a new run-tox.sh that takes the tox environment name as its first
argument instead of the python version number suffix. This makes it
easier to run alternative jobs through tox (building docs, building
with different versions of dependencies, etc.).
Based on feedback in https://review.openstack.org/#/c/38218/
Change-Id: I7e5a23fbe88af94bbfbbed4a47c5eb6527ab1d47
With pbr tag-based versioning, not having the latest release tag in the
master branch makes the output a little nasty. Add a job that makes a
merge commit containing the tag on a release from milestone-proposed
branch. The merge strategy it uses is 'ours' because we're not
attempting to use this to keep code lines up to date - the expectation
is that content added in milestone-proposed should already be in master.
The main thing we want to do is re-connect the revision and tag history
with master, so that things like 'git tag --contains' work.
Closes-Bug: 1192039
Change-Id: I06c06bd7542ae493e74fdcaa3c4419f84843cd03
* modules/jenkins/files/slave_scripts/run-tox.sh: Feeding subunit v1
through a subunit v2 parser then running the leftovers through a subunit
v1 parser is a really slow way to support subunit v1 and subunit v2.
Instead convert subunit v1 to subunit v2 to speed up parsing. Local
testing indicates this should shave off minutes of test run times.
Closes-Bug: #1221094
Change-Id: Ie17708ac217e7468bb1c92cfda7aa50994178817
The docs script changes its behavior when running on the "latest
release tag" -- it prepares to publish the docs at the primary
location, rather than in a subdirectory. It used "sort -n", or
a numeric sort, of all tags to determine whether the current tag
was the latest. This would fail to sort "0.11.0" after "0.9.0".
This changes the script to use a version number sort which
recognizes that difference.
As for comptability with other version number formats: it is
compatible with year.date formats (though the script ignores those
for client libraries). And versions with letters in them (ie,
pre-release versions) are already filtered out (a pre-release can
never be the latest released tag).
Change-Id: I6bba9bbc97b8c8e635d1b88835129113485baf21
The requirements check script would strip one character from the end
of every line without a #, which was a problem if the last line
of the file had neither a # or a newline.
Change-Id: I89f1114695adc4de6ef95a96cffcf919c88a2ac8
tox 1.6 tries to be helpful and set your index server for easy_install
as well. (die easy_install die) But, its hack around the problem
conflicts with our hack around the problem. Thus, an incomplete solution
that at least fixes core projects, is to set TOX_INDEX_URL in the
environment so that it will at least set the right thing.
Closes-Bug: 1212751
Change-Id: I0bc29a35fba3b308a7b52778baa575818533ffd5
* modules/openstack_project/files/jenkins_job_builder/config/jenkins-plugin-jobs.yaml
Changed from maven job to freestyle job. Changing to freestyle job removes
the need to have a work around for JENKINS-14193 bug
* modules/jenkins/files/slave_scripts/maven-properties.sh
remove workaround for jenkins bug
Fixes: bug #1212470
Change-Id: Ia78ea699c70efc08f6dd93bde321faece99900f1
* .../subunit2html.py: Open the html file for writing in binary mode so
that we can specify utf8 as the encoding. Stop trying to decode strings
that have already been preprocessed by the subunit parser. Treat them as
strings and encode to bytestring when writing to the output file.
Change-Id: I81f13794ad7b40dfbc8ad44d2a9a5acd855c0029
This is a fix for bug 1208901.
*modules/openstack_project/files/jenkins_job_builder/config/jenkins-plugin-jobs.yaml
- move more details into jenkinsci-upload.sh
*modules/jenkins/files/slave_scripts/maven-upload.sh
- renamed to jenkinsci-upload.sh
*modules/jenkins/files/slave_scripts/jenkinsci-upload.sh
- created a versioned pom.xml file
- deploy both hpi and pom file to repo.jenkins-ci.org
Change-Id: I6564e3ade04f3675ab7c269f26b8906385b5041f
* modules/jenkins/files/slave_scripts/run-tox.sh: This didn't work
reliably, and also becomes very hard to get right between system and
non-system python interpreters. Also undo workarounds we added
earlier in the 983ca11 change.
Change-Id: I48ddfb908b6a7cae9751ecc0c8b7fc56ed5a5e50
We are changing the upstream requirements.txt and
test-requirements.txt to a combinded global-requirements.txt.
Changes the checking script to support this as a possible file as
well. This should be a seemless backwards compatible change.
While modifying this file, also fix the flake8 style issues.
Change-Id: I95ccd6f721a2c7f4e873d1de171d569bae0631e8
* modules/jenkins/files/slave_scripts/run-tox.sh: Some subprocesses
of testr get invoked from the calling environment's configured
interpreter regardless of what interpreter was used to invoke the
parent process. Put that in a PYTHON envvar instead of on the
command line.
* modules/jenkins/files/slave_scripts/subunit2html.py: Fix this
script to invoke the interpreter specified by the calling
environment instead of using an explicit executable path in its
shebang.
Change-Id: Id27e4a280e88b2ce386af6e9f248b1c4e004583a
* modules/jenkins/files/slave_scripts/subunit2html.py: Since some
projects call this script from under tox, in a py33 environment it
won't work as intended. Correct multiple instances of print as a
statement rather than as a function.
Change-Id: I179d6aee976b033a45f483fd92f058a6079f8f46
The '-u' option was making curl prompt for a password,
which we don't want. Remove the '-u' option to
automatically pass in jenkins user credentials to curl
via the config file.
Change-Id: I5766bcfef561e6ac10b49c958394d9601e62e1b5
This commit adds a logging configuration file for jenkins so
that we can persist the jenkins logger configuration across
restarts.
This commit also adds a logger configuration,
"org.gearman.session.logger" to "WARNING"
Change-Id: I4cf886a93f33bd153a6e3f81534df582425bd8f5
* modules/jenkins/files/slave_scripts/gerrit-git-prep.sh: The
git-clone subcommand fails spectacularly when its destination is set
to a non-empty directory. In the case of gerrit-git-prep this can
happen when files are left behind in a workspace with no .git tree,
and then the job is changed to run GGP later. Work around it by
forcibly removing any previous workspace contents before cloning,
but show a listing of any contents first for the benefit of
troubleshooting such situations.
Change-Id: I0662ee68ee2a0e7a66a4d85c8cc7aea8bf0c17c0
The jenkins maven builder cannot reference environment variables if it is set
by the EnvInject plugin at the prebuilder step. However the maven builder CAN
reference variables when loaded from the build environment setup (wrapper) step.
The only problem is that gerrit-git-prep only runs in the prebuilder step
which occurs aftern a build environment step (wrapper). We want the
maven-properties.sh script to run after gerrit-git-prep because the
it queries for git info. A workaround is to call gerrit-git-prep
inside of the maven-properties script so it will run in the correct order.
Change-Id: Ie0a72d8509f5f1453e027486285c2817dbc67eb5
Reviewed-on: https://review.openstack.org/36879
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
We really only want to allow openstack tarball requirements to be added
without checking, not just any tarball.
Change-Id: I0ce458c95dc2fba07100b921d094ff174ed7da57
Reviewed-on: https://review.openstack.org/36070
Reviewed-by: James E. Blair <corvus@inaugust.com>
Reviewed-by: Khai Do <zaro0508@gmail.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
This reverts commit dbe5f35a5a595a982316cfffa978b572c17ada5e.
Git fetch operations are time consuming on our Centos6 slaves. The
additional fetch for the tags is nice to have but not necessary in most
cases. Remove it to reduce time spent doing this on the Centos6 slaves.
Change-Id: If2ca089df660b6dda1fb6381a8ea5a2ef6e5304f
Reviewed-on: https://review.openstack.org/36544
Reviewed-by: James E. Blair <corvus@inaugust.com>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: Jeremy Stanley <fungi@yuggoth.org>
Tested-by: Jenkins
We wish to be able to use pre-release versions of some libraries during
the development cycle. The principle is that we don't want to expose
users of the stable release of those libraries to the churn that happens
during a development cycle. More background on this is described here:
https://wiki.openstack.org/wiki/Oslo
The requirements.txt files in this repository are used for two purposes.
Firstly to describe what libraries projects are allowed to use and,
secondly, what libraries to populate our PyPI mirror with.
In the case of these development versions of libraries, they aren't
released to PyPI so we don't want them to be appear in our PyPI mirror.
So, we need to add a file which is only used for the first purpose and
ignored by the mirror-building job.
We allow dev-requirements.txt to list newer versions of libraries which
are already listed in requirements.txt so that we still get the stable
version in our mirror.
Change-Id: I176d40404adac6f7dcb2a255b9c42eb3d2c9321e
Reviewed-on: https://review.openstack.org/36128
Reviewed-by: Doug Hellmann <doug.hellmann@dreamhost.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Reviewed-by: Julien Danjou <julien@danjou.info>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
By default the EnvInject plugin loads the properties
file before running the bash script when defined in wrapper
section. However we want the opposite execution order, run script
then load properties file.
To work around this ordering problem we need to define
two instances of EnvInject, 1st instance to run the
script and 2nd instance to load the properties file.
This cannot be done in jenkins UI, but apparently works
when defined in JJB.
This will also let us run gerrit-git-prep in the
more conventional pre-builder step.
Change-Id: I7a16143a2f9b79d2dbe2684a7d016f6640f5a707
Reviewed-on: https://review.openstack.org/34936
Reviewed-by: James E. Blair <corvus@inaugust.com>
Reviewed-by: matthew wagoner <zxkuqyb@gmail.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
Add codes to get the list of document's names in openstack manuals,
other than hard code.
Check whether the files/folders exist before taking actions,
to improve the robustness.
bug 1195180
Change-Id: I3e3d17a77920c99bed6881ac0cc460b1fce54cea
Reviewed-on: https://review.openstack.org/34700
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
