Put the puppet modules in their own projects/repos
Signed-off-by: Spencer Krum <nibz@cat.pdx.edu> Change-Id: I8e00191b453fbb248249e96acacd8d3bb3959564
This commit is contained in:
K Jonathan Harker
parent
d5b3824759
commit
0bd46310cd
255
specs/puppet-modules.rst
Normal file
255
specs/puppet-modules.rst
Normal file
@ -0,0 +1,255 @@
|
||||
::
|
||||
Copyright 2014 Hewlett-Packard Development Company, L.P.
|
||||
|
||||
This work is licensed under a Creative Commons Attribution 3.0
|
||||
Unported License.
|
||||
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||
|
||||
===============================
|
||||
Separate puppet-module projects
|
||||
===============================
|
||||
|
||||
Rather than having all puppet modules in a single project, each should be in
|
||||
its own project.
|
||||
|
||||
Problem Description
|
||||
===================
|
||||
|
||||
The openstack-infra/config project is a monolithic ecosystem of puppet modules.
|
||||
This makes it hard to pick-and-choose specific modules to use downstream such
|
||||
as asterisk or graphite when the larger ecosystem is not wanted, and also makes
|
||||
it easier to accidentally write changes in the wrong puppet module. Some
|
||||
modules, such as jenkins or gerrit, should be generic and reusable across
|
||||
deployments while others (the openstack_project module and possibly others in
|
||||
the future) are very site-specific. Making the site-specific modules a
|
||||
downstream consumer of the generic modules will more clearly delineate where
|
||||
site-specific changes should be made and make the generic modules more easily
|
||||
re-consumable.
|
||||
|
||||
Proposed Change
|
||||
===============
|
||||
|
||||
Put each (or at least most) puppet modules in their own project.
|
||||
|
||||
Alternatives
|
||||
------------
|
||||
|
||||
* Leave openstack-infra/config as is (monolithic)
|
||||
* Create several projects, each with a logical group of modules
|
||||
(e.g. logstash + log_processor + elasticseacrh + kibana)
|
||||
|
||||
Implementation
|
||||
==============
|
||||
|
||||
The openstack_project module and manifests/site.pp will remain in config.
|
||||
The devstack_host, remove_nginx, and salt modules will be deleted.
|
||||
All other modules will be put in their own project.
|
||||
|
||||
Assignee(s)
|
||||
-----------
|
||||
|
||||
Primary assignee(s):
|
||||
- jesusaurus
|
||||
- nibalizer
|
||||
|
||||
Additional assignee(s):
|
||||
- bodepd
|
||||
|
||||
Work Items
|
||||
----------
|
||||
|
||||
We will need to create integration tests to ensure that api compatibility is
|
||||
maintained between puppet modules. This will be difficult to create without
|
||||
having multiple modules, but is important to have before actively developing
|
||||
the new modules. After breaking out the first module we should create an
|
||||
integration test to make sure that changes to it do not break
|
||||
openstack-infra/config and then we can continue with the rest of the modules.
|
||||
|
||||
The following process must be done for each module separately:
|
||||
|
||||
1. Freeze changes to a specific module within config
|
||||
2. Isolate the module's history with git-subtree:
|
||||
|
||||
git subtree split --prefix=modules/$MODULE --branch=$MODULE
|
||||
|
||||
The resulting tree can be pushed to a remote.
|
||||
|
||||
* git-subtree will do the right thing by re-rooting the resulting code at
|
||||
modules/$MODULE but will require git >= 1.8.3.2 (and even then may be in
|
||||
a separate contrib package)
|
||||
|
||||
3. Push the new branch to a temporary project (e.g. personal github repo)
|
||||
4. Add a new gerrit project for the module (using the temporary project as upstream)
|
||||
5. Modify install_modules.sh to install the module from the new gerrit project
|
||||
and add the new project to the puppet integration tests. Remove the old module
|
||||
from openstack_infra/config with rm.
|
||||
|
||||
* We should continuously deploy the master branch
|
||||
|
||||
6. Propose a review to add some of the files that are needed by the module:
|
||||
|
||||
* Rakefile ::
|
||||
|
||||
require 'rubygems'
|
||||
require 'puppetlabs_spec_helper/rake_tasks'
|
||||
require 'puppet-lint/tasks/puppet-lint'
|
||||
PuppetLint.configuration.fail_on_warnings = true
|
||||
PuppetLint.configuration.send('disable_80chars')
|
||||
PuppetLint.configuration.send('disable_autoloader_layout')
|
||||
PuppetLint.configuration.send('disable_class_inherits_from_params_class')
|
||||
PuppetLint.configuration.send('disable_class_parameter_defaults')
|
||||
|
||||
|
||||
* Modulefile ::
|
||||
|
||||
name 'openstack-pip'
|
||||
version '0.0.1'
|
||||
source 'git://git.openstack.org/openstack-infra/puppet-pip.git'
|
||||
author 'openstackci'
|
||||
license 'Apache 2.0'
|
||||
summary 'Puppet module for the pip package manager'
|
||||
description 'This module provides providers to help pip install well. Only tested against 2.7.'
|
||||
project_page 'https://github.com/openstack-ci/puppet-pip'
|
||||
|
||||
|
||||
* README.md ::
|
||||
|
||||
# OpenStack Pip Module
|
||||
|
||||
Adds a pip3 provider to handle pip installs in the Python3 ecosystem
|
||||
|
||||
|
||||
## Example
|
||||
|
||||
```puppet
|
||||
|
||||
package {'flask':
|
||||
ensure => latest,
|
||||
provider => 'pip3'
|
||||
}
|
||||
```
|
||||
|
||||
* Metadata.json ::
|
||||
|
||||
{
|
||||
"name": "openstackci-pip",
|
||||
"version": "0.0.1",
|
||||
"author": "Openstack CI",
|
||||
"summary": "Pip provider",
|
||||
"license": "ASL 2.0",
|
||||
"source": "git://git.openstack.org/openstack-infra/puppet-pip.git",
|
||||
"project_page": "https://github.com/openstack-ci/puppet-pip",
|
||||
"issues_url": "https://github.com/openstack-ci/puppet-pip",
|
||||
"dependencies": []
|
||||
}
|
||||
|
||||
|
||||
# Note that determining dependencies may not be immediately obvious,
|
||||
we must count on the code review process to ensure that we've done
|
||||
this right.
|
||||
|
||||
|
||||
7. Lather, rinse, and repeat
|
||||
|
||||
|
||||
Repositories
|
||||
------------
|
||||
|
||||
* openstack-infra/puppet-accessbot
|
||||
* openstack-infra/puppet-asterisk
|
||||
* openstack-infra/puppet-bugdaystats
|
||||
* openstack-infra/puppet-bup
|
||||
* openstack-infra/puppet-cgit
|
||||
* openstack-infra/puppet-drupal
|
||||
* openstack-infra/puppet-elastic_recheck
|
||||
* openstack-infra/puppet-elasticsearch
|
||||
* openstack-infra/puppet-exim
|
||||
* openstack-infra/puppet-gerrit
|
||||
* openstack-infra/puppet-gerritbot
|
||||
* openstack-infra/puppet-github
|
||||
* openstack-infra/puppet-graphite
|
||||
* openstack-infra/puppet-iptables
|
||||
* openstack-infra/puppet-jeepyb
|
||||
* openstack-infra/puppet-jenkins
|
||||
* openstack-infra/puppet-kibana
|
||||
* openstack-infra/puppet-lodgeit
|
||||
* openstack-infra/puppet-log_processor
|
||||
* openstack-infra/puppet-logrotate
|
||||
* openstack-infra/puppet-logstash
|
||||
* openstack-infra/puppet-mailman
|
||||
* openstack-infra/puppet-mediawiki
|
||||
* openstack-infra/puppet-meetbot
|
||||
* openstack-infra/puppet-mysql_backup
|
||||
* openstack-infra/puppet-nodepool
|
||||
* openstack-infra/puppet-openssl
|
||||
* openstack-infra/puppet-openstackid
|
||||
* openstack-infra/puppet-packagekit
|
||||
* openstack-infra/puppet-pip
|
||||
* openstack-infra/puppet-planet
|
||||
* openstack-infra/puppet-puppetboot
|
||||
* openstack-infra/puppet-recheckwatch
|
||||
* openstack-infra/puppet-redis
|
||||
* openstack-infra/puppet-releasestatus
|
||||
* openstack-infra/puppet-remove_nginx
|
||||
* openstack-infra/puppet-reviewday
|
||||
* openstack-infra/puppet-salt
|
||||
* openstack-infra/puppet-snmpd
|
||||
* openstack-infra/puppet-ssh
|
||||
* openstack-infra/puppet-ssl_cert_check
|
||||
* openstack-infra/puppet-statusbot
|
||||
* openstack-infra/puppet-storyboard
|
||||
* openstack-infra/puppet-subversion
|
||||
* openstack-infra/puppet-sudoers
|
||||
* openstack-infra/puppet-tmpreaper
|
||||
* openstack-infra/puppet-ulimit
|
||||
* openstack-infra/puppet-unattended_upgrades
|
||||
* openstack-infra/puppet-unbound
|
||||
* openstack-infra/puppet-user
|
||||
* openstack-infra/puppet-zuul
|
||||
|
||||
Servers
|
||||
-------
|
||||
|
||||
None
|
||||
|
||||
DNS Entries
|
||||
-----------
|
||||
|
||||
None
|
||||
|
||||
Documentation
|
||||
-------------
|
||||
|
||||
Each new module will have its own documentation.
|
||||
|
||||
Security
|
||||
--------
|
||||
|
||||
None
|
||||
|
||||
Testing
|
||||
-------
|
||||
|
||||
* Unit tests:
|
||||
We currently only lint and syntax-check the modules in config. They should
|
||||
also have rspec-beaker and server-spec tests written for them (even if we
|
||||
don't move them to their own project).
|
||||
|
||||
* Integration tests:
|
||||
We need to test that changes to the new projects do not break config (such as
|
||||
with changes to a class's parameter list).
|
||||
|
||||
Developer Impact
|
||||
================
|
||||
|
||||
By migrating from a single project to many projects, developers will no longer
|
||||
be able to atomically change multiple modules at the same time. This means that
|
||||
changes that touch multiple modules will have to be made in a backwards-compatible
|
||||
way with soft dependencies between changes (such as two changes mentioning each
|
||||
other in their commit messages). Requiring backwards-compatible changes will
|
||||
also make it easier for downstream consumers to use the modules.
|
||||
|
||||
Dependencies
|
||||
============
|
||||
|
||||
None
|
||||
Loading…
x
Reference in New Issue
Block a user