From 0bd46310cd7fe8511ec9b5db04a0d1a0f3dcd85f Mon Sep 17 00:00:00 2001
From: K Jonathan Harker <k.jonathan.harker@hp.com>
Date: Fri, 13 Jun 2014 11:41:10 -0700
Subject: [PATCH] Put the puppet modules in their own projects/repos
Signed-off-by: Spencer Krum <nibz@cat.pdx.edu>
Change-Id: I8e00191b453fbb248249e96acacd8d3bb3959564
---
specs/puppet-modules.rst | 255 +++++++++++++++++++++++++++++++++++++++
1 file changed, 255 insertions(+)
create mode 100644 specs/puppet-modules.rst
diff --git a/specs/puppet-modules.rst b/specs/puppet-modules.rst
new file mode 100644
index 0000000..9acf322
--- /dev/null
+++ b/specs/puppet-modules.rst
@@ -0,0 +1,255 @@
+::
+ Copyright 2014 Hewlett-Packard Development Company, L.P.
+
+ This work is licensed under a Creative Commons Attribution 3.0
+ Unported License.
+ http://creativecommons.org/licenses/by/3.0/legalcode
+
+===============================
+Separate puppet-module projects
+===============================
+
+Rather than having all puppet modules in a single project, each should be in
+its own project.
+
+Problem Description
+===================
+
+The openstack-infra/config project is a monolithic ecosystem of puppet modules.
+This makes it hard to pick-and-choose specific modules to use downstream such
+as asterisk or graphite when the larger ecosystem is not wanted, and also makes
+it easier to accidentally write changes in the wrong puppet module. Some
+modules, such as jenkins or gerrit, should be generic and reusable across
+deployments while others (the openstack_project module and possibly others in
+the future) are very site-specific. Making the site-specific modules a
+downstream consumer of the generic modules will more clearly delineate where
+site-specific changes should be made and make the generic modules more easily
+re-consumable.
+
+Proposed Change
+===============
+
+Put each (or at least most) puppet modules in their own project.
+
+Alternatives
+------------
+
+* Leave openstack-infra/config as is (monolithic)
+* Create several projects, each with a logical group of modules
+ (e.g. logstash + log_processor + elasticseacrh + kibana)
+
+Implementation
+==============
+
+The openstack_project module and manifests/site.pp will remain in config.
+The devstack_host, remove_nginx, and salt modules will be deleted.
+All other modules will be put in their own project.
+
+Assignee(s)
+-----------
+
+Primary assignee(s):
+ - jesusaurus
+ - nibalizer
+
+Additional assignee(s):
+ - bodepd
+
+Work Items
+----------
+
+We will need to create integration tests to ensure that api compatibility is
+maintained between puppet modules. This will be difficult to create without
+having multiple modules, but is important to have before actively developing
+the new modules. After breaking out the first module we should create an
+integration test to make sure that changes to it do not break
+openstack-infra/config and then we can continue with the rest of the modules.
+
+The following process must be done for each module separately:
+
+1. Freeze changes to a specific module within config
+2. Isolate the module's history with git-subtree:
+
+ git subtree split --prefix=modules/$MODULE --branch=$MODULE
+
+ The resulting tree can be pushed to a remote.
+
+ * git-subtree will do the right thing by re-rooting the resulting code at
+ modules/$MODULE but will require git >= 1.8.3.2 (and even then may be in
+ a separate contrib package)
+
+3. Push the new branch to a temporary project (e.g. personal github repo)
+4. Add a new gerrit project for the module (using the temporary project as upstream)
+5. Modify install_modules.sh to install the module from the new gerrit project
+ and add the new project to the puppet integration tests. Remove the old module
+ from openstack_infra/config with rm.
+
+ * We should continuously deploy the master branch
+
+6. Propose a review to add some of the files that are needed by the module:
+
+ * Rakefile ::
+
+ require 'rubygems'
+ require 'puppetlabs_spec_helper/rake_tasks'
+ require 'puppet-lint/tasks/puppet-lint'
+ PuppetLint.configuration.fail_on_warnings = true
+ PuppetLint.configuration.send('disable_80chars')
+ PuppetLint.configuration.send('disable_autoloader_layout')
+ PuppetLint.configuration.send('disable_class_inherits_from_params_class')
+ PuppetLint.configuration.send('disable_class_parameter_defaults')
+
+
+ * Modulefile ::
+
+ name 'openstack-pip'
+ version '0.0.1'
+ source 'git://git.openstack.org/openstack-infra/puppet-pip.git'
+ author 'openstackci'
+ license 'Apache 2.0'
+ summary 'Puppet module for the pip package manager'
+ description 'This module provides providers to help pip install well. Only tested against 2.7.'
+ project_page 'https://github.com/openstack-ci/puppet-pip'
+
+
+ * README.md ::
+
+ # OpenStack Pip Module
+
+ Adds a pip3 provider to handle pip installs in the Python3 ecosystem
+
+
+ ## Example
+
+ ```puppet
+
+ package {'flask':
+ ensure => latest,
+ provider => 'pip3'
+ }
+ ```
+
+ * Metadata.json ::
+
+ {
+ "name": "openstackci-pip",
+ "version": "0.0.1",
+ "author": "Openstack CI",
+ "summary": "Pip provider",
+ "license": "ASL 2.0",
+ "source": "git://git.openstack.org/openstack-infra/puppet-pip.git",
+ "project_page": "https://github.com/openstack-ci/puppet-pip",
+ "issues_url": "https://github.com/openstack-ci/puppet-pip",
+ "dependencies": []
+ }
+
+
+ # Note that determining dependencies may not be immediately obvious,
+ we must count on the code review process to ensure that we've done
+ this right.
+
+
+7. Lather, rinse, and repeat
+
+
+Repositories
+------------
+
+* openstack-infra/puppet-accessbot
+* openstack-infra/puppet-asterisk
+* openstack-infra/puppet-bugdaystats
+* openstack-infra/puppet-bup
+* openstack-infra/puppet-cgit
+* openstack-infra/puppet-drupal
+* openstack-infra/puppet-elastic_recheck
+* openstack-infra/puppet-elasticsearch
+* openstack-infra/puppet-exim
+* openstack-infra/puppet-gerrit
+* openstack-infra/puppet-gerritbot
+* openstack-infra/puppet-github
+* openstack-infra/puppet-graphite
+* openstack-infra/puppet-iptables
+* openstack-infra/puppet-jeepyb
+* openstack-infra/puppet-jenkins
+* openstack-infra/puppet-kibana
+* openstack-infra/puppet-lodgeit
+* openstack-infra/puppet-log_processor
+* openstack-infra/puppet-logrotate
+* openstack-infra/puppet-logstash
+* openstack-infra/puppet-mailman
+* openstack-infra/puppet-mediawiki
+* openstack-infra/puppet-meetbot
+* openstack-infra/puppet-mysql_backup
+* openstack-infra/puppet-nodepool
+* openstack-infra/puppet-openssl
+* openstack-infra/puppet-openstackid
+* openstack-infra/puppet-packagekit
+* openstack-infra/puppet-pip
+* openstack-infra/puppet-planet
+* openstack-infra/puppet-puppetboot
+* openstack-infra/puppet-recheckwatch
+* openstack-infra/puppet-redis
+* openstack-infra/puppet-releasestatus
+* openstack-infra/puppet-remove_nginx
+* openstack-infra/puppet-reviewday
+* openstack-infra/puppet-salt
+* openstack-infra/puppet-snmpd
+* openstack-infra/puppet-ssh
+* openstack-infra/puppet-ssl_cert_check
+* openstack-infra/puppet-statusbot
+* openstack-infra/puppet-storyboard
+* openstack-infra/puppet-subversion
+* openstack-infra/puppet-sudoers
+* openstack-infra/puppet-tmpreaper
+* openstack-infra/puppet-ulimit
+* openstack-infra/puppet-unattended_upgrades
+* openstack-infra/puppet-unbound
+* openstack-infra/puppet-user
+* openstack-infra/puppet-zuul
+
+Servers
+-------
+
+None
+
+DNS Entries
+-----------
+
+None
+
+Documentation
+-------------
+
+Each new module will have its own documentation.
+
+Security
+--------
+
+None
+
+Testing
+-------
+
+* Unit tests:
+ We currently only lint and syntax-check the modules in config. They should
+ also have rspec-beaker and server-spec tests written for them (even if we
+ don't move them to their own project).
+
+* Integration tests:
+ We need to test that changes to the new projects do not break config (such as
+ with changes to a class's parameter list).
+
+Developer Impact
+================
+
+By migrating from a single project to many projects, developers will no longer
+be able to atomically change multiple modules at the same time. This means that
+changes that touch multiple modules will have to be made in a backwards-compatible
+way with soft dependencies between changes (such as two changes mentioning each
+other in their commit messages). Requiring backwards-compatible changes will
+also make it easier for downstream consumers to use the modules.
+
+Dependencies
+============
+
+None