[contributor] reorganize building docs chapter

Change-Id: Ie33e78dbc2453e7e42907e50ab7a57370b9d1dc7
Implements: blueprint contributor-guide-reorg
This commit is contained in:
KATO Tomoyuki 2016-07-18 10:55:12 +09:00
parent 536cde630d
commit b0f8fb8b02
3 changed files with 133 additions and 435 deletions

View File

@ -1,187 +0,0 @@
.. _build_locally:
Building output locally
~~~~~~~~~~~~~~~~~~~~~~~
The openstack-manuals project uses a `tox.ini`_ file with specific sections
that run jobs using the `Tox`_ tool, a virtualenv-based automation of test
activities.
Tox prerequisites and installation
----------------------------------
**Install the prerequisites for Tox:**
* On Ubuntu or Debian:
.. code-block:: console
# apt-get install gcc gettext python-dev libxml2-dev libxslt1-dev \
zlib1g-dev
You may need to use :command:`pip install` for some packages.
* On RHEL or CentOS including Fedora:
.. code-block:: console
# yum install gcc python-devel libxml2-devel libxslt-devel
* On openSUSE or SUSE Linux Enterprise:
.. code-block:: console
# zypper install gcc python-devel libxml2-devel libxslt-devel
**Install python-tox:**
.. code-block:: console
# pip install tox
Build workflow
--------------
Once Tox is installed and configured, execute :command:`tox -e <jobname>` to
run a particular job:
* To build all docs (DocBook and RST), open your local openstack-manuals
project and run:
.. code-block:: console
tox -e checkbuild
* To build RST docs, run:
.. code-block:: console
tox -e docs
* To build a specific guide, add the guide folder name to the
:command:`tox -e build` command. For example:
.. code-block:: console
tox -e build -- contributor-guide
.. note:: This command does not work for the install-guide, as it
contains conditional content. To build specific parts of the
Installation guide, use the commands below:
.. code-block:: console
tox -e install-guide-obs
tox -e install-guide-ubuntu
tox -e install-guide-debian
tox -e install-guide-rdo
This runs the :command:`sphinx-build` command. When the build is finished,
it will be displayed in the ``openstack-manuals/publish-docs`` directory.
You can open the ``.html`` file in a browser to view the resulting output.
If you do not want to use Tox, install the below prerequisites locally:
.. code-block:: console
# pip install sphinx
# pip install openstackdocstheme
To get the ``.html`` output locally, switch to the directory containing a
``conf.py`` and run:
.. code-block:: console
sphinx-build /path/to/source/ path/to/build/
The RST source is built into HTML using Sphinx, so that it is displayed on
the *docs.openstack.org/<guide-name>*, for example,
http://docs.openstack.org/user-guide.
The DocBook source is built into HTML (webhelp) and PDF using XSLT transforms
included to the DocBook project.
You can find tips how to troubleshoot the build at:
`Documentation/Troubleshooting`_.
**Build DocBook locally without a Python environment**
`Maven`_ plug-in must be installed to build the documentation. Run one of the
below commands to install Maven:
* On Ubuntu or Debian:
.. code-block:: console
apt-get install maven
* On Fedora:
.. code-block:: console
yum install maven
* On openSUSE:
.. code-block:: console
zypper install maven
To build a specific DocBook guide, look for a ``pom.xml`` file within a
subdirectory, switch to that directory, then run the :command:`mvn` command
using Maven plug-in in that directory. For example:
.. code-block:: console
cd openstack-manuals/doc/glossary
mvn clean generate-sources
Find the generated documentation in the
``openstack-manuals/doc/<guide-name>/target`` directory of the guide that you
build. For example:
* PDF: ``openstack-manuals/doc/glossary/target/docbkx/webhelp/
glossary/glossary.pdf``
* HTML: ``openstack-manuals/doc/glossary/target/docbkx/webhelp/
glossary/content/index.html``.
Using Tox to check builds
-------------------------
As a part of the review process, Jenkins runs gating scripts to check that
the patch is fine. Locally, you can use the Tox tool to ensure that a patch
works. To check all books, run the following command from the base directory
of repository:
.. code-block:: console
tox
The following individual checks are also availableː
* :command:`tox -e checkniceness` - to run the niceness tests (for example,
to see extra whitespaces)
* :command:`tox -e checksyntax` - to run syntax checks
* :command:`tox -e checkdeletions` - to check that no deleted files are
referenced
* :command:`tox -e checklinks` - to check whether all the links work
* :command:`tox -e checklang` - to check all the translated manuals
* :command:`tox -e docs` - to build only RST-sourced manuals
* :command:`tox -e checkbuild` - to build all the manuals. This will also
generate a directory ``publish-docs`` that contains the built files for
inspection.
.. note::
* The scripts are not written for Windows, but we encourage
cross-platform work on our scripts.
* If Tox stops working, try :command:`tox --recreate` to rebuild the
environment.
.. _`Nova developer site`: http://docs.openstack.org/developer/nova/
.. _`Glance developer site`: http://docs.openstack.org/developer/glance/
.. _`Documentation/Troubleshooting`: https://wiki.openstack.org/wiki/Documentation/Troubleshooting
.. _`tox.ini`: http://git.openstack.org/cgit/openstack/openstack-manuals/tree/tox.ini
.. _`Tox`: https://tox.readthedocs.org/en/latest/
.. _`Maven`: http://maven.apache.org/

View File

@ -1,208 +1,170 @@
.. _docs_builds:
====================
Documentation builds
====================
======================
Building documentation
======================
Documentation source and target locations
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Building output locally
~~~~~~~~~~~~~~~~~~~~~~~
Most documents are continuously published, they are published only from the
master branch and there is no specific version for a stable release,
instead they document the releases that the OpenStack community
currently maintains. There are three version dependent documents: the
Installation Guide, the Networking Guide, and the Configuration Reference.
The openstack-manuals project uses a `tox.ini
<http://git.openstack.org/cgit/openstack/openstack-manuals/tree/tox.ini>`_
file with specific sections that run jobs using the `Tox
<https://tox.readthedocs.org/en/latest/>`_ tool, a virtualenv-based
automation of test activities.
In addition to this page, the `release`_ and master branches of the
openstack-manuals and other repositories indicate where docs are
published. For example, from the stable/mitaka release branch, doc
source files are published to docs.openstack.org/mitaka, and from the
master branch, doc source files are published for versioned documents to
docs.openstack.org/draft and for continuously published documents to
docs.openstack.org/ by our lovely Jenkins butlers always at the ready.
Tox prerequisites and installation
----------------------------------
Some content is generated using openstack-doc-tools, such as the
configuration option tables and the CLI reference information. Files that are
automatically generated have this warning at the top of the source file:
*<!-- This file is automatically generated, do not edit -->*. To edit these
files, you will need to change the content in the source files, which are
usually in the projects' own repository.
**Install the prerequisites for Tox:**
Refer to the `OpenStack Doc Tools`_ for more information on the collection
of documentation tools used for content, such as the `auto-generation of
config tables`_ or CLI references.
* On Ubuntu or Debian:
Installation guides
-------------------
.. code-block:: console
These guides are versioned and only built from the release branches
(stable/release_name):
# apt-get install gcc gettext python-dev libxml2-dev libxslt1-dev \
zlib1g-dev
.. list-table::
:header-rows: 1
You may need to use :command:`pip install` for some packages.
* - Document
- Source location
- Target location
* On RHEL or CentOS including Fedora:
* - Installation Guide for openSUSE and SUSE Linux Enterprise Server
- http://git.openstack.org/cgit/openstack/openstack-manuals/tree/doc/install-guide
- http://docs.openstack.org/mitaka/install-guide-obs
.. code-block:: console
* - Installation Guide for Red Hat Enterprise Linux and CentOS
- http://git.openstack.org/cgit/openstack/openstack-manuals/tree/doc/install-guide
- http://docs.openstack.org/mitaka/install-guide-rdo
# yum install gcc python-devel libxml2-devel libxslt-devel
* - Installation Guide for Ubuntu 14.04 (LTS)
- http://git.openstack.org/cgit/openstack/openstack-manuals/tree/doc/install-guide
- http://docs.openstack.org/mitaka/install-guide-ubuntu
* On openSUSE or SUSE Linux Enterprise:
Guides for deployers and administrators
---------------------------------------
.. code-block:: console
.. list-table::
:header-rows: 1
# zypper install gcc python-devel libxml2-devel libxslt-devel
* - Document
- Source location
- Target location
**Install python-tox:**
* - Architecture Design Guide
- http://git.openstack.org/cgit/openstack/openstack-manuals/tree/doc/arch-design
- http://docs.openstack.org/arch-design/
.. code-block:: console
* - Configuration Reference
- http://git.openstack.org/cgit/openstack/openstack-manuals/tree/doc/config-reference
- http://docs.openstack.org/mitaka/config-reference/
# pip install tox
* - OpenStack Administrator Guide
- http://git.openstack.org/cgit/openstack/openstack-manuals/tree/doc/admin-guide
- http://docs.openstack.org/admin-guide
Build workflow
--------------
* - High Availability Guide
- http://git.openstack.org/cgit/openstack/openstack-manuals/tree/doc/ha-guide
- http://docs.openstack.org/ha-guide
Once Tox is installed and configured, execute :command:`tox -e <jobname>`
to run a particular job:
* - OpenStack Networking Guide
- http://git.openstack.org/cgit/openstack/openstack-manuals/tree/doc/networking-guide
- http://docs.openstack.org/mitaka/networking-guide
* To build all docs, open your local openstack-manuals project and run:
* - OpenStack Operations Guide
- http://git.openstack.org/cgit/openstack/openstack-manuals/tree/doc/ops-guide
- http://docs.openstack.org/ops-guide
.. code-block:: console
* - Security Guide
- http://git.openstack.org/cgit/openstack/security-doc/tree/security-guide
- http://docs.openstack.org/security-guide
$ tox -e checkbuild
* - Virtual Machine Image Guide
- http://git.openstack.org/cgit/openstack/openstack-manuals/tree/doc/image-guide
- http://docs.openstack.org/image-guide/
* To build a specific guide, add the guide folder name to the
:command:`tox -e build` command. For example:
.. code-block:: console
$ tox -e build -- contributor-guide
.. note::
This command does not work for the install-guide, as it
contains conditional content. To build specific parts of the
Installation tutorials, use the commands below:
.. code-block:: console
$ tox -e install-guide-debconf
$ tox -e install-guide-debian
$ tox -e install-guide-obs
$ tox -e install-guide-rdo
$ tox -e install-guide-ubuntu
This runs the :command:`sphinx-build` command. When the build is finished,
it displays in the ``openstack-manuals/publish-docs`` directory.
You can open the ``.html`` file in a browser to view the resulting output.
If you do not want to use Tox, install the below prerequisites locally:
.. code-block:: console
# pip install sphinx
# pip install openstackdocstheme
To get the ``.html`` output locally, switch to the directory containing a
``conf.py`` and run:
.. code-block:: console
$ sphinx-build /path/to/source/ path/to/build/
The RST source is built into HTML using Sphinx, so that it is displayed on
the *docs.openstack.org/<guide-name>*. For example:
http://docs.openstack.org/user-guide/.
Using Tox to check builds
-------------------------
As a part of the review process, Jenkins runs gating scripts to check that
the patch is fine. Locally, you can use the Tox tool to ensure that a patch
works. To check all books, run the following command from the base directory
of repository:
.. code-block:: console
$ tox
The following individual checks are also availableː
* :command:`tox -e checkniceness` - to run the niceness tests (for example,
to see extra whitespaces)
* :command:`tox -e checksyntax` - to run syntax checks
* :command:`tox -e checklang` - to check all the translated manuals
* :command:`tox -e docs` - to build only RST-sourced manuals
* :command:`tox -e checkbuild` - to build all the manuals. This will also
generate a directory ``publish-docs`` that contains the built files for
inspection.
.. note::
The Configuration Reference and the Networking Guide are versioned,
all other guides are continuously published.
Guides for end users
--------------------
* The scripts are not written for Windows, but we encourage
cross-platform work on our scripts.
* If Tox stops working, try :command:`tox --recreate` to rebuild the
environment.
.. list-table::
:header-rows: 1
Build an existing patch locally
-------------------------------
* - Document
- Source location
- Target location
To build a patch locally:
* - API Guide
- http://git.openstack.org/cgit/openstack/api-site/tree/api-quick-start
- http://developer.openstack.org/api-guide/quick-start/
#. Change to the directory containing the appropriate repository:
* - End User Guide
- http://git.openstack.org/cgit/openstack/openstack-manuals/tree/doc/user-guide
- http://docs.openstack.org/user-guide
* openstack-manuals
* security-doc
* api-site
* - Command Line Interface Reference
- http://git.openstack.org/cgit/openstack/openstack-manuals/tree/doc/cli-reference
- http://docs.openstack.org/cli-reference
For example:
Developer guides
----------------
.. code-block:: console
Each project maintains its own developer guide. We do not publish these on
the docs site.
See http://docs.openstack.org/developer/openstack-projects.html.
$ cd openstack-manuals
API Complete Reference pages (HTML)
-----------------------------------
#. Create a local branch that contains the particular patch.
We are migrating away from WADL in 2016. Refer to :ref:`api-docs` for details.
Once the migration is complete, the API reference pages will be stored in the
project repositories and we will have redirects in place for these API pages.
.. code-block:: console
**POM file location**: api-site/api-ref/pom.xml
$ git review -d PATCH_ID
**Source file location**:
Where the value of ``PATCH_ID`` is a Gerrit commit number.
You can find this number on the patch link,
``https://review.openstack.org/#/c/PATCH_ID``.
- api-site/api-ref/src/docbkx for DocBook files
- api-site/api-ref/src/wadls/<project-name> for WADL files and code samples
#. Build all the books that are affected by changes in the patch set:
**Target file location**: http://developer.openstack.org/api-ref.html. See
the navigation bar on the left side for links to all API reference pages.
.. code-block:: console
API Complete Reference guides (PDFs)
------------------------------------
$ tox -e checkbuild
**POM file location**: api-site/api-ref-guides/pom.xml
**Source file location**:
- api-site/api-ref-guides/src for DocBook files for books
- api-site/api-ref/src/docbkx for DocBook files for chapters
- api-site/api-ref/src/wadls/<project-name> for WADL files and code samples
**Target file location**: http://developer.openstack.org/api-ref-guides/bk-api-ref.pdf.
See the navigation bar on the left side for links to all API reference pages.
On each reference page, a link for the PDF appears in the upper right corner.
Contributor guides
------------------
Generally, the docs.openstack.org/developer documentation is meant for
contributors to OpenStack projects. Each project's repo has a doc/source
directory where RST source files are stored. They are built automatically
with Sphinx when the patch is merged. For example, see
http://git.openstack.org/cgit/openstack/horizon/tree/doc/source for the
horizon contributor documentation source and http://docs.openstack.org/developer/horizon/
for the built documentation.
.. list-table::
:header-rows: 1
* - Document
- Source location
- Target location
* - Documentation Contributor Guide
- http://git.openstack.org/cgit/openstack/openstack-manuals/tree/doc/contributor-guide
- http://docs.openstack.org/contributor-guide/
* - Python Developer Documentation
- http://git.openstack.org/cgit/openstack/<project>/tree/master/doc/source/,
such as http://git.openstack.org/cgit/openstack/nova/tree/doc/source
- http://docs.openstack.org/developer/openstack-projects.html
* - Language Bindings and Python Clients
- http://git.openstack.org/cgit/openstack/python-<project>client/tree/master/doc/source/,
such as http://git.openstack.org/cgit/openstack/python-novaclient/tree/doc/source
- http://docs.openstack.org/developer/language-bindings.html
* - OpenStack Project Infrastructure
- http://git.openstack.org/cgit/openstack-infra/system-config/tree/doc/source
- http://docs.openstack.org/infra/system-config/
* - Tempest Testing Project
- http://git.openstack.org/cgit/openstack/tempest/tree/doc/source
- http://docs.openstack.org/developer/tempest/
#. Find the build result in ``publish-docs/index.html``.
Build jobs
~~~~~~~~~~
@ -210,7 +172,7 @@ Build jobs
The build jobs for documentation are stored in the
http://git.openstack.org/cgit/openstack-infra/project-config
repository. The :file:`zuul/layout.yaml` file and the
:file:`jenkins/jobs/manual-jobs.yaml` or :file:`jenkins/jobs/api-jobs.yaml`
``jenkins/jobs/manual-jobs.yaml`` or ``jenkins/jobs/api-jobs.yaml``
file contain the Jenkins build jobs that build to the docs.openstack.org
and developer.openstack.org sites, copying built files via FTP.
@ -218,95 +180,26 @@ The release specific books are built for the currently supported branches
(current and previous releases), development happens on the master branch.
The continuously released books are only built on the master branch.
.. _mvn:
Like other projects, the documentation projects use a number of jobs
that do automatic testing of patches.
Maven plug-in
~~~~~~~~~~~~~
The current jobs are:
The Maven plug-in is updated periodically with features we may want to
incorporate in the OpenStack build process. Specifically, 2.1.4 is what we
use for Kilo documentation as it contains features designed to make life
easier. These changes also required some changes in pom.xml for each book.
All these changes have been incorporated, so this information is to describe
the settings in pom.xml. A major new feature of this version of the plug-in is
that images are automatically handled for you. This saves two steps and adds
a level of validation.
You no longer have to add a postProcess section to your pom.xml configuration
to copy image files into the webhelp output directory unless you want to do
a clean up step of deleting the renamed directory. Instead, these settings
tell the build where to place the built files.
::
<targetDirectory>${basedir}/target/docbkx/webhelp/glossary<targetDirectory>
<webhelpDirname>/</webhelpDirname>
<pdfFilenameBase>bk-glossary</pdfFilenameBase>
The clouddocs-maven-plugin automatically detects which images you use in your
document and copies them to the output directory. When you use .svg graphics,
you do not have to create a .png version. Now, when you generate web help,
the clouddocs-maven-plugin automatically converts the .svg to a .png file
and uses it instead. You want to ensure all images have the <figure> tag
and use contentwidth="6in" as an attribute on the <imageobject>. The system
also checks for the availability of images before proceeding with the build,
but you may still see "Figure not found" errors that you can safely ignore.
When you generate web help, by default the plug-in now automatically generates
a PDF and puts it in the webhelp directory, so links will no longer break in
the PDF. You can also remove any pdf processing instructions from the book
file itself.
SNAPSHOT builds
---------------
To build with the latest SNAPSHOT version of the plug-in, do the following:
#. Clone the clouddocs-maven-plugin::
git clone https://git.openstack.org/openstack/clouddocs-maven-plugin
#. Open the repository::
cd clouddocs-maven-plugin
#. Build the plug-in::
mvn clean install
#. Edit your document's pom.xml file to depend on the current snapshot
version of the plugin. For example, 1.12.1-SNAPSHOT.
#. Build the document::
mvn clean generate-sources
Gates
~~~~~
Like other projects, the documentation projects use a number of gates that do
automatic testing of patches.
The current gates are:
* gate-openstack-manuals-tox-checklinks
* gate-openstack-manuals-tox-checkniceness
* gate-openstack-manuals-tox-checksyntax
* gate-openstack-manuals-tox-checkdeletions
* gate-openstack-manuals-tox-doc-publish-checkbuild
* gate-openstack-manuals-tox-checklang
Checklang gate
--------------
Checklang job
-------------
We only gate on manual/language combinations that are translated
sufficiently. For example, in openstack-manuals this includes Japanese with
the Security Guide, HA Guide and Install Guides.
* If an import from Zanata fails, we do not approve the import.
* If any other patch fails, the failure might get ignored.
* In any case of failure, a bug gets reported against the i18n project
(`launchpad link`_).
* In any case of failure, a bug gets reported against the `i18n project
<https://bugs.launchpad.net/openstack-i18n>`_.
If you want to manually run this check on your local workstation you can use
the checklang environment (:command:`tox -e checklang`). To use this
@ -314,10 +207,3 @@ environment, you first have to install the *xml2po* utility on your local
workstation. xml2po is part of the gnome-doc-utils and can be installed with
:command:`yum install gnome-doc-utils` (on RedHat-based distributions), or
:command:`zypper install xml2po` (on SUSE-based distributions).
.. Links:
.. _`release`: https://wiki.openstack.org/wiki/Releases
.. _`OpenStack Doc Tools`: http://git.openstack.org/cgit/openstack/openstack-doc-tools/
.. _`auto-generation of config tables`: http://git.openstack.org/cgit/openstack/openstack-doc-tools/tree/autogenerate_config_docs/README.rst
.. _`Documentation/Migrate`: https://wiki.openstack.org/wiki/Documentation/Migrate#Installation_Guide_Migration
.. _`launchpad link`: https://bugs.launchpad.net/openstack-i18n

View File

@ -33,7 +33,6 @@ Contents
json-conv.rst
tools-and-content-overview.rst
docs-builds.rst
build-locally.rst
doc-tools.rst
Glossary