openstack/openstacksdk
Code Issues Proposed changes

Build up contributor documentation section

Browse Source 
This change shifts a few things from the old contributing.rst file and
begins to expand in a few areas. A lot of the contents are broken out
into separate files, and there's a start on guidelines of how to
contribute new features for services.

Change-Id: I9e04c4ee6bdb25026defb3cccebaef4df61bb3d3
Partial-Bug: 1416553
This commit is contained in:
Brian Curtin 2015-01-31 18:32:49 -06:00
parent 9b2729cdd5
commit e2f198d688
7 changed files with 511 additions and 150 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

29
doc/source/contributors/create/examples/resource/fake.py Normal file
View File

@ -0,0 +1,29 @@
# Apache 2 header omitted for brevity
from openstack.fake import fake_service
from openstack import resource
class Fake(resource.Resource):
resource_key = "resource"
resources_key = "resources"
base_path = "/fake"
service = fake_service.FakeService()
id_attribute = "name"
allow_create = True
allow_retrieve = True
allow_update = True
allow_delete = True
allow_list = True
allow_head = True
#: The transaction date and time.
timestamp = resource.prop("x-timestamp")
#: The name of this resource.
name = resource.prop("name")
#: The value of the resource. Also available in headers.
value = resource.prop("value", alias="x-resource-value")
#: Is this resource cool? If so, set it to True.
#: This is a multi-line comment about cool stuff.
cool = resource.prop("cool", type=bool)

13
doc/source/contributors/create/examples/resource/fake_service.py Normal file
View File

@ -0,0 +1,13 @@
# Apache 2 header omitted for brevity
from openstack.auth import service_filter
class FakeService(service_filter.ServiceFilter):
"""The fake service."""
valid_versions = [service_filter.ValidVersion('v2')]
def __init__(self, version=None):
"""Create a fake service."""
super(FakeService, self).__init__(service_type='fake', version=version)

191
doc/source/contributors/create/resource.rst Normal file
View File

@ -0,0 +1,191 @@
Creating a New Resource
=======================
This guide will walk you through how to add resources for a service.
Naming Conventions
------------------
Above all, names across this project conform to Python's naming standards,
as laid out in `PEP 8 <https://www.python.org/dev/peps/pep-0008/>`_.
The relevant details we need to know are as follows:
* Module names are lower case, and separated by underscores if more than
one word. For example, ``openstack.object_store``
* Class names are capitalized, with no spacing, and each subsequent word is
capitalized in a name. For example, ``ServerMetadata``.
* Attributes on classes, including methods, are lower case and separated
by underscores. For example, ``allow_list`` or ``get_data``.
Services
********
Services in the OpenStack SDK are named after their program name, not their
code name. For example, the project often known as "Nova" is always called
"compute" within this SDK.
This guide walks through creating service for an OpenStack program called
"Fake". Following our guidelines, the code for its service would
live under the ``openstack.fake`` namespace. What follows is the creation
of a :class:`~openstack.resource.Resource` class for the "Fake" service.
Resources
*********
Resources are named after the server-side resource, which is set in the
``base_path`` attribute of the resource class. This guide creates a
resouce class for the ``/fake`` server resource, so the resource module
is called ``fake.py`` and the class is called ``Fake``.
An Example
----------
``openstack/fake/fake_service.py``
.. literalinclude:: examples/resource/fake_service.py
:language: Python
:linenos:
``openstack/fake/v2/fake.py``
.. literalinclude:: examples/resource/fake.py
:language: Python
:linenos:
``fake.Fake`` Attributes
------------------------
Each service's resources inherit from :class:`~openstack.resource.Resource`,
so they can override any of the base attributes to fit the way their
particular resource operates.
``resource_key`` and ``resources_key``
**************************************
These attributes are set based on how your resource responds with data.
The default values for each of these are ``None``, which works fine
when your resource returns a JSON body that can be used directly without a
top-level key, such as ``{"name": "Ernie Banks", ...}"``.
However, our ``Fake`` resource returns JSON bodies that have the details of
the resource one level deeper, such as
``{"resources": {"name": "Ernie Banks", ...}, {...}}``. It does a similar
thing with single resources, putting them inside a dictionary keyed on
``"resource"``.
By setting ``Fake.resource_key`` on *line 8*, we tell the ``Resource.create``,
``Resource.get``, and ``Resource.update`` methods that we're either sending
or receiving a resource that is in a dictionary with that key.
By setting ``Fake.resources_key`` on *line 9*, we tell the ``Resource.list``
method that we're expecting to receive multiple resources inside a dictionary
with that key.
``base_path``
*************
The ``base_path`` is the URL we're going to use to make requests for this
resource. In this case, *line 10* sets ``base_path = "/fake"``, which also
corresponds to the name of our class, ``Fake``.
Most resources follow this basic formula. Some cases are more complex, where
the URL to make requests to has to contain some extra data. The volume service
has several resources which make either basic requests or detailed requests,
so they use ``base_path = "/volumes/%s(detailed)"``. Before a request is made,
if ``detailed = True``, they convert it to a string so the URL becomes
``/volumes/detailed``. If it's ``False``, they only send ``/volumes/``.
``service``
***********
*Line 11* is an instance of the service we're implementing. Each resource
ties itself to the service through this setting, so that the proper URL
can be constructed.
In ``fake_service.py``, we specify the valid versions as well as what this
service is called in the service catalog. When a request is made for this
resource, the Session now knows how to construct the appropriate URL using
this ``FakeService`` instance.
``id_attribute``
****************
*Line 12* specifies that this resource uses a different identifier than
the default of ``id``. While IDs are used internally, such as for creating
request URLs to interact with an individual resource, they are exposed for
consistency so users always have one place to find the resource's identity.
Supported Operations
--------------------
The base :class:`~openstack.resource.Resource` disallows all types of requests
by default, requiring each resource to specify which requests they support.
On *lines 14-19*, our ``Fake`` resource specifies that it'll work with all
of the operations.
In order to have the following methods work, you must allow the corresponding
value by setting it to ``True``:
+----------------------------------------------+----------------+
| :class:`~openstack.resource.Resource.create` | allow_create |
+----------------------------------------------+----------------+
| :class:`~openstack.resource.Resource.delete` | allow_delete |
+----------------------------------------------+----------------+
| :class:`~openstack.resource.Resource.head` | allow_head |
+----------------------------------------------+----------------+
| :class:`~openstack.resource.Resource.list` | allow_list |
+----------------------------------------------+----------------+
| :class:`~openstack.resource.Resource.get` | allow_retrieve |
+----------------------------------------------+----------------+
| :class:`~openstack.resource.Resource.update` | allow_update |
+----------------------------------------------+----------------+
An additional attribute to set is ``put_update`` if your service uses ``PUT``
requests in order to update a resource. By default, ``PATCH`` requests are
used for ``Resource.update``.
Properties
----------
The way resource classes communicate values between the user and the server
are :class:`~openstack.resource.prop` objects. These act similarly to Python's
built-in property objects, but they share only the name - they're not the same.
Properties are set based on the contents of a response body or headers.
Based on what your resource returns, you should set ``prop``\s to map
those those values to ones on your :class:`~openstack.resource.Resource`
object.
*Line 22* sets a prop for ``timestamp`` , which will cause the
``Fake.timestamp`` attribute to contain the value returned in an
``X-Timestamp`` header, such as from a ``Fake.head`` request.
*Line 24* sets a prop for ``name``, which is a value returned in a body, such
as from a ``Fake.get`` request. Note from *line 12* that ``name`` is
specified its ``id`` attribute, so when this resource
is populated from a response, ``Fake.name`` and ``Fake.id`` are the same
value.
*Line 26* sets a prop which contains an alias. ``Fake.value`` will be set
when a response body contains a ``value``, or when a header contains
``X-Resource-Value``.
*Line 28* specifies a type to be checked before sending the value in a request.
In this case, we can only set ``Fake.cool`` to either ``True`` or ``False``,
otherwise a TypeError will be raised if the value can't be converted to the
expected type.
Documentation
-------------
We use Sphinx's ``autodoc`` feature in order to build API documentation for
each resource we expose. The attributes we override from
:class:`~openstack.resource.Resource` don't need to be documented, but any
:class:`~openstack.resource.prop` attributes must be. All you need to do is
add a comment *above* the line to document, with a colon following the
pound-sign.
*Lines 21, 23, 25, and 27-28* are comments which will then appear in the API
documentation. As shown in *lines 27 & 28*, these comments can span multiple
lines.

76
doc/source/contributors/index.rst
View File

@ -1,2 +1,74 @@
Contributors
============
Contributing to the OpenStack SDK
=================================
This section of documentation pertains to those who wish to contribute to the
development of this SDK. If you're looking for documentation on how to use
the SDK to build applications, please see the `user <../users>`_ section.
About the Project
-----------------
The OpenStack SDK is a Stackforge project aimed at providing a complete
software development kit for the programs which make up the OpenStack
community. It is a set of Python-based libraries, documentation, examples,
and tools released under the Apache 2 license.
Contacting the Developers
-------------------------
IRC
***
The developers of this project are available in the
`#openstack-sdks <http://webchat.freenode.net?channels=%23openstack-sdks>`_
channel on Freenode. This channel includes conversation on SDKs and tools
within the general OpenStack community, including OpenStackClient as well
as occasional talk about SDKs created for languages outside of Python.
Email
*****
The `openstack-dev <mailto:openstack-dev@openstack.org?subject=[python-openstacksdk]%20Question%20about%20the%20python-openstacksdk>`_
mailing list fields questions of all types on OpenStack. Using the
``[python-openstacksdk]`` filter to begin your email subject will ensure
that the message gets to SDK developers.
Development Environment
-----------------------
The first step towards contributing code and documentation is to setup your
development environment. We use a pretty standard setup, but it is fully
documented in our `setup <setup>`_ section.
.. toctree::
:maxdepth: 2
setup
Project Layout
--------------
The project contains a top-level ``openstack`` package, which houses several
modules that form the foundation upon which each service's API is built on.
Under the ``openstack`` package are packages for each of those services,
such as ``openstack.compute``.
.. toctree::
layout
Adding Features
---------------
Does this SDK not do what you need it to do? Is it missing a service? Are you
a developer on another project who wants to add their service? You're in the
right place. Below are examples of how to add new features to the
OpenStack SDK.
.. toctree::
:maxdepth: 2
create/resource
.. TODO(briancurtin): document how to create a proxy
.. TODO(briancurtin): document how to create auth plugins

170
doc/source/contributors/contributing.rst → doc/source/contributors/layout.rst
View File

@ -1,123 +1,12 @@
============
Contributing
============
How the SDK is organized
========================
python-openstacksdk is a Stackforge project, mirrored on `GitHub`_. Bugs and
Blueprints are handled on `Launchpad`_. Code reviews are hosted on `Gerrit`_.
The following diagram shows how the project is laid out.
.. _GitHub: https://github.com/stackforge/python-openstacksdk
.. _Launchpad: https://launchpad.net/python-openstacksdk
.. _Gerrit: https://review.openstack.org/#/q/project:stackforge/python-openstacksdk,n,z
Getting Setup
-------------
Python
******
The python-openstacksdk project supports Python versions 2.6, 2.7, 3.3+, and
pypy, so you'll need to have at least one of those to get started.
virtualenv
**********
Rather than installing the project's dependencies into your system-wide Python
installation, you should create a virtual environment for this project.
Install
^^^^^^^
Debian based platforms::
apt-get install -y python-virtualenv
RedHat based platforms::
yum install -y python-virtualenv
Other::
pip install virtualenv
Setup
^^^^^
::
$ virtualenv sdk
New python executable in sdk/bin/python
Installing setuptools, pip...done.
$ source sdk/bin/activate
(sdk)$
Getting the code
****************
If you haven't contributed in the openstack community before, be sure to read:
http://docs.openstack.org/infra/manual/developers.html
http://docs.openstack.org/infra/manual/developers.html#development-workflow
and then you'll be ready to::
git clone https://github.com/stackforge/python-openstacksdk.git
tox
***
We use `tox <https://tox.readthedocs.org>`_ as our test runner, as it provides
the ability to run your test against multiple versions. Going back to the
`Python`_ section, ideally you have all of the versions installed so tox
will accurately reflect how your code will run through the
`continuous integration <http://ci.openstack.org/>`_ system.::
(sdk)$ pip install tox
To run tox, just execute the ``tox`` command. With no arguments, it runs
everything in our ``tox.ini`` file. You can also give it a specific
environment to run.::
(sdk)$ tox
(sdk)$ tox -e py33
Using the code
**************
To run the examples or otherwise use the SDK within your environment, you'll
need to get the project's dependencies.::
(sdk)$ python setup.py develop
...
(sdk)$ python
>>> import openstack
Project Layout
--------------
The code is laid out in the following structure. This example shows files
relevant to working with code for the compute service's servers.::
openstack/
connection.py
resource.py
session.py
transport.py
auth/
identity/
v2.py
v3.py
compute/
compute_service.py
v2/
server.py
_proxy.py
tests/
compute/
v2/
test_server.py
.. literalinclude:: layout.txt
Session
*******
-------
The :class:`openstack.session.Session` manages an authenticator,
transport, and user preferences. It exposes methods corresponding to
@ -125,8 +14,8 @@ HTTP verbs, and injects your authentication token into a request,
determines any service preferences callers may have set, gets the endpoint
from the authenticator, and sends the request out through the transport.
Authenticator
^^^^^^^^^^^^^
Auth
----
As the `Session`_ needs a way to get a token and endpoint, it is constructed
with either a ``v2.Auth`` or ``v3.Auth`` object from
@ -135,7 +24,7 @@ service and are able to handle things like authentication tokens and their
expiration, and the service catalog.
Transport
^^^^^^^^^
---------
The :class:`openstack.transport.Transport` class in is built on
`requests.Session <http://docs.python-requests.org/en/latest/user/advanced/>`_
@ -151,7 +40,7 @@ follows a
that isn't suitable for this library.
Resource
********
--------
The :class:`openstack.resource.Resource` base class is the building block
of any service implementation. ``Resource`` objects correspond to the
@ -187,17 +76,25 @@ string replacement is used, e.g., ``base_path = "/servers/%(server_id)s/ips"``.
``resource_key`` and ``resources_key`` are attributes to set when a
``Resource`` returns more than one item in a response, or otherwise
requires a key to obtain the response value. For example, the ``Server``
class sets ``resource_key = "server"`` and ``resource_keys = "servers"``
to support the fact that multiple ``Server``\s can be returned, and each
is identified with a singular noun in the response.
class sets ``resource_key = "server"`` as an individual ``Server`` is
stored in a dictionary keyed with the singular noun,
and ``resource_keys = "servers"`` as multiple ``Server``\s are stored in
a dictionary keyed with the plural noun in the response.
Proxy
*****
-----
Each service implements a ``Proxy`` class, within the
``openstack/<program_name>/vX/_proxy.py`` module. For example, the v2 compute
service's ``Proxy`` exists in ``openstack/compute/v2/_proxy.py``.
This ``Proxy`` class manages a :class:`~openstack.sessions.Session` and
provides a higher-level interface for users to work with via a
:class:`~openstack.connection.Connection` instance. Rather than requiring
users to maintain their own session and work with lower-level
:class:`~openstack.resource.Resource` objects, the ``Proxy`` interface
offers a place to make things easier for the caller.
Each ``Proxy`` class implements methods which act on the underlying
``Resource`` classes which represent the service. For example::
@ -214,7 +111,7 @@ under construction, as we figure out the best way to implement them in a
way which will apply nicely across all of the services.
Connection
**********
----------
The :class:`openstack.connection.Connection` class builds atop a ``Session``
object, and provides a higher level interface constructed of ``Proxy``
@ -227,26 +124,3 @@ to this SDK, managing the lower level connecton bits and exposing the
If you've built proper ``Resource`` objects and implemented methods on the
corresponding ``Proxy`` object, the high-level interface to your service
should now be exposed.
Contacting the Team
-------------------
IRC
***
The developers of this project are available in the
`#openstack-sdks <http://webchat.freenode.net?channels=%23openstack-sdks>`_
channel on Freenode.
Email
*****
The `openstack-dev <mailto:openstack-dev@openstack.org?subject=[python-openstacksdk]%20Question%20about%20the%20python-openstacksdk>`_
mailing list fields questions of all types on OpenStack. Using the
``[python-openstacksdk]`` filter to begin your email subject will ensure
that the message gets to SDK developers.
If you're interested in communicating one-on-one, the following developers
of the project are available:
* Brian Curtin <brian@python.org>

18
doc/source/contributors/layout.txt Normal file
View File

@ -0,0 +1,18 @@
openstack/
connection.py
resource.py
session.py
transport.py
auth/
identity/
v2.py
v3.py
compute/
compute_service.py
v2/
server.py
_proxy.py
tests/
compute/
v2/
test_server.py

164
doc/source/contributors/setup.rst Normal file
View File

@ -0,0 +1,164 @@
Creating a Development Environment
==================================
Required Tools
--------------
Python
******
As the OpenStack SDK is developed in Python, you will need at least one
version of Python installed. It is strongly preferred that you have at least
one of version 2 and one of version 3 so that your tests are run against both.
Our continuous integration system runs against several versions, so ultimately
we will have the proper test coverage, but having multiple versions locally
results in less time spent in code review when changes unexpectedly break
other versions.
Python can be downloaded from https://www.python.org/downloads.
virtualenv
**********
In order to isolate our development environment from the system-based Python
installation, we use `virtualenv <https://virtualenv.pypa.io/en/latest/>`_.
This allows us to install all of our necessary dependencies without
interfering with anything else, and preventing others from interfering with us.
Virtualenv must be installed on your system in order to use it, and it can be
had from PyPI, via pip, as follows. Note that you may need to run this
as an administrator in some situations.::
$ apt-get install python-virtualenv # Debian based platforms
$ yum install python-virtualenv # Red Hat based platforms
$ pip install virtualenv # Mac OS X and other platforms
You can create a virtualenv in any location. A common usage is to store all
of your virtualenvs in the same place, such as under your home directory.
To create a virtualenv for the default Python, likely a version 2, run
the following::
$ virtualenv $HOME/envs/sdk
To create an environment for a different version, such as Python 3, run
the following::
$ virtualenv -p python3.4 $HOME/envs/sdk3
When you want to enable your environment so that you can develop inside of it,
you *activate* it. To activate an environment, run the /bin/activate
script inside of it, like the following::
$ source $HOME/envs/sdk3/bin/activate
(sdk3)$
Once you are activated, you will see the environment name in front of your
command prompt. In order to exit that environment, run the ``deactivate``
command.
tox
***
We use `tox <https://tox.readthedocs.org/en/latest/>`_ as our test runner,
which allows us to run the same test commands against multiple versions
of Python. Inside any of the virtualenvs you use for working on the SDK,
run the following to install ``tox`` into it.::
(sdk3)$ pip install tox
Git
***
The source of the OpenStack SDK is stored in Git. In order to work with our
source repository, you must have Git installed on your system. If your
system has a package manager, it can likely be had from there. If not,
you can find downloads or the source at http://git-scm.com.
Getting the Source Code
-----------------------
.. TODO(briancurtin): We should try and distill the following document
into the minimally necessary parts to include directly in this section.
I've talked to several people who are discouraged by that large of a
document to go through before even getting into the project they want
to work on. I don't want that to happen to us because we have the potential
to be more public facing than a lot of other projects.
.. note:: Before checking out the code, please read the OpenStack
`Developer's Guide <http://docs.openstack.org/infra/manual/developers.html>`_
for details on how to use the continuous integration and code
review systems that we use.
The canonical Git repository is hosted on openstack.org at
http://git.openstack.org/cgit/stackforge/python-openstacksdk/, with a
mirror on GitHub at https://github.com/stackforge/python-openstacksdk.
Because of how Git works, you can create a local clone from either of those,
or your own personal fork.::
(sdk3)$ git clone git@github.com:briancurtin/python-openstacksdk.git
(sdk3)$ cd python-openstacksdk
Installing Dependencies
-----------------------
In order to work with the SDK locally, such as in the interactive interpreter
or to run example scripts, you need to install the project's dependencies.::
(sdk3)$ pip install -r requirements.txt
After the downloads and installs are complete, you'll have a fully functional
environment to use the SDK in. This step installs the following dependencies.
* `oslo.utils <https://pypi.python.org/pypi/oslo.utils>`_, which we use
for its ``timeutils`` module when calculating if or when authentication
tokens are considered expired.
* `pbr <https://pypi.python.org/pypi/pbr>`_, or the Python Build
Reasonableness project. pbr injects a set of common defaults which are used
throughout the OpenStack project.
* `requests <https://pypi.python.org/pypi/requests>`_, which we use in the
:class:`~openstack.transport.Transport` class to handle HTTP requests and
responses.
* `six <https://pypi.python.org/pypi/six>`_, which we use for compatibility
across Python 2 and 3.
* `stevedore <https://pypi.python.org/pypi/stevedore>`_, which we use for
working with plugins. stevedore builds on setuptools ``entry_points``.
Running the Tests
-----------------
In order to run the entire test suite, simply run the ``tox`` command inside
of your source checkout. This will attempt to run every test command listed
inside of ``tox.ini``, which includes Python 2.6, 2.7, 3.3, 3.4, PyPy, and
a PEP 8 check. You should run the full test suite on all versions before
submitting changes for review in order to avoid unexpected failures in
the continuous integration system.::
(sdk3)$ tox
...
py33: commands succeeded
py34: commands succeeded
py26: commands succeeded
py27: commands succeeded
pypy: commands succeeded
pep8: commands succeeded
congratulations :)
During development, it may be more convenient to run a subset of the tests
to keep test time to a minimum. You can choose to run the tests only on one
version. A step further is to run only the tests you are working on.::
(sdk3)$ tox -e py34 # Run run the tests on Python 3.4
(sdk3)$ tox -e py34 TestContainer # Run only the TestContainer tests on 3.4
Building the Documentation
--------------------------
Our documentation is written in reStructured Text and is built using
Sphinx. A ``docs`` command is availble in our ``tox.ini``, allowing you
to build the documentation like you'd run tests. The ``docs`` command is
not evaluated by default.::
(sdk3)$ tox -e docs
That command will cause the documentation, which lives in the ``docs`` folder,
to be built. HTML output is the most commonly referenced, which is located
in ``docs/build/html``.