openstack/operations-guide
Code Issues Proposed changes

Added HA/Neutron architecture.

Browse Source 
Includes overview, node types and diagrams, and example configuration.

Closes-Bug: #1268784

Change-Id: I91d66dc3fb7dff2ad54e8293497901bbdd620f12
This commit is contained in:
Summer Long 2014-01-20 15:42:23 +10:00 committed by annegentle
parent a8c683d1d9
commit 56f3dff89f
10 changed files with 706 additions and 39 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
doc/openstack-ops/ch_arch_examples.xml Normal file
View File

@ -0,0 +1,19 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE chapter [
<!-- Some useful entities borrowed from HTML -->
<!ENTITY ndash "&#x2013;">
<!ENTITY mdash "&#x2014;">
<!ENTITY hellip "&#x2026;">
<!ENTITY plusmn "&#xB1;">
]>
<chapter xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink" version="5.0"
xml:id="example_architecture">
<?dbhtml stop-chunking?>
<title>Example Architectures</title>
<xi:include href="section_arch_example-nova.xml"/>
<xi:include href="section_arch_example-neutron.xml"/>
</chapter>

BIN
doc/openstack-ops/figures/ha_network_diagram_basic.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 48 KiB

BIN
doc/openstack-ops/figures/ha_network_diagram_performance.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 50 KiB

BIN
doc/openstack-ops/figures/ha_node_compute.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 20 KiB

BIN
doc/openstack-ops/figures/ha_node_controller.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 62 KiB

BIN
doc/openstack-ops/figures/ha_node_network.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 29 KiB

BIN
doc/openstack-ops/figures/ha_node_storage.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 24 KiB

2
doc/openstack-ops/part_architecture.xml
View File

@ -21,7 +21,7 @@
decisions you will need to make during the process.
</para>
</partintro>
<xi:include href="ch_arch_example.xml"/>
<xi:include href="ch_arch_examples.xml"/>
<xi:include href="ch_arch_provision.xml"/>
<xi:include href="ch_arch_cloud_controller.xml"/>
<xi:include href="ch_arch_compute_nodes.xml"/>

645
doc/openstack-ops/section_arch_example-neutron.xml Normal file
View File

@ -0,0 +1,645 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE section [
<!-- Some useful entities borrowed from HTML -->
<!ENTITY ndash "&#x2013;">
<!ENTITY mdash "&#x2014;">
<!ENTITY hellip "&#x2026;">
<!ENTITY plusmn "&#xB1;">
]>
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude" xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="example_architecture-neutron">
<?dbhtml stop-chunking?>
<title>Example Architecture - OpenStack Networking</title>
<para>This chapter provides an example architecture using OpenStack Networking in a highly
available environment.</para>
<section xml:id="overview-neutron">
<title>Overview</title>
<para>A highly-available environment can be put into place if you require an environment
that can scale horizontally, or want your cloud to continue to be operational in case of
node failure. This example architecture has been written based on the current default feature
set of OpenStack Havana, with an emphasis on high availability.</para>
<section xml:id="overview_components_neutron">
<title>Components</title>
<informaltable rules="all">
<col width="27%"/>
<col width="73%"/>
<tbody>
<tr>
<td><para>OpenStack release</para></td>
<td><para><emphasis role="bold">Havana</emphasis></para></td>
</tr>
<tr>
<td><para>Host operating system</para></td>
<td><para><emphasis role="bold">Red Hat Enterprise Linux 6.5</emphasis></para></td>
</tr>
<tr>
<td><para>OpenStack package repository</para></td>
<td><para><link
xlink:href="http://repos.fedorapeople.org/repos/openstack/openstack-havana/rdo-release-havana-7.noarch.rpm"
>Red Hat Distributed OpenStack (RDO)</link>
(http://repos.fedorapeople.org/repos/openstack/openstack-havana/rdo-release-havana-7.noarch.rpm)</para></td>
</tr>
<tr>
<td><para>Hypervisor</para></td>
<td><para><emphasis role="bold">KVM</emphasis></para></td>
</tr>
<tr>
<td><para>Database</para></td>
<td><para><emphasis role="bold">MySQL</emphasis></para></td>
</tr>
<tr>
<td><para>Message queue</para></td>
<td><para><emphasis role="bold">Qpid</emphasis></para></td>
</tr>
<tr>
<td><para>Networking service</para></td>
<td><para><emphasis role="bold">OpenStack Networking</emphasis></para></td>
</tr>
<tr>
<td><para>Tenant Network Separation</para></td>
<td><para><emphasis role="bold">VLAN</emphasis></para>
</td>
</tr>
<tr>
<td><para>Image Service (glance)
back-end</para></td>
<td><para><emphasis role="bold">GlusterFS</emphasis></para></td>
</tr>
<tr>
<td><para>Identity Service (keystone)
driver</para></td>
<td><para><emphasis role="bold">SQL</emphasis></para></td>
</tr>
<tr>
<td><para>Block Storage Service (cinder)
back-end</para></td>
<td><para><emphasis role="bold">GlusterFS</emphasis></para></td>
</tr>
</tbody>
</informaltable>
</section>
<section xml:id="rational-neutron"><title>Rationale</title>
<para>This example architecture has been selected based on the current default feature set
of OpenStack Havana, with an emphasis on high availability. This architecture is currently
being deployed in an internal Red Hat OpenStack cloud, and used to run hosted and shared
services which by their nature must be highly available.
</para>
<para>This architecture's components have been selected for the following reasons:</para>
<itemizedlist>
<listitem>
<para><emphasis role="bold">Red Hat Enterprise Linux</emphasis> - You must
choose an operating system that can run on all of the physical nodes.
This example architecture is based on Red Hat Enterprise Linux, which
offers reliability, long-term support, certified testing, and is
hardened. Enterprise customers, now moving into OpenStack usage,
typically require these advantages.</para>
</listitem>
<listitem>
<para><emphasis role="bold">RDO</emphasis> - The Red Hat Distributed
OpenStack package offer an easy way to download the most current
OpenStack release that is built for the Red Hat Enterprise Linux
platform.</para>
</listitem>
<listitem>
<para><emphasis role="bold">KVM</emphasis> - KVM is the supported hypervisor
of choice for Red Hat Enterprise Linux (and included in distribution).
It is feature complete, and free from licensing charges and
restrictions.</para>
</listitem>
<listitem>
<para><emphasis role="bold">MySQL</emphasis> - Mysql is used as the database
backend for all databases in the OpenStack environment. MySQL is the
supported database of choice for Red Hat Enterprise Linux (and included
in distribution); the database is open source, scalable, and handles
memory well.</para>
</listitem>
<listitem>
<para><emphasis role="bold">Qpid</emphasis> - Apache Qpid offers 100 percent
compatibility with the Advanced Message Queuing Protocol Standard, and
its broker is available for both C++ and Java.</para>
</listitem>
<listitem>
<para><emphasis role="bold">OpenStack Networking</emphasis> - OpenStack
Networking offers sophisticated networking functionality, including
Layer 2 (L2) network segregation and provider networks.</para>
</listitem>
<listitem>
<para><emphasis role="bold">VLAN</emphasis> - Using a virtual local area
network offers broadcast control, security, and physical layer
transparency. If needed, use VXLAN to extend your address space.</para>
</listitem>
<listitem>
<para><emphasis role="bold">GlusterFS</emphasis> - GlusterFS offers scalable
storage. As your environment grows, you can continue to add more storage
nodes (instead of being restricted, for example, by an expensive storage
array).</para>
</listitem>
</itemizedlist>
</section>
</section>
<section xml:id="detailed_description">
<title>Detailed Description</title>
<section xml:id="node_types">
<title>Node Types</title>
<para>This section gives you a breakdown of the different nodes that make up
the OpenStack environment. A node is a physical machine that is provisioned
with an operating system, and running a defined software stack on top of it.
The following table provides node descriptions and specifications.</para>
<table rules="all">
<caption>Node Types</caption>
<col width="11%"/>
<col width="62%"/>
<col width="27%"/>
<thead>
<tr>
<th>Type</th>
<th>Description</th>
<th>Example Hardware</th>
</tr>
</thead>
<tbody>
<tr>
<td>Controller</td>
<td><para>Controller nodes are responsible for running the management software services needed for the
OpenStack environment to function. These nodes:</para>
<para><itemizedlist>
<listitem>
<para>Provide the front door that people access as well as the API
services which all other components in the environment talk
to.</para></listitem>
<listitem>
<para>Run a number of services in a highly available fashion,
utilising Packemaker and HAProxy to provide a virtual IP and
load-balancing functions so all controller nodes are being
used.</para></listitem>
<listitem>
<para>Supply highly available "infrastructure" services as MySQL and
Qpid which underpin all the services.</para></listitem>
<listitem>
<para>Provide what is known as "persistent storage" through services
run on the host as well. This persistent storage is backed onto
the storage nodes for reliability.</para></listitem>
</itemizedlist></para><para>See <xref linkend="node_controller-diagram"/>.</para></td>
<td>
<para>Model: Dell R620</para>
<para>CPU: 2 x Intel® Xeon® CPU E5-2620 0 @ 2.00GHz</para>
<para>Memory: 32GB</para>
<para>Disk: 2 x 300GB 10000 RPM SAS Disks</para>
<para>Network: 2 x 10G network ports</para>
</td>
</tr>
<tr>
<td>Compute</td>
<td><para>Compute nodes run the virtual machine instances in OpenStack. They:</para>
<para><itemizedlist>
<listitem>
<para>Run the bare minimum of services needed to facilitate
these instances.</para>
</listitem>
<listitem>
<para>Use local storage on the node for the virtual machines, so
that no VM migration or instance recovery at node failure is
possible.</para>
</listitem>
</itemizedlist>See <xref linkend="node_compute-diagram"/>.</para></td>
<td>
<para>Model: Dell R620</para>
<para>CPU: 2x Intel® Xeon® CPU E5-2650 0 @ 2.00GHz</para>
<para>Memory: 128GB</para>
<para>Disk: 2 x 600GB 10000 RPM SAS Disks</para>
<para>Network: 4 x 10G network ports (For future proofing expansion)</para>
</td>
</tr>
<tr>
<td>Storage</td>
<td><para>Storage nodes store all the data required for the environment, including disk images in the
Image Service library, and the persistent storage volumes created by the
Block Storage service. Storage nodes use GlusterFS technology to
keep the data highly available and scalable.</para>
<para>See <xref linkend="node_storage-diagram"/>.</para></td>
<td>
<para>Model: Dell R720xd</para>
<para>CPU: 2 x Intel® Xeon® CPU E5-2620 0 @ 2.00GHz</para>
<para>Memory: 64GB</para>
<para>Disk: 2 x 500GB 7200 RPM SAS Disks + 24 x 600GB 10000 RPM SAS
Disks</para>
<para>Raid Controller: PERC H710P Integrated RAID Controller, 1GB NV
Cache</para>
<para>Network: 2 x 10G network ports</para>
</td>
</tr>
<tr>
<td>Network</td>
<td><para>Network nodes are responsible for doing all the virtual networking needed
for people to create public or private networks, and uplink their virtual
machines into external networks. Network nodes:</para><para>
<itemizedlist>
<listitem><para>Form the only ingress and egress point for instances running on top of OpenStack.</para></listitem>
<listitem><para>Run all of the environment's networking services with the exception of the networking API
service (which runs on the controller node).</para></listitem>
</itemizedlist></para>
<para>See <xref linkend="node_network-diagram"/>.</para></td>
<td>
<para>Model: Dell R620</para>
<para>CPU: 1 x Intel® Xeon® CPU E5-2620 0 @ 2.00GHz</para>
<para>Memory: 32GB</para>
<para>Disk: 2 x 300GB 10000 RPM SAS Disks</para>
<para>Network: 5 x 10G network ports</para>
</td>
</tr>
<tr>
<td>Utility</td>
<td><para>Utility nodes are used by internal administration staff only to provide a
number of basic system administration functions needed to get the
environment up and running, and to maintain the hardware, OS, and software
on which it runs.</para>
<para>These nodes run services such as provisioning, configuration
management, monitoring, or GlusterFS management software. They are not
required to scale although these machines are usually backed up.</para></td>
<td>
<para>Model: Dell R620</para>
<para>CPU: 2x Intel® Xeon® CPU E5-2620 0 @ 2.00GHz</para>
<para>Memory: 32 GB</para>
<para>Disk: 2 x 500GB 7200 RPM SAS Disks</para>
<para>Network: 2 x 10G network ports</para>
</td>
</tr>
</tbody>
</table>
</section>
<section xml:id="networking_layout">
<title>Networking Layout</title>
<para>The network contains all the management devices for all hardware in the environment
(for example, by including Dell iDrac7 devices for the hardware nodes, and management
interfaces for network switches). The network is accessed by internal staff only when
diagnosing or recovering a hardware issue.</para>
<section xml:id="OS_network">
<title>OpenStack internal network</title>
<para>This network is used for OpenStack management functions and traffic, including
services needed for the provisioning of physical nodes
(<systemitem>pxe</systemitem>, <systemitem>tftp</systemitem>,
<systemitem>kickstart</systemitem>), traffic between various OpenStack node
types using OpenStack APIs and messages (for example,
<systemitem>nova-compute</systemitem> talking to
<systemitem>keystone</systemitem> or <systemitem>cinder-volume</systemitem>
talking to <systemitem>nova-api</systemitem>), and all traffic for storage data to
the storage layer underneath by the Gluster protocol. All physical nodes have at
least one network interface (typically <systemitem>eth0</systemitem>) in this
network. This network is only accessible from other VLANs on port 22 (for
<systemitem>ssh</systemitem> access to manage machines).</para>
</section>
<section xml:id="public_network">
<title>Public Network</title>
<para>This network is a combination of:
<itemizedlist>
<listitem><para>IP addresses for public-facing interfaces on the
controller nodes (which end users will access the OpenStack services)</para></listitem>
<listitem><para>A range of publicly routable, IPv4 network addresses to be used
by OpenStack Networking for floating IPs. You may be restricted in your access
to IPv4 addresses; a large range of IPv4 addresses is not necessary.
</para></listitem>
<listitem><para>Routers for private networks created within OpenStack.</para></listitem>
</itemizedlist>
</para>
<para>This network is connected to the controller nodes so users can access the OpenStack
interfaces, and connected to the network nodes to provide VMs with publicly routable
traffic functionality. The network is also connected to the utility machines so that
any utility services that need to be made public (such as system monitoring) can be
accessed.</para>
</section>
<section xml:id="vm_network">
<title>VM traffic network</title>
<para>This is a closed network that is not publicly routable and is simply used as a
private, internal network for traffic between virtual machines in OpenStack, and
between the virtual machines and the network nodes which provide l3 routes out to
the public network (and floating IPs for connections back in to the VMs). Because
this is a closed network, we are using a different address space to the others to
clearly define the separation. Only Compute and OpenStack Networking nodes need to
be connected to this network.</para>
</section>
</section>
<section xml:id="node_connectivity">
<title>Node connectivity</title>
<para>The following section details how the nodes are connected to the different networks
(see <xref linkend="networking_layout"/>), and what other considerations need to take
place (for example, bonding) when connecting nodes to the networks.</para>
<section xml:id="node_connectivity-initial">
<title>Initial deployment</title>
<para>Initially, the connection setup should revolve around keeping the connectivity simple
and straightforward, in order to minimise deployment complexity and time to deploy. The
following deployment aims to have 1x10G connectivity available to all Compute nodes, while
still leveraging bonding on appropriate nodes for maximum performance.</para>
<figure>
<title>Basic node deployment</title>
<mediaobject>
<imageobject>
<imagedata fileref="figures/ha_network_diagram_basic.png" width="5in"/>
</imageobject>
</mediaobject>
</figure>
</section>
<section xml:id="node_connectivity-performance">
<title>Connectivity for maximum performance</title>
<para>If the networking performance of the basic layout is not enough, you can move to
the following layout which provides 2x10G network links to all instances in the
environment, as well as providing more network bandwidth to the storage
layer.</para>
<figure>
<title>Performance node deployment</title>
<mediaobject>
<imageobject>
<imagedata fileref="figures/ha_network_diagram_performance.png" width="5in"/>
</imageobject>
</mediaobject>
</figure>
</section>
</section>
<section xml:id="node_diagrams">
<title>Node Diagrams</title>
<para>The following diagrams include logical information about the different types of nodes,
indicating what services will be running on top of them, and how they interact with each
other. The diagrams also illustrate how the availability and scalability of services are
achieved.</para>
<figure xml:id="node_controller-diagram">
<title>Controller node</title>
<mediaobject>
<imageobject>
<imagedata fileref="figures/ha_node_controller.png" width="5in"/>
</imageobject>
</mediaobject>
</figure>
<figure xml:id="node_compute-diagram">
<title>Compute Node</title>
<mediaobject>
<imageobject>
<imagedata fileref="figures/ha_node_compute.png" width="5in"/>
</imageobject>
</mediaobject>
</figure>
<figure xml:id="node_network-diagram">
<title>Network Node</title>
<mediaobject>
<imageobject>
<imagedata fileref="figures/ha_node_network.png" width="5in"/>
</imageobject>
</mediaobject>
</figure>
<figure xml:id="node_storage-diagram">
<title>Storage Node</title>
<mediaobject>
<imageobject>
<imagedata fileref="figures/ha_node_storage.png" width="5in"/>
</imageobject>
</mediaobject>
</figure>
</section>
</section>
<section xml:id="software_config">
<title>Example component configuration</title>
<para>The following tables includes example configuration and considerations for both third-party and OpenStack
components: <table rules="all">
<caption>Third-party component configuration</caption>
<col width="10%"/>
<col width="20%"/>
<col width="40%"/>
<col width="30%"/>
<thead>
<tr>
<th>Component</th>
<th>Tuning</th>
<th>Availability</th>
<th>Scalability</th>
</tr>
</thead>
<tbody>
<tr>
<td>MySQL</td>
<td><systemitem>binlog-format = row</systemitem></td>
<td>Master-Master replication. However, both nodes are not used at the same
time. Replication keeps all nodes as close to being up to date as
possible (although the asynchronous nature of the replication means a
fully consistent state is not possible). Connections to the database
only happen through a Pacemaker virtual IP, ensuring that most problems that
occur with master-master replication can be avoided.</td>
<td>Not heavily considered. Once load on the MySQL server increases enough
that scalability needs to be considered, multiple masters or a
master/slave setup can be used.</td>
</tr>
<tr>
<td>Qpid</td>
<td><systemitem>max-connections=1000 worker-threads=20
connection-backlog=10</systemitem>, sasl security enabled with
SASL-BASIC authentication</td>
<td>Qpid is added as a resource to the Pacemaker software that runs on
Controller nodes where Qpid is situated. This ensures only one Qpid
instance is running at one time, and the node with the Pacemaker virtual IP will
always be the node running Qpid.</td>
<td>Not heavily considered. However, Qpid can be changed to run on all
controller nodes for scalability and availability purposes, and removed
from Pacemaker.</td>
</tr>
<tr>
<td>Haproxy</td>
<td><systemitem>maxconn 3000</systemitem></td>
<td>Haproxy is a software layer-7 load balancer used to front door all
clustered OpenStack API components and do SSL termination. Haproxy can
be added as a resource to the Pacemaker software that runs on the
Controller nodes where HAProxy is situated. This ensures that only one
HAProxy instance is running at one time, and the node with the Pacemaker
virtual IP will always be the node running HAProxy.</td>
<td>Not considered. Haproxy has small enough performance overheads that a
single instance should scale enough for this level of workload. If extra
scalability is needed, <systemitem>keepalived</systemitem> or other
Layer-4 load balancing can be introduced to be placed in front of
multiple copies of Haproxy.</td>
</tr>
<tr>
<td>Memcached</td>
<td><systemitem>MAXCONN="8192" CACHESIZE="30457"</systemitem></td>
<td>Memcached is a fast in-memory key/value cache software that is used by
OpenStack components for caching data and increasing performance.
Memcached runs on all controller nodes, ensuring that should one go
down, another instance of memcached is available.</td>
<td>Not considered. A single instance of memcached should be able to scale
to the desired workloads. If scalability is desired, Haproxy can be
placed in front of Memcached (in raw <systemitem>tcp</systemitem> mode)
to utilise multiple <systemitem>memcached</systemitem> instances for
scalability. However, this might cause cache consistency issues.</td>
</tr>
<tr>
<td>Pacemaker</td>
<td>Configured to use <systemitem>corosync</systemitem> and
<systemitem>cman</systemitem> as a cluster communication
stack/quorum manager, and as a two-node cluster.</td>
<td><para>Pacemaker is the clustering software used to ensure the availability of services running on
the controller and network nodes: <itemizedlist>
<listitem>
<para>Because Pacemaker is cluster software, the software
itself handles its own availability, leveraging
<systemitem>corosync</systemitem> and
<systemitem>cman</systemitem> underneath.</para>
</listitem>
<listitem>
<para>If you use the GlusterFS native client, no virtual IP
is needed since the client knows all about nodes after
initial connection, and automatically routes around
failures on the client side.</para>
</listitem>
<listitem>
<para>If you use the NFS or SMB adaptor, you will need a
virtual IP on which to mount the GlusterFS
volumes.</para>
</listitem>
</itemizedlist>
</para></td>
<td>If more nodes need to be made cluster aware, Pacemaker can scale to 64
nodes.</td>
</tr>
<tr>
<td>GlusterFS</td>
<td><systemitem>glusterfs</systemitem> performance profile "virt" enabled on
all volumes. Volumes are setup in 2-node replication.</td>
<td>Glusterfs is a clustered file system that is run on the storage
nodes to provide persistent scalable data storage the the environment.
Because all connections to gluster use the
<systemitem>gluster</systemitem> native mount points, the
<systemitem>gluster</systemitem> instances themselves provide
availability and failover functionality.</td>
<td>The scalability of GlusterFS storage can be achieved by adding in more
storage volumes.</td>
</tr>
</tbody>
</table>
<table rules="all">
<caption>OpenStack component configuration</caption>
<col width="110px"/>
<col width="8%"/>
<col width="19%"/>
<col width="35%"/>
<col width="30%"/>
<thead>
<tr>
<th>Component</th>
<th>Node type</th>
<th>Tuning</th>
<th>Availability</th>
<th>Scalability</th>
</tr>
</thead>
<tbody>
<tr>
<td>Dashboard (horizon)</td>
<td>Controller</td>
<td>Configured to use <systemitem>memcached</systemitem> as a session store,
<systemitem>neutron</systemitem> support is enabled,
<systemitem>can_set_mount_point = False</systemitem></td>
<td>The Dashboard is run on all controller nodes, ensuring at least once
instance will be available in case of node failure. It also sits behind
HAProxy, which detects when the software fails and routes requests
around the failing instance.</td>
<td>The Dashboard is run on all controller nodes, so scalability can be achieved with additional
controller nodes. Haproxy allows scalability for the Dashboard as more nodes are added.</td>
</tr>
<tr>
<td>Identity (keystone)</td>
<td>Controller</td>
<td>Configured to use memcached for caching, and use PKI for tokens.</td>
<td>Identity is run on all controller nodes, ensuring at least once instance will be available
in case of node failure. Identity also sits behind HAProxy, which detects when the software fails
and routes requests around the failing instance.</td>
<td>Identity is run on all controller nodes, so scalability can be achieved with additional
controller nodes. Haproxy allows scalability for Identity as more nodes are added.</td>
</tr>
<tr>
<td>Image Service (glance)</td>
<td>Controller</td>
<td><systemitem>/var/lib/glance/images</systemitem> is a GlusterFS native
mount to a Gluster volume off the storage layer.</td>
<td>The Image Service is run on all controller nodes, ensuring at least once
instance will be available in case of node failure. It also sits behind
HAProxy, which detects when the software fails and routes requests
around the failing instance.</td>
<td>The Image Service is run on all controller nodes, so scalability can be achieved with additional controller
nodes. HAProxy allows scalability for the Image Service as more nodes are added.</td>
</tr>
<tr>
<td>Compute (nova)</td>
<td>Controller, Compute</td>
<td><para>Configured to use Qpid. <systemitem>qpid_heartbeat = 10</systemitem>,
configured to use <systemitem>memcached</systemitem> for caching,
configured to use <systemitem>libvirt</systemitem>, configured to use
<systemitem>neutron</systemitem>.</para>
<para>Configured <systemitem>nova-consoleauth</systemitem> to use
<systemitem>memcached</systemitem> for session management (so
that it can have multiple copies and run in a load balancer).</para></td>
<td><para>The nova API, scheduler, objectstore, cert, consoleauth, conductor, and vncproxy services are
run on all controller nodes, ensuring at least once instance will be
available in case of node failure. Compute is also behind HAProxy,
which detects when the software fails and routes requests around the
failing instance.</para>
<para>Compute's compute and conductor services, which run on the compute nodes, are only needed to run services on
that node, so availability of those services is coupled tightly to the nodes that are available. As long
as a compute node is up, it will have the needed services running on top of it.</para>
</td>
<td>The nova API, scheduler, objectstore, cert, consoleauth, conductor, and
vncproxy services are run on all controller nodes, so scalability can be
achieved with additional controller nodes. HAProxy allows scalability
for Compute as more nodes are added. The scalability of services running
on the compute nodes (compute, conductor) is achieved linearly by adding
in more compute nodes.</td>
</tr>
<tr>
<td>Block Storage (cinder)</td>
<td>Controller</td>
<td>Configured to use Qpid, <systemitem>qpid_heartbeat = 10</systemitem>,
configured to use a Gluster volume from the storage layer as the backend
for Block Storage, using the Gluster native client.</td>
<td>Block Storage API, scheduler, and volume services are run on all controller nodes,
ensuring at least once instance will be available in case of node failure. Block
Storage also sits behind HAProxy, which detects if the software fails and routes
requests around the failing instance.</td>
<td>Block Storage API, scheduler and volume services are run on all
controller nodes, so scalability can be achieved with additional
controller nodes. HAProxy allows scalability for Block Storage as more
nodes are added.</td>
</tr>
<tr>
<td>OpenStack Networking (neutron)</td>
<td>Controller, Compute, Network</td>
<td>Configured to use QPID. <systemitem>qpid_heartbeat = 10</systemitem>,
kernel namespace support enabled, <systemitem>tenant_network_type =
vlan</systemitem>, <systemitem>allow_overlapping_ips =
true</systemitem>, <systemitem>tenant_network_type =
vlan</systemitem>, <systemitem>bridge_uplinks =
br-ex:em2</systemitem>, <systemitem>bridge_mappings =
physnet1:br-ex</systemitem></td>
<td><para>The OpenStack Networking service is run on all controller nodes,
ensuring at least one instance will be available in case of node
failure. It also sits behind HAProxy, which detects if the software
fails and routes requests around the failing instance.</para>
<para>OpenStack Networking's <systemitem>ovs-agent</systemitem>,
<systemitem>l3-agent-dhcp-agent</systemitem>, and
<systemitem>metadata-agent</systemitem> services run on the
network nodes, as <systemitem>lsb</systemitem> resources inside of
Pacemaker. This means that in the case of network node failure,
services are kept running on another node. Finally, the
<systemitem>ovs-agent</systemitem> service is also run on all
compute nodes, and in case of compute node failure, the other nodes
will continue to function using the copy of the service running on
them.</para></td>
<td>The OpenStack Networking server service is run on all controller nodes,
so scalability can be achieved with additional controller nodes. HAProxy
allows scalability for OpenStack Networking as more nodes are added.
Scalability of services running on the network nodes is not currently
supported by OpenStack Networking, so they are not be considered. One
copy of the services should be sufficient to handle the workload.
Scalability of the <systemitem>ovs-agent</systemitem> running on compute
nodes is achieved by adding in more compute nodes as necessary.</td>
</tr>
</tbody>
</table>
</para>
</section>
</section>

79
doc/openstack-ops/ch_arch_example.xml → doc/openstack-ops/section_arch_example-nova.xml
View File

@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE chapter [
<!DOCTYPE section [
<!-- Some useful entities borrowed from HTML -->
<!ENTITY ndash "&#x2013;">
<!ENTITY mdash "&#x2014;">
@ -7,12 +7,12 @@
<!ENTITY plusmn "&#xB1;">
]>
<chapter xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink" version="5.0"
xml:id="example_architecture">
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude" xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="example_architecture-nova">
<?dbhtml stop-chunking?>
<title>Example Architecture</title>
<title>Example Architecture - Legacy Networking (nova)</title>
<para>Because OpenStack is highly configurable, with many different
<glossterm>back-ends</glossterm> and network configuration options,
it is difficult to write documentation that covers all possible
@ -34,6 +34,8 @@
architecture does not dictate a particular number of nodes, but
shows the thinking and considerations that went into choosing this
architecture including the features offered.</para>
<section xml:id="overview_components-nova">
<title>Components</title>
<informaltable rules="all">
<col width="40%"/>
<col width="60%"/>
@ -142,20 +144,22 @@
<section xml:id="rationale">
<title>Rationale</title>
<para>This example architecture has been selected based on the current
default feature set of OpenStack <glossterm>Folsom</glossterm>, with
an emphasis on stability. We believe that many clouds that
currently run OpenStack in production have made similar
choices.</para>
<para>You must first choose the operating system that runs on all of the
physical nodes. While OpenStack is supported on several
distributions of Linux, we used <emphasis role="bold">Ubuntu 12.04
LTS (Long Term Support)</emphasis>, which is used by the
majority of the development community, has feature completeness
compared with other distributions, and has clear future support
plans.</para>
<para>We recommend that you do not use the default Ubuntu OpenStack
install packages and instead use the <link
<para>This example architecture has been selected based on the
current default feature set of OpenStack
<glossterm>Havana</glossterm>, with an emphasis on
stability. We believe that many
clouds that currently run OpenStack in production have
made similar choices.</para>
<para>You must first choose the operating system that runs on
all of the physical nodes. While OpenStack is supported on
several distributions of Linux, we used <emphasis
role="bold">Ubuntu 12.04 LTS (Long Term
Support)</emphasis>, which is used by the majority of
the development community, has feature completeness
compared with other distributions, and has clear future
support plans.</para>
<para>We recommend that you do not use the default Ubuntu
OpenStack install packages and instead use the <link
xlink:href="https://wiki.ubuntu.com/ServerTeam/CloudArchive"
>Ubuntu Cloud Archive</link>
(https://wiki.ubuntu.com/ServerTeam/CloudArchive). The Cloud Archive
@ -219,15 +223,15 @@
indispensable, not just for user interaction with the cloud, but
also as a tool for operators. Additionally, the dashboard's use of
Django makes it a flexible framework for extension.</para>
<section xml:id="neutron">
<title>Why Not Use the OpenStack Network Service (neutron)?</title>
<para>We do not discuss the OpenStack Network Service (neutron) in
this guide, because the authors of this guide only have
production deployment experience using
<code>nova-network</code>. Additionally, it does not yet support
multi-host networking.</para>
</section>
<section xml:id="neutron">
<title>Why Not Use the OpenStack Network Service
(neutron)?</title>
<para>This example architecture does not use the OpenStack
Network Service (neutron), because it does not yet support
multi-host networking and our organizations (university,
government) have access to a large range of
publicly-accessible IPv4 addresses.</para>
</section>
<section xml:id="multi-host-networking">
<title>Why Use Multi-host Networking?</title>
<para>In a default OpenStack deployment, there is a single
@ -241,17 +245,16 @@
become a bottleneck if excessive network traffic comes in and
goes out of the cloud.</para>
<para>
<!-- FIXME - need to find the place for multu-host in havana -->
<link xlink:title="Multi-host"
xlink:href="http://docs.openstack.org/folsom/openstack-compute/admin/content/existing-ha-networking-options.html#d6e8906"
<link xlink:href="http://docs.openstack.org/havana/install-guide/install/apt/content/nova-network.html"
>Multi-host</link>
(http://docs.openstack.org/folsom/openstack-compute/admin/content/existing-ha-networking-options.html#d6e8906)
is a high-availability option for the network configuration
where the nova-network service is run on every compute node
instead of running on only a single node.</para>
(http://docs.openstack.org/havana/install-guide/install/apt/content/nova-network.html)
is a high-availability option for the network
configuration where the nova-network service is run on
every compute node instead of running on only a single
node.</para>
</section>
</section>
</section>
<section xml:id="detailed_desc">
<title>Detailed Description</title>
<para>The reference architecture consists of multiple compute
@ -330,4 +333,4 @@
</listitem>
</itemizedlist>
</section>
</chapter>
</section>