1b7058a15c
- Also includes these changes: GlusterFS supports Block Storage Fix options for nova flavor-access-list Fixes bug 1316040 - reference a generic SQL database "once instance" -> "one instance" recover sheepdog doc Change-Id: I81f57c189bb5138c89f0208404f31c4453ec0aac
484 lines
20 KiB
XML
484 lines
20 KiB
XML
<?xml version="1.0" encoding="UTF-8"?>
|
|
<section version="5.0" xml:id="example_architecture-nova"
|
|
xmlns="http://docbook.org/ns/docbook"
|
|
xmlns:xlink="http://www.w3.org/1999/xlink"
|
|
xmlns:xi="http://www.w3.org/2001/XInclude"
|
|
xmlns:ns5="http://www.w3.org/2000/svg"
|
|
xmlns:ns4="http://www.w3.org/1998/Math/MathML"
|
|
xmlns:ns3="http://www.w3.org/1999/xhtml"
|
|
xmlns:ns="http://docbook.org/ns/docbook">
|
|
<?dbhtml stop-chunking?>
|
|
|
|
<title>Example Architecture—Legacy Networking (nova)</title>
|
|
|
|
<para>This particular example architecture has been upgraded from Grizzly to
|
|
Havana and tested in production environments where many public IP addresses
|
|
are available for assignment to multiple instances. You can find a second
|
|
example architecture that uses OpenStack Networking (neutron) after this
|
|
section. Each example offers high availability, meaning that if a particular
|
|
node goes down, another node with the same configuration can take over the
|
|
tasks so that service continues to be available.<indexterm class="singular">
|
|
<primary>Havana</primary>
|
|
</indexterm><indexterm class="singular">
|
|
<primary>Grizzly</primary>
|
|
</indexterm></para>
|
|
|
|
<section xml:id="overview">
|
|
<title>Overview</title>
|
|
|
|
<para>The simplest architecture you can build upon for Compute has a
|
|
single cloud controller and multiple compute nodes. The simplest
|
|
architecture for Object Storage has five nodes: one for identifying users
|
|
and proxying requests to the API, then four for storage itself to provide
|
|
enough replication for eventual consistency. This example architecture
|
|
does not dictate a particular number of nodes, but shows the thinking
|
|
<phrase role="keep-together">and considerations</phrase> that went into
|
|
choosing this architecture including the features <phrase
|
|
role="keep-together">offered</phrase>.<indexterm class="singular">
|
|
<primary>CentOS</primary>
|
|
</indexterm><indexterm class="singular">
|
|
<primary>RDO (Red Hat Distributed OpenStack)</primary>
|
|
</indexterm><indexterm class="singular">
|
|
<primary>Ubuntu</primary>
|
|
</indexterm><indexterm class="singular">
|
|
<primary>legacy networking (nova)</primary>
|
|
|
|
<secondary>component overview</secondary>
|
|
</indexterm><indexterm class="singular">
|
|
<primary>example architectures</primary>
|
|
|
|
<see>legacy networking; OpenStack networking</see>
|
|
</indexterm><indexterm class="singular">
|
|
<primary>Object Storage</primary>
|
|
|
|
<secondary>simplest architecture for</secondary>
|
|
</indexterm><indexterm class="singular">
|
|
<primary>Compute</primary>
|
|
|
|
<secondary>simplest architecture for</secondary>
|
|
</indexterm></para>
|
|
|
|
<section xml:id="overview_components-nova">
|
|
<title>Components</title>
|
|
|
|
<informaltable rules="all">
|
|
<col width="40%" />
|
|
|
|
<col width="60%" />
|
|
|
|
<thead>
|
|
<tr>
|
|
<th>Component</th>
|
|
|
|
<th>Details</th>
|
|
</tr>
|
|
</thead>
|
|
|
|
<tbody>
|
|
<tr>
|
|
<td><para>OpenStack release</para></td>
|
|
|
|
<td><para>Havana</para></td>
|
|
</tr>
|
|
|
|
<tr>
|
|
<td><para>Host operating system</para></td>
|
|
|
|
<td><para>Ubuntu 12.04 LTS or Red Hat Enterprise Linux 6.5,
|
|
including derivatives such as CentOS and Scientific
|
|
Linux</para></td>
|
|
</tr>
|
|
|
|
<tr>
|
|
<td><para>OpenStack package repository</para></td>
|
|
|
|
<td><para><link xlink:href="http://opsgui.de/NPHp7s">Ubuntu Cloud
|
|
Archive</link> or <link
|
|
xlink:href="http://opsgui.de/1eLCZcm">RDO</link>*</para></td>
|
|
</tr>
|
|
|
|
<tr>
|
|
<td><para>Hypervisor</para></td>
|
|
|
|
<td><para>KVM</para></td>
|
|
</tr>
|
|
|
|
<tr>
|
|
<td><para>Database</para></td>
|
|
|
|
<td><para>MySQL*</para></td>
|
|
</tr>
|
|
|
|
<tr>
|
|
<td><para>Message queue</para></td>
|
|
|
|
<td><para>RabbitMQ for Ubuntu; Qpid for Red Hat Enterprise Linux
|
|
and derivatives</para></td>
|
|
</tr>
|
|
|
|
<tr>
|
|
<td><para>Networking service</para></td>
|
|
|
|
<td><para><literal>nova-network</literal></para></td>
|
|
</tr>
|
|
|
|
<tr>
|
|
<td><para>Network manager</para></td>
|
|
|
|
<td><para>FlatDHCP</para></td>
|
|
</tr>
|
|
|
|
<tr>
|
|
<td><para>Single <literal>nova-network</literal> or
|
|
multi-host?</para></td>
|
|
|
|
<td><para>multi-host*</para></td>
|
|
</tr>
|
|
|
|
<tr>
|
|
<td><para>Image Service (glance) backend</para></td>
|
|
|
|
<td><para>file</para></td>
|
|
</tr>
|
|
|
|
<tr>
|
|
<td><para>Identity Service (keystone) driver</para></td>
|
|
|
|
<td><para>SQL</para></td>
|
|
</tr>
|
|
|
|
<tr>
|
|
<td><para>Block Storage Service (cinder) backend</para></td>
|
|
|
|
<td><para>LVM/iSCSI</para></td>
|
|
</tr>
|
|
|
|
<tr>
|
|
<td><para>Live Migration backend</para></td>
|
|
|
|
<td><para>Shared storage using NFS*</para></td>
|
|
</tr>
|
|
|
|
<tr>
|
|
<td><para>Object storage</para></td>
|
|
|
|
<td><para>OpenStack Object Storage (swift)</para></td>
|
|
</tr>
|
|
</tbody>
|
|
</informaltable>
|
|
|
|
<para>An asterisk (*) indicates when the example architecture deviates
|
|
from the settings of a default installation. We'll offer explanations
|
|
for those deviations next.<indexterm class="singular">
|
|
<primary>objects</primary>
|
|
|
|
<secondary>object storage</secondary>
|
|
</indexterm><indexterm class="singular">
|
|
<primary>storage</primary>
|
|
|
|
<secondary>object storage</secondary>
|
|
</indexterm><indexterm class="singular">
|
|
<primary>migration</primary>
|
|
</indexterm><indexterm class="singular">
|
|
<primary>live migration</primary>
|
|
</indexterm><indexterm class="singular">
|
|
<primary>IP addresses</primary>
|
|
|
|
<secondary>floating</secondary>
|
|
</indexterm><indexterm class="singular">
|
|
<primary>floating IP address</primary>
|
|
</indexterm><indexterm class="singular">
|
|
<primary>storage</primary>
|
|
|
|
<secondary>block storage</secondary>
|
|
</indexterm><indexterm class="singular">
|
|
<primary>block storage</primary>
|
|
</indexterm><indexterm class="singular">
|
|
<primary>dashboard</primary>
|
|
</indexterm><indexterm class="singular">
|
|
<primary>legacy networking (nova)</primary>
|
|
|
|
<secondary>features supported by</secondary>
|
|
</indexterm></para>
|
|
|
|
<note>
|
|
<para>The following features of OpenStack are supported by the example
|
|
architecture documented in this guide, but are optional:<itemizedlist
|
|
role="compact">
|
|
<listitem>
|
|
<para><glossterm>Dashboard</glossterm>: You probably want to
|
|
offer a dashboard, but your users may be more interested in API
|
|
access only.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><glossterm>Block storage</glossterm>: You don't have to
|
|
offer users block storage if their use case only needs ephemeral
|
|
storage on compute nodes, for example.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><glossterm>Floating IP address</glossterm>: Floating IP
|
|
addresses are public IP addresses that you allocate from a
|
|
predefined pool to assign to virtual machines at launch.
|
|
Floating IP address ensure that the public IP address is
|
|
available whenever an instance is booted. Not every organization
|
|
can offer thousands of public floating IP addresses for
|
|
thousands of instances, so this feature is considered
|
|
optional.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><glossterm>Live migration</glossterm>: If you need to move
|
|
running virtual machine instances from one host to another with
|
|
little or no service interruption, you would enable live
|
|
migration, but it is considered optional.</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para><glossterm>Object storage</glossterm>: You may choose to
|
|
store machine images on a file system rather than in object
|
|
storage if you do not have the extra hardware for the required
|
|
replication and redundancy that OpenStack Object Storage
|
|
offers.</para>
|
|
</listitem>
|
|
</itemizedlist></para>
|
|
</note>
|
|
</section>
|
|
|
|
<section xml:id="rationale">
|
|
<title>Rationale</title>
|
|
|
|
<para>This example architecture has been selected based on the current
|
|
default feature set of OpenStack <glossterm>Havana</glossterm>, with an
|
|
emphasis on stability. We believe that many clouds that currently run
|
|
OpenStack in production have made similar choices.<indexterm
|
|
class="singular">
|
|
<primary>legacy networking (nova)</primary>
|
|
|
|
<secondary>rationale for choice of</secondary>
|
|
</indexterm></para>
|
|
|
|
<para>You must first choose the operating system that runs on all of the
|
|
physical nodes. While OpenStack is supported on several distributions of
|
|
Linux, we used <emphasis>Ubuntu 12.04 LTS (Long Term
|
|
Support)</emphasis>, which is used by the majority of the development
|
|
community, has feature completeness compared with other distributions
|
|
and has clear future support plans.</para>
|
|
|
|
<para>We recommend that you do not use the default Ubuntu OpenStack
|
|
install packages and instead use the <link
|
|
xlink:href="http://opsgui.de/NPHp7s">Ubuntu Cloud Archive</link>. The
|
|
Cloud Archive is a package repository supported by Canonical that allows
|
|
you to upgrade to future OpenStack releases while remaining on Ubuntu
|
|
12.04.</para>
|
|
|
|
<para><emphasis>KVM</emphasis> as a <glossterm>hypervisor</glossterm>
|
|
complements the choice of Ubuntu—being a matched pair in terms of
|
|
support, and also because of the significant degree of attention it
|
|
garners from the OpenStack development community (including the authors,
|
|
who mostly use KVM). It is also feature complete, free from licensing
|
|
charges and restrictions.<indexterm class="singular">
|
|
<primary>kernel-based VM (KVM) hypervisor</primary>
|
|
</indexterm><indexterm class="singular">
|
|
<primary>hypervisors</primary>
|
|
|
|
<secondary>KVM</secondary>
|
|
</indexterm></para>
|
|
|
|
<para><emphasis>MySQL</emphasis> follows a similar trend. Despite its
|
|
recent change of ownership, this database is the most tested for use
|
|
with OpenStack and is heavily documented. We deviate from the default
|
|
database, <emphasis>SQLite</emphasis>, because SQLite is not an
|
|
appropriate database for production usage.</para>
|
|
|
|
<para>The choice of <emphasis>RabbitMQ</emphasis> over other AMQP
|
|
compatible options that are gaining support in OpenStack, such as ZeroMQ
|
|
and Qpid, is due to its ease of use and significant testing in
|
|
production. It also is the only option that supports features such as
|
|
Compute cells. We recommend clustering with RabbitMQ, as it is an
|
|
integral component of the system and fairly simple to implement due to
|
|
its inbuilt nature.<indexterm class="singular">
|
|
<primary>Advanced Message Queuing Protocol (AMQP)</primary>
|
|
</indexterm></para>
|
|
|
|
<para>As discussed in previous chapters, there are several options for
|
|
networking in OpenStack Compute. We recommend
|
|
<emphasis>FlatDHCP</emphasis> and to use <emphasis>Multi-Host</emphasis>
|
|
networking mode for high availability, running one
|
|
<code>nova-network</code> daemon per OpenStack compute host. This
|
|
provides a robust mechanism for ensuring network interruptions are
|
|
isolated to individual compute hosts, and allows for the direct use of
|
|
hardware network gateways.</para>
|
|
|
|
<para><emphasis>Live Migration</emphasis> is supported by way of shared
|
|
storage, with <emphasis>NFS</emphasis> as the distributed file
|
|
system.</para>
|
|
|
|
<para>Acknowledging that many small-scale deployments see running Object
|
|
Storage just for the storage of virtual machine images as too costly, we
|
|
opted for the file backend in the OpenStack Image Service (Glance). If
|
|
your cloud will include Object Storage, you can easily add it as a
|
|
backend.</para>
|
|
|
|
<para>We chose the <emphasis>SQL backend for Identity Service
|
|
(keystone)</emphasis> over others, such as LDAP. This backend is simple
|
|
to install and is robust. The authors acknowledge that many
|
|
installations want to bind with existing directory services and caution
|
|
careful understanding of the <link xlink:href="http://opsgui.de/1eLCZJr"
|
|
xlink:title="LDAP config options">array of options
|
|
available</link>.</para>
|
|
|
|
<para>Block Storage (cinder) is installed natively on external storage
|
|
nodes and uses the <emphasis>LVM/iSCSI plug-in</emphasis>. Most Block
|
|
Storage Service plug-ins are tied to particular vendor products and
|
|
implementations limiting their use to consumers of those hardware
|
|
platforms, but LVM/iSCSI is robust and stable on commodity
|
|
hardware.</para>
|
|
|
|
<?hard-pagebreak ?>
|
|
|
|
<para>While the cloud can be run without the <emphasis>OpenStack
|
|
Dashboard</emphasis>, we consider it to be indispensable, not just for
|
|
user interaction with the cloud, but also as a tool for operators.
|
|
Additionally, the dashboard's use of Django makes it a flexible
|
|
framework for <phrase role="keep-together">extension</phrase>.</para>
|
|
</section>
|
|
|
|
<section xml:id="neutron">
|
|
<title>Why not use the OpenStack Network Service (neutron)?</title>
|
|
|
|
<para>This example architecture does not use the OpenStack Network
|
|
Service (neutron), because it does not yet support multi-host networking
|
|
and our organizations (university, government) have access to a large
|
|
range of publicly-accessible IPv4 addresses.<indexterm class="singular">
|
|
<primary>legacy networking (nova)</primary>
|
|
|
|
<secondary>vs. OpenStack Network Service (neutron)</secondary>
|
|
</indexterm></para>
|
|
</section>
|
|
|
|
<section xml:id="multi-host-networking">
|
|
<title>Why use multi-host networking?</title>
|
|
|
|
<para>In a default OpenStack deployment, there is a single
|
|
<code>nova-network</code> service that runs within the cloud (usually on
|
|
the cloud controller) that provides services such as network address
|
|
translation (NAT), DHCP, and DNS to the guest instances. If the single
|
|
node that runs the <code>nova-network</code> service goes down, you
|
|
cannot access your instances, and the instances cannot access the
|
|
Internet. The single node that runs the <literal>nova-network</literal>
|
|
service can become a bottleneck if excessive network traffic comes in
|
|
and goes out of the cloud.<indexterm class="singular">
|
|
<primary>networks</primary>
|
|
|
|
<secondary>multi-host</secondary>
|
|
</indexterm><indexterm class="singular">
|
|
<primary>multi-host networking</primary>
|
|
</indexterm><indexterm class="singular">
|
|
<primary>legacy networking (nova)</primary>
|
|
|
|
<secondary>benefits of multi-host networking</secondary>
|
|
</indexterm></para>
|
|
|
|
<tip>
|
|
<para><link xlink:href="http://opsgui.de/NPHqbu">Multi-host</link> is
|
|
a high-availability option for the network configuration, where the
|
|
<literal>nova-network</literal> service is run on every compute node
|
|
instead of running on only a single node.</para>
|
|
</tip>
|
|
</section>
|
|
</section>
|
|
|
|
<section xml:id="detailed_desc">
|
|
<title>Detailed Description</title>
|
|
|
|
<para>The reference architecture consists of multiple compute nodes, a
|
|
cloud controller, an external NFS storage server for instance storage, and
|
|
an OpenStack Block Storage server for <glossterm>volume</glossterm>
|
|
storage.<indexterm class="singular">
|
|
<primary>legacy networking (nova)</primary>
|
|
|
|
<secondary>detailed description</secondary>
|
|
</indexterm> A network time service (Network Time Protocol, or NTP)
|
|
synchronizes time on all the nodes. FlatDHCPManager in multi-host mode is
|
|
used for the networking. A logical diagram for this example architecture
|
|
shows which services are running on each node:</para>
|
|
|
|
<informalfigure>
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata fileref="figures/osog_01in01.png"></imagedata>
|
|
</imageobject>
|
|
</mediaobject>
|
|
</informalfigure>
|
|
|
|
<para>The cloud controller runs the dashboard, the API services, the
|
|
database (MySQL), a message queue server (RabbitMQ), the scheduler for
|
|
choosing compute resources (<literal>nova-scheduler</literal>), Identity
|
|
services (keystone, <code>nova-consoleauth</code>), Image services
|
|
(<code>glance-api</code>, <code>glance-registry</code>), services for
|
|
console access of guests, and Block Storage services, including the
|
|
scheduler for storage resources (<code>cinder-api</code> and
|
|
<code>cinder-scheduler</code>).<indexterm class="singular">
|
|
<primary>cloud controllers</primary>
|
|
|
|
<secondary>duties of</secondary>
|
|
</indexterm></para>
|
|
|
|
<para>Compute nodes are where the computing resources are held, and in our
|
|
example architecture, they run the hypervisor (KVM), libvirt (the driver
|
|
for the hypervisor, which enables live migration from node to node),
|
|
<code>nova-compute</code>, <code>nova-api-metadata</code> (generally only
|
|
used when running in multi-host mode, it retrieves instance-specific
|
|
metadata), <code>nova-vncproxy</code>, and
|
|
<code>nova-network</code>.</para>
|
|
|
|
<para>The network consists of two switches, one for the management or
|
|
private traffic, and one that covers public access, including floating
|
|
IPs. To support this, the cloud controller and the compute nodes have two
|
|
network cards. The OpenStack Block Storage and NFS storage servers only
|
|
need to access the private network and therefore only need one network
|
|
card, but multiple cards run in a bonded configuration are recommended if
|
|
possible. Floating IP access is direct to the Internet, whereas Flat IP
|
|
access goes through a NAT. To envision the network traffic, use this
|
|
diagram:</para>
|
|
|
|
<informalfigure>
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata fileref="figures/osog_01in02.png"></imagedata>
|
|
</imageobject>
|
|
</mediaobject>
|
|
</informalfigure>
|
|
</section>
|
|
|
|
<section xml:id="optional_extensions">
|
|
<title>Optional Extensions</title>
|
|
|
|
<para>You can extend this reference architecture as<indexterm
|
|
class="singular">
|
|
<primary>legacy networking (nova)</primary>
|
|
|
|
<secondary>optional extensions</secondary>
|
|
</indexterm> follows:</para>
|
|
|
|
<itemizedlist role="compact">
|
|
<listitem>
|
|
<para>Add additional cloud controllers (see <xref
|
|
linkend="maintenance" />).</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Add an OpenStack Storage service (see the Object Storage chapter
|
|
in the <emphasis>OpenStack Installation Guide</emphasis> for your
|
|
distribution).</para>
|
|
</listitem>
|
|
|
|
<listitem>
|
|
<para>Add additional OpenStack Block Storage hosts (see <xref
|
|
linkend="maintenance" />).</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</section>
|
|
</section> |