6df4d84cb1
- Does not add a Detailed Description section to the first arch, as the request is for hardware suggestions and details on the high availability information for Pacemaker and the like, which are not present in the first arch. Would like reviewers to add or weigh in with suggestions. Change-Id: Ica3f19485a74af9359a7057e57df58c33c0675b3
317 lines
18 KiB
XML
317 lines
18 KiB
XML
<?xml version="1.0" encoding="UTF-8"?>
|
|
<!DOCTYPE section [
|
|
<!-- Some useful entities borrowed from HTML -->
|
|
<!ENTITY ndash "–">
|
|
<!ENTITY mdash "—">
|
|
<!ENTITY hellip "…">
|
|
<!ENTITY plusmn "±">
|
|
|
|
]>
|
|
<section xmlns="http://docbook.org/ns/docbook"
|
|
xmlns:xi="http://www.w3.org/2001/XInclude" xmlns:xlink="http://www.w3.org/1999/xlink"
|
|
version="5.0"
|
|
xml:id="example_architecture-nova">
|
|
<?dbhtml stop-chunking?>
|
|
<title>Example Architecture - Legacy Networking (nova)</title>
|
|
<para>This particular example architecture has been upgraded from Grizzly to Havana and tested in production environments where many public IP addresses are available for assignment to multiple instances. You can find a second example architecture that uses OpenStack Networking (neutron) after this section.</para>
|
|
<section xml:id="overview">
|
|
<title>Overview</title>
|
|
<para>The simplest architecture you can build upon for Compute has a single cloud controller and multiple compute nodes. The simplest architecture for Object Storage has five nodes: one for identifying users and proxying requests to the API, then four for storage itself to provide enough replication for eventual consistency. This example architecture does not dictate a particular number of nodes, but shows the thinking and considerations that went into choosing this architecture including the features offered.</para>
|
|
<section xml:id="overview_components-nova">
|
|
<title>Components</title>
|
|
<informaltable rules="all">
|
|
<col width="40%"/>
|
|
<col width="60%"/>
|
|
<tbody>
|
|
<tr>
|
|
<td><para>OpenStack release</para></td>
|
|
<td><para>Havana</para></td>
|
|
</tr>
|
|
<tr>
|
|
<td><para>Host operating system</para></td>
|
|
<td><para>Ubuntu 12.04 LTS or Red Hat Enterprise Linux 6.5
|
|
including derivatives such as CentOS and Scientific Linux</para></td>
|
|
</tr>
|
|
<tr>
|
|
<td><para>OpenStack package repository</para></td>
|
|
<td><para><link
|
|
xlink:href="https://wiki.ubuntu.com/ServerTeam/CloudArchive"
|
|
>Ubuntu Cloud Archive</link>
|
|
(https://wiki.ubuntu.com/ServerTeam/CloudArchive)
|
|
or <link xlink:href="http://openstack.redhat.com/Frequently_Asked_Questions">RDO</link>
|
|
(http://openstack.redhat.com/Frequently_Asked_Questions)
|
|
*</para></td>
|
|
</tr>
|
|
<tr>
|
|
<td><para>Hypervisor</para></td>
|
|
<td><para>KVM</para></td>
|
|
</tr>
|
|
<tr>
|
|
<td><para>Database</para></td>
|
|
<td><para>MySQL*</para></td>
|
|
</tr>
|
|
<tr>
|
|
<td><para>Message queue</para></td>
|
|
<td><para>RabbitMQ for Ubuntu, Qpid for Red Hat Enterprise
|
|
Linux and derivatives</para></td>
|
|
</tr>
|
|
<tr>
|
|
<td><para>Networking service</para></td>
|
|
<td><para>nova-network</para></td>
|
|
</tr>
|
|
<tr>
|
|
<td><para>Network manager</para></td>
|
|
<td><para>FlatDHCP</para></td>
|
|
</tr>
|
|
<tr>
|
|
<td><para>Single nova-network or
|
|
multi-host?</para></td>
|
|
<td><para>multi-host*</para></td>
|
|
</tr>
|
|
<tr>
|
|
<td><para>Image Service (glance)
|
|
back-end</para></td>
|
|
<td><para>file</para></td>
|
|
</tr>
|
|
<tr>
|
|
<td><para>Identity Service (keystone)
|
|
driver</para></td>
|
|
<td><para>SQL</para></td>
|
|
</tr>
|
|
<tr>
|
|
<td><para>Block Storage Service (cinder)
|
|
back-end</para></td>
|
|
<td><para>LVM/iSCSI</para></td>
|
|
</tr>
|
|
<tr>
|
|
<td><para>Live Migration back-end</para></td>
|
|
<td><para>shared storage using NFS *</para></td>
|
|
</tr>
|
|
<tr>
|
|
<td><para>Object storage</para></td>
|
|
<td><para>OpenStack Object Storage
|
|
(swift)</para></td>
|
|
</tr>
|
|
</tbody>
|
|
</informaltable>
|
|
<para>An asterisk (*) indicates when the example architecture
|
|
deviates from the settings of a default
|
|
installation. We'll offer explanations for those deviations next.</para>
|
|
<?hard-pagebreak?>
|
|
<note>
|
|
<para>The following features of OpenStack are supported by
|
|
the example architecture documented in this guide, but
|
|
are optional:<itemizedlist role="compact">
|
|
<listitem>
|
|
<para><glossterm>dashboard</glossterm>: You probably want to offer a dashboard, but your users may be more interested in API access only.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><glossterm>block
|
|
storage</glossterm>: You don't have to offer users block storage if their use case only needs ephemeral storage on compute nodes, for example.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><glossterm>floating IP
|
|
address</glossterm>es: Not every organization can offer public floating IP addresses for thousands of instances, so this feature is considered optional.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><glossterm>live
|
|
migration</glossterm>: If you need to move running virtual machine instances from one host to another with little or no service interruption you would enable live migration, but it is considered optional.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para><glossterm>object
|
|
storage</glossterm>: You may chose to store machine images on a file system rather than in object storage if you do not have the extra hardware for the required replication and redundancy that OpenStack Object Storage offers.</para>
|
|
</listitem>
|
|
</itemizedlist></para>
|
|
</note>
|
|
</section>
|
|
<section xml:id="rationale">
|
|
<title>Rationale</title>
|
|
<para>This example architecture has been selected based on the
|
|
current default feature set of OpenStack
|
|
<glossterm>Havana</glossterm>, with an emphasis on
|
|
stability. We believe that many
|
|
clouds that currently run OpenStack in production have
|
|
made similar choices.</para>
|
|
<para>You must first choose the operating system that runs on
|
|
all of the physical nodes. While OpenStack is supported on
|
|
several distributions of Linux, we used <emphasis
|
|
role="bold">Ubuntu 12.04 LTS (Long Term
|
|
Support)</emphasis>, which is used by the majority of
|
|
the development community, has feature completeness
|
|
compared with other distributions, and has clear future
|
|
support plans.</para>
|
|
<para>We recommend that you do not use the default Ubuntu
|
|
OpenStack install packages and instead use the <link
|
|
xlink:href="https://wiki.ubuntu.com/ServerTeam/CloudArchive"
|
|
>Ubuntu Cloud Archive</link>
|
|
(https://wiki.ubuntu.com/ServerTeam/CloudArchive). The Cloud Archive
|
|
is a package repository supported by Canonical that allows you to
|
|
upgrade to future OpenStack releases while remaining on Ubuntu
|
|
12.04.</para>
|
|
<para><emphasis role="bold">KVM</emphasis> as a
|
|
<glossterm>hypervisor</glossterm> complements the choice of
|
|
Ubuntu - being a matched pair in terms of support, and also because
|
|
of the significant degree of attention it garners from the OpenStack
|
|
development community (including the authors, who mostly use KVM).
|
|
It is also feature complete, free from licensing charges and
|
|
restrictions.</para>
|
|
<para><emphasis role="bold">MySQL</emphasis> follows a similar trend.
|
|
Despite its recent change of ownership, this database is the most
|
|
tested for use with OpenStack and is heavily documented. We deviate
|
|
from the default database, <emphasis>SQLite</emphasis>, because
|
|
SQLite is not an appropriate database for production usage.</para>
|
|
<para>The choice of <emphasis role="bold"> RabbitMQ</emphasis> over
|
|
other AMQP compatible options that are gaining support in OpenStack,
|
|
such as ZeroMQ and Qpid is due to its ease of use and
|
|
significant testing in production. It also is the only option which
|
|
supports features such as Compute cells. We recommend clustering
|
|
with RabbitMQ, as it is an integral component of the system, and
|
|
fairly simple to implement due to its inbuilt nature.</para>
|
|
<para>As discussed in previous chapters, there are several options for
|
|
networking in OpenStack Compute. We recommend <emphasis role="bold"
|
|
>FlatDHCP</emphasis> and to use <emphasis role="bold"
|
|
>Multi-Host</emphasis> networking mode for high availability,
|
|
running one <code>nova-network</code> daemon per OpenStack Compute
|
|
host. This provides a robust mechanism for ensuring network
|
|
interruptions are isolated to individual compute hosts, and allows
|
|
for the direct use of hardware network gateways.</para>
|
|
<para><emphasis role="bold">Live Migration</emphasis> is supported by
|
|
way of shared storage, with <emphasis role="bold">NFS</emphasis> as
|
|
the distributed file system.</para>
|
|
<para>Acknowledging that many small-scale deployments see running Object
|
|
Storage just for the storage of virtual machine images as too
|
|
costly, we opted for the file back-end in the OpenStack Image
|
|
Service (Glance). If your cloud will include Object Storage, you can
|
|
easily add it as a back-end.</para>
|
|
<para>We chose the <emphasis role="bold">SQL back-end for Identity
|
|
Service (keystone)</emphasis> over others, such as LDAP. This
|
|
back-end is simple to install and is robust. The authors acknowledge
|
|
that many installations want to bind with existing directory
|
|
services, and caution careful understanding of the <link
|
|
xlink:title="LDAP config options"
|
|
xlink:href="http://docs.openstack.org/havana/config-reference/content/ch_configuring-openstack-identity.html#configuring-keystone-for-ldap-backend"
|
|
>array of options available</link>
|
|
(http://docs.openstack.org/havana/config-reference/content/ch_configuring-openstack-identity.html#configuring-keystone-for-ldap-backend).</para>
|
|
<para>Block Storage (cinder) is installed natively on
|
|
external storage nodes and uses the <emphasis role="bold">LVM/iSCSI
|
|
plugin</emphasis>. Most Block Storage Service plugins are tied
|
|
to particular vendor products and implementations limiting their use
|
|
to consumers of those hardware platforms, but LVM/iSCSI is robust
|
|
and stable on commodity hardware.</para>
|
|
<para>While the cloud can be run without the <emphasis role="bold"
|
|
>OpenStack Dashboard</emphasis>, we consider it to be
|
|
indispensable, not just for user interaction with the cloud, but
|
|
also as a tool for operators. Additionally, the dashboard's use of
|
|
Django makes it a flexible framework for extension.</para>
|
|
</section>
|
|
<section xml:id="neutron">
|
|
<title>Why Not Use the OpenStack Network Service
|
|
(neutron)?</title>
|
|
<para>This example architecture does not use the OpenStack
|
|
Network Service (neutron), because it does not yet support
|
|
multi-host networking and our organizations (university,
|
|
government) have access to a large range of
|
|
publicly-accessible IPv4 addresses.</para>
|
|
</section>
|
|
<section xml:id="multi-host-networking">
|
|
<title>Why Use Multi-host Networking?</title>
|
|
<para>In a default OpenStack deployment, there is a single
|
|
<code>nova-network</code> service that runs within the cloud
|
|
(usually on the cloud controller) that provides services such as
|
|
network address translation (NAT), DHCP, and DNS to the guest
|
|
instances. If the single node that runs the
|
|
<code>nova-network</code> service goes down, you cannot
|
|
access your instances and the instances cannot access the
|
|
Internet. The single node that runs the nova-network service can
|
|
become a bottleneck if excessive network traffic comes in and
|
|
goes out of the cloud.</para>
|
|
<tip><para>
|
|
<link
|
|
xlink:href="http://docs.openstack.org/havana/install-guide/install/apt/content/nova-network.html"
|
|
>Multi-host</link>
|
|
(http://docs.openstack.org/havana/install-guide/install/apt/content/nova-network.html)
|
|
is a high-availability option for the network
|
|
configuration where the nova-network service is run on
|
|
every compute node instead of running on only a single
|
|
node.</para></tip>
|
|
</section>
|
|
</section>
|
|
<section xml:id="detailed_desc">
|
|
<title>Detailed Description</title>
|
|
<para>The reference architecture consists of multiple compute
|
|
nodes, a cloud controller, an external NFS storage server
|
|
for instance storage and an OpenStack Block Storage server
|
|
for <glossterm>volume</glossterm> storage. A network time
|
|
service (Network Time Protocol, NTP) synchronizes time on
|
|
all the nodes. FlatDHCPManager in multi-host mode is used
|
|
for the networking. A logical diagram for this example
|
|
architecture shows which services are running on each node:</para>
|
|
<informalfigure>
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata width="4in"
|
|
fileref="figures/os-ref-arch.png"/>
|
|
</imageobject>
|
|
</mediaobject>
|
|
</informalfigure>
|
|
<para>The cloud controller runs: the dashboard, the API
|
|
services, the database (MySQL), a message queue server
|
|
(RabbitMQ), the scheduler for choosing compute resources
|
|
(nova-scheduler), Identity services (keystone,
|
|
<code>nova-consoleauth</code>), Image services
|
|
(<code>glance-api</code>,
|
|
<code>glance-registry</code>), services for console access
|
|
of guests, and Block Storage services including the
|
|
scheduler for storage resources (<code>cinder-api</code>
|
|
and <code>cinder-scheduler</code>).</para>
|
|
<para>Compute nodes are where the computing resources are
|
|
held, and in our example architecture they run the
|
|
hypervisor (KVM), libvirt (the driver for the hypervisor,
|
|
which enables live migration node to node),
|
|
<code>nova-compute</code>,
|
|
<code>nova-api-metadata</code> (generally only used
|
|
when running in multi-host mode, it retrieves
|
|
instance-specific metadata), <code>nova-vncproxy</code>,
|
|
and <code>nova-network</code>.
|
|
</para>
|
|
<para>The network consists of two switches, one for the management or
|
|
private traffic, and one which covers public access including
|
|
Floating IPs. To support this, the cloud controller and the compute
|
|
nodes have two network cards. The OpenStack Block Storage and NFS
|
|
storage servers only need to access the private network and
|
|
therefore only need one network card, but multiple cards run in a
|
|
bonded configuration are recommended if possible. Floating IP access
|
|
is direct to the internet, whereas Flat IP access goes through a
|
|
NAT. To envision the network traffic use this diagram:</para>
|
|
<informalfigure>
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata width="4in"
|
|
fileref="figures/os_physical_network.png"/>
|
|
</imageobject>
|
|
</mediaobject>
|
|
</informalfigure>
|
|
</section>
|
|
<?hard-pagebreak?>
|
|
<section xml:id="optional_extensions">
|
|
<title>Optional Extensions</title>
|
|
<para>You can extend this reference architecture as
|
|
follows:</para>
|
|
<itemizedlist role="compact">
|
|
<listitem>
|
|
<para>Add additional cloud controllers (see <xref
|
|
linkend="maintenance"/>).</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>Add an OpenStack Storage service (see the Object Storage
|
|
chapter in the <citetitle>OpenStack Installation
|
|
Guide</citetitle> for your distribution.</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>Add additional OpenStack Block Storage hosts
|
|
(see <xref linkend="maintenance"/>).</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</section>
|
|
</section>
|