vlan_interface
specifies what interface OpenStack should attach all VLANs
to. The correct setting should have been:
- vlan_interface=bond0.nova database and saw the
instance's entry in the nova.instances table.
- The image that the instance was using matched what
- _base files not in use.
+ and see if there were any _base files not in use.
If there
- were, nova would delete them. This idea sounds innocent
+ were, Nova would delete them. This idea sounds innocent
enough and has some good qualities to it. But how did this
feature end up turned on? It was disabled by default in
Essex. As it should be. It was decided to enable it in Folsom
+ >decided to be turned on in Folsom
(https://bugs.launchpad.net/nova/+bug/1029674). I cannot
emphasize enough that:XXX.nova-* components that represent the global state of
- the cloud, talks to services such as authentication, maintains
- information about the cloud in a database, communicates to all compute
- nodes and storage | Consideration | -Ramification | -
|---|---|
nova-api
- services do you run at once for your
- cloud? |
- |
glance-* servers
- on the swift-proxy
- servernova-api and
- swift-proxy servers on
- different physical servers and used the
- load balancer to switch between
- them.nova-compute, swift-proxy
- and swift-object servers, should not be virtualized.
- However, control servers can often be happily virtualized—the
- performance penalty can usually be offset by simply
- running more of the service.nova-conductor process, either on the
- same server or across multiple servers.nova-api
- service.glance-api and glance-registry. The
- former is responsible for the delivery of images; the compute
- node uses it to download images from the back-end. The latter
- maintains the metadata information associated with virtual machine
- images and requires a database.glance-api part is an abstraction layer
- that allows a choice of back-end. Currently, it
- supports:httpd. Therefore, you may treat it the same
- as any other web application, provided it can reach the
- API servers (including their admin endpoints) over the
- network.nova-* components that represent the global state of the cloud;
+ talks to services such as authentication; maintains information about the
+ cloud in a database; communicates to all compute nodes and storage
+ | Consideration | + +Ramification | +
|---|---|
nova-api services do you run at once
+ for your cloud? |
+
+ |
| Scenario | + +Justification | +
|---|---|
glance-* servers on the
+ swift-proxy server. |
+
+ |
nova-api and
+ swift-proxy servers on different physical servers and
+ used the load balancer to switch between them. |
+
nova-compute, swift-proxy and
+ swift-object servers, should not be virtualized. However,
+ control servers can often be happily virtualized—the performance penalty
+ can usually be offset by simply running more of the service.nova-conductor process, either on the same server or across
+ multiple servers.nova-api service.glance-api and glance-registry. The former is
+ responsible for the delivery of images; the compute node uses it to
+ download images from the backend. The latter maintains the metadata
+ information associated with virtual machine images and requires a
+ database.glance-api part is an abstraction layer that allows
+ a choice of backend. Currently, it supports:httpd.
+ Therefore, you may treat it the same as any other web application,
+ provided it can reach the API servers (including their admin endpoints)
+ over the swift-proxy,
- nova-api,
- glance-api, and horizon
- come to these addresses, which could be on
- one side of a load balancer or pointing
- at individual machines.swift-proxy,
+ nova-api, glance-api, and horizon come to
+ these addresses, which could be on one side of a load balancer or
+ pointing at individual machines.| Network deployment model | + +Strengths | + +Weaknesses | + +Neutron equivalent | +
|---|---|---|---|
| Configure a single bridge as the integration bridge (br-int) and + connect it to a physical network interface with the Modular Layer 2 + (ML2) plug-in, which uses Open vSwitch by default. | +|||
| Configure DHCP agents and routing agents. Network Address + Translation (NAT) performed outside of compute nodes, typically on + one or more network nodes. | +|||
| Network Deployment Model | -Strengths | -Weaknesses | -Neutron Equivalent | -
|---|---|---|---|
|
- |
-
- |
-
- |
- Configure a single bridge as the integration bridge - (br-int) and connect it to a physical network - interface with the Modular Layer 2 (ML2) plug-in, - which uses Open vSwitch by default. | -
|
- |
-
- |
-
- |
- Configure DHCP agents and routing agents. Network - Address Translation (NAT) performed outside of - compute nodes, typically on one or more network nodes. | -
|
- |
-
- |
-
- |
-
- |
-
|
- |
-
- |
-
- |
-
- |
-
nova-network service. All compute
- nodes forward traffic from the instances to the cloud
- controller. The cloud controller then forwards traffic
- to the Internet. The cloud controller hosts the
- floating IPs and security groups for all instances on
- all compute nodes in the cloud.nova-network hosts. You
- could consider providing a dynamic DNS service to
- allow instances to update a DNS entry with new IP
- addresses. You can also consider making a generic
- forward and reverse DNS mapping for instances' IP
- addresses, such as
- vm-203-0-113-123.example.com.nova-network service. All compute nodes forward traffic
+ from the instances to the cloud controller. The cloud controller then
+ forwards traffic to the Internet. The cloud controller hosts the
+ floating IPs and security groups for all instances on all compute nodes
+ in the cloud.nova-network hosts. You
+ could consider providing a dynamic DNS service to allow instances to
+ update a DNS entry with new IP addresses. You can also consider making a
+ generic forward and reverse DNS mapping for instances' IP addresses,
+ such as vm-203-0-113-123.example.com./boot partition mirror. You can make partition 2 of all
- disks the root partition mirror. You can use partition 3 of all disks for a
- cinder-volumes LVM partition running on a RAID 10
- array./boot partition mirror. You can make
+ partition 2 of all disks the root partition mirror. You can use
+ partition 3 of all disks for a cinder-volumes LVM
+ partition running on a RAID 10 array.((overcommit fraction × cores) / virtual cores per instance),
- (flavor disk size × number of instances).
- /var/lib/nova/instances:cpu_allocation_ratio in
- nova.conf) of 16:1| Name | + +Virtual cores | + +Memory | + +Disk | + +Ephemeral | +
|---|---|---|---|---|
((overcommit fraction × cores) / virtual cores per
+ instance)(flavor disk size
+ × number of instances)/var/lib/nova/instances:cpu_allocation_ratio
+ in nova.conf) of 16:1.nova-scheduler and nova-console—on a
+ new server for expansion. However, other integral parts require more
+ care.nova-api, or the Object Storage proxy. Use any standard HTTP
+ load-balancing method (DNS round robin, hardware load balancer, or
+ software such as Pound or HAProxy). One caveat with dashboard is the VNC
+ proxy, which uses the WebSocket protocol—something that an L7 load
+ balancer might struggle with. See also Horizon session
+ storage.nova-api and
+ glance-api, to use multiple processes by changing a flag in
+ their configuration file—allowing them to share work between multiple
+ cores on the one machine.| + + | Cells | + +Regions | + +Availability zones | + +Host aggregates | +
|---|---|---|---|---|
nova-api |
+
+
nova-api service, but no
+ nova-compute services. Each child cell runs all of the
+ other typical nova-* services found in a regular
+ installation, except for the nova-api service. Each cell
+ has its own message queue and database service and also runs
+ nova-cells, which manages the communication between the API
+ cell and child cells.nova-scheduler selection of hosts, provides greater
+ flexibility to control where virtual machines are run.nova-scheduler and
- nova-console—on a new server for
- expansion. However, other integral parts require more
- care.nova-api, or the Object Storage
- proxy. Use any standard HTTP load-balancing method (DNS
- round robin, hardware load balancer, software such as Pound
- or HAProxy). One caveat with Dashboard is the VNC proxy,
- which uses the WebSocket protocol—something that an L7
- load balancer might struggle with. See also Horizon session storage
- (http://docs.openstack.org/developer/horizon/topics/deployment.html#session-storage).nova-api and glance-api, to
- use multiple processes by changing a flag in their
- configuration file—allowing them to share work between
- multiple cores on the one machine.| - | Cells | -Regions | -Availability Zones | -Host Aggregates | -
|---|---|---|---|---|
nova-cells.nova-api. |
- ||||
nova-api
- |
-
nova-api service, but no
- nova-compute services. Each child
- cell runs all of the other typical nova-*
- services found in a regular installation, except for
- the nova-api service. Each cell has its
- own message queue and database service, and also runs
- nova-cells — which manages the
- communication between the API cell and child
- cells.nova-scheduler selection of hosts,
- provides greater flexibility to control where virtual
- machines are run.nova-scheduler will
- take care of differences in sizing having to do with core
- count and RAM amounts, however you should consider that the
- user experience changes with differing CPU speeds.
- When adding object storage nodes, a
- nova-scheduler will take care of
+ differences in sizing having to do with core count and RAM amounts;
+ however, you should consider that the user experience changes with
+ differing CPU speeds. When adding object storage nodes, a
+ /var/lib/nova/instances.| - | Ephemeral storage | -Block storage | -Object storage | -
|---|---|---|---|
|
- |
-
/var/lib/nova/instances when
- designing your cloud, since you must have a shared
- file system if you want to support live
- migration./var/lib/nova/instances.| - | Object | -Block | -File-level* | -
|---|---|---|---|
| + + | Ephemeral storage | + +Block storage | + +Object storage | +
|---|---|---|---|
/var/lib/nova/instances when designing your
+ cloud, since you must have a shared file system if you want to support
+ live migration.| + + | Object | + +Block | + +File-level |
+
|---|---|---|---|
nova is the database you want to back
- up.nova is the database you want to back up./var/log/nova does not need to be backed up if you
+ have all logs going to a central area. It is highly recommended to use a
+ central logging server or back up the log directory./var/lib/nova is another important directory to back
+ up. The exception to this is the /var/lib/nova/instances
+ subdirectory on compute nodes. This subdirectory contains the KVM images
+ of running instances. You would want to back up this directory only if
+ you need to maintain backup copies of all instances. Under most
+ circumstances, you do not need to do this, but this can vary from cloud
+ to cloud and your service levels. Also be aware that making a backup of
+ a live KVM instance can cause that instance to not boot properly if it
+ is ever restored from a backup./var/log/nova does not need to be backed up if
- you have all logs going to a central area. It is
- highly recommended to use a central logging server or
- back up the log directory./var/lib/nova is another important
- directory to back up. The exception to this is the
- /var/lib/nova/instances subdirectory
- on compute nodes. This subdirectory contains the KVM
- images of running instances. You would want to
- back up this directory only if you need to maintain backup
- copies of all instances. Under most circumstances, you
- do not need to do this, but this can vary from cloud
- to cloud and your service levels. Also be aware that
- making a backup of a live KVM instance can cause that
- instance to not boot properly if it is ever restored
- from a backup./etc/glance and
- /var/log/glance follow the same rules
- as their nova counterparts./var/lib/glance should also be backed up.
- Take special notice of
- /var/lib/glance/images. If you are
- using a file-based backend of glance,
- /var/lib/glance/images is where the
- images are stored and care should be taken./etc/keystone and
- /var/log/keystone follow the same
- rules as other components./var/lib/keystone, although it should not
- contain any data being used, can also be backed up
- just in case./etc/cinder and
- /var/log/cinder follow the same rules
- as other components./var/lib/cinder should also be backed
- up./etc/swift is very important to have
- backed up. This directory contains the swift
- configuration files as well as the ring files and ring
- /etc/glance and /var/log/glance follow
+ the same rules as their nova counterparts./var/lib/glance should also be backed up. Take
+ special notice of /var/lib/glance/images. If you are using
+ a file-based backend of glance, /var/lib/glance/images is
+ where the images are stored and care should be taken.nova
- services:/etc/keystone and /var/log/keystone
+ follow the same rules as other components./var/lib/keystone, although it should not contain any
+ data being used, can also be backed up just in case./etc/cinder and /var/log/cinder follow
+ the same rules as other components./var/lib/cinder should also be backed up./etc/swift is very important to have backed up. This
+ directory contains the swift configuration files as well as the ring
+ files and ring nova services:nova client,
- specify --flavor 3 for the nova
- boot command to get adequate memory and
- disk sizes.stack.sh script
- takes a while to run. Perhaps take
- this opportunity to join the OpenStack
- Foundation
- (http://www.openstack.org/join/).nova client, specify
+ --flavor 3 for the nova boot command to get
+ adequate memory and disk sizes.stack.sh script takes a while to run.
+ Perhaps you can take this opportunity to join the OpenStack
+ Foundation.stack.sh starts
- with screen -r stack, you see a screen for each service
- running, which can be a few or several depending on how many
- services you configured DevStack to run.shellkey*horizons-{name}/opt/stack. Go to the swift
- directory in the shell screen and edit your
- middleware module.stack.sh starts
+ with screen -r stack, you see a screen for each service
+ running, which can be a few or several, depending on how many services you
+ configured DevStack to run.shellkey*horizons-{name}/opt/stack. Go
+ to the swift directory in the shell screen and edit your
+ middleware module.env and conf that
- you can use to decide what to do with the request.
- To find out more about what properties are
- available, you can insert the following log
- statement into the __init__
- method__call__ method[filter:ratelimit] section in
- env and
+ conf that you can use to decide what to do with the
+ request. To find out more about what properties are available, you can
+ insert the following log statement into the __init__
+ method:__call__
+ method:[filter:ratelimit] section in
+ [pipeline:main] section in
- ip_whitelist after ratelimit to the list like
- so. When you're done, save and close the
- file.[pipeline:main] section in
+ ip_whitelist after ratelimit to the list like so. When
+ you're done, save and close the file:swift CLI. Start
- by switching to the shell screen and finish by
- switching back to the swift-proxy screen
- to check the log output.swift CLI. Start
+ by switching to the shell screen and finish by switching back to the
+ swift-proxy screen to check the log output:keystone and swift clients on your local machine.keystone and swift
+ clients on your local machine:pipeline value in the project's
- conf or ini configuration
- files in /etc/<project> to identify
- projects that use Paste.nova.scheduler.driver.Scheduler. Of the five
- methods that you can override, you update_service_capabilities
- hosts_up
- group_hosts
- schedule_run_instance
- select_destinations
- stack.sh starts with screen -r stack,
- you are greeted with many screen windows.shellkeyhorizonn-{name}n-sch/opt/stack,
- so go to the nova directory and edit your scheduler module.
- Change to the directory where nova is installed:pipeline value in
+ the project's conf or ini configuration files in
+ /etc/<project> to identify projects that use
+ Paste.nova.scheduler.driver.Scheduler. Of the five methods that you
+ can override, you update_service_capabilitieshosts_upgroup_hostsschedule_run_instanceselect_destinationsstack.sh starts
+ with screen -r stack, you are greeted with many screen
+ windows:shellkeyhorizonn-{name}n-sch/opt/stack, so go
+ to the context, request_spec, and
- filter_properties that you can use to
- decide where to schedule the instance. To find out more
- about what properties are available, you can insert the
- following log statements into the
- schedule_run_instance method of the
- scheduler above.context,
+ request_spec, and filter_properties that you
+ can use to decide where to schedule the instance. To find out more
+ about what properties are available, you can insert the following log
+ statements into the schedule_run_instance method of the
+ scheduler above:scheduler_driver
- config and change it like so.n-sch
- screen.shell screen and finish by switching
- back to the n-sch screen to check the log
- output.n-sch screen. Among the
- log statements, you'll see the line:/etc/<project> to identify
- projects that use a driver architecture.scheduler_driver config and change it like
+ so:n-sch screen:n-sch screen.shell screen and finish by switching back to the
+ n-sch screen to check the log output:n-sch screen. Among the log
+ statements, you'll see the line:/etc/<project> to identify projects that use a driver
+ architecture.-e
+ flag. You must specify a name for the Python egg that is installed. For
+ example:-e flag. You must specify a
- name for the Python egg that is installed. For
- example:*-manage tools must be run from the
- cloud controller, as root, because they need read
- access to the config files such as
- /etc/nova/nova.conf and to make queries
- directly against the database rather than against the
- OpenStack *-manage tools is
- a legacy issue. It is a goal of the OpenStack project
- to eventually migrate all of the remaining
- functionality in the *-manage tools into
- the API-based tools. Until that day, you need to
- SSH into the *-manage tools.*-manage
+ tools must be run from the cloud controller, as root, because they need
+ read access to the config files such as /etc/nova/nova.conf
+ and to make queries directly against the database rather than against
+ the OpenStack *-manage tools is a legacy
+ issue. It is a goal of the OpenStack project to eventually migrate all
+ of the remaining functionality in the *-manage tools into
+ the API-based tools. Until that day, you need to SSH into the
+ *-manage
+ toolsOS_PASSWORD. It is
- important to note that this does require interactivity. It
- is possible to store a value directly in the script if you
- require a noninteractive operation, but you then need to be
- extremely cautious with the security and permissions of this
- file.user-openrc. Extract the zip file here. You
- should have OS_PASSWORD. It is important to note that this does
+ require interactivity. It is possible to store a value directly in the
+ script if you require a noninteractive operation, but you then need to
+ be extremely cautious with the security and permissions of this
+ file.user-openrc. Extract the ZIP file here. You should have
+ ec2rc.sh file.--debug flag to them. For example:--os-cache flag or set the
- environment variable OS_CACHE=1.openrc.sh discussed above. The token allows
- you to interact with your other service endpoints without
- needing to reauthenticate for every request. Tokens are
- typically good for 24 hours, and when the token expires, you
- are alerted with a 401 (Unauthorized) response and you can
- request another token.ec2rc.sh file.--debug flag to
+ them.--os-cache flag or set the environment variable
+ OS_CACHE=1.openrc.sh discussed above. The token allows you to
+ interact with your other service endpoints without needing to
+ reauthenticate for every request. Tokens are typically good for 24
+ hours, and when the token expires, you are alerted with a 401
+ (Unauthorized) response and you can request another -s flag used in the cURL
- commands above are used to prevent the progress
- meter from being shown. If you are having trouble
- running cURL commands, you'll want to remove it.
- Likewise, to help you troubleshoot cURL commands
- you can include the -v flag to show
- you the verbose output. There are many more
- extremely useful features in cURL; refer to the
- man page for all the options.-s flag used in the cURL commands above are
+ used to prevent the progress meter from being shown. If you are
+ having trouble running cURL commands, you'll want to remove it.
+ Likewise, to help you troubleshoot cURL commands, you can include the
+ -v flag to show you the verbose output. There are many
+ more extremely useful features in cURL; refer to the man page for all
+ the options.:-) which indicates that the services
- are up and running. If a service is no
- longer available, the :-) symbol changes to
- XXX. This is an indication that you
- should troubleshoot why the service is down.:-), which
+ indicates that the services are up and running. If a service is no
+ longer available, the :-) symbol changes to
+ XXX. This is an indication that you should troubleshoot why
+ the service is down./var/log
- directory, as listed in OpenStack Log Locations.| Node Type | -Service | -Log Location | -
|---|---|---|
nova-*
- |
- /var/log/nova
- |
- |
glance-*
- |
- /var/log/glance
- |
- |
cinder-*
- |
- /var/log/cinder
- |
- |
keystone-*
- |
- /var/log/keystone
- |
- |
neutron-*
- |
- /var/log/neutron
- |
- |
/var/log/apache2/
- |
- ||
/var/log/syslog
- |
- ||
/var/log/libvirt/libvirtd.log
- |
- ||
/var/lib/nova/instances/instance-<instance
- id>/console.log
- |
- ||
/var/log/cinder/cinder-volume.log
- |
-
logger_root and handler_file
- sections./var/log directory, as listed in | Node type | + +Service | + +Log location | +
|---|---|---|
nova-* |
+
+ /var/log/nova |
+ |
glance-* |
+
+ /var/log/glance |
+ |
cinder-* |
+
+ /var/log/cinder |
+ |
keystone-* |
+
+ /var/log/keystone |
+ |
neutron-* |
+
+ /var/log/neutron |
+ |
/var/log/apache2/ |
+ ||
/var/log/syslog |
+ ||
/var/log/libvirt/libvirtd.log |
+ ||
/var/lib/nova/instances/instance-<instance id>/console.log
+ |
+ ||
/var/log/cinder/cinder-volume.log
+ |
+
logger_root and handler_file
+ sections.nova-*
- services, and across both the cloud controller and compute
- nodes.nova-* services and across both the cloud controller
+ and compute nodes.faf7ded8-4a46-413b-b113-f19590746ffe. If
- you search for this string on the cloud controller in the
- nova-* services.faf7ded8-4a46-413b-b113-f19590746ffe. If you search for this
+ string on the cloud controller in the
+ nova-*
+ services.nova.
- Notifications are essentially the same as logs but can be
- much more detailed. A good overview of notifications can
- be found at System Usage Data
- (https://wiki.openstack.org/wiki/SystemUsageData).nova. Notifications are essentially the
+ same as logs but can be much more detailed. A good overview of
+ notifications can be found at System Usage Data.nova to send notifications, add the following
+ to nova is sending notifications, install
- and configure StackTach. Since StackTach is relatively new
- and constantly changing, installation instructions would
- quickly become outdated. Please refer to the StackTach GitHub repo
- (https://github.com/rackerlabs/stacktach) for instructions
- as well as a demo video.nova-api service is
- running on the cloud controller:nova is sending notifications, install and
+ configure StackTach. Since StackTach is relatively new and constantly
+ changing, installation instructions would quickly become outdated. Please
+ refer to the StackTach GitHub
+ repo for instructions as well as a demo video.nova-api
+ service is running on the cloud controller:nova-compute process is
- running on compute nodes, create an alert on your
- Nagios server that looks like this:nova-compute process is running on compute nodes, create an
+ alert on your Nagios server that looks like this:nova
- command:nova database contains three
- tables that store usage information.nova.quotas and
- nova.quota_usages tables store quota
- information. If a tenant's quota is different from the
- default quota settings, its quota is stored in the
- nova.quotas table. For
- example:nova
+ command:nova database contains three tables that
+ store usage information.nova.quotas and nova.quota_usages
+ tables store quota information. If a tenant's quota is different from
+ the default quota settings, its quota is stored in the nova.quotasnova.quota_usages table keeps track
- of how many resources the tenant currently has in
- use:nova.quota_usages table keeps track of how many
+ resources the tenant currently has in use:glance-api
- and glance-registry processes are running
- or by seeing whether glace-api is responding
- on port 9292.glance-api and glance-registry
+ processes are running or by seeing whether glace-api is
+ responding on port 9292.nova-api usage
- can allow you to track the need to scale your cloud
- controller. By keeping an eye on nova-api
- requests, you can determine whether you need to spawn more
- nova-api processes or go as far as introducing an
- entirely new server to run nova-api. To
- get an approximate count of the requests, look for
- standard INFO messages in
- /var/log/nova/nova-api.log:nova-api usage is increasing,
- decreasing, or keeping steady.nova-api usage can allow you
+ to track the need to scale your cloud controller. By keeping an eye on
+ nova-api requests, you can determine whether you need to
+ spawn more nova-api. To get
+ an approximate count of the requests, look for standard INFO messages in
+ /var/log/nova/nova-api.log:nova-api usage is increasing, decreasing, or keeping
+ steady.ps and
- grep to determine if nova, glance,
- and keystone are currently running:ps and grep to determine if nova, glance, and
+ keystone are currently running:openrc file, then runs some basic
- glance, nova, and keystone commands. If the commands
- work as expected, you can be confident that those
- services are in working condition:openrc file, then runs some basic
+ glance, nova, and keystone commands. If the commands work as expected,
+ you can be confident that those services are in working
+ condition:nova live-migration
- command. First, get a list of instances that need to
- be moved:--block-migrate option:nova-compute service has
- stopped:nova-compute
- service is always running, you can temporarily move
- the init files:nova live-migration command. First,
+ get a list of instances that need to be moved:--block-migrate option:nova-compute service has nova-compute service is always running, you can
+ temporarily move the nova-compute service by
- undoing the previous commands:nova-compute
+ service by undoing the previous commands:nova-compute
- service:nova-compute service is
- running:nova-compute service:nova-compute
+ service is running:fsck on the
- root partition. If this happens, the user can use
- the dashboard VNC console to fix this.virsh
- list never shows the instance as even
- attempting to boot, do the following on the compute
- node:nova reboot command
- again. You should see an error message about why the
- instance was not able to boot/etc/libvirt/qemu/instance-xxxxxxxx.xml)
- that no longer exists. You can enforce re-creation of
- the XML file as well as rebooting the instance by
- running the following command:fsck on the root partition. If this happens, the user can
+ use the dashboard VNC console to fix this.virsh list
+ never shows the instance as even attempting to boot, do the following on
+ the compute node:nova reboot command again. You
+ should see an error message about why the instance was not able to
+ boot/etc/libvirt/qemu/instance-xxxxxxxx.xml) that no
+ longer exists. You can enforce re-creation of the XML file as well as
+ rebooting the instance by running the following command:/mnt, which correspond to the
- first partition of the instance's disk./mnt, which
+ correspond to the first partition of the instance's disk./var/lib/nova/instances./var/lib/nova/instances contains two
- types of directories._base directory. This
- contains all the cached base images from glance for
- each unique image that has been launched on that
- compute node. Files ending in _20 (or a
- different number) are the ephemeral base
- images.instance-xxxxxxxx. These directories
- correspond to instances running on that compute node.
- The files inside are related to one of the files in
- the _base directory. They're essentially
- differential-based files containing only the changes
- made from the original _base
- directory./var/lib/nova/instances are uniquely
- named. The files in _base are uniquely titled for the
- glance image that they are based on, and the directory
- names instance-xxxxxxxx are uniquely
- titled for that particular instance. For example, if
- you copy all data from
- /var/lib/nova/instances on one
- compute node to another, you do not overwrite any
- files or cause any damage to images that have the same
- unique name, because they are essentially the same
- file./var/lib/nova/instances./var/lib/nova/instances contains two types of
+ directories._base directory. This contains all
+ the cached base images from glance for each unique image that has been
+ launched on that compute node. Files ending in _20 (or a
+ different number) are the ephemeral base images.instance-xxxxxxxx.
+ These directories correspond to instances running on that compute node.
+ The files inside are related to one of the files in the
+ _base directory. They're essentially differential-based
+ files containing only the changes made from the original
+ _base directory./var/lib/nova/instances
+ are uniquely named. The files in _base are uniquely titled for the
+ glance image that they are based on, and the directory names
+ instance-xxxxxxxx are uniquely titled for that particular
+ instance. For example, if you copy all data from
+ /var/lib/nova/instances on one compute node to another, you
+ do not overwrite any files or cause any damage to images that have the
+ same unique name, because they are essentially the same file.swift-ring-builder heavily depends on
- the original options used when you originally created
- your cluster. Please refer back to those
- commands./dev/sdb has
- failed./dev/sdb.sql_connection
- or simply connection. The following
- command uses grep to display the SQL
- connection string for nova, glance, cinder, and
- keystone:nova-api,
- glance-api, glance-registry,
- keystone, and potentially swift-proxy. As a
- result, it is sometimes difficult to determine exactly
- where problems lie. Assisting in this is the purpose of
- this section.nova list is failing, try tailing a
- nova log file and running the command again:glance-api service refuses to start
- and stay running, try launching the daemon from the
- command line:libvirtd daemon was run on
- the command line. Finally a helpful error message: it
- could not connect to d-bus. As ridiculous as it
- sounds, libvirt, and thus nova-compute,
- relies on d-bus and somehow d-bus crashed. Simply
- starting d-bus set the entire chain back on track and
- soon everything was back up and running.swift-ring-builder heavily depends on the original
+ options used when you originally created your cluster. Please refer back
+ to those commands./dev/sdb has failed./dev/sdb.| Priority | + +Services | +
|---|---|
sql_connection or simply connection. The
+ following command uses grep to display the SQL connection
+ string for nova, glance, cinder, and keystone:nova-api, glance-api,
+ glance-registry, keystone, and potentially
+ swift-proxy. As a result, it is sometimes difficult to
+ determine exactly where problems lie. Assisting in this is the purpose of
+ this section.nova list is failing,
+ try tailing a nova log file and running the command again:glance-api service refuses to start and stay running, try
+ launching the daemon from the command line:libvirtd daemon was run on the command
+ line. Finally a helpful error message: it could not connect to d-bus.
+ As ridiculous as it sounds, libvirt, and thus
+ nova-compute, relies on d-bus and somehow d-bus crashed.
+ Simply starting d-bus set the entire chain back on track, and soon
+ everything was back up and running.br100.
- flat_interface_bridge
- option.br-int. This bridge connects all
- the instance TAP devices and any other bridges on the
- system. In this example, we have
- int-br-eth1 and
- patch-tun. int-br-eth1 is
- one half of a veth pair connecting to the bridge
- br-eth1, which handles VLAN networks
- trunked over the physical Ethernet device
- eth1. patch-tun is an Open
- vSwitch internal port that connects to the
- br-tun bridge for GRE networks.patch-tun, are only
- visible within the Open vSwitch environment. If you
- try to run snooper0.snooper0 to bridge
- br-int.patch-tun to
- snooper0 (returns UUID of mirror port).br100.flat_interface_bridge option.br-int. This bridge connects all the instance TAP devices
+ and any other bridges on the system. In this example, we have
+ int-br-eth1 and patch-tun.
+ int-br-eth1 is one half of a veth pair connecting to the
+ bridge br-eth1, which handles VLAN networks trunked over
+ the physical Ethernet device eth1. patch-tun
+ is an Open vSwitch internal port that connects to the
+ br-tun bridge for GRE networks.patch-tun, are only visible
+ within the Open vSwitch environment. If you try to run
+ snooper0:snooper0 to bridge
+ br-int:patch-tun to
+ snooper0 (returns UUID of mirror port):patch-tun by running
- br-int and deleting the dummy
- interface.br-int,
- incoming packets from the int-br-eth1 are translated from external
- tags to internal tags. Other translations also happen on the other bridges and
- will be discussed in those sections.provider:segmentation_id as
- returned by the networking service:patch-tun by
+ running br-int and
+ deleting the dummy interface:br-int, incoming packets from the
+ int-br-eth1 are translated from external tags to internal
+ tags. Other translations also happen on the other bridges and will be
+ discussed in those sections.provider:segmentation_id as returned
+ by the networking service:provider:segmentation_id, 2113 in this
- case, in the output of provider:segmentation_id, 2113 in
+ this case, in the output of int-br-eth1.int-br-eth1:int-br-eth1
- and arrive on the bridge br-eth1 on the
- other member of the veth pair
- phy-br-eth1. Packets on this interface
- arrive with internal VLAN tags and are translated to
- external tags in the reverse of the process described
- above.
- eth1. The
- Layer2 switch this interface is connected to must be
- configured to accept traffic with the VLAN ID used.
- The next hop for this packet must also be on the
- same layer-2 network.patch-tun to the tunnel bridge
- br-tun on interface
- patch-int. This bridge also
- contains one port for each GRE tunnel peer, so one
- for each compute node and network node in your
- network. The ports are named sequentially from
- gre-1 onward.gre-<n> interfaces to
- tunnel endpoints is possible by looking at the Open
- vSwitch state:int-br-eth1 and arrive on the bridge
+ br-eth1 on the other member of the veth pair
+ phy-br-eth1. Packets on this interface arrive with
+ internal VLAN tags and are translated to external tags in the
+ reverse of the process described above:eth1. The Layer2 switch
+ this interface is connected to must be configured to accept
+ traffic with the VLAN ID used. The next hop for this packet must
+ also be on the same layer-2 network.patch-tun to
+ the tunnel bridge br-tun on interface
+ patch-int. This bridge also contains one port for
+ each GRE tunnel peer, so one for each compute node and network
+ node in your network. The ports are named sequentially from
+ gre-1 onward.gre-<n> interfaces to tunnel
+ endpoints is possible by looking at the Open vSwitch state:gre-1 is a tunnel from
- IP 10.10.128.21, which should match a local
- interface on this node, to IP 10.10.128.16 on the
- remote side.br-tun are
- internal to Open vSwitch. To monitor traffic on them
- you need to set up a mirror port as described above
- for patch-tun in the
- br-int bridge.provider:segmentation_id of the network
- you're interested in. This is the same field used for the VLAN
- ID in VLAN-based networks:gre-1 is a tunnel from IP
+ 10.10.128.21, which should match a local interface on this node,
+ to IP 10.10.128.16 on the remote side.br-tun are internal to
+ Open vSwitch. To monitor traffic on them, you need to set up a
+ mirror port as described above for patch-tun in the
+ br-int bridge.provider:segmentation_id of the
+ network you're interested in. This is the same field used for the
+ VLAN ID in VLAN-based networks:provider:segmentation_id>,
- 0x3 in this case, in the output of
- provider:segmentation_id>,
+ 0x3 in this case, in the output of eth1 in
- this example. Just as on the compute node, this
- interface is a member of the br-eth1
- bridge.br-tun, which behaves just like the
- GRE interfaces on the compute node.qrouter-<router-uuid>. Running
- eth1 in this example.
+ Just as on the compute node, this interface is a member of the
+ br-eth1 bridge.br-tun, which behaves just like the GRE interfaces on
+ the compute node.qrouter-<router-uuid>. Running qg-<n> interface in the
- l3-agent router namespace sends the packet on to its
- next hop through device eth0 on the
- external bridge br-ex. This bridge is
- constructed similarly to br-eth1 and may
- be inspected in the same way.eth0 in this example,
- which finally lands the packet on the external network
- destined for an external router or destination.
- qdhcp-<uuid> and have a TAP
- device on the integration bridge. Debugging of DHCP
- issues usually involves working inside this network
- namespace.tcpdump -i any -n -v 'icmp[icmptype] =
- icmp-echoreply or icmp[icmptype] = icmp-echo'
- qg-<n> interface in the l3-agent router
+ namespace sends the packet on to its next hop through device
+ eth0 on the external bridge br-ex. This
+ bridge is constructed similarly to br-eth1 and may be
+ inspected in the same way.eth0 in this example, which finally lands the packet on
+ the external network destined for an external router or
+ destination.qdhcp-<uuid> and have a TAP device on the
+ integration bridge. Debugging of DHCP issues usually involves working
+ inside this network namespace.tcpdump on the interfaces to
- determine where the packets are getting lost.tcpdump to
- listen to ports 67 and 68 on br100, you would do:ip a and brctl
- show to ensure that the interfaces are
- actually up and configured the way that you think that
- they are.host command. If DNS is working, you
- should see:tcpdump on the interfaces to determine where
+ the packets are getting lost.tcpdump to listen to ports 67 and 68 on br100, you would
+ do:ip a and brctl show to ensure that the
+ interfaces are actually up and configured the way that you think that they
+ are.host command. If DNS is
+ working, you should see:tcpdump to look at the packets to trace where the
+ failure is. The DNS server listens on UDP port 53. You should see the DNS
+ request on the bridge (such as, br100) of your compute node. Let's say you
+ start listening with tcpdump on the compute node:ping openstack.org, you should see
- something like:ping
+ openstack.org, you should see something like:br-int, br-tun,
- br-ex, etc...) exist and have the proper ports
- connected to them.eth1 network interface:
- br-int,
+ br-tun, br-ex, etc.) exist and have the proper
+ ports connected to them.eth1 network interface:eth1-br, which contains the physical network
- interface eth1 and the virtual interface
- phy-eth1-br.
- eth1-br, which contains
+ the physical network interface phy-eth1-br:br-int, contains
- int-eth1-br, which pairs with
- phy-eth1-br to connect to the physical network shown
- in the previous bridge, patch-tun, which is used
- to connect to the GRE tunnel bridge and the TAP devices that
- connect to the instances currently running on the system.
- br-int, contains
+ int-eth1-br, which pairs with phy-eth1-br to
+ connect to the physical network shown in the previous bridge,
+ patch-tun, which is used to connect to the GRE tunnel bridge
+ and the TAP devices that connect to the instances currently running on the
+ system:br-tun, contains the
- patch-int interface and
- gre-<N> interfaces for each peer it
- connects to via GRE, one for each compute and network node in
- your cluster.
- br-tun, contains the
+ patch-int interface and gre-<N> interfaces
+ for each peer it connects to via GRE, one for each compute and network
+ node in your cluster:--description
- tenant-description , which can be very useful. You can also
- create a group in a disabled state by appending --enabled false to the command.
- By default, projects are created in an enabled state.--description
+ tenant-description , which can be very
+ useful. You can also create a group in a disabled state by appending
+ --enabled false to the command. By default, projects are
+ created in an enabled state.image_member_quota, set to 128 by default. That setting is a
- different quota from the storage quota.image_member_quota, set to 128
+ by default. That setting is a different quota from the storage
+ quota.| Quota | +Description | -Property Name | + +Property name | ||
|---|---|---|---|---|---|
|
- |
-
- |
-
- |
+ |||
|
- |
-
- |
-
- |
+ |||
|
- |
-
- |
-
- |
+ |||
|
- |
-
- |
-
- |
+ |||
|
- |
-
- |
-
- |
+ |||
|
- |
-
- |
-
- |
+ |||
|
- |
-
- |
-
- |
+ |||
|
- |
-
- |
-
- |
+ |||
|
- |
-
- |
-
- |
+ |||
|
- |
-
- |
-
- |
+ |||
|
- |
-
- |
-
- |
+ |||
|
- |
-
- |
-
- |
+
container_quotas or account_quotas (or
- both) added to the [pipeline:main] pipeline. Each quota type also requires its own section
- in the container_quotas
+ or account_quotas (or both) added to the
+ swift command provided by
- the python-swiftclient package. Any user included in the project can view the
- quotas placed on their project. To update Object Storage quotas on a project, you must have
- the role of ResellerAdmin in the project that the quota is being applied to.swift command provided by the
+ python-swiftclient package. Any user included in the
+ project can view the quotas placed on their project. To update Object
+ Storage quotas on a project, you must have the role of ResellerAdmin in
+ the project that the quota is being applied to.| Property Name | -Description | +Property name | + +Description |
|---|---|---|---|
|
- |
-
- |
+ ||
|
- |
-
- |
+ ||
|
- |
-
- |
+
policy.json file. The actual
- location of this file might vary from distribution to distribution: for nova, it is
- typically in /etc/nova/policy.json. You can update entries while the system is
- running, and you do not have to restart services. Currently, the only way to update such
- policies is to edit the policy file.compute:create:
- [["rule:admin_or_owner"]] statement, the
- policy is compute:create, and the rule is
- admin_or_owner.create:compute policy every time a
- user sends a POST /v2/{tenant_id}/servers
- request to the OpenStack Compute API server. Policies
- can be also related to specific compute_extension:rescue the
- attributes defined by the provider extensions trigger
- the rule test for that operation."role:admin" is
- successful if the user submitting the request is an administrator."field:networks:shared=True" is successful if the attribute shared of the
- network resource is set to true."tenant_id:%(tenant_id)s" is successful if the tenant identifier in the
- resource is equal to the tenant identifier of the user submitting the request.policy.json
+ file. The actual location of this file might vary from distribution to
+ distribution: for nova, it is typically in
+ /etc/nova/policy.json. You can update entries while the
+ system is running, and you do not have to restart services. Currently,
+ the only way to update such policies is to edit the policy file.compute:create: [["rule:admin_or_owner"]]
+ statement, the policy is compute:create, and the rule is
+ admin_or_owner.create:compute policy every time a user sends a POST
+ /v2/{tenant_id}/servers request to the OpenStack Compute API
+ server. Policies can be also related to specific compute_extension:rescue, the attributes defined by the
+ provider extensions trigger the rule test for that operation."role:admin" is
+ successful if the user submitting the request is an
+ administrator."field:networks:shared=True" is successful if the
+ attribute shared of the network resource is set to
+ "tenant_id:%(tenant_id)s" is successful if the tenant
+ identifier in the resource is equal to the tenant identifier of
+ the user submitting the request.policy.json.policy.json.tcpdump Tutorial and Primer”dist-upgrade may restart services supplemental
- to your OpenStack environment. For example, if you use
- Open-iSCSI for Block Storage volumes and the upgrade includes
- a new open-scsi package, the package manager
- will restart Open-iSCSI services, which may cause the
- volumes for your users to be disconnected..dpkg-dist to the
- end of newer versions of existing configuration files. You
- should consider adopting conventions associated with the
- newer configuration files and merging them with your existing
- configuration files after completing the upgrade process.dist-upgrade may restart services supplemental to your
+ OpenStack environment. For example, if you use Open-iSCSI for Block
+ Storage volumes and the upgrade includes a new open-scsi
+ package, the package manager will restart Open-iSCSI services, which
+ may cause the volumes for your users to be disconnected..dpkg-dist to the end of newer versions
+ of existing configuration files. You should consider adopting
+ conventions associated with the newer configuration files and merging
+ them with your existing configuration files after completing the upgrade
+ process..dpkg-dist file after completing
- the upgrade process..dpkg-dist file after completing the upgrade
+ process..dpkg-dist file after completing
- the upgrade process..dpkg-dist file after completing the upgrade
+ process..rpmnew to
- the end of newer versions of existing configuration files.
- You should consider adopting conventions associated with
- the newer configuration files and merging them with your
- existing configuration files after completing the upgrade
- process..rpmnew to the end
+ of newer versions of existing configuration files. You should consider
+ adopting conventions associated with the newer configuration files and
+ merging them with your existing configuration files after completing
+ the upgrade process."context_is_admin": "role:admin", which limits
- access to private images for projects.deinstall state, and saving the final
- output to a file. For example, the following
- command covers a controller node with keystone,
- glance, nova, neutron, and cinder:"context_is_admin": "role:admin", which limits access to
+ private images for projects.deinstall state, and saving
+ the final output to a file. For example, the following command
+ covers a controller node with keystone, glance, nova, neutron, and
+ cinder:1:2013.1.4-0ubuntu1~cloud0. The
- process of manually picking through this list of
- packages is rather tedious and prone to errors.
- You should consider using the following script
- to help with this process:neutron to quantum where
- applicable.<package-name>=<version>.
- The script in the previous step conveniently created
- a list of package=version pairs for
- you.neutron to quantum where
+ applicable.<package-name>=<version>. The script in
+ the previous step conveniently created a list of
+ package=version pairs for you:openstack@lists.openstack.org.
- The scope of this list is the current state of OpenStack.
- This is a very high-traffic mailing list, with many, many
- emails per day.openstack-operators@lists.openstack.org.
- This list is intended for discussion among
- existing OpenStack cloud operators, such as
- yourself. Currently, this list is relatively low
- traffic, on the order of one email a day.openstack-dev@lists.openstack.org.
- The scope of this list is the future state of OpenStack.
- This is a high-traffic mailing list, with multiple emails
- per day.#openstack
- on irc.freenode.net.glance image-create command
- provides a large set of options for working with your image.
- For example, the min-disk option is
- useful for images that require root disks of a certain
- size (for example, large Windows images). To view
- these options, do:location option is important to
- note. It does not copy the entire image into the Image Service,
- but references an original location where the image
- can be found. Upon launching an instance of that
- image, the Image Service accesses the image from the location
- specified.copy-from option copies the image
- from the location specified into the
- /var/lib/glance/images directory. The
- same thing is done when using the STDIN redirection with <
- such as shown in the example.glance image-create command provides a large set
+ of options for working with your image. For example, the
+ min-disk option is useful for images that require root
+ disks of a certain size (for example, large Windows images). To view
+ these options, do:location option is important to note. It does not
+ copy the entire image into the Image Service, but references an original
+ location where the image can be found. Upon launching an instance of
+ that image, the Image Service accesses the image from the location
+ specified.copy-from option copies the image from the
+ location specified into the /var/lib/glance/images
+ directory. The same thing is done when using the STDIN redirection with
+ <, as shown in the example.compute_extension:flavormanage in
- /etc/nova/policy.json on the nova-api server). To get the
- list of available flavors on your system, run:compute_extension:flavormanage in
+ /etc/nova/policy.json on the nova-api server).
+ To get the list of available flavors on your system, run:nova flavor-create command allows authorized users to create new flavors. Additional flavor manipulation commands can be shown with the command: |
- |
-
- |
-
|
- |
-
- |
-
|
- |
-
- |
-
|
- |
-
- |
-
|
- |
-
- |
-
|
- |
-
- |
-
|
- |
-
- |
-
|
- |
-
- |
-
|
- |
-
- |
-
|
- |
-
- |
-
|
- |
-
- |
-
nova flavor-create command allows authorized users
+ to create new flavors. Additional flavor manipulation commands can be
+ shown with the command: nova.conf option
- allow_same_net_traffic (which defaults to
- true) globally controls whether the rules apply to hosts
- that share a network. When set to true, hosts on the same
- subnet are not filtered and are allowed to pass all types
- of traffic between them. On a flat network, this allows
- all instances from all projects unfiltered communication.
- With VLAN networking, this allows access between instances
- within the same project. If
- allow_same_net_traffic is set to false,
- security groups are enforced for all connections. In this
- case, it is possible for projects to simulate
- allow_same_net_traffic by configuring
- their default security group to allow all traffic from
- their subnet.quota_security_group_rules and the number of allowed
- security groups per project is controlled by the
- quota_security_groups quota.nova.conf option
+ allow_same_net_traffic (which defaults to
+ allow_same_net_traffic is set to allow_same_net_traffic by
+ configuring their default security group to allow all traffic from their
+ subnet.quota_security_group_rules, and the number of allowed
+ security groups per project is controlled by the
+ quota_security_groups quota.nova secgroup-add-group-rule <secgroup> <source-group> <ip-proto> <from-port> <to-port>nova
+ secgroup-add-group-rule <secgroup> <source-group>
+ <ip-proto> <from-port> <to-port>. An example
+ usage is shown here:nova show on the faulted
- instance:nova show on the faulted instance:nova show does not sufficiently
- explain the failure, searching for the instance UUID in
- the nova-compute.log on the compute node
- it was scheduled on or the
- nova-scheduler.log on your scheduler
- hosts is a good place to start looking for lower-level
- problems.nova show as an admin user will
- show the compute node the instance was scheduled on as
- hostId. If the instance failed during
- scheduling this field is blank.--key_name mykey to your command line.
- For example:--meta option with a key=value
- pair, where you can make up the string for both
- the key and the value. For example, you could add
- a description and also the creator of the
- server:nova show does not sufficiently explain the
+ failure, searching for the instance UUID in the
+ nova-compute.log on the compute node it was scheduled on or
+ the nova-scheduler.log on your scheduler hosts is a good
+ place to start looking for lower-level problems.nova show as an admin user will show the
+ compute node the instance was scheduled on as hostId. If
+ the instance failed during scheduling, this field is blank.--key_name mykey to your command line. For
+ example:--meta option with a key-value pair, where you
+ can make up the string for both the key and the value. For example,
+ you could add a description and also the creator of the server:user-data key is a special key in the
- metadata service that holds a file that cloud-aware
- applications within the guest instance can access.
- For example cloudinit (https://help.ubuntu.com/community/CloudInit)
- is an open source package from Ubuntu, but available in most
- distributions, that handles early initialization of a cloud
- instance that makes use of this user data.--user-data <user-data-file>.
- For example:--file
- <dst-path=src-path> option. You may store up to
- five files. For example, let's say you have a special
- user-data key is a special key in the metadata
+ service that holds a file that cloud-aware applications within the
+ guest instance can access. For example, cloudinit is an open
+ source package from Ubuntu, but available in most distributions, that
+ handles early initialization of a cloud instance that makes use of
+ this user data.--user-data
+ <user-data-file>. For example:--file
+ <dst-path=src-path> option. You may store up to five
+ files.--security-groups with a comma-separated list of
- security groups.<dev-name>=<id>:<type>:<size(GB)>:<delete-on-terminate>,
- where:/dev/dev_name
- ./dev/vdc. It is not a snapshot,
- does not specify a size, and will not be deleted when the
- instance is terminated:/dev/vda:sync writes dirty buffers
- (buffered blocks that have been modified but not
- written yet to the disk block) to disk.sync is not enough to
- ensure that the file system is consistent. We recommend that you
- use the fsfreeze tool, which halts new
- access to the file system, and create a stable image on
- disk that is suitable for snapshotting. The fsfreeze
- tool supports several file systems, including ext3, ext4,
- and XFS. If your virtual machine instance is running
- on Ubuntu, install the util-linux package to get
- fsfreeze:--security-groups with a comma-separated list of security
+ groups.<dev-name>=<id>:<type>:<size(GB)>:<delete-on-terminate>,
+ where:/dev/dev_name /dev/vdc. It is not a snapshot, does not specify a size, and
+ will not be deleted when the instance is terminated:/dev/vda:sync writes dirty buffers (buffered blocks
+ that have been modified but not written yet to the disk block) to
+ disk.sync is not enough to ensure that the
+ file system is consistent. We recommend that you use the
+ fsfreeze tool, which halts new access to the file system,
+ and create a stable image on disk that is suitable for snapshotting.
+ The fsfreeze tool supports several file systems,
+ including ext3, ext4, and XFS. If your virtual machine instance is
+ running on Ubuntu, install the util-linux package to get
+ (Alice,
- delete) for a file gives Alice permission to
- delete the file.
- (Alice, delete) for a file gives
+ Alice permission to delete the file.| Type | + +Description | + +Example hardware | +
|---|---|---|
| Controller | + +||
| Compute | +||
| Storage | +||
| Network | +||
| Utility | +
| Type | -Description | -Example Hardware | -
|---|---|---|
| Controller | -
- |
- |
| Compute | -
- |
- |
| Storage | -
- |
- |
| Network | -
- |
- |
| Utility | -
- |
-
| Component | -Tuning | -Availability | -Scalability | -
|---|---|---|---|
| MySQL | -Master-Master replication. However, both nodes are not used at the same - time. Replication keeps all nodes as close to being up to date as - possible (although the asynchronous nature of the replication means a - fully consistent state is not possible). Connections to the database - only happen through a Pacemaker virtual IP, ensuring that most problems that - occur with master-master replication can be avoided. | -Not heavily considered. Once load on the MySQL server increases enough - that scalability needs to be considered, multiple masters or a - master/slave setup can be used. | -|
| Qpid | -Qpid is added as a resource to the Pacemaker software that runs on - Controller nodes where Qpid is situated. This ensures only one Qpid - instance is running at one time, and the node with the Pacemaker virtual IP will - always be the node running Qpid. | -Not heavily considered. However, Qpid can be changed to run on all - controller nodes for scalability and availability purposes, and removed - from Pacemaker. | -|
| Haproxy | -Haproxy is a software layer-7 load balancer used to front door all - clustered OpenStack API components and do SSL termination. Haproxy can - be added as a resource to the Pacemaker software that runs on the - Controller nodes where HAProxy is situated. This ensures that only one - HAProxy instance is running at one time, and the node with the Pacemaker - virtual IP will always be the node running HAProxy. | -Not considered. Haproxy has small enough performance overheads that a
- single instance should scale enough for this level of workload. If extra
- scalability is needed, |
- |
| Memcached | -Memcached is a fast in-memory key/value cache software that is used by - OpenStack components for caching data and increasing performance. - Memcached runs on all controller nodes, ensuring that should one go - down, another instance of memcached is available. | -Not considered. A single instance of memcached should be able to scale
- to the desired workloads. If scalability is desired, Haproxy can be
- placed in front of Memcached (in raw |
- |
| Pacemaker | -Configured to use |
- If more nodes need to be made cluster aware, Pacemaker can scale to 64 - nodes. | -|
| GlusterFS | -Glusterfs is a clustered file system that is run on the storage nodes to
- provide persistent scalable data storage in the environment. Because all
- connections to gluster use the |
- The scalability of GlusterFS storage can be achieved by adding in more - storage volumes. | -
| Component | -Node type | -Tuning | -Availability | -Scalability | -
|---|---|---|---|---|
| Dashboard (horizon) | -Controller | -Configured to use |
- The Dashboard is run on all controller nodes, ensuring at least one - instance will be available in case of node failure. It also sits behind - HAProxy, which detects when the software fails and routes requests - around the failing instance. | -The Dashboard is run on all controller nodes, so scalability can be achieved with additional - controller nodes. Haproxy allows scalability for the Dashboard as more nodes are added. | -
| Identity (keystone) | -Controller | -Configured to use memcached for caching, and use PKI for tokens. | -Identity is run on all controller nodes, ensuring at least one instance will be available - in case of node failure. Identity also sits behind HAProxy, which detects when the software fails - and routes requests around the failing instance. | -Identity is run on all controller nodes, so scalability can be achieved with additional - controller nodes. Haproxy allows scalability for Identity as more nodes are added. | -
| Image Service (glance) | -Controller | -The Image Service is run on all controller nodes, ensuring at least one - instance will be available in case of node failure. It also sits behind - HAProxy, which detects when the software fails and routes requests - around the failing instance. | -The Image Service is run on all controller nodes, so scalability can be achieved with additional controller - nodes. HAProxy allows scalability for the Image Service as more nodes are added. | -|
| Compute (nova) | -Controller, Compute | -The nova API, scheduler, objectstore, cert, consoleauth, conductor, and - vncproxy services are run on all controller nodes, so scalability can be - achieved with additional controller nodes. HAProxy allows scalability - for Compute as more nodes are added. The scalability of services running - on the compute nodes (compute, conductor) is achieved linearly by adding - in more compute nodes. | -||
| Block Storage (cinder) | -Controller | -Configured to use Qpid, |
- Block Storage API, scheduler, and volume services are run on all controller nodes, - ensuring at least one instance will be available in case of node failure. Block - Storage also sits behind HAProxy, which detects if the software fails and routes - requests around the failing instance. | -Block Storage API, scheduler and volume services are run on all - controller nodes, so scalability can be achieved with additional - controller nodes. HAProxy allows scalability for Block Storage as more - nodes are added. | -
| OpenStack Networking (neutron) | -Controller, Compute, Network | -Configured to use QPID. |
- The OpenStack Networking server service is run on all controller nodes,
- so scalability can be achieved with additional controller nodes. HAProxy
- allows scalability for OpenStack Networking as more nodes are added.
- Scalability of services running on the network nodes is not currently
- supported by OpenStack Networking, so they are not be considered. One
- copy of the services should be sufficient to handle the workload.
- Scalability of the |
-
| Component | + +Tuning | + +Availability | + +Scalability | +
|---|---|---|---|
| MySQL | + +Master/master replication. However, both nodes are not used at + the same time. Replication keeps all nodes as close to being up to + date as possible (although the asynchronous nature of the + replication means a fully consistent state is not possible). + Connections to the database only happen through a Pacemaker + virtual IP, ensuring that most problems that occur with + master-master replication can be avoided. | + +Not heavily considered. Once load on the MySQL server + increases enough that scalability needs to be considered, multiple + masters or a master/slave setup can be used. | +|
| Qpid | + +Qpid is added as a resource to the Pacemaker software that + runs on Controller nodes where Qpid is situated. This ensures only + one Qpid instance is running at one time, and the node with the + Pacemaker virtual IP will always be the node running Qpid. | + +Not heavily considered. However, Qpid can be changed to run on + all controller nodes for scalability and availability purposes, + and removed from Pacemaker. | +|
| HAProxy | + +HAProxy is a software layer-7 load balancer used to front door + all clustered OpenStack API components and do SSL termination. + HAProxy can be added as a resource to the Pacemaker software that + runs on the Controller nodes where HAProxy is situated. This + ensures that only one HAProxy instance is running at one time, and + the node with the Pacemaker virtual IP will always be the node + running HAProxy. | + +Not considered. HAProxy has small enough performance overheads
+ that a single instance should scale enough for this level of
+ workload. If extra scalability is needed,
+ |
+ |
| Memcached | + +Memcached is a fast in-memory key-value cache software that is + used by OpenStack components for caching data and increasing + performance. Memcached runs on all controller nodes, ensuring that + should one go down, another instance of Memcached is + available. | + +Not considered. A single instance of Memcached should be able
+ to scale to the desired workloads. If scalability is desired,
+ HAProxy can be placed in front of Memcached (in raw
+ |
+ |
| Pacemaker | + +Configured to use |
+
+ If more nodes need to be made cluster aware, Pacemaker can + scale to 64 nodes. | +|
| GlusterFS | + +Glusterfs is a clustered file system that is run on the
+ storage nodes to provide persistent scalable data storage in the
+ environment. Because all connections to gluster use the
+ |
+
+ The scalability of GlusterFS storage can be achieved by adding + in more storage volumes. | +
| Component | + +Node type | + +Tuning | + +Availability | + +Scalability | +
|---|---|---|---|---|
| Dashboard (horizon) | + +Controller | + +Configured to use Memcached as a session store,
+ |
+
+ The dashboard is run on all controller nodes, ensuring at least + one instance will be available in case of node failure. It also + sits behind HAProxy, which detects when the software fails and + routes requests around the failing instance. | + +The dashboard is run on all controller nodes, so scalability + can be achieved with additional controller nodes. HAProxy allows + scalability for the dashboard as more nodes are added. | +
| Identity (keystone) | + +Controller | + +Configured to use Memcached for caching and PKI for + tokens. | + +Identity is run on all controller nodes, ensuring at least one + instance will be available in case of node failure. Identity also + sits behind HAProxy, which detects when the software fails and + routes requests around the failing instance. | + +Identity is run on all controller nodes, so scalability can be + achieved with additional controller nodes. HAProxy allows + scalability for Identity as more nodes are added. | +
| Image Service (glance) | + +Controller | + +The Image Service is run on all controller nodes, ensuring at + least one instance will be available in case of node failure. It + also sits behind HAProxy, which detects when the software fails + and routes requests around the failing instance. | + +The Image Service is run on all controller nodes, so + scalability can be achieved with additional controller nodes. + HAProxy allows scalability for the Image Service as more nodes are + added. | +|
| Compute (nova) | + +Controller, Compute | + +The nova API, scheduler, objectstore, cert, consoleauth, + conductor, and vncproxy services are run on all controller nodes, + so scalability can be achieved with additional controller nodes. + HAProxy allows scalability for Compute as more nodes are added. + The scalability of services running on the compute nodes (compute, + conductor) is achieved linearly by adding in more compute + nodes. | +||
| Block Storage (cinder) | + +Controller | + +Configured to use Qpid, |
+
+ Block Storage API, scheduler, and volume services are run on all + controller nodes, ensuring at least one instance will be available + in case of node failure. Block Storage also sits behind HAProxy, + which detects if the software fails and routes requests around the + failing instance. | + +Block Storage API, scheduler and volume services are run on + all controller nodes, so scalability can be achieved with + additional controller nodes. HAProxy allows scalability for Block + Storage as more nodes are added. | +
| OpenStack Networking (neutron) | + +Controller, Compute, Network | + +Configured to use QPID, |
+
+ The OpenStack Networking server service is run on all
+ controller nodes, so scalability can be achieved with additional
+ controller nodes. HAProxy allows scalability for OpenStack
+ Networking as more nodes are added. Scalability of services
+ running on the network nodes is not currently supported by
+ OpenStack Networking, so they are not be considered. One copy of
+ the services should be sufficient to handle the workload.
+ Scalability of the |
+
nova-network daemon per OpenStack Compute
- host. This provides a robust mechanism for ensuring network
- interruptions are isolated to individual compute hosts, and allows
- for the direct use of hardware network gateways.nova-network daemon per OpenStack compute host. This
+ provides a robust mechanism for ensuring network interruptions are
+ isolated to individual compute hosts, and allows for the direct use of
+ hardware network gateways.nova-network service that runs within the cloud
- (usually on the cloud controller) that provides services such as
- network address translation (NAT), DHCP, and DNS to the guest
- instances. If the single node that runs the
- nova-network service goes down, you cannot
- access your instances and the instances cannot access the
- Internet. The single node that runs the nova-network service can
- become a bottleneck if excessive network traffic comes in and
- goes out of the cloud.nova-consoleauth), Image services
- (glance-api,
- glance-registry), services for console access
- of guests, and Block Storage services including the
- scheduler for storage resources (cinder-api
- and cinder-scheduler).nova-compute,
- nova-api-metadata (generally only used
- when running in multi-host mode, it retrieves
- instance-specific metadata), nova-vncproxy,
- and nova-network.
- nova-network service that runs within the cloud (usually on
+ the cloud controller) that provides services such as network address
+ translation (NAT), DHCP, and DNS to the guest instances. If the single
+ node that runs the nova-network service goes down, you
+ cannot access your instances, and the instances cannot access the
+ Internet. The single node that runs the nova-consoleauth), Image services
+ (glance-api, glance-registry), services for
+ console access of guests, and Block Storage services, including the
+ scheduler for storage resources (cinder-api and
+ cinder-scheduler).nova-compute, nova-api-metadata (generally only
+ used when running in multi-host mode, it retrieves instance-specific
+ metadata), nova-vncproxy, and
+ nova-network.