operations-guide/doc/openstack-ops/preface_ops.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE preface [
<!-- Some useful entities borrowed from HTML -->
<!ENTITY ndash  "&#x2013;">
<!ENTITY mdash  "&#x2014;">
<!ENTITY hellip "&#x2026;">
<!ENTITY plusmn "&#xB1;">
]>
<preface xmlns="http://docbook.org/ns/docbook"
    xmlns:xi="http://www.w3.org/2001/XInclude"
    xmlns:xlink="http://www.w3.org/1999/xlink" version="5.0"
    xml:id="openstack-ops_preface">
    <?dbhtml stop-chunking?>
    <title>Preface</title>
    <para>OpenStack is an open source platform that lets you build an
        Infrastructure as a Service (IaaS) cloud that runs on commodity
        hardware.</para>
    <section xml:id="introduction-to-openstack">
        <title>Introduction to OpenStack</title>
        <para>OpenStack believes in open source, open design, open development, all in an open
            community encouraging participation by anyone. The long-term vision for OpenStack
            is to produce a ubiquitous open source cloud computing platform that meets the needs of public and
            private cloud providers regardless of size. OpenStack services control large pools of
            compute, storage, and networking resources throughout a data center.</para>
        <para>The technology behind OpenStack consists of a series of interrelated projects
            delivering various components for a cloud infrastructure solution.
            Each service provides an open API so that all of these resources can be managed through a
            dashboard that gives administrators control while empowering users to provision
            resources through a web interface, a command-line client, or software
            development kits that support the API. Many OpenStack APIs are extensible, meaning you
            can keep compatibility with a core set of calls while providing access to more resources
            and innovating through API extensions. The OpenStack project is a global collaboration
            of developers and cloud computing technologists producing the open standard cloud
            computing platform for both public and private clouds. By focusing on ease of
            implementation, massive scalability, a variety of rich features and tremendous
            extensibility, the project aims to deliver a practical and reliable cloud solution for
            all types of organisations.</para>
        <section xml:id="preface_getting_started"><title>Getting Started with OpenStack</title>
        <para>As an open source project, one of the unique aspects about OpenStack is that there are
            many different levels you can begin to engage with it &mdash; you don't have to do everything
            yourself.</para>
            <section xml:id="preface_using_openstack">
                <title>Using OpenStack</title>
                <para>You could ask, "do I even need to build a cloud?". If you
                    want to start using a Compute or Storage service by
                    just swiping your credit card, you can go to eNovance, HP, Rackspace and other
                    organisations to start using their public OpenStack clouds.
                    Using their OpenStack cloud resources is similar to accessing
                    the publically-available Amazon Web Services Elastic Compute
                    Cloud (EC2) or Simple Storage Solution (S3).</para>
            </section>
                <section xml:id="preface_plug_and_play"><title>Plug and Play OpenStack</title>
                <para>However, the enticing part of OpenStack might be to build your own private
            cloud, and there are several ways to accomplish this goal. Perhaps the simplest of all is an
            appliance-style solution. You purchase an appliance, un-box it, plug in the power and the
            network and watch it transform into an OpenStack cloud with minimal additional
            configuration. Few, if any, other open source cloud products that have such turn-key options.
            If a turn-key solution is interesting to you, take a look at Nebula One.</para>
                <para>However, hardware choice is important for many applications, so if that applies to you,
            consider that there are several software distributions available that you can run on
            servers, storage and network products of your choosing. Canonical (where OpenStack replaced Eucalyptus as
            the default cloud option in 2011), Red Hat and SUSE offer enterprise OpenStack
            solutions and support. You may also want to take a look also at some of the
            specialized distributions, such as those from Rackspace, Piston, SwiftStack or
            Cloudscaling. Also, a hat tip to Apache CloudStack, which Citrix donated to the Apache
            Foundation after its US $200 million purchase of Cloud.com. While not currently packaged
            in any distributions, like Eucalyptus it is an example of an alternative private
            cloud software developed in an open source-like manner.</para>
                <para>Alternately, if you want someone to help guide you through the decisions from the
            underlying hardware up to your applications, perhaps adding in a few features or integrating
            components along the way, consider contacting one of the system integrators with
            OpenStack experience like Mirantis or Metacloud.</para>
                <para>If your preference is to build your own OpenStack expertise internally, a good way to
            kick start that might be to attend or arrange a training session. The OpenStack
            Foundation recently launched a <link
                        xlink:href="http://www.openstack.org/marketplace/training"
                        >Training Marketplace
            (http://www.openstack.org/marketplace/training)</link>, where you can look for nearby events.
            Also the OpenStack community is <link
                        xlink:href="https://wiki.openstack.org/wiki/Training-manuals"
                        >working to produce
            (https://wiki.openstack.org/wiki/Training-manuals)</link>open source training materials.</para>
                </section>
                <section xml:id="preface_roll_your_own_openstack"><title>Roll your Own OpenStack</title>
                    <para>However, this guide has a different audience &mdash; those seeking to derive the most
            flexibility from the OpenStack framework; conducting 'Do-It-Yourself' solutions, if you
            will.</para>
                <para>OpenStack is designed for scalability, so you can easily add new
        compute, network and storage resources to grow your cloud over time. In addition to several
        massive OpenStack public clouds, a considerable number of other organisations (such as
        Paypal, Intel and Comcast) have built large-scale private clouds. OpenStack offers much
        more than a typical software package because it lets you integrate a number of different
        technologies to construct a cloud. This approach provides great flexibility, but the
        number of options might be bewildering at first.</para>
                </section>
        </section>
            </section>
    <section xml:id="who-this-book-is-for">
        <title>Who This Book Is For</title>
        <para>This book is for those of you starting to run OpenStack clouds as
            well as those of you who were handed an operational one and want to keep
            it running well. Perhaps you're on a devops team, perhaps you are a
            system administrator starting to dabble in the cloud, or maybe you
            want to get on that OpenStack cloud team at your company. This book
            is for all of you.</para>
        <para>This guide assumes that you are familiar with a Linux
            distribution supporting OpenStack, SQL databases, and virtualization.
             You must be comfortable administering and configuring multiple Linux machines
            for networking. You must install and maintain a MySQL database, and
            occasionally run SQL queries against it.</para>
        <para>One of the most complex aspects of an OpenStack cloud is the
            networking configuration. You should be familiar with concepts such
            as DHCP, Linux bridges, VLANs, and iptables. You must also have
            access to a network hardware expert who can configure the switches
            and routers required in your OpenStack cloud.</para>
        <tip><para>As cloud computing is a quite advanced topic, this book
            requires a lot of background knowledge. However, if you are fairly
            new to cloud computing, we recommend that you make use of the
            <xref linkend="openstack_glossary"/> at the back of
            the book, as well as refer to the online documentation for OpenStack,
            and additional resources mentioned
            in this book in the <xref linkend="recommended-reading"/> list.</para></tip>
        <section xml:id="further_reading"><title>Further Reading</title>
        <para>There are other books on the OpenStack documentation web site at
                <link xlink:href="http://docs.openstack.org"
                >docs.openstack.org</link> that can help you get the job
            done.</para>
        <variablelist>
            <title>OpenStack Guides</title>
            <varlistentry>
                <term>OpenStack Installation Guides</term>
                <listitem>
                    <para>Describe a manual install process, as in, by hand, no
                        automation, for multiple distributions based on packaging
                        system:</para>
                    <itemizedlist>
                        <listitem>
                            <para><link
                                    xlink:href="http://docs.openstack.org/havana/install-guide/install/apt-debian/content/"
                                    >Installation Guide for Debian 7.0</link>
                            </para>
                        </listitem>
                        <listitem>
                            <para><link
                                    xlink:href="http://docs.openstack.org/havana/install-guide/install/zypper/content/"
                                    >Installation Guide for openSUSE and SUSE
                                    Linux Enterprise Server</link>
                            </para>
                        </listitem>
                        <listitem>
                            <para><link
                                    xlink:href="http://docs.openstack.org/havana/install-guide/install/yum/content/"
                                    >Installation Guide for Red Hat Enterprise
                                    Linux, CentOS, and Fedora</link></para>
                        </listitem>
                        <listitem>
                            <para><link
                                    xlink:href="http://docs.openstack.org/havana/install-guide/install/apt/content/"
                                    >Installation Guide for Ubuntu 12.04 (LTS)
                                    Server</link></para>
                        </listitem>
                    </itemizedlist>
                </listitem>
            </varlistentry>
            <varlistentry>
                <term><link
                        xlink:href="http://docs.openstack.org/havana/config-reference/content/"
                        >OpenStack Configuration Reference</link>
                </term>
                <listitem>
                    <para>Contains a reference listing of all configuration
                        options for core and integrated OpenStack services by
                        release version.</para>
                </listitem>
            </varlistentry>
            <varlistentry>
                <term><link
                        xlink:href="http://docs.openstack.org/admin-guide-cloud/content/"
                        >OpenStack Cloud Administrator Guide</link></term>
                <listitem>
                    <para>Contains how-to information for managing an OpenStack
                        cloud as needed for your use cases, such as storage,
                        computing, or software-defined-networking.</para>
                </listitem>
            </varlistentry>
            <varlistentry>
                <term><link
                        xlink:href="http://docs.openstack.org/high-availability-guide/content/index.html"
                        >OpenStack High Availability Guide</link>
                </term>
                <listitem>
                    <para>Describes potential strategies for making your
                        OpenStack services and related controllers and data
                        stores highly available.</para>
                </listitem>
            </varlistentry>
            <varlistentry>
                <term><link xlink:href="http://docs.openstack.org/sec/"
                        >OpenStack Security Guide</link></term>
                <listitem>
                    <para>Provide best practices and conceptual information
                        about securing an OpenStack cloud.</para>
                </listitem>
            </varlistentry>
            <varlistentry>
                <term><link xlink:href="http://docs.openstack.org/image-guide/content/">Virtual Machine Image Guide</link></term>
                <listitem>
                    <para>Shows you how to obtain, create, and modify virtual
                        machine images that are compatible with
                        OpenStack.</para>
                </listitem>
            </varlistentry>
            <varlistentry>
                <term><link xlink:href="http://docs.openstack.org/user-guide/content/">OpenStack End User Guide</link></term>
                <listitem>
                    <para>Shows OpenStack end users how to create and manage
                        resources in an OpenStack cloud with the OpenStack
                        dashboard and OpenStack client commands.</para>
                </listitem>
            </varlistentry>
            <varlistentry>
                <term><link xlink:href="http://docs.openstack.org/user-guide-admin/content/">OpenStack Admin User Guide</link></term>
                <listitem>
                    <para>Shows OpenStack administrators how to create and
                        manage resources in an OpenStack cloud with the
                        OpenStack dashboard and OpenStack client
                        commands.</para>
                </listitem>
            </varlistentry>
            <varlistentry>
                <term><link xlink:href="http://docs.openstack.org/api/quick-start/content/">OpenStack API Quick Start</link></term>
                <listitem>
                    <para>A brief overview of how to send REST API requests to
                        endpoints for OpenStack services.</para>
                </listitem>
            </varlistentry>
        </variablelist>
        </section>
    </section>
    <section xml:id="how-this-book-is-organized">
        <title>How This Book Is Organized</title>
        <para>This book is organized in two parts, the architecture decisions for designing
            OpenStack clouds and the repeated operations for running OpenStack clouds.</para>
        <para>Part I</para>
        <para><xref linkend="example_architecture"/>: Because of all the decisions the other
            chapters discuss, this chapter describes the decisions made for this particular book and
            much of the justification for the example architecture.</para>
        <para><xref linkend="section_arch_provision"/>: While this book doesn't describe
            installation, we do recommend automation for deployment and configuration, discussed in
            this chapter.</para>
        <para><xref linkend="cloud_controller_design"/>: The cloud controller is an invention for
            the sake of consolidating and describing which services run on which nodes. The chapter
            discusses hardware and network considerations as well as how to design the cloud
            controller for performance and separation of services.</para>
        <para><xref linkend="compute_nodes"/>: This chapter describes the compute nodes, which are
            dedicated to run virtual machines. Some hardware choices come into play here as well as
            logging and networking descriptions.</para>
        <para><xref linkend="scaling"/>: This chapter discusses the growth of your cloud resources
            through scaling and segregation considerations.</para>
        <para><xref linkend="storage_decision"/>: Along with other architecture decisions, storage
            concepts within OpenStack take a lot of consideration, and this chapter lays out the
            choices for you.</para>
        <para><xref linkend="network_design"/>: Your OpenStack cloud networking needs to fit into
            your existing networks while also enabling the best design for your users and
            administrators, and this chapter gives you in-depth information about networking
            decisions.</para>
        <para>Part II</para>
        <para><xref linkend="lay_of_the_land"/>: This chapter is written to let you get your hands
            wrapped around your OpenStack cloud through command line tools and understanding what is
            already set up in your cloud.</para>
        <para><xref linkend="projects_users"/>: This chapter walks through those user-enabling
            processes that all admins must face to manage users, give them quotas to parcel out
            resources, and so on.</para>
        <para><xref linkend="user_facing_operations"/>: This chapter moves along to show you how to
            use OpenStack cloud resources and train your users as well.</para>
        <para><xref linkend="maintenance"/>: This chapter goes into the common failures the authors
            have seen while running clouds in production, including troubleshooting.</para>
        <para><xref linkend="network_troubleshooting"/>: Because network troubleshooting is
            especially difficult with virtual resources, this chapter is chock-full of helpful tips
            and tricks to tracing network traffic, finding the root cause of networking failures,
            and debugging related services like DHCP and DNS.</para>
        <para><xref linkend="logging_monitoring"/>: This chapter shows you where OpenStack places
            logs and how to best to read and manage logs for monitoring purposes.</para>
        <para><xref linkend="backup_and_recovery"/>: This chapter describes what you need to back up
            within OpenStack as well as best practices for recovering backups.</para>
        <para><xref linkend="customize"/>: When you need to get a specialized feature into
            OpenStack, this chapter describes how to use DevStack to write custom middleware or a
            custom scheduler to rebalance your resources.</para>
        <para><xref linkend="upstream_openstack"/>: Because OpenStack is so, well, open, this
            chapter is dedicated to helping you navigate the community and find out where you can
            help and where you can get help.</para>
        <para><xref linkend="advanced_configuration"/>: Much of OpenStack is driver-oriented, where
            you can plug in different solutions to the base set of services. This chapter describes
            some advanced configuration topics.</para>
        <para><xref linkend="ch_ops_upgrades"/>: This chapter provides upgrade information based on
            the architectures in this book.</para>
        <para><xref linkend="use-cases"/>: You can read a small selection of use cases from the
            OpenStack community with some technical detail and further resources.</para>
        <para><xref linkend="app_crypt"/>: These are shared legendary tales of image disappearances,
            VM massacres, and crazy troubleshooting techniques to share those hard-learned lessons
            and wisdom.</para>
        <para><xref linkend="working-with-roadmaps"/>: Read about how to track the OpenStack
        roadmap through the open and transparent development processes.</para>
        <para><xref linkend="recommended-reading"/>: So many OpenStack resources are available
            online due to the fast-moving nature of the project, but there are also listed resources
            the authors found helpful while learning themselves.</para>
        <para><xref linkend="openstack_glossary"/>: A list of terms used in this book is included, which is a subset of the
            larger OpenStack Glossary available online.</para>
    </section>
    <section xml:id="why-and-how-we-wrote-this-book">
        <title>Why and How We Wrote This Book</title>
        <para>We wrote this book because we have deployed and maintained
            OpenStack clouds for at least a year, and wanted to be able to
            distribute this knowledge to others. After months of being the point
            people for an OpenStack cloud, we also wanted to have a document to
            hand to our system administrators so they'd know how to operate the
            cloud on a daily basis — both reactively and proactively. We wanted
            to provide more detailed technical information about the decisions
            that deployers make along the way.</para>
        <?hard-pagebreak?>
        <para>We wrote this book to help you:<itemizedlist role="compact">
                <listitem>
                    <para>Design and create an architecture for your first
                        non-trivial OpenStack cloud. After you read this guide,
                        you'll know which questions to ask and how to organize
                        your compute, networking, storage resources, and the
                        associated software packages.</para>
                </listitem>
                <listitem>
                    <para>Perform the day-to-day tasks required to administer a
                        cloud.</para>
                </listitem>
            </itemizedlist></para>
        <para>We wrote this book in a Book Sprint, which is a facilitated rapid
            development production method for books. For more information see
            the <link xlink:href="http://www.booksprints.net">Book Sprint
                site</link>. Your authors cobbled this book together in five
            days during February 2013, fueled by caffeine and the best take-out
            food that Austin, Texas could offer.</para>
        <para>On the first day we filled white boards with colorful sticky notes
            to start to shape this nebulous book about how to architect and
            operate clouds. <informalfigure>
                <mediaobject>
                    <imageobject>
                        <imagedata width="5in" fileref="figures/1-IMG_4895.JPG"
                        />
                    </imageobject>
                </mediaobject>
            </informalfigure></para>
        <para>We wrote furiously from our own experiences and bounced ideas
            between each other. At regular intervals we reviewed the shape and
            organization of the book and further molded it, leading to what you
            see today.</para>
        <?hard-pagebreak?>
        <para>The team includes:</para>
        <itemizedlist role="compact">
            <listitem>
                <para><emphasis role="bold">Tom Fifield</emphasis>. After
                    learning about scalability in computing from particle
                    physics experiments like ATLAS at the LHC, Tom worked on
                    OpenStack clouds in production to support the Australian
                    public research sector. Tom currently serves as an OpenStack
                    community manager and works on OpenStack documentation in
                    his spare time.</para>
            </listitem>
            <listitem>
                <para><emphasis role="bold">Diane Fleming</emphasis>. Diane
                    works on the OpenStack API documentation tirelessly. She
                    helped out wherever she could on this project.</para>
            </listitem>
            <listitem>
                <para><emphasis role="bold">Anne Gentle</emphasis>. Anne is the
                    documentation coordinator for OpenStack and also served as
                    an individual contributor to the Google Doc Summit in 2011,
                    working with the Open Street Maps team. Anne has worked on
                    doc sprints in the past with FLOSS Manuals’ Adam Hyde
                    facilitating. Anne lives in Austin, Texas.</para>
            </listitem>
            <listitem>
                <para><emphasis role="bold">Lorin Hochstein</emphasis>. An
                    academic turned software developer-slash-operator, Lorin
                    currently works as the Lead Architect for Cloud Services at
                    Nimbis Services where he deploys OpenStack for technical
                    computing applications. He has been working with OpenStack
                    since the Cactus release. Previously, he worked on
                    high-performance computing extensions for OpenStack at
                    University of Southern California's Information Sciences
                    Institute (USC-ISI).</para>
            </listitem>
            <listitem>
                <para><emphasis role="bold">Adam Hyde</emphasis>. Adam
                    facilitated this Book Sprint. He also founded the Book
                    Sprint methodology and is the most experienced Book Sprint
                    facilitator around. See
                        <uri>http://www.booksprints.net/</uri> for more
                    information. Adam founded FLOSS Manuals—a community of some
                    3,000 individuals developing Free Manuals about Free
                    Software. He is also the founder and project manager for
                    Booktype, an open source project for writing, editing, and
                    publishing books online and in print.</para>
            </listitem>
            <listitem>
                <para><emphasis role="bold">Jonathan Proulx</emphasis>. Jon has
                    been piloting an OpenStack cloud as a senior technical
                    architect at the MIT Computer Science and Artificial
                    Intelligence Lab for his researchers to have as much
                    computing power as they need. He started contributing to
                    OpenStack documentation and reviewing the documentation so
                    that he could accelerate his learning.</para>
            </listitem>
            <listitem>
                <para><emphasis role="bold">Everett Toews</emphasis>. Everett is
                    a Developer Advocate at Rackspace making OpenStack and the
                    Rackspace Cloud easy to use. Sometimes developer, sometimes
                    advocate, and sometimes operator. He's built web
                    applications, taught workshops, given presentations around
                    the world, and deployed OpenStack for production use by
                    academia and business.</para>
            </listitem>
            <listitem>
                <para><emphasis role="bold">Joe Topjian</emphasis>. Joe has
                    designed and deployed several clouds at Cybera, where, as a
                    non-profit, they are building e-infrastructure to support
                    entrepreneurs and local researchers in Alberta, Canada. He
                    also actively maintains and operates these clouds as a
                    systems architect, and his experiences have generated a
                    wealth of troubleshooting skills for cloud
                    environments.</para>
            </listitem>
        </itemizedlist>
    </section>

    <section xml:id="how-to-contribute-to-ops-guide">
        <title>How to Contribute to This Book</title>
        <para>The genesis of this book was an in-person event, but now that the
            book is in your hands we want you to contribute to it. OpenStack
            documentation follows the coding principles of iterative work, with
            bug logging, investigating, and fixing. We also store the source
            content on Github and invite collaborators through the OpenStack
            Gerrit installation, which offers reviews. For the O'Reilly edition
            of this book, we are using their Atlas system which also stores
            source content on Github and enables collaboration among
            contributors.</para>
        <para>Learn more about how to contribute to the OpenStack docs
            at <link
                xlink:href="http://wiki.openstack.org/Documentation/HowTo"
                >Documentation How To</link>
            (http://wiki.openstack.org/Documentation/HowTo).</para>
        <para>If you find a bug and can't fix it or aren't sure it's
            really a doc bug, log a bug at <link
                xlink:href="http://bugs.launchpad.net/openstack-manuals"
                >OpenStack Manuals</link>
            (http://bugs.launchpad.net/openstack-manuals). Tag the bug
            under <guilabel>Extra</guilabel> options with
                <literal>ops-guide</literal> tag to indicate that the
            bug is in this guide. You can assign the bug to yourself
            if you know how to fix it. Also, a member of the OpenStack
            doc-core team can triage the doc bug.</para>
    </section>

</preface>