[OpenStack Beginner’s Guide for Ubuntu 11.04] – Installation and Configuration

Installation and configuration

The following section describes how to set up a minimal cloud infrastructure based on OpenStack using 3 machines. These machines are referred to in this and subsequent chapters as Server1 and Server2 and Client1. Server1 runs all the 7 components of Nova as well as Glance and OpenStack dashboard. Server2 runs only nova-compute. Since OpenStack components follow a shared-nothing policy, each component or any group of components can be installed any server.

Client1 is not a required component. In our sample setup, it is used for bundling images and for using as a client for web interface and to run euca commands for managing the infrastructure. Having this client ensures that you do not need to meddle with the servers for tasks such as bundling. Also, bundling of Desktop Systems including Windows will require GUI and it is better to have a dedicated machine for this purpose. We would recommend this machine to be VT-Enabled so that KVM can be run and Windows VMs can be run during image creation for bundling.

The installation steps use certain specifics such as hostnames/IP addresses etc. Modify them to suit your environment before using them. The following table summarizes these specifics.

	Server 1	Server 2	Client 1
Functionality	All components of Open Stack including nova-compute	nova-compute	Client
No of NICs	eth0 -Public N/W,eth1 -Private N/W	eth0 -Public N/W,eth1 -Private N/W	eth0- Public N/W
Ipaddresses	Eth0-10.10.10.2,eth1-192.168.3.1	Eth0-10.10.10.3,eth1-192.168.3.2	Eth0-10.10.10.4
Hostname	server1.example.com	server2.example.com	client.example.com
DNS servers	10.10.10.3	10.10.10.3	10.10.10.3
Gateway IP	10.10.10.1	10.10.10.1	10.10.10.1

Server1

Base OS

Boot the server off the Ubuntu server 11.04 CD. At the graphical menu select Install Ubuntu server and proceed with basic installation steps.
We will also be running nova-volume on this server and it is ideal to have a dedicated partition for the use of nova-volume. So, ensure that you choose manual partitioning scheme while installing Ubuntu Server and create a dedicated partition with adequate amount of space for this purpose. We have referred to this partition in the rest of the chapter as /dev/sda6. You can substitute the correct device name of this dedciated partition based on your local setup while following the instructions. Also ensure that the parition type is set as Linux LVM ( 8e ) using fdisk either during install or immediately after installation is over.

• Create the first user with the name ‘localadmin’ .
• Installation lets you setup the IP address for the first interface i.e. eth0. Set the IP address details.
• During installation select only Openssh-server in the packages menu.

Nova and Glance have been included in Universe repository . Enable Universe repository in your /etc/apt/sources.list.

Update the machine using the following commands.

sudo apt-get update

sudo apt-get upgrade

Install bridge-utils:

sudo apt-get install bridge-utils

Reboot the server and login as the admin user(localadmin) created during the OS installation.

Networking Configuration

Edit the /etc/network/interfaces file so as to looks like this:

auto lo
iface lo inet loopback
auto eth0
iface eth0 inet static
address 10.10.10.2
netmask 255.255.255.0
broadcast 10.10.10.255
gateway 10.10.10.1
dns-nameservers 10.10.10.100

auto br100
iface br100 inet static
bridge_ports eth1
bridge_stp off
bridge_maxwait 0
bridge_fd 0
address 192.168.3.1
netmask 255.255.0.0
broadcast 192.168.255.255

Restart the network now

sudo /etc/init.d/networking restart

NTP Server

Install NTP package. This server is going to act as an NTP server for the nodes. The time on all components of OpenStack will have to be in sync. We can run NTP server on this and have other components sync to it.

sudo apt-get install ntp

Open the file /etc/ntp.conf and add the following 2 lines to make sure that the server serves time even when its connectivity to the Internet is down. The following settings ensure that the NTP server uses its own clock as the clock source:

server 127.127.1.0
fudge 127.127.1.0 stratum 10

Restart NTP service to make the changes effective

sudo /etc/init.d/ntp restart

Glance

Glance is an image Server that Nova can use to pickup images from. Glance is very modular and can use several types of storage backends such as filestore, s3 etc. We are installing Glance before installing Nova, so that when we get to configuring Nova, glance is ready to be used by Nova.

sudo apt-get install glance

The default config file at /etc/glance/glance.conf is good to use for a simple file store as the storage backend. Glance can be configured to use other storage backends such as swift. This will be covered in more detail in the chapter on “Image Management”.

Glance uses sqlite as the default database backend. While sqlite offers a quick and easy way to get started, for production use, you may consider a database such as MySQL or Postgresql. This will be covered in more detail in the chapter on “Image Management”

Glance has two components – glance-api and glance-registry. These can be controlled using the concerned upstart jobs.

MySQL Server

Install mysql-server package

sudo apt-get install -y mysql-server

Configuration

Set a variable called “MYSQL_PASS” for use in the various commands below:

MYSQL_PASS="mygreatsecret"

Change the bind address from 127.0.0.1 to 0.0.0.0 in /etc/mysql/my.cnf and it will look like this:

bind-address = 0.0.0.0

Restart mysql server to ensure that it starts listening on all interfaces.

sudo restart mysql

If you did not set the mysql root password during installation, set it now.

mysqladmin -u root password $MYSQL_PASS

Create a database named nova.

sudo mysql -uroot -p$MYSQL_PASS -e 'CREATE DATABASE nova;'

Update the database to grant super user privileges for root user to login from any IP.

sudo mysql -uroot -p$MYSQL_PASS -e "GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' WITH GRANT OPTION;"

Set mysql root password for login from any IP.

sudo mysql -uroot -p$MYSQL_PASS -e "SET PASSWORD FOR 'root'@'%' = PASSWORD('$MYSQL_PASS');"

Nova Components

Install the messaging queue server, RabbitMQ and various nova components.

sudo apt-get install -y rabbitmq-server nova-common nova-doc python-nova nova-api nova-network nova-volume nova-objectstore nova-scheduler nova-compute

Install euca2ools package for command line tools to interact with nova.

sudo apt-get install -y euca2ools

Install unzip for extracting archives.

sudo apt-get install -y unzip

Edit the /etc/nova/nova.conf file to look like this.

--dhcpbridge_flagfile=/etc/nova/nova.conf
--dhcpbridge=/usr/bin/nova-dhcpbridge
--logdir=/var/log/nova
--lock_path=/var/lock/nova
--state_path=/var/lib/nova
--verbose
--s3_host=10.10.10.2
--rabbit_host=192.168.3.1
--cc_host=192.168.3.1
--ec2_url=http://10.10.10.2:8773/services/Cloud
--fixed_range=192.168.0.0/16
--network_size=8
--FAKE_subdomain=ec2
--routing_source_ip=192.168.3.1
--sql_connection=mysql://root:mygreatsecret@10.10.10.2/nova
--glance_host=192.168.3.1
--image_service=nova.image.glance.GlanceImageService
--iscsi_ip_prefix=192.168.

Enable iscsi target.

sudo sed -i 's/false/true/g' /etc/default/iscsitarget

Restart the iscsitarget service.

sudo service iscsitarget restart

Create a Physical Volume.

sudo pvcreate /dev/sda6

Create a Volume Group named.

sudo vgcreate nova-volumes /dev/sda6

Create a group called “nova”.

sudo groupadd nova

Change the ownership of the /etc/nova folder and permissions for /etc/nova/nova.conf:

sudo chown -R root:nova /etc/nova
sudo chmod 644 /etc/nova/nova.conf

Restart all the nova related services.

sudo /etc/init.d/libvirt-bin restart; sudo restart nova-network; sudo restart nova-compute; sudo restart nova-api; sudo restart nova-objectstore; sudo restart nova-scheduler; sudo restart nova-volume; sudo restart glance-api; sudo restart glance-registry

Create nova schema in the Mysql Database.

sudo nova-manage db sync

Create a list of Ips to be used from the network of fixed Ips set inside nova.conf.

sudo nova-manage network create 192.168.3.0/24 1 255

Allocate 32 Pubic IP addresses for use with the instances starting from 10.10.10.225.

sudo nova-manage floating create 10.10.10.2 10.10.10.224/27

Create a user with admin rights on nova.

sudo nova-manage user admin novaadmin

Create a project named proj.

sudo nova-manage project create proj novaadmin

Create a directory to download nova credentials and download the zip file.

mkdir /home/localadmin/creds

Generate and save credentials for accessing/managing the nova cloud.

sudo nova-manage project zipfile proj novaadmin /home/localadmin/creds/novacreds.zip

Contents of novacreds.zip are required to use euca2ools to manage the cloud infrastructure and you will need to transfer this zip file to any machine from where you want to run the commands from euca2ools. We will be using these credentials from client1 as well.

Navigate in to the folder created and extract the files and change their ownership.

cd /home/localadmin/creds
unzip novacreds.zip
sudo chown localadmin:localadmin /home/localadmin/creds/ -R

Here are the files extracted:
cacert.pem, cert.pem, novarc, pk.pem
novarc contains several environmental variables including your nova credentials to be set before you can use the commands from euca2ools such euca-describe-images, euca-describe-instances etc. these variables can be set by sourcing novarc file.

source /home/localadmin/creds/novarc

Restart all the nova related services.

sudo /etc/init.d/libvirt-bin restart; sudo restart nova-network; sudo restart nova-compute; sudo restart nova-api; sudo restart nova-objectstore; sudo restart nova-scheduler; sudo restart nova-volume; sudo restart glance-api; sudo restart glance-registry

Check if the credentials are working and if nova has been setup properly by running:

euca-describe-availability-zones verbose

If you see something like the following with all components happy, it means that the set up is ready to be used.

AVAILABILITYZONE nova available
AVAILABILITYZONE |- server1
AVAILABILITYZONE | |- nova-compute enabled : -) 2011-04-03 07:48:50
AVAILABILITYZONE | |- nova-scheduler enabled : -) 2011-04-03 07:48:48
AVAILABILITYZONE | |- nova-network enabled : -) 2011-04-03 07:48:49
AVAILABILITYZONE | |- nova-volume enabled : -) 2011-04-03 07:48:49

Nova Dashboard

OpenStack-dashboard is a web interface for managing users, user credentials, key pairs, images, instances etc.
Install bazaar version control system to fetch required software from the repository at launchpad.

sudo apt-get install -y bzr
sudo easy_install virtualenv

You have already finished setting up credentials for a user called localadmin in the Nova configuration section above. The credentials of this user will need to embedded into the dashboard’s configuration file.

Checkout the source of OpenStack-dashboard from bzr and run run_tests.sh, which does not only test the installation, but also installs several dependencies of the dashboard.

sudo bzr init-repo .
sudo bzr branch lp:openstack-dashboard -r 46 /opt/osdb
cd /opt/osdb
sudo sh run_tests.sh
cd openstack-dashboard

Note:It has been tested with version 46 of OpenStack Dashboard

Since you are trying to checkout from bzr anonymously, a message saying “You have not informed bzr of your Launchpad ID…” is displayed. You can safely ignore that.

Edit /opt/osdb/openstack-dashboard/local/local_settings.py to include certain details required for connecting to nova-api.

NOVA_DEFAULT_ENDPOINT = 'http://localhost:8773/services/Cloud'
NOVA_DEFAULT_REGION = 'nova'
NOVA_ACCESS_KEY = 'b6a7e3ca-f894-473b-abca-84329d9829fa:proj'
NOVA_SECRET_KEY = '2d61a361-965a-4ed6-966a-d9f543b42531'
NOVA_ADMIN_USER = 'novaadmin'
NOVA_PROJECT = 'proj'

A simple way of doing this will be to copy the relevant lines from novarc file that we discussed above.

Setting Up E-mail service for the web interface

In order to to have mails generated by OpenStack dashboard delivered, we need to configure dashboard with the details of an smtp server by editing local_settings.py file.

EMAIL_HOST = 'server1.example.com'
EMAIL_PORT = 25
If the mail server provides only authenticated SMTP, add the following lines:
EMAIL_USER =
EMAIL_PASSWORD =

If the mail server requires a TLS connection, add the following lines:

EMAIL_USE_TLS = 'True'

Create a openstack-dashboard database and its schema with the syncdb command. Provide the name/email address/desired password of the administrative user when prompted.

sudo tools/with_venv.sh dashboard/manage.py syncdb

While creating the schema, the above command asks you to create an admin account for the dashboard. Choose the user name as the project admin’s user name you chose above while creating the project ( novadmin in our case). You can choose any password you like.

Launch the default python-django server. If you want the dashboard application to be available on port 8000 :

sudo tools/with_venv.sh dashboard/manage.py runserver 10.10.10.2:8000

To check the installation open a browser and enter the following URL

http://10.10.10.2:8000

You should be able to login as “novaadmin” using the password chosen above. Any other user trying to access the interface for the first time, will need to sign up and will be able to use the interface after the account is approved by the administrator.

A successful login and display of the project named “proj” on the dashboard will indicate that the dashboard has been setup successfully.

OpenStack Dashboard with Mysql Database

Dashboard uses sqllite database by default. For a production use, MySQL or PostgreSQL may be more preferable. The procedure for MySQL is given below. Procedure for PostgreSQL will be very similar.

Install python-dev and libhmysqlclient-dev

sudo apt-get install libmysqlclient-dev
sudo apt-get install python-dev

Activate virtualenv and install mysql-python package inside the virtual environment of Dashboard.

cd /opt/osdb/openstack-dashboard
sudo bash
source .dashboard-venv/bin/activate
easy_install mysql-python

Create a Mysql database user with all privileges on Openstack Dashboard database

mysql -uroot -p <rootpasswd>

>create database dashboarddb;
>grant ALL on dashboarddb.* to nova@localhost identified by 'mygreatsecret';

Update the DATABASES section of the Django’s local_settings.py file (/opt/osdb/openstack-dashboard/local/local_settings.py) with the MySQL database settings. Here is the relevant extract from the updated file:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'dashboarddb',
        'USER': 'nova',
        'PASSWORD': 'mygreatsecret',
        'HOST': 'localhost',
        'default-character-set': 'utf8',
    }
}

Create the schema in the database

sudo tools/with_venv.sh dashboard/manage.py syncdb

Running Dashboard on apache2 with mod_wsgi

While the webserver that is included in Django is good for testing, for production use, it is recommended to use a web server like Apache with mod_wsgi.

Install apache2 and wsgi module.

sudo apt-get install apache2 libapache2-mod-wsgi

Dashboard includes a file django.wsgi(/opt/osdb/openstack-dashboard/dashboard/wsgi/django.wsgi) to help in running dashboard under Apache with WSGI. You can replace the default file with the file below.

# Ref: http://jmoiron.net/blog/deploying-django-mod-wsgi-virtualenv/

import sys
import site
import os

#we are adding virtual enviornment path.
vepath = '/opt/osdb/openstack-dashboard/.dashboard-venv/lib/python2.7/site-packages'
os.environ['PYTHON_EGG_CACHE'] = '/opt/osdb/openstack-dashboard/.dashboard-venv/lib/python2.7/site-packages'

prev_sys_path = list(sys.path)

# add the site-packages of our virtualenv as a site dir
site.addsitedir(vepath)

# reorder sys.path so new directories from the addsitedir show up first

new_sys_path = [p for p in sys.path if p not in prev_sys_path]

for item in new_sys_path:
    sys.path.remove(item)
sys.path[:0] = new_sys_path

# import from down here to pull in possible virtualenv django install

from django.core.handlers.wsgi import WSGIHandler
os.environ['DJANGO_SETTINGS_MODULE'] = 'dashboard.settings'
application = WSGIHandler()

Setting up the virtual host and WSGI alias in Apache

Create /etc/apache2/sites-available/openstack with the following contents:

Listen 8000
<VirtualHost 10.10.10.2:8000>
    ServerName 10.10.10.2
    WSGIScriptAlias / /opt/osdb/openstack-dashboard/dashboard/wsgi/django.wsgi
    Alias /media/admin/  /opt/osdb/openstack-dashboard/.dashboard-venv/lib/python2.7/site-packages/django/contrib/admin/media/
</VirtualHost>

Enable virtual host.

sudo a2ensite openstack
sudo /etc/init.d/apache2 reload

Dashboard should now be available at http://10.10.10.2:8000

Server 2

BaseOS

Install 64-bit version of Natty Server

Networking Configuration

sudo apt-get install bridge-utils

Edit the /etc/network/interfaces file so as to looks like this:

auto lo
 iface lo inet loopback
 auto eth0
 iface eth0 inet static
 address 10.10.10.3
 netmask 255.255.255.0
 broadcast 10.10.10.255
 gateway 10.10.10.1
 dns-nameservers 10.10.10.100

auto br100
 iface br100 inet static
 bridge_ports eth1
 bridge_stp off
 bridge_maxwait 0
 bridge_fd 0
 address 192.168.3.2
 netmask 255.255.0.0
 broadcast 192.168.255.255

Restart the network now

sudo /etc/init.d/networking restart

NTP Client

Install NTP package. This will sync to server1

sudo apt-get install ntp

Open the file /etc/ntp.conf and add the following line to sync to server1.

server 10.10.10.2

Restart NTP service to make the changes effective

sudo /etc/init.d/ntp restart

Nova Components (nova-compute alone)

Install the nova-components and dependencies.

sudo apt-get install -y nova-common python-nova nova-compute vlan

Install euca tools, for command line tools

sudo apt-get install -y euca2ools

Install unzip for extracting archives

sudo apt-get install -y unzip

Edit the /etc/nova/nova.conf file to look like this. This file is essentially similar to the configuration file (/etc/nova/nova.conf) of Server1

 --dhcpbridge_flagfile=/etc/nova/nova.conf
 --dhcpbridge=/usr/bin/nova-dhcpbridge
 --logdir=/var/log/nova
 --lock_path=/var/lock/nova
 --state_path=/var/lib/nova
 --verbose
 --s3_host=10.10.10.2
 --rabbit_host=192.168.3.1
 --cc_host=192.168.3.1
 --ec2_url=http://10.10.10.2:8773/services/Cloud
 --fixed_range=192.168.0.0/16
 --network_size=8
 --FAKE_subdomain=ec2
 --routing_source_ip=192.168.3.2
 --sql_connection=mysql://root:nova@10.10.10.2/nova
 --glance_host=192.168.3.1
 --image_service=nova.image.glance.GlanceImageService

Client1

BaseOS

Install 64-bit Desktop Version of Natty

Networking Configuration

Edit the /etc/network/interfaces file so as to looks like this:

auto lo
 iface lo inet loopback
auto eth0
 iface eth0 inet static
 address 10.10.10.4
 netmask 255.255.255.0
 broadcast 10.10.10.255
 gateway 10.10.10.1
 dns-nameservers 10.10.10.100

NTP Client

Install NTP package. This will sync to server1

sudo apt-get install ntp

Open the file /etc/ntp.conf and add the following line to sync to server1.

server 10.10.10.2

Restart NTP service to make the changes effective

sudo /etc/init.d/ntp restart

Client Tools

As mentioned above, this is a desktop installation of Natty to be used for tasks such as bundling of images. It will also be used for managing the cloud infrastructure using euca2ools.
Install euca tools, for command line tools

sudo apt-get install -y euca2ools

Install qemu-kvm

sudo apt-get install qemu-kvm

Download the credentials we created for localadmin to this machine:

mkdir /home/localadmin/creds
cd /home/localadmin/creds
scp localadmin@10.10.10.2:/home/localadmin/creds/novacreds.zip .
unzip creds.zip

Source novarc file and see if connectivity to api server is working correctly:

source novarc
euca-describe-availability-zones verbose

The output should be similar to what is shown above in the configuration section for server1.

Note: If you want to avoid manually sourcing the novarc file everytime, the user can add the following line to the .profile file in his home directory:

 source /home/localadmin/creds/novarc