[Eucalyptus Beginner’s Guide – UEC edition] Chapter 4 – Image Management

Chapter 3 Web Interface

Chapter 5 Instance Management

Eucalyptus Machine Image(EMI)

A Eucalpyptus Machine Image(EMI) is a combination of a virtual disk image(s), kernel and ramdisk images as well as an xml file containing meta data about the image. These images reside on WS3 and used as templates for creating instances on UEC. Each Linux EMI is a combination of the following:

• An XML file with a name like “karmic.img.manifest.xml” with information about one or more hard disk images, a kernel image and a ram disk image (id – emi-65440E7E)
• An XML file with a name like “vmlinuz-2.6.28-11-server.manifest.xml” with information about the corresponding kernel image(id – eki-39FC1244)
• An XML file with a name like “initrd.img-2.6.28-11-server.manifest.xml” with information about the corresponding ramdisk image(id – eri-71ED1322)

Each of these images has its own ID that can be used while running the instances. More on this in the chapter on “Managing Instances”

From the web interface of Eucalyptus, you can view a list of EMIs in the “Store” tab of the web UI. These are the EMIs listed from Canonical or partners. You can choose to download and install any of these images directly from Canonical’s site.

Since most enterprise/individual users of Eucalyptus have a need for bringing up instances based on custom images, image management plays a key role in Eucalyptus administration. Such images could be based on a preferred version/variant of a preferred OS distribution with a set of required applications pre-installed.

Bundling an EMI is a multi-step process involving the following:

1. creating a virtual disk image
2. installing the OS
3. installing required applications
4. making the OS ready to run under UEC
5. registering the images with UEC
6. testing the image etc.

Bundling process differs between Linux and Windows images and we will discuss the process in both cases in some detail below.

Bundling Linux Image

We will use Client1 for the purpose of working on creation of the image. Please note that we had installed KVM on Client1.

Creating a new disk Image

This will represent the main HDD of the virtual machine, so make sure to give it as much space as you will need. Since we are building a kvm image, we can use a qcow2 format for disk images. Qcow2 is an expandable image format, so it will only occupy as much storage space as is actually filled by the data within the image.

uecadmin@client1:~$ kvm-img create -f qcow2 image.img 5G

OS Installation

Download the iso file of the Linux distribution you want installed in the image. The example below refers to creating a Karmic 64-bit server image.

uecadmin@client1:~$ wget http://releases.ubuntu.com/karmic/ubuntu-9.10-server-amd64.iso

And start the installation process:

uecadmin@client1:~$ sudo kvm -m 256 -cdrom ubuntu-9.10-server-amd64.iso -drive file=image.img,if=scsi,index=0 -boot d \
-net nic -net user -nographic -vnc :0

If your installation process requires more than 256MB of RAM change the -m option, and if you need more processors available, you can use the ‘-c’ option.
The command above will boot a new KVM instance, with the disk image you’ve created as the primary HDD and the iso as the first bootable device. Also the ‘-nographic’ option will not display any graphical output. You can connect to the instance through VNC (use display number :0) and finish the installation.

For Ex: vncviewer A.B.C.D :0, where A.B.C.D is the IP address of Client1.

After finishing the installation, relaunch the VM by executing the following command.

uecadmin@client1:~$ sudo kvm -m 256 -drive file=image.img,if=scsi,index=0,boot=on -boot c \
-net nic -net user -nographic -vnc :0

At this point you can add all the packages you want to have installed, update the installation, add users and any settings that need to be present in your new UEC instances.

$ sudo apt-get update
$ sudo apt-get upgrade
$ sudo apt-get install mediawiki

Integrating with Eucalyptus

An instance running under Eucalyptus needs to know what IP it has and also, it needs to have the public key of the user allowed to do a passwordless access through SSH. This is accomplished by using a restful interface provided by the cloud. The interface is available under this URL: http://169.254.169.254/latest/meta-data and accessible from within the Instance.

Eucalyptus node controller is set up to prevent automatic key injection if the system is in MANAGED or MANAGED-NOVLAN mode. Instead, it is assumed that the instance will use the above meta-data service to retrieve the public keys when running in these modes. You will need to facilitate this by installing curl and adding a script that will run on each boot.

Install curl on the VM.

$ sudo apt-get install curl

Now add the following lines to /etc/rc.local of the image.

depmod -a
modprobe acpiphp

# simple attempt to get the user ssh key using the meta-data service
# assuming “user” is the username of an account that has been created

mkdir -p /home/user/.ssh
echo >> /home/user/.ssh/authorized_keys
curl -m 10 -s http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key | grep 'ssh-rsa' >> \
                          /home/user/.ssh/authorized_keys
echo "AUTHORIZED_KEYS:"
echo "************************"
cat /home/user/.ssh/authorized_keys
echo "************************"

Add the above lines before the “exit 0” in /etc/rc.local

Also remove the network persistent rules from /etc/udev/rules.d, so that the instance always comes up with eth0 as the interface name as expected by eucalyptus.

$ sudo rm -rf /etc/udev/rules.d/70-persistent-net.rules

This completes the process of customizing the OS installed as a reference image.

Registering with Eucalyptus

The last step would be to upload the images to Eucalyptus. The files that need to be uploaded for the above sample setup are: vmlinuz-2.6.28-11-server, initrd.img-2.6.28-11-server, image.img.

Copy the kernel and the initrd image from the VM image to some place outside. These will be used later for creating and uploading a complete virtual image to Eucalyptus.

$ scp /boot/initrd.img-2.6.28-11-server user@A.B.C.D:
$ scp /boot/vmlinuz-2.6.28-11-server user@A.B.C.D:

Before starting the upload process shut down the VM.

Registering kernel image

Execute the following commands to bundle and register the kernel image (vmlinuz-2.6.28-11-server)

uecadmin@client1:~$ euca-bundle-image -i vmlinuz-2.6.28-11-server --kernel true
uecadmin@client1:~$ euca-upload-bundle -b mybucket -m /tmp/vmlinuz-2.6.28-11-server.manifest.xml
uecadmin@client1:~$ euca-register mybucket/vmlinuz-2.6.28-11-server.manifest.xml

Save the output produced by the last command above (eki-XXXXXXXX), which will be needed while registering the disk image.

Registering ramdisk image

Execute the following commands to bundle and register the ramdisk image (initrd.img-2.6.28-11-server)

uecadmin@client1:~$ euca-bundle-image -i  initrd.img-2.6.28-11-server
uecadmin@client1:~$ euca-upload-bundle -b mybucket -m /tmp/initrd.img-2.6.28-11-server.manifest.xml
uecadmin@client1:~$ euca-register mybucket/initrd.img-2.6.28-11-server.manifest.xml

Save the output produced by the last command above (eri-XXXXXXXX), which will be needed while registering the disk image.

Registering disk image

Execute the following commands to bundle and register the ramdisk image (image.img)

uecadmin@client1:~$ euca-bundle-image -i image.img --kernel eki-XXXXXXXX --ramdisk eri-XXXXXXXX
uecadmin@client1:~$ euca-upload-bundle -b mybucket -m /tmp/image.img.manifest.xml
uecadmin@client1:~$ euca-register mybucket/image.img.manifest.xml

Replace eki-XXXXXXXX and eri-XXXXXXXX with the exact values you have saved earlier.

Image Listing

The new images that have been uploaded can be seen by using euca-describe-images command.

For Ex:

uecadmin@client1:~$ euca-describe-images
IMAGE   emi-70B70EC0    mybucket/image.img.manifest.xml                     admin   available        public  x86_64  machine
IMAGE   eri-A2BE13EC    mybucket/initrd.img-2.6.28-11-server.manifest.xml   admin    available       public  x86_64  ramdisk
IMAGE   eki-685F1306    mybucket/vmlinuz-2.6.28-11-server.manifest.xml      admin    available       public  x86_64  kernel

More details on managing the instances in later chapters.

Bundling Windows Image

Creating new disk image

This will represent the main HDD of the virtual machine, so make sure to give it as much space as you will need. Since we are building a kvm image, we can use a qcow2 format for disk images. Qcow2 is an expandable image format, so it will only occupy as much storage space as is actually filled by the data within the image.

uecadmin@client1:~$ kvm-img create -f qcow2 win-2k3.img 20G

OS Installation

Create an ISO image of the relevant Windows installation CD. And start the installation process.

uecadmin@client1:~$ sudo kvm -m 1024 -cdrom Win2003ServerR2x86_cd1.iso -drive file=win-2k3.img,if=scsi,boot=on -nographic -vnc :0

After finishing the installation and in case you need to install from the 2nd CD as well, reboot the VM and launch the VM by the following command.

uecadmin@client1:~$ sudo kvm -m 1024 -boot c -cdrom Win2003ServerR2x86_cd2.iso -drive file=win-2k3.img,if=scsi,boot=on -nographic -vnc :0

Once installation is complete, we need to create the boot disk needed for Windows 2003 server

For Windows XP bundling

While bundling Windows XP, the OS doesn’t recognize SCSI driver during installation process.

The following workaround is suggested:
1. Attach the disk as IDE disk.

$ sudo kvm -m 1024 -cdrom winxpcd.iso -drive file=winxp.img,boot=on -nographic -vnc :0

2. Finish the installation of Windows XP.
3. Now create another disk image using kemu-img

$ kvm-img create -f qcow2 newdisk.img 5G

4. Start the instance by attaching the second disk as SCSI disk.

$ sudo kvm -m 1024 -drive file=winxp.img,boot=on -drive file=newdisk.img,if=scsi -nographic -vnc :0

5. After Windows XP boots, it detects and installs the driver for the SCSI disk (sys_hi.sys)

The remaining procedure is same as that of bundling Windows 2003 Server.

Copying the boot files

Re-launch the VM using the following command, so that we can access the first CD, in order to copy some files from it.

uecadmin@client1:~$ sudo kvm -m 1024 -boot c -cdrom Win2003ServerR2x86_cd1.iso -drive file=win-2k3.img,if=scsi,boot=on -nographic -vnc :0

Copy the following files to a specific location to create the boot disk, say C:\bootfiles

• ntldr
• ntdetect.com
• sym_hi.sys

Rename sym_hi.sys to Ntbootdd.sys (SCSI driver needed to boot the hard disk).

In case, sym_hi.sys is not present in the CD, copy it from the following location of the installation

C:\WINDOWS\system32\drivers

Write the following lines to a new file and save it as Boot.ini in the same location, i.e, C:\bootfiles

[boot loader]
timeout=30
Default= multi(0)disk(0)rdisk(0)partition(1)\Windows

[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows Server 2003"

Shut down the VM and proceed for the boot disk creation

Boot disk creation

Execute the following command to create the boot disk.

uecadmin@client1:~$ dd bs=512 count=2880 if=/dev/zero of=win-boot.img

Now attach this disk as a floppy disk and start the Windows VM.

uecadmin@client1:~$ sudo kvm -m 1024 -boot c -drive file=win-2k3.img,if=scsi,boot=on,index=1 -fda win-boot.img -nographic -vnc :0

On the Windows VM, format the floppy disk and copy all the files from C:\bootfiles to the floppy.
Shutdown the VM and test the boot disk created with the following command

uecadmin@client1:~$ sudo kvm -m 1024 -boot a -drive file=win-2k3.img,if=scsi,boot=on,index=1 -fda win-boot.img -nographic -vnc :0

This should boot you into windows, if you have rightly created the boot floppy.

Network Configuration

Now you need to download the e1000 drivers, since e1000 is the default interface that is supported by Eucalyptus. You can download the drivers from the following location:

http://downloadcenter.intel.com/Detail_Desc.aspx?agr=Y&DwnldId=8659&lang=ENG

For 32-bit VM, download PRO2KXP.exe
For 64-bit Vm, download PROEM64T .exe

Once the download is finished, shutdown the VM and restart the VM with the following command.

uecadmin@client1:~$ sudo kvm -m 1024 -boot a -drive file=win-2k3.img,if=scsi,boot=on,index=1 -fda win-boot.img \
-net nic,model=e1000 -net tap -nographic -vnc :0

The above command tell the VM to make use of e1000 model as its NIC.
After executing the above command, you will get this warning.

Warning: vlan 0 is not connected to host network

You can safely ignore this warning message.

After the VM boots, install the network drivers.

Enable remote desktop for all users on the VM. This is very important. As this is the only way, without installing a third party tool, you allow get access to a Windows instance running on UEC. You can enable remote desktop by navigating to Start → Settings → Control Panel → System → Remote and selecting Remote Desktop option for all users. This completes the Windows disk setup. Shut down the VM.

Memdisk

The kernel memdisk is needed to load Windows. The default syslinux memdisk could be used for this purpose. If you cannot locate memdisk on CC, you need to install syslinux.

uecadmin@client1:~$ sudo apt-get install syslinux

After installing, memdisk will be found in /usr/lib/syslinux

uecadmin@client1:~$ cd /usr/lib/syslinux
uecadmin@client1:/usr/lib/syslinux$ ls -l memdisk
-rw-r--r-- 1 root root 20068 2008-07-15 20:25 memdisk

Copy it to the location where you have all windows files.

Registering with Eucalyptus

The last step would be to upload the images to UEC. The files that need to be uploaded are:

• memdisk
• win-boot.img
• win-2k3.img

Registering memdisk

Execute the following commands to bundle and register the memdisk.

uecadmin@client1:~$ euca-bundle-image -i memdisk --kernel true
uecadmin@client1:~$ euca-upload-bundle -b win2k3_bucket -m /tmp/memdisk.manifest.xml
uecadmin@client1:~$ euca-register win2k3_bucket/memdisk.manifest.xml

Save the output produced by the last command above (eki-XXXXXXXX), which will be needed while registering the disk image.

Registering boot disk

Execute the following commands to bundle and register the boot disk (win-boot.img).

uecadmin@client1:~$ euca-bundle-image -i win-boot.img --ramdisk true
uecadmin@client1:~$ euca-upload-bundle -b win2k3_bucket -m /tmp/win-boot.img.manifest.xml
uecadmin@client1:~$ euca-register win2k3_bucket/win-boot.img.manifest.xml

Save the output produced by the last command above (eri-XXXXXXXX), which will be needed while registering the disk image.

Registering disk image

Execute the following commands to bundle and register the disk image (win-2k3.img)

uecadmin@client1:~$ euca-bundle-image -i win-2k3.img --kernel eki-XXXXXXXX --ramdisk eri-XXXXXXXX
uecadmin@client1:~$ euca-upload-bundle -b win2k3_bucket -m /tmp/win-2k3.img.manifest.xml
uecadmin@client1:~$ euca-register win2k3_bucket/win-2k3.img.manifest.xml

Replace eki-XXXXXXXX and eri-XXXXXXXX with the exact values as noted down earlier.

Image Listing

The new images that have been uploaded can be seen by using euca-describe-images command.
For Ex:

uecadmin@client1:~$ euca-describe-images
IMAGE   emi-55470DE2    win-2k3/win-2k3.img.manifest.xml        admin   available    public  x86_64  machine
IMAGE   eki-2BC30D1B    win-2k3/memdisk.manifest.xml    admin   available   public   x86_64  kernel
IMAGE   eri-76C20ED5    win-2k3/win-boot.img.manifest.xml       admin   available    public  x86_64  ramdisk

Running instance of custom image

To launch a new instance of the custom image, execute euca-run-instances command.

uecadmin@client1:~$ cd .euca
uecadmin@client1:~/.euca$ euca-run-instances  emi-55470DE2 -k mykey -t c1.xlarge

Since this is a windows instance the VM type is specified as c1.xlarge, which has 20GB of hard disk space.

The above procedure for bundling Windows images worked on Ubuntu 9.10 (Karmic Koala). But it fails on Ubuntu 10.04(Lucid Lynx). Please refer to Chapter 11 for a hack to get Windows images running on Lucid Lynx.

Chapter 3 Web Interface

Chapter 5 Instance Management

About these ads