hp technology forum & expo get connected. people. technology. solutions

HP Technology Forum & ExpoGet connected. People. Technology. Solutions.

Security Agility Reduced Security Agility Reduced CostCost

Best Practices using Xen Virtualization with Red Hat

Enterprise Linux 5

Jan Mark [email protected] Engineer Emerging Technology GroupCTO Office

mailto:[email protected]

HP Technology Forum June 2007 / RHEL5/Virt Best Practices

Product features subject to change prior to availability3

Outline • Best Practices

• Performance Expectations

• Key directories and files

• Some common examples

• Q&A



Basic Information before you start



Before you start• Read the documentation and QuickStart guides

− RHEL5 Virt Guide• http://www.redhat.com/docs/manuals/enterprise/RHEL-5-manual/

Virtualization-en-US/index.html• http://www.redhat.com/docs/

− Fedora Core 6 Xen QuickStart• http://fedoraproject.org/wiki/FedoraXenQuickstartFC6

− Kbase articles• http://kbase.redhat.com• http://kbase.redhat.com/faq/topten_108_0.shtm for virtualization specific

topics

− Virtualization “brain dump”• http://et.redhat.com/~jmh/docs/

− Look for Installing_RHEL5_Virt.pdf/odt− Red Hat Summit Slides

• http://et.redhat.com/~jmh/RH_Summit_2007/

− XenWiki• http://wiki.xensource.com/xenwiki/



RHEL5/Xen Architecture

Hardware

Hypervisor

dom0

Device Driver

Back End

domU/VM1

Front end

domU/VM2

Front end

Virtual CPU & Memory



Guest/Hypervisor Matrixwith PAE

32bit PAE paravirt Guest

32bit HVM Guest

64bit paravirt Guest

64bit HVM Guest

32bit (PAE) Hypervisor / dom0

64bit Hypervisor / dom0

For Paravirtual, the guest has to be equal to dom0 For HVM, the guest has to be equal or less than dom0 RHEL5/Xen Hypervisor itself must be EQUAL to dom0

Goal to support 32bit para-virt on 64bit dom0/HV for RHEL 5.1

8HP Technology Forum June 2007 / RHEL5/Virt Best Practices

Product features subject to change prior to availability

Platform/Guest Release Support in RHEL5 Fully supported by Red Hat

Para-virtualized guest support

RHEL5 at GA (Q1/2007)

RHEL4 U5 / RHEL 4.5

SMP and UP support for all RHEL para-virt guests

Fully-virtualized support (with appropriate hardware features(VTx or AMD-V)

All existing and supported RHEL releases

RHEL3, RHEL4 (SMP and UP support)

“Will/Should” work operating systems (best effort support by Red Hat)

Fully virt support only for any “foreign” operating systems

SUSE SLES9/SLES10

OpenSolaris

Windows

Other x86/x86_64 OS



Basic Information and Hints



Prerequisites for RHEL5/Xen Use RHEL5 Xen packages, DO NOT install Xen RPMs from XS !! Apply Day “0” errata updates for RHEL5 and Xen components

Fix for 'dual' cursor mouse in virt-manager

Need Intel-VT/AMD-V capabilities for HVM/Fully-Virt guests Install VNC and VSFTP packages

VNC for remote desktop access VSFTP for access to local/remote para-virt installation tree

At least 2GB of memory for a “working” system Use at least 300MB for guest installations

More memory will speed up the installation process 256MB may potentially cause installation failures

Plan for enough disk space on the host Rough guideline for a guest installation is ~5GB Guest image includes encapsulated swap space



Installing RHEL5/Xen Considerations

Secure RHEL5 platform layer before installing any virtual machines or applications

Enable SElinux to run in 'enforcing' mode Remove or disable any unwanted services

AutoFS, NFS, FTP, WWW, NIS, telnetd, sendmail etc... Only add minimum number of user accounts needed for platform

management Avoid running applications on dom0/Hypervisor Running applications in dom0 may impact virtual machine

performance Use central location for virtual machine installations

Will make it easier to move to shared storage later on If laptop with wireless adapter it used as virt platform need to perform

special steps to make Xen networking functional Setup dummy network interface and NAT traffic through WiFi



Planning for a Xen Infrastructure Naming conventions for Virtual Machines, Network Interfaces,

Filesystems, Volume Groups At least choose different name for dom0/host VolGroup00

Consider shared storage to store Virtual Machine images Foundation for live migration Make /var/lib/xen/images a seperate filesystem and as of RHEL5.1

use GFS2 even on a single/local host Make /var/lib/xen/dumps a seperate filesystem to avoid out-of-space

for VM core dumps Private network for Xen traffic

Secure migration traffic and improve performance Central location for installation sources

Network based installation tree for Para-Virt installations ISO file(s) or DVD/CD driver for HVM/Fully-virt installations



RHEL5/Xen Considerations

Most of the standard Linux/RHEL tools for troubleshooting still work in a virtualized environment

top, iostat, vmstat, lsof, etc... Very few Xen specific tools (mostly xm and virsh commands)

“xm top” for HyperVisor view of performance data XenOprofile for profiling of active and inactive domains

Leverage RHEL5/Xen capabilities to streamline provisioning Golden Images for VMs and applications Fast Provisioning with Kickstart files



/etc/xen xend-config.sxp

xend configuration file Domain / guest config files

/etc/xen/scripts (customizable) control scripts

Device management -eg. Networking /etc/xen/auto

Softlinks to guests that should start automatically xendomains service starts these guests

Typically disabled by default

Configuration – Key files and directories



/var/lib/xen dump

Kernel dumps generated with “xm dump-core <Domain>”

images Guest images (covered by SElinux policy)

xend-db Xen internal database

/var/lib/xend Sockets used by xend (xend-socket / relocation-socket)

/var/lib/xenstored xend database (tdb) Can be “read” with xenstore-ls

Configuration – Key files and directories



/var/log/xen xend.log

Logfile used by xend for logging Primary file to review in case of problems

xend-debug.log Debug output from xend

xend-hotplug.log Logfile for hotplug events Will record information in case of hotplug failure of devices

qemu-dm.{PID}.log Logfile used by the qemu-dm process

All logfiles are human readbale and can be view with the standard Linux utilities (or “xm log” for xend.log)

Logging – Key files and directories



/proc/cpuinfo shows whether pae, svm, vmx enabled CPU

/proc/xen various information about xen

/sys/hypervisor more information about domain, including UUID

Xen Python scripts are logging to debug facility (not enabled by default)

Add *.* /var/log/debug.log to syslog config file

Logging/Capabilities – Key files and directories



• Scripts are stored in /etc/xen/scripts

• Configured in /etc/xen/xend-config.sxp

Networking

(network-script network-bridge) (vif-script vif-bridge)

This configuration declares that

• “network-bridge” should handle the network setup

• “vif-bridge” should handle each domU add/remove



• Multi Network Bridge Solution : − Modify /etc/xen/xend-config.sxp

• Replace call to network-bridge script with call to custom script (ie multi-network-bridge)

• Create custom script in /etc/xen/scripts− Inside the custom script call network-bridge script with addtl

parameters/interfaces

Networking / Multi-bridge config



Networking / Example for multi-bridge config

#!/bin/sh# network-xen-multi-bridgeset -e# First arg is the operation.OP=$1shiftscript=/etc/xen/scripts/network-bridge.xencase ${OP} in start)

$script start vifnum=0 bridge=xenbr0 netdev=eth1 $script start vifnum=1 bridge=xenbr1 netdev=eth2 $script start vifnum=2 bridge=xenbr2 netdev=eth3 ;; stop) $script stop vifnum=0 bridge=xenbr0 netdev=eth1 $script stop vifnum=1 bridge=xenbr1 netdev=eth2 $script stop vifnum=2 bridge=xenbr2 netdev=eth3 ;; status)

$script status vifnum=0 bridge=xenbr0 netdev=eth1 $script status vifnum=1 bridge=xenbr1 netdev=eth2 $script status vifnum=2 bridge=xenbr2 netdev=eth3 ;; *) echo 'Unknown command: ' ${OP} echo 'Valid commands are: start, stop, status' exit 1esac



Migration

• Requires shared storage− Both dom0's must access same disk image or

physical device

• Using Physical device− LUN on SAN− Exported block device

• ISCSI, GNBD− Both dom0's should use same name for device

•eg. /dev/sdg•Use UDEV rules for mapping if required



Migration

• Using File based disk image− Disk image stored on shared file system

•NFS *•Samba•GFS

− Should be mounted on same mount point•Both domain 0's see same directory structure

− eg. /var/lib/xen/images



Migration• Enable relocation server

/etc/xen/xend-config.sxp(xend-relocation-server yes)(xend-relocation-port 8002)(xend-relocation-address '')(xend-relocation-hosts-allow '')

• Enables relocation server

• Listens on port 8002 (default)

− Use lsof to verify relocation port is active

[root@grumble]$ sudo lsof -i :8002

python 3873 root 5u IPv4 12329 TCP *:teradataordbms (LISTEN)

• Binds to all IP addresses

• Allows any host to migrate to this domain 0

− Authorized hosts can be listed by IP or name

− '^myserver.mydomain.com$ ^.*\.redhat\.com$'



Basic Xen commands

•Once you have your first guest installed you can use the following commands for some basic management•To startup a guest

# /usr/sbin/xm create -c GuestName −Where GuestName is the name you gave for your guest during the installation−The -c will attach a xen console to your vm

•A variety of other commands are available via xm including# /usr/sbin/xm help or # virsh help

−For a list of commands that can be run−Use '--long' in addition for extended help text−You can also use

#/usr/sbin/xm help –help −for a specific command



Basic Xen commands contd.

# /usr/sbin/xm list (--long) or # virsh listList running domains/guest and their status/accumulated CPU time

# /usr/sbin/xm topfor a display showing what your virtual machines are doing similar to that provided by top

# /usr/sbin/xm shutdown GuestName or # virsh shutdown GuestName

to nicely shut down a guest OS where foo is the name of your guest.

# /usr/sbin/xm destroy GuestName or # virsh destroy GuestNameTo power down a guest (hard reset)



Basic Xen commands (Suspend/Resume)

# /usr/sbin/xm save GuestName GuestName.sav or # virsh save GuestName GuestName.sav

to save the state of the guest 'GuestName' to the file GuestName.restore

# /usr/sbin/xm restore GuestName.sav or # virsh restore GuestName.sav

to restore the above saved guest

# /usr/sbin/xm pause GuestName or # virsh pause GuestNameto suspend a running guest (release CPU cycles but retain memory footprint)

# /usr/sbin/xm unpause GuestName or # virsh unpause GuestName

to resume a previously suspended guest



Basic Xen commands (Resource Management)# /usr/sbin/xm vcpu-set <dom> <value> or # virsh setvcpus <dom> <vcpus>

set the number of CPUs available to <dom> to <value> (only works for dom0/paravirtualized guests)

# /usr/sbin/xm vcpu-list or # virsh vcpuinfoList the physical-virtual CPU bindings

# /usr/sbin/xm mem-set <dom> <value> or # virsh setmem <dom> <value>

balloon <dom> up or down to <value> (only works for dom0/paravirtualized guests)

# /usr/sbin/xm sched-credit -d <DomainID>Display credit schedule information and set cap/weight for individual domain



Other Useful Commands and ToolsBasic Xen commands

xm logxm dmesgxm info / virsh nodeinfoxm top

virsh commands (extract)virsh dominfo <dom>virsh domstate <dom>Virsh dumpxml <dom>

Tools strace, lsof, iostat/vmstatSystemtap/var/log/messages/var/log/xenAVC messages (setroubleshoot)sosreport has a plugin that automatically gathers all of the above information

Plugin not included in RHEL-5 GA, should be included in day0 errata



RHEL5 Virt Performance Data



Red Hat Virt Single-guest Performance

0

50

100

150

200

250

300

350

400

00.0050.010.0150.020.0250.030.0350.040.0450.05

0.017391304347826

0.0353982300884956

0.0418250950570342

0.04682622268470340.0454545454545455

domU vs dom0 RHEL5 RC1 2.6.18-7 3.2 ghz em64T 1GB mem

dom0domU(1)dom0/domU



Red Hat Virt Performance Para-Virt vs Fully Virt VTx

Copy 10 MB/sec

Linpackd 1kx1k Mflops

GeoMean 1k-1G

External Mbit/sec

Internal MB/sec

0

100

200

300

400

500

600

700

800

900

1000

RHEL5, R4 PV, R4/R3 FV Performance Xen 2.6.17-1.2519.1 Woodcrest

R5-FVR4U4 PVR4U4 FVR3 U7 FV

Perf

orm

ance



Red Hat Virt Multi-guest Performance

0

50

100

150

200

250

300

350

400

Multiiple XenU Scalability 1,2,4 on RHEL5 GA 2.6.18-8

3.2 Ghz 2-cpu/2ht em64T 4GB memory, 1GB/xen guest

xenU-1xenU-2XenU-4



Red Hat Virt Multi-guest Efficiency

0

0.2

0.4

0.6

0.8

1

1.2

Efficiency of Multiple XenU 1,2,4 on RHEL5 GA 2.6.18-8

3.2 Ghz em64T 4GB memory, 1GB/xen guest

XenU-1 BaseXen2/Xen1Xen4/Xen1

Sca

ling

eff

icie

ncy



Red Hat Virt Java Multiguest

Performance



Red Hat Virt Multiple Guest

Performance

Ora TPM(Fiber)0

10

20

30

40

50

60

70

80

Oracle 10G tpm RHEL5 Multi-Instance Xen1,2,4

RHEL5 GA 2.6.18-8 4-cpu 3 ghz em64T 1GB /vm

RHEL4 U4RHEL5 RC1R5 – Dom0Xen1Xen2Xen4

Tra

ns/

min

(tp

m)



SMP in Guest OSes

• Takes great care to get good performance while remaining secure

• Paravirtualized approach yields many important benefits− Avoids many virtual IPIs− Enables ‘bad preemption’ avoidance− Auto hot plug/unplug of CPUs

• SMP scheduling (at hypervisor)− Strict gang scheduling not optimal− Credit Scheduler in Xen 3.0.2 (RHEL5)



Red Hat Virt SMP guest Performance

0

5000

10000

15000

20000

25000

30000

RHEL5 GA Aim 4-cpu, 4GB Woodcrest

r5xen 1-cpur5xen 2-cpur5xen 4-cpur5-Base

Processes

Jo

bs/m

in



Red Hat Virt SMP Performance

1 2 40

5000

10000

15000

20000

25000

30000

35000

0.82

0.83

0.84

0.85

0.86

0.87

0.88

0.89

Oracle 10G tpm SMP Scaling - R4, R5, Xen

2.6.18-8 4-cpu 3 ghz em64T 1GB/vm

RHEL4 U4RHEL5 B1R5 – Dom0R5 – Xen%diff Xen/R5

Number of CPUs

OA

ST -

TPM



1 20

5

10

15

20

25

30

35

40

0.81

0.815

0.82

0.825

0.83

0.835

0.84

0.845

0.85

0.855

Sybase 12.5 tpm SMP Scaling - R4, R5, Xen

2.6.18-8 4-cpu 3 ghz em64T 1GB/vm

RHEL4 U4RHEL5 B1R5 – Dom0R5 – Xen%diff Xen/R5

Number of CPUs

TPM

(k)

Red Hat Virt SMP guest Performance



Red Hat Virt Network Performance

Dom0->External (raw) Dom0->-Guest External->Guest

0

20

40

60

80

100

120

RHEL5 RC1 NetPerf Performance x86_64 (@3k = ave size of SPECweb2005)

PV5 3k

PV4.5 3k



Red Hat Virt Network Performance

Guest-Dom0 Guest->Guest Guest->External0

100

200

300

400

500

600

RHEL5 RC1 NetPerf Performance x86_64

(@3k = ave size of SPECweb2005)

PV5 3kPV4.5 3k

Th

rou

gh

pu

t M

B/s



Red Hat Xen FV Performance

Linpackd 1kx1k Mflops Copy MB/sec0

500

1000

1500

2000

2500

3000

3500

Fully Virtualized VT Performance Xen 2.6.18-7 Woodcrest

FC5-BaseR4U4 i386R4U4 x86_64R3 U7

Perf

orm

an

ce



Red Hat Virt Storage Alternatives

Fibe

r(xvd

)

iSCSI

(xvd

)

ISCSI

(gue

st)

NFSfiler

(xvd

)0

5000

10000

15000

20000

25000

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

RHEL5 RC1 Xen Application Perfor-mance

w/ various Storage (2-cpu AMD64 2.2)

R5(base)R5 – Dom0R5 – Xen4-cpu %diff Xen/Dom0

Ora

cle 1

0G

Tra

ns/

min

(tp

m)



Tuning using PCI domain

Hypervisor can dedicate PCI-domain's to individual guest.

Default Xen shared HBA / NICs within guest config.

Disk – tested up to 6 Fiber Channel HBAs Network – tested up to 3 1-Gibit network

cards Pros - Dedicate physical PCI resource to guest

Done in guest configuration file Improve I/O performance 10-15%

Cons - Limit throughput when VM's are idle. Limit configuration if planning to use

migration



Red Hat Virt Performance

Enhance CPU Perf w/ Vcpu-pin Tuning



Tuning using PCI domain



A few examples



Installation Issues

Require different installation source for PV vs FV installations

PV requires network based install treeFV require local boot.iso or DVD

In upcoming version of virt-manager/virt-install can specify a network tree for boot.iso

VM image not in /var/lib/xen/images with SElinux enabled in enforcing mode

Will cause backend device not being “connected” and generate a hotplug eventInstallation seems to hang

If iSCSI is used for root device for VM need to create a local boot slab

Also needs a custom network script if iSCSI is not reachable via default network



Installation Issues

If multiple Xen bridges are configured use virt-install to specify specific bridge

virt-manager will use the bridge attached to the default network/route



Installation Issues / VT/AMD-V Extensions

Intel/VT extensions not enabled in BIOSUnable to install HVM guest (box grayed out)

You can use the following commands to verify whether the virtualization extensions have been enabledOn an Intel/VT based system, look for “VMX”

[root@woodie ~]# xm dmesg | grep VMX(XEN) VMXON is done(XEN) VMXON is done

and to verify the CPU flags have been set [root@woodie ~]# cat /proc/cpuinfo |grep vmxflags: fpu tsc msr pae mce cx8 apic mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall lm constant_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr lahf_lmYou should have a VMXON for each reported processor. If you have any other messages visit your BIOS settingsThere is no reason to go any further until you have VMXON reported - it just isn't going to work



Installation Issues / VT/AMD-V Extensions

AMD/AMD-V extensions not enabled in BIOSUnable to install HVM guest (box grayed out)

On an AMD-V based system, look for “SVM”[root@perf3 ~]# xm dmesg|grep SVM(XEN) AMD SVM Extension is enabled for cpu 0(XEN) AMD SVM Extension is enabled for cpu 1and to verify the CPU flags have been set

[root@perf3 ~]# cat /proc/cpuinfo |grep svmflags : fpu tsc msr pae mce cx8 apic mtrr mca cmov pat pse36

clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni cx16 lahf_lm cmp_legacy svm cr8legacy ts fid vid ttp tm stc

You should have a AMD SVM for each reported processor. If you have any other messages visit your BIOS settingsThere is no reason to go any further until you have VMXON reported - it just isn't going to work



Troubleshooting (Memory Ballooning)

Failed domain creation due to memory shortage, ie unable to balloon domainA domain might fail to start if there's either not enough memory available or if dom0 has not ballooned down enough to provide space for the newly created/started guestA typical error message in your /var/log/xen/xend.log would be

[2006-11-21 20:33:31 xend 3198] DEBUG (balloon:133) Balloon: 558432 KiB free; 0 to scrub; need 1048576; retries: 20.[2006-11-21 20:33:52 xend.XendDomainInfo 3198] ERROR (XendDomainInfo:202) Domain construction failed

You can verify the amount of memory currently used by dom0 using the command “xm list Domain-0” , if dom0 indeed is not ballooned down you can use the command “xm mem-set Domain-0 NewMemSize” (where NewMemSize should be a small(er) value



Troubleshooting (non-Xen/PAE kernel)

Wrong kernel image (ie non-Xen kernel in a para-virt guest)If you try to boot a non-xen kernel in a para-virtualized guest you will see the following error message

[root@grumble]# xm create testVMUsing config file "./testVM".Going to boot Red Hat Enterprise Linux Server (2.6.18-1.2839.el5)kernel: /vmlinuz-2.6.18-1.2839.el5initrd: /initrd-2.6.18-1.2839.el5.imgError: (22, 'Invalid argument')

In the above example you can see that the kernel line shows that it's trying to boot a non-xen kernel. The correct entry would be ”kernel: /vmlinuz-2.6.18-1.2839.el5xen”



Troubleshooting (networking/bridging)

Wrong bridge configured in guest configuration file causing Xen Hotplug scripts to timeoutIf you have moved configuration files between different hosts you may to make sure your guest configuration files have been updated to reflect any change in network topology/configuration such as Xen bridge numbering etc

[root@grumble xen]# xm create r5b2-mySQL01Using config file "r5b2-mySQL01".Going to boot Red Hat Enterprise Linux Server (2.6.18-

1.2747.el5xen)kernel: /vmlinuz-2.6.18-1.2747.el5xeninitrd: /initrd-2.6.18-1.2747.el5xen.imgError: Device 0 (vif) could not be connected. Hotplug scripts not

workingIn /var/log/xen/xen-hotplug.log you will see the following error being logged

bridge xenbr1 does not exist!



Troubleshooting (networking/bridging) contd

In /var/log/xen/xend.log you will see the following messages being logged[2006-12-14 15:07:08 xend 3874] DEBUG (DevController:143)

Waiting for devices vif.[2006-12-14 15:07:08 xend 3874] DEBUG (DevController:149)

Waiting for 0.[2006-12-14 15:07:08 xend 3874] DEBUG (DevController:464)

hotplugStatusCallback /local/domain/0/backend/vif/2/0/hotplug-status.

[2006-12-14 15:07:08 xend 3874] DEBUG (DevController:464) hotplugStatusCallback /local/domain/0/backend/vif/2/0/hotplug-status.

[2006-12-14 15:08:48 xend 3874] DEBUG (DevController:464) hotplugStatusCallback /local/domain/0/backend/vif/2/0/hotplug-status.

[2006-12-14 15:08:48 xend 3874] DEBUG (DevController:464)



Zombie Domains

When shutting down a domain or migrating, it doesn't actually die but ends up leaving a domain named Zombie-<dom>

Typically caused by restarting xend while domains are running Xend does not properly reconnect certain devices (most notably frame

buffers), so when the domain goes to shut down, xend does not know how to find all of the resources for the domain to destroy

If it happens during migration likely cause is either networking problem or remote relocation server not enabled (status can also change to “migrating-<dom>”

[root@dhcp78-237 ~]# xm list

Name ID Mem(MiB) VCPUs State Time(s)

Domain-0 0 14627 8 r----- 18883.9

Zombie-rhel4u5pv01 9 511 1 -p---- 1072.2

rhel5gapv01 4 511 1 -b---- 1961.1 To resolve the issue

Reboot (only reliable solution) Balloon down Zombie domain to make room for other domains



Storage Troubleshooting (Filebased)

Unable to start more than 8 file based guests If file based guest images are used one may have to increase

the number of configured loop devices. The default configuration allows up to 8 loop devices to be active, if more than 8 file based guests/loop devices are needed the number of loop devices configured can be adjusted in /etc/modprobe.conf

Simply edit /etc/modprobe.conf and add the following line to it

options loop max_loop=64 You can substitute the number '64' with a number which fits

your local configuration



Storage Troubleshooting (Filebased)

Check if tapdisk process is actually running

[root@dhcp78-237 ~]# ps auxw|grep tap

root 3338 0.0 0.0 95644 680 ? Ssl Apr29 0:00 blktapctrl

root 1053 0.0 0.0 60268 708 pts/7 S+ 06:09 0:00 grep tapd

root 5238 0.0 0.0 30176 612 ? Sl Apr29 0:04 tapdisk /dev/xen/tapctrlwrite2

/dev/xen/tapctrlread2

root 6843 0.0 0.0 30172 612 ? Sl Apr29 0:03 tapdisk /dev/xen/tapctrlwrite3

/dev/xen/tapctrlread3



Storage Troubleshooting (Filebased) For each disk there should be a corresponding entry in your domain

configuration[root@woodie ~]# virsh dumpxml 4

<domain type='xen' id='4'>

<name>rhel5gapv01</name>

<os>

<type>linux</type>

<kernel>/var/lib/xen/vmlinuz.Q0Psrs</kernel>

<initrd>/var/lib/xen/initrd.B_e0Gm</initrd>

<cmdline>ro root=/dev/VolGroup00/LogVol00 rhgb quiet</cmdline>

<disk type='file' device='disk'>

<driver name='tap' type='aio'/>

<source file='/var/lib/xen/images/rhel5gapv01.dsk'/>

<target dev='xvda'/>

</disk>

<console tty='/dev/pts/3'/>

</devices>

</domain>



Other random bits

A fully virtualized x86_64 guest fails to boot with “Your CPU does not support long mode. Use a 32bit distribution”.

Resolution: Make sure “pae=1” is set in the domain configuration file

Over time, the /var/lib/xen directory becomes full of kernel.xxxxx and initrd.xxxxx files

Xen tools don't always clean up after errors Resolution:

Safe to just delete all of the files

Message: “FATAL: Module microcode not found” during domain boot

It's not really a fatal error, it is just warning that the virtual machine couldn't update the CPU microcode

Resolution: Nothing required To get rid of error message, disable microcode_ctl service

61HP Technology forum June 2007 / RHEL5/Virt Workshop


Consider the possibilities...



Resources

• Red Hat − http://www.redhat.com/

• Virtualization Infocenter− http://www.openvirtualization.com/

• Libvirt− http://www.libvirt.org/

• Virt-Manager− http://virt-manager.et.redhat.com/

• Red Hat Cluster Suite− http://www.redhat.com/solutions/gfs/

• Red Hat Emerging Technology Group− http://et.redhat.com/



QuestionQuestions?s?

ThanksThanks

HP Technology Forum & ExpoGet connected. People. Technology. Solutions.



Highly Available RHEL5 Host

RHEL5

Host A

Guest

RHEL5

Host B

Shared

Storage

Guest running as a RHCS serviceGuest 1

Guest 2

Guest X




RHEL5

Host A

Guest

RHEL5

Host B

Shared

Storage

Guest running as a RHCS serviceGuest 1

Guest 2

Guest X

Automatic failover upon Hypervisor failure




RHEL5

Host A

Guest

RHEL5

Host B

Shared

Storage

App

Guest 2

Guest XGuest 1Guests running as independent cluster

Hypervisor clustered via RHCS




RHEL5

Host A

RHEL5

Host B

Shared

Storage

Guest 2

App


Hypervisor clustered via RHCSApplication failover upon hosts/guest failure




RHEL5

Host A

Guest

RHEL5

Host B

Shared

Storage

Guest 2

App


Hypervisor clustered via RHCSApplication failover upon hosts/guest failure




RHEL5

Host A

RHEL5

Host B

Shared

Storage

App

Guest 2

Guest X

Guests running as independent cluster

Hypervisor and bare metal host clustered via RHCS

RHEL5

Host C

Guest 1 Guest X




RHEL5

Host A

RHEL5

Host B

Shared

Storage

Guest 2

Guest X



RHEL5

Host C

App

Guest 1 Guest XApplication can migrate to another Guest/VM




RHEL5

Host A

RHEL5

Host B

Shared

Storage

Guest 2

Guest X



RHEL5

Host CApp

Guest 1 Guest X

Application can migrate to a bare metal system



RHEL5 Disaster Recovery

RHEL5

Site A

RHEL5

Site B

Shared

Storage

Guest 1

Guest 2

Guest XGuestImage

GuestImage

GuestImage

GuestImage

GuestImage

GuestImage

XP/CA, EVA/CA




RHEL5

Site A

RHEL5

Site B

Shared

Storage

Guest 1

Guest 2

Guest XGuestImage

GuestImage

GuestImage

GuestImage

GuestImage

GuestImage

XP/CA, EVA/CA




RHEL5

Site A

RHEL5

Site B

Shared

Storage

Guest 1

Guest 2

Guest X

GuestImage

GuestImage

GuestImage

GuestImage

GuestImage

GuestImage

XP/CA, EVA/CA




RHEL5

Site A

RHEL5

Site B

Shared

Storage

Guest 1

Guest 2

Guest X

GuestImage

GuestImage

GuestImage

XP/CA, EVA/CA

hp technology forum & expo get connected. people. technology. solutions

Documents

cost best practices

hp technology forum

virt support

wifi slide

os slide

comxenwiki slide

xen quickstart http

virt platform