acs 4.1.1+ | kvm agent networking schema & upgrading
DESCRIPTION
1.) KVM agent networking schema changes 2.) Known upgrade issues & the common upgrade method 3.) Troubleshooting and work-aroundsTRANSCRIPT
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading
Kelcey Damage | 11/24/2013
Foreword
We are going to take a quick look at the networking schema changes introduced in 4.2 for KVM agents, as well as the upgrade process existing users.
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading
Agenda
1.) KVM agent networking schema changes
2.) Known upgrade issues & the common upgrade method
3.) Troubleshooting and work-arounds
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading
KVM networking schema changes
There are two key areas of interest in regards to the changes made to networking with KVM and Cloudstack
Schema Changes
Impact On Production
The KVM Agent
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading
Bridge SchemaACS 4.1.0+ | KVM Agent Networking Schema And Upgrading
bridge name bridge id STP enabled interfaces
brbondg-1205 8000.0023aef93c7c no bondg.1205 vnet7brbondg-1213 8000.0023aef93c7c no bondg.1213
vnet12cloudVirBr1205 8000.0023aef93c7c no vnet5cloudVirBr1213 8000.0023aef93c7c no vnet9
cloud0 8000.fe00a9fe014f no vnet0vnet4
vnet8cloudbrguest 8000.0023aef93c7c yes bondgcloudbrman 8000.0026b9fdb774 yes bondm vnet1 vnet3cloudbrpub 8000.0023aef93c80 yes bondp vnet2 vnet6virbr0 8000.525400368385 yes virbr0-nic
KVM Agent Schema as of 4.1.0+
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading
InterfacesWhat’s Changed:
Interfaces now require names shorter them 8 characters.
Why:
Because the new distributed port-group names are concatenations of the VLAN ID and the interface name.
“br{interface_name}-{VLAN#}”
What this replaces:
“cloudVirBr{VLAN#}”
Important note on interfaces
• Longer names will get truncated.
• Routing will break and traffic may be shared within VLANs.
Example:
• Naming a bond as ‘bondguest’ with a VLAN of 1204 will create the port-group ‘brbondguest-120’. VLANs 1200 – 1209 will all get assigned to this port-group
Effect on production systems
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading
If using any name other then the default:
“eth/em/p1p”…
… such as bonds or aliases then the bond or alias name ..//
//.. must be shorter then 8 characters.
Upgrading to 4.2+ and beyond
There are two key areas of interest in the upgrade process for anyone moving to the latest version.
Documentation
The New System VM
The Upgrade Process
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading
In the official release notes, the steps for upgrading from 4.0 forward to 4.2+ were missing the section on how and where to download the latest templates.
We now use a virtual-IO socket for communication with the system VM, instead of the prior ‘patchdisk’ method of passing cmdline and authorized_keys.
Documentation The New System VM1 2
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading
Major changes in the templates require updating them to the
latest version prior to upgrading the CloudStack.
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading
KVM
Name: systemvm-kvm-4.2 Description: systemvm-kvm-4.2 URL: http://download.cloud.com/templates/4.2/systemvmtemplate-2013-06-12-master-kvm.qcow2.bz2 Zone: Choose the zone where this hypervisor is used Hypervisor: KVM Format: QCOW2 OS Type: Debian GNU/Linux 7.0 (32-bit) (or the highest Debian release number available in the dropdown) Extractable: no Password Enabled: no Public: no Featured: no
Oh No! I Broke My Cloud
If you’ve gone ahead and upgraded your CloudStack version before upgrading your templates, there are ways to fix your cloud.
The Database Method
The Java Method
The Fix
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading
The Database Method
o Step 1): Mount your secondary storage to your management server
mount -t nfs {ip_of_storage_server}:[path_to_secondary_storage] /mnt
o Step 2): Download the latest version of the templates:
/usr/share/cloudstack-common/scripts/storage/secondary/cloud- install-sys- tmplt -m /mnt -u http://download.cloud.com/templates/4.2/systemvmtemplate- 2013-06-12-master-kvm.qcow2.bz2 -h kvm -F
o Step 3): Find the name of the old template in the database:
USE cloud; SELECT install_path FROM template_host_ref WHERE template_id=3;
o Step 4): write down the name of the .qcow2 file that your given in the previous step.
o Step 5): from the management server locate the new template on the mounted secondary storage:
cd /mnt/template/tmpl/1/3/
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading
The Database Method
o Step 6): rename the .qcow2 file in that folder to the name we copied from the database.
o Step 7): edit the template.properties file in the same folder and change both instances of the old name to the new one.
o Step 8): we need to reset the cached template in the database:
UPDATE template_spool_ref SET download_pct='0',download_state='NOT_DOWNLOADED',state='NULL',local_path='NULL‘,install_path='NULL',template_size='0' WHERE template_id='3‘;
o Step 9): Unmount your secondary storage from the management server:
umount /mnt
o Step 10): disable the zone from the management UI.
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading
The Database Method
o Step 11): update the database records for your system VMs to be ‘Stopped’. You will need to do this for both the Secondary Storage VM and the Console Proxy. The ID of the system VM is the number in it’s name, for example; s-34-VM,, would have an ID of ’34 .′
UPDATE vm_instance SET state='Stopped' where id='{id_of_system_vm}';
o Step 12): From the management UI, destroy both the system VMs.
o Step 13): Once both system VMs have been destroyed, re-enable the zone.
o Step 14): Tail the management log and watch for the VMs to start.
tail -f /var/log/cloudstack/management/management-server.log
o Step 15): Enjoy your 4.2+ Cloud!
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading
IYSE Method *(If Your Smart Enough)
For those in the USR tract that understand gobble-de-gook below is the JAVA method that some prefer.
(1) register a new template (such as systemvmtemplate-4.2) on UI or in Upgrade*.java file
(2) Change template type from User to System in Upgrade*.java file.
(3) cloud-install-sys-tmplt download the system-vm template
(4) cloudstack-sysvmadm re-deploy system-vms
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading
Referencehttps://issues.apache.org/jira/browse/CLOUDSTACK-4826
http://cloud.kelceydamage.com/cloudfire/blog/2013/10/08/conquering-the-cloudstack-4-2-dragon-kvm/
Get Involved
Web: http://cloudstack.apache.org/
Mailing Lists: cloudstack.apache.org/mailing-lists.html
IRC: irc.freenode.net: 6667 #cloudstack
Twitter: @cloudstack
LinkedIn: www.linkedin.com/groups/CloudStack-Users-Group-3144859
If it didn’t happen on the mailing list, it didn’t happen.
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading