virtual router in cloudstack 4.4
DESCRIPTION
Presented at Apache CloudStack Collabration Conference 2014, Denver, CO. Talked about recently Virtual Router improvement in CloudStack 4.4 to unify and significantly speed up VR command execution, as well as some further improvement ideas.TRANSCRIPT
Virtual Router in CloudStack 4.4Sheng Yang
Virtual Router in CloudStack 4.4Sheng Yang
Virtual Router(VR) in CloudStack
VPC VRPublic network
Web Tier: 10.1.0.1/24
App Tier: 10.1.1.1/24
DB Tier: 10.1.2.1/24
Web 1 Web 3Web 2
App 1 App 2
DB Master DB Slave
10.1.0.1/16
Site-to-site VPN
Remote Access VPN
Public Load balancing
VLAN 1001
VLAN 1000
VLAN 1002
Remote NetworkRouter
VR code flow – VirtualRouterElement
– Interface for NetworkOrchestrator– VirtualNetworkApplianceManager
– applyRules()– finalizeVirtualMachineProfile()– finalizeCommandsOnStart()
– In VR– /etc/init.d/cloud-early-config– /opt/cloud/bin/
Before 4.4 – Different hypervisor resource would implement different
way to execute commands– Because VR is not a “host” in CloudStack– Commands can be executed in different environment
– On the host, or in the VR– Commands can only be executed one by one
– It would take significant time to program large numbers of commands
New in 4.4 – Introduce the new VirtualRoutingResource
– Handle all the commands for VR– One instance for one hypervisor resource– All execution would be done inside VR
– A new VirtualRouterDeployer interface for hypervisor which need to programming VR
– PrepareCommand()– ExecuteInVR()– CopyFileToVR()– CleanupCommand()
New in 4.4 (cont.) – Introducing aggregation mechanism to accelerate
commands execution in VR– Involved when VR booting/recreating– Delay and merge all the commands execution at
VirtualRoutingResource level– One SSH for all initialization commands
AggregationControlCommand– Action
– Start, Finish, Cleanup– Delay execution initialization commands to a queue
after start– Return success immediately for commands during
the aggregated period– Merge and execute the commands when finish
– By generating and applying a single configuration file to VR
– Clean up the queue at last– In case things goes wrong in merge commands
AggregationControlCommand (cont.)– Only support commands return true or false
– No query command– Only used when boot up or recreate VR– One failure fails all the following commands
– No rollback available– Detail logs in /var/log/cloud.log
– Rotated
Performance measurement– Create a unit test for performance testing
– Implement VirtualRouterDeployer interface– Using VirtualRoutingResource– Test against pre-created VR– Execute a large numbers of commands
– Simulate different command execution flow for hypervisors
– Direct access to VR for VMware– One hop on host for XenServer
Performance result
1000 DHCP and VMData
1000 Port Forwarding Rules
0 200 400 600 800 1000 1200 1400
Aggregated
VMware Original
Xen Original
Time consumed in seconds, smaller is better
More in the plan– Further optimize VR configuration
– Current implementation is simply execute the command queue in the VR
– In the future the configuration file can be generated in mgmt server then replace in VR, which make VR update an almost O(1) action
– Better way to update software on VR– Debian repository may would be the choice
Thanks!Thanks!