scenario-based distributed virtualization management architecture
DESCRIPTION
TRANSCRIPT
Scenario-based Distributed Virtualization Management Architecture for Multi-Host Environments
F. Galán (Telefónica I+D), D. Fernández (UPM), M. Ferrer (UPM), Fco. J. Martín (UPM)
DMTF 2nd Workshop on Systems and Virtualization Management (SVM 2008)Munich, 21-22 October 2008
© 2008 Telefónica Investigación y Desarrollo, S.A. Unipersonal
2TELEFÓNICA I+D
© 2008 Telefónica Investigación y Desarrollo, S.A. Unipersonal
01 Introduction
02 EDIV Design
- General architecture
- Deployment controller
- Operation
03 Practical Results
04 Alignment with DMTF’s Standards
- CIM-based virtual management
- OVF alignment
05 Summary
Index
3TELEFÓNICA I+D
© 2008 Telefónica Investigación y Desarrollo, S.A. Unipersonal
Introduction (1/3)Scenario-based virtualization management principles
Scenario-based virtualization management tools
— Scenario specification can be defined as a format representation (e.g., XML) of a set of virtual machines along with their interconnections in a given topology.
— Scenario-based management consist in processing scenario specifications in order to be deployed in physical infrastructure (hosts) and provide further management (e.g. monitoring, execute command sequences, undeploy, etc.).
Some tools nowadays are scenario-based, focused on experimentation environments, such as research and educational testbeds
— VNUML, NetKit, MLN
Conventional virtualization management tools (e.g. VMware VirtualCenter) for production environments use not to be scenario-based (out of our scope)
01
4TELEFÓNICA I+D
© 2008 Telefónica Investigación y Desarrollo, S.A. Unipersonal
Introduction (2/3)Scenario-based management example: VNUML
01
Scenario-based tool(e.g. VNUML)
Desired scenario (designed by the
user)
Physical host
VMs and interconnection virtual networks
Scenario specificati
on(in XML)
<vnuml>
</vnuml>
5TELEFÓNICA I+D
© 2008 Telefónica Investigación y Desarrollo, S.A. Unipersonal
Introduction (3/3)Problem Statement
Scenario-based virtualization tools use to consider just one physical host deployment
— Scalability problems (e.g. a 1000 VM scenario)
We have designed and implemented a distributed virtualization management tool named EDIV (Distributed Scenarios with VNUML, in Spanish) which purpose is to deploy scenarios in a cluster of N physical hosts.
— Based on VNUML (in other words, VNUML scenarios can be used in EDIV without modifications)
— Modular and extensible partition algorithm
— A partnership project between Telefónica I+D and Universidad Politécnica de Madrid (Spain) which prototype results are publicly available (GPL).
01
6TELEFÓNICA I+D
© 2008 Telefónica Investigación y Desarrollo, S.A. Unipersonal
01 Introduction
02 EDIV Design
- General architecture
- Deployment controller
- Operation
03 Practical Results
04 Alignment with DMTF’s Standards
- CIM-based virtual management
- OVF alignment
05 Summary
Index
7TELEFÓNICA I+D
© 2008 Telefónica Investigación y Desarrollo, S.A. Unipersonal
EDIV General Architecture02
Switch-based interconnection backplane(supporting 802.1q VLAN)
…
Deployment hosts (N)
VLAN802.1qtrunk
Deployment controller
managementnetwork
VNUML VNUMLVNUML
Deployment hosts
— GNU/Linux systems with VNUML installed along with other utilities (vconfig, sshd, etc.)
— VNUML provide three operations: deploy, execution and undeploy
Switch-based interconnection backplane
— One or serveral Ethernet switches providing end-to-end 802.1q VLAN trunks
— A management network (physical of VLAN) must be implemented
8TELEFÓNICA I+D
© 2008 Telefónica Investigación y Desarrollo, S.A. Unipersonal
EDIV Deployment Controller02
VNUMLVNUML
VNUML
Coordinator
Switches
Deployment hosts
DB
Deployment controller
U
DS
H
W
SegmentatorSegmentatorSegmentator
Scenario specification (VNUML language) and
operation (deploy,execution or undeploy)
(ssh/scp)
(SQL)(vendor CLI)
(Perl module)
<vnuml></
vnuml>
managementnetwork
9TELEFÓNICA I+D
© 2008 Telefónica Investigación y Desarrollo, S.A. Unipersonal
EDIV OperationDeployment
02
Coor Seg H1 H2 SW
SPLIT_REQ(SC, param)segmentator algorithm maps VMs in SC tohosts (generating a mapping table)
split SC XML in two fragment (SC1, SC2) as specified in
mappingCREATE_VLAN_SUBIF (…)RUN_VNUML(‘deploy’,SC1)
CREATE_VLAN_SUBIF (…)RUN_VNUML(‘deploy’,SC2)
ENABLE_VLAN (…)
get pre-deployment information
RESULT(‘OK’)
wait all responses
store SC1 y SC2 associated with the distributed scenario name (eg., ‘tutorial’)store VLAN IDs allocated to the scenario
SPLIT_RSP(mapping)
OPER(‘deploy’,SC)
10TELEFÓNICA I+D
© 2008 Telefónica Investigación y Desarrollo, S.A. Unipersonal
01 Introduction
02 EDIV Design
- General architecture
- Deployment controller
- Operation
03 Practical Results
04 Alignment with DMTF’s Standards
- CIM-based virtual management
- OVF alignment
05 Summary
Index
11TELEFÓNICA I+D
© 2008 Telefónica Investigación y Desarrollo, S.A. Unipersonal
Practical ResultsExperiment Setup
03
H1 H2 H3
Switch
eth1
eth1eth1
eth0 eth0 eth0
Deployment controller
Large scenario (51 VMs)Management network
Small scenario (6 VMs)
EDIV tool implemented in Perl Physical management network Partition algorithms
— Round robin— Weighted round robin (based on CPU load)— Deployment constraints (“VM A and B in the same
physical host”)
12TELEFÓNICA I+D
© 2008 Telefónica Investigación y Desarrollo, S.A. Unipersonal
Practical ResultsVirtual machine distribution
03
Deployment host loads
Round-RobinWeighted
Round Robin
L1 L2 L3 H1 H2 H3 H1 H2 H3
Small scenari
o (6 VMs)
L L L 2 2 2 2 2 2
M M M 2 2 2 2 2 2
H L L 2 2 2 1 3 2
H M M 2 2 2 2 2 2
Large scenario (51 VMs)
L L L 17 17 17 17 17 17
M M M 17 17 17 17 17 17
H L L 17 17 17 7 22 22
H M M 17 17 17 13 19 19L (low) 0% load M (medium) 50% load H (high) 100% load
13TELEFÓNICA I+D
© 2008 Telefónica Investigación y Desarrollo, S.A. Unipersonal
Practical ResultsTime comparison
How much does it take to deploy each scenario?
— With VNUML (reference time)
– Small scenario: 42 s
– Large scenario: 282 s
With EDIV (L/L/L, low mean load)
– Small scenario: 17 s
– Large scenario: 130 s
03
EDIV can achieve a 50-60% saving regarding
the mono-host case with VNUML
14TELEFÓNICA I+D
© 2008 Telefónica Investigación y Desarrollo, S.A. Unipersonal
01 Introduction
02 EDIV Design
- General architecture
- Deployment controller
- Operation
03 Practical Results
04 Alignment with DMTF’s Standards
- CIM-based virtual management
- OVF alignment
05 Summary
Index
15TELEFÓNICA I+D
© 2008 Telefónica Investigación y Desarrollo, S.A. Unipersonal
DMTF’s Standards AlignmentCIM-based virtual machine management
04
Deploymentcontroller
VNUML VNUML-based deployment host
(UML virtual machines)
U
H
CIMVirt. Model
WBEMServer
Provider(e.g libvirt-cim)
CIM-baseddeployment host
(CIM-manageable VMs: Xen, KVM, OpenVZ, etc.)
H’
(CIM-XML orCIM-SOAP)
CMPI, Java, …
(ssh/scp)
Scenario specification (VNUML language) and
operation (deploy, execution or undeploy)
<vnuml></
vnuml>
16TELEFÓNICA I+D
© 2008 Telefónica Investigación y Desarrollo, S.A. Unipersonal
DMTF’s Standards Alignment OVF-based virtual machines in EDIV scenarios
Open Virtualization Format (OVF)— Recently published DSP0243 1.0.0d (September 2008)
Why introduce OVF in EDIV?— To use virtual appliances highly optimized for specific purposes in EDIV
scenarios (e.g., firewalls, dynamic routing stacks, etc.)
— To smooth the evolution of virtual machines from development/pre-production environments to production
How to do it?
04
Deploymentcontroller
U H/H’
(ssh/scp or CIM-XML orCIM-SOAP)
<vnuml></
vnuml>
OVF-complaintdeployer
physical hostsOVF descriptors in
scenario specifications
OVF-complaintdeployerOVF-complaint
deployerOVF-compliantdeployer
17TELEFÓNICA I+D
© 2008 Telefónica Investigación y Desarrollo, S.A. Unipersonal
Summary
Main conclusions— EDIV architecture solves the problem of scenario-based
virtualization management in distributed multi-host environment (as checked by ours experiments)– Scalability
– Dramatic reduction (50-60% saving) in deployment time
— Simple and open API to develop new partition algorithms
— Transparency to VNUML users (the specification language is the same)
Work in progress— Wide-area deployment environment (instead of a local cluster)
– This is being working out in PASITO (a distributed experimentation platform within RedIris, the Spanish National Research and Education Network)
Future working lines— Alignment with DMTF’s standards
— Virtual machine images management
05
© 2008 Telefónica Investigación y Desarrollo, S.A. Unipersonal
EDIV tool and documentation isfreely available at
http://www.dit.upm.es/vnuml
19TELEFÓNICA I+D
© 2008 Telefónica Investigación y Desarrollo, S.A. Unipersonal
OperationCommand sequences execution
02
Coor Seg H1 H2 SW
OPER(‘exec’, cmd, ’tutorial’)
get XML1 and XML2 (which were associated to ‘tutorial’ at deployment time)
RUN_VNUML(‘exec’,cmd,XML1) RUN_VNUML(‘exec’,cmd,XML2)
RESULT(‘OK’)
wait all responses
20TELEFÓNICA I+D
© 2008 Telefónica Investigación y Desarrollo, S.A. Unipersonal
OperationUndeployment
02
Coor H1 H2 SW
OPER(‘udeploy’, ‘tutorial’) RUN_VNUML(‘undeploy’,SC1)
DESTROY_VLAN_SUBIF (…)RUN_VNUML(‘udeploy’,SC2)DESTROY_VLAN_SUBIF (…)
DISABLE_VLAN (…)
remove SC1 y SC2 release VLAN IDs used by the scenario
RESULT(‘OK’)
get SC1 and SC2 and VLAN IDs (which were associated to ‘tutorial’ at deployment time)
wait all responses
21TELEFÓNICA I+D
© 2008 Telefónica Investigación y Desarrollo, S.A. Unipersonal
Interconnection backplane
brA brA Deployment controller
host1 host2Net0 Net1 Net2
vm1vm2
vm3
vm5
vm4
.2
.1
.3
10.0.0.0/24 10.0.1.0/24 10.0.2.0/24
.1 .1.2
.2
<vnuml>
</vnuml>
Net1 VLAN (e.g, ID=120)
Segmentation— {vm1,vm2,vm3} → host1
— {vm4,vm5} → host2
— Inter-host networks: Net1 (vm3-vm4)
Net2vm5
vm4
vm3
Net0
vm1
vm2
vm5
vm4Net2
vm2
vm3
vm1
Net0
Environment configuration— VLAN creation (e.g., ID 120)
Environment configuration— Inter-host network bridge creation
Per-host VNUML execution
OperationExample
02
22TELEFÓNICA I+D
© 2008 Telefónica Investigación y Desarrollo, S.A. Unipersonal
Segmentation: {vm1,vm2,vm3} → host1, {vm4,vm5} → host2
<vnuml> <net name="Net0" /> <net name="Net1" external="brA" /> <vm name="vm1"> <if id="1" net="Net0"><ipv4>10.0.0.1</ipv4></if> </vm> <vm name="vm2"> <if id="1" net="Net0"><ipv4>10.0.0.2</ipv4></if> </vm> <vm name="vm3"> <if id="1" net="Net0"><ipv4>10.0.0.3</ipv4></if> <if id="2" net="Net1"><ipv4>10.0.1.1</ipv4></if> </vm></vnuml>
Host1Sub-Spec
<vnuml> <net name="Net1" external="brA" /> <net name="Net2" /> <vm name="vm4"> <if id="1" net="Net1"><ipv4>10.0.1.2</ipv4></if> <if id="2" net="Net2"><ipv4>10.0.2.1</ipv4></if> </vm> <vm name="vm5"> <if id="1" net="Net2"><ipv4>10.0.2.2</ipv4></if> </vm></vnuml>
Host2Sub-Spec
<vnuml> <net name="Net0" /> <net name="Net1" /> <net name="Net2" /> <vm name="vm1"> <if id="1" net="Net0"><ipv4>10.0.0.1</ipv4></if> </vm> <vm name="vm2"> <if id="1" net="Net0"><ipv4>10.0.0.2</ipv4></if> </vm> <vm name="vm3"> <if id="1" net="Net0"><ipv4>10.0.0.3</ipv4></if> <if id="2" net="Net1"><ipv4>10.0.1.1</ipv4></if> </vm> <vm name="vm4"> <if id="1" net="Net1"><ipv4>10.0.1.2</ipv4></if> <if id="2" net="Net2"><ipv4>10.0.2.1</ipv4></if> </vm> <vm name="vm5"> <if id="1" net="Net2"><ipv4>10.0.2.2</ipv4></if> </vm></vnuml>
ScenarioSpec
Net1 is inter-host, becauseit connect vms in host1 (vm3)and host 2 (vm4)
OperationVNUML Specification Segmentation
02