performance differentiation of network i/o in xen
DESCRIPTION
by Kuriakose Mathew (08305062) under the supervision of Prof. Purushottam Kulkarni and Prof. Varsha Apte. PERFORMANCE DIFFERENTIATION OF NETWORK I/O in XEN. Outline. Introduction Need for Network I/O differentiation in Xen - PowerPoint PPT PresentationTRANSCRIPT
1
PERFORMANCE DIFFERENTIATION OF NETWORK
I/O in XENby
Kuriakose Mathew(08305062)
under the supervision
of
Prof. Purushottam Kulkarni
and
Prof. Varsha Apte
2
Outline
Introduction
Need for Network I/O differentiation in Xen
Implementation of the Tool for Network I/O configuration in Xen
Validation of the Tool
Conclusion and Future work
3
Virtualization
Virtualization – Abstraction of resources
Single OS systemVirtualized system
Virtualization[http://software.intel.com]
5
Xen- Open Source VMM
VMM
Hardware
Dom-UDom-0
Xen Architecture [David Chisnall.The Definitive Guide]
6
Performance Differentiation in Xen
CPU credit scheduler – Weights – share of the
physical CPU time that the domain gets
– Caps – maximum physical CPU time that the domain gets
– Weights – relative eg 128:256 256:512
– Caps – absolute 20% 50%– Eg xm sched-credit –d
<domain_name> -w 256 –c 20
Tools exist for configuration
No means of configurationno weighted schedulingProvides fair scheduling
7
Network I/O Differentiation in Xen
With existing methods in Xen, network bandwidth utilization cannot be controlled
25% CPU, 2MBps Domain 135% CPU, 2MBps Domain 2
Dom1 – 20% CPU limitDom2 – 80% CPU limit
25% CPU, 2MBps Domain 1 35% CPU, 5MBps Domain 2
Dom1 – 20% CPU limitDom2 – 80% CPU limit
Need for a separate control mechanism for Network I/O Differentiation
8
Previous Work at IIT B
DDP 08-09 work proposed and implemented a weighted Network I/O scheduler providing bandwidth limits and guarantees.
Implemented bandwidth limits and guarantees for VMs in Xen
limits – Maximum bandwidth usage for a VM guarantees – Amount of available bandwidth that a
VM is guaranteed
9
Short Comings of Previous Work
Shortcomings Hard coding of values in Netback driver
Recompilation of kernel for change of parameter
Lack of dynamic configuration tool Difficulty in doing experimentation and validation
Issues with interference of CPU scheduler Complex sharing of credit values
10
Overall MTP Goal
Design and implement a tool for specification and dynamic reconfiguration of limits for Network I/O in Xen
Simplification of the existing algorithm to provide bandwidth guarantees to Network I/O
Study the interference of Xen CPU scheduler with the Network I/O configuration
Validation with realistic I/O intensive applications
11
High-level Specification of the Tool
Dynamically specify the bandwidth usage limits and guarantees of a virtual machine
Usage
Xmsetbw –d <domain_name> -g <guarantee> -l <limits>
domain_name - name of the domain whose bandwidth parameters need to be set
guarantee – the guarantee in bandwidth provided to the domain
limit – the maximum limit in bandwidth usage for the domain
12
Xen Device Driver Model
Front-end Back-end
Xen device driver model [David Chisnall. The Definitive Guide]
Tool input
13
Packet Transmission in Xen
Packet Transmission/Reception [Sriram Govindan. Xen and co.: Communication-aware CPU Scheduling]
Xenstore
14
Xenstore and Xenbus
Xenstore Exchange out of band information Hierarchical filesystem like database used for
sharing small amount of info between domains Contains 3 main paths
/vm /local/domain /tools
16
Xenbus(cont.)
Provide APIs for reading and writing data Support Transactions for group execution of
operations Watches
Associated with a key in xenstore Change in value causes a call-back to be triggered User-kernel interaction register_xenbus_watch register_xenstore_notifier
17
Implementation of Tool for Dynamic Network Configuration
Userspace tool Domain Name
Domain id
Xenstore
Format string and write to
predefined-path
Write
User inputCommand
18
Implementation of Tool (cont.)
Register initialCall-back
Register pre-defined
path callback
Read pathUpdate variable
Read initial value
If structures modified
Xenstore write
Read new value
Transmit packetsArray of
structures and flags
Xenstore
Register Call-back1
RegisterCall-back2
Call-back1
Call-back2
Start-up
Y N
XENBUS NETBACK
19
Modification in Netback driver
• Credit scheduling in the netback driver
• By default, network scheduler assigns equal credits
• Fair sharing of bandwidth
• Modified to assign weighted credits
Network Scheduler[ddp]
20
Can the bandwidth limit for each domain be set for step increase in
bandwidth limit?
C1- All doms 1MBps
C2- dom4 2MBps
C3 - dom3 2MBps dom4 3MBps
C4 - dom2 2MBps dom3 3MBps dom4 4MBps
Bandwidth is limited to within 96.4%
0.96
1.92
2.91
3.87
21
Can the bandwidth limit for each domain be set for step decrease in
bandwidth limit?
C1 –dom1 4 MBps dom2 3MBps
dom3 2MBps dom4 1MBps
C2 –dom1,2 3MBps dom3 2MBps
dom4 1MBps
C3- dom1,2,3 2MBpsdom4 1MBps
C4- All doms 1MBps
Bandwidth is limited to
within 96.4%
3.87
2.91
1.92
0.96
22
How is the bandwidth shared for domains which are not bandwidth
limited?
C1 –No limit set
C2 –dom5 1MBps
C3- dom5 2MBps
Bandwidth is fairly shared for domains that are not bandwidth
limited
0.95
1.93
23
How does the bandwidth change with time for a step change in limit for a
single domain?
Bandwidth settles in 6.7 sec (on an avg) for a step change
24
How does the bandwidth change with time for a step change in limit for
multiple domains?
Bandwidth limits set for multiple domains
independently
25
How does the CPU utilization in DomU and Dom-0 vary for different
bandwidth limits for a DomU?
DomU utilization increases proportionallyWhile Dom-0 utilization does not
1.6
54.5
3.2
144.7
4.7
169.6
6.7 8.1 9.7 11.4 12.4
182.3 190.3 193.4 195.2206.6
26
Summary of Work Done
Implemented a tool for specification and dynamic reconfiguration of bandwidth limits for VMs in Xen
Modified the Netback driver of Xen to provide bandwidth limits
Experimentation and Validation to verify the bandwidth limits configuration
27
Implementation Problems
Compilation, Installation & Debugging of Xen and domains
Code Exploration to understand Xen network I/O model.
Not much documentation available Xenstore generally used for guest domain to domain-0
Direct calling of xenbus API in Netback were giving problems
2 Levels of call-back needed
28
Future Work
Simplification of the algorithm to implement the bandwidth guarantee
Study the effect of CPU scheduler on the bandwidth limits and guarantee
Validation with realistic I/O intensive applications