towards vm consolidation using idle...
Post on 09-Jun-2020
0 Views
Preview:
TRANSCRIPT
Towards VM Consolidation Using Idle States
Rayman Preet Singh, Tim Brecht, S. Keshav University of Waterloo
@ACM VEE ‘15
1
Traditional VM Consolidation
• Re-package and save
• Power(idle machine) > 50% of peak power [Gandhi ‘09]
Hardware
Hypervisor
VM VM
Hardware
Hypervisor
VM Hardware
Hypervisor
VM VM VM
2
Consolidating Further
• More inactive states
– Frozen [LXC]
– Substrate [Wang ‘11], Fast-resume [Zhang ’11]
Booted
Inactive
VM
VM
VM
VM
3
VM
VM
1
Inactive 2
VM
VM
VM
VM
VM
VM
VM
VM
VM
Example: DreamServer
• Web-hosting • Density improvement: over 46%
• Miss penalty: ~1 sec
Booted
VM Suspended
VM
VM
VM
VM
VM
VM
VM
VM
[Knauth et al. DreamServer: Truly On-Demand Cloud Services. SYSTOR ’14]
4
Question
Goals – Maximize VM density
– Minimize average miss penalties
What policy should we adopt to manage VMs across the different inactive states?
5
Low Duty-cycle Workloads
• High idle times
• Relatively uncorrelated active times
• Only a small fraction simultaneously active – Large fraction inactive
VM 1
VM 2
25 % duty-cycle
6
Low Duty-cycle Workloads
• Notable examples – Web hosting [Knauth ‘14]
– Personal servers [Elsmore ‘12, Mortier ’10]
– Cyber-foraging [Satyanarayanan ’09, Ha ‘13]
App VEE
7
Problem Formulation
VM
VM
VM
Booted
Inactive 1
Inactive i
Inactive N
Bi VM
Ti,0
• Find policy P – Maximize #VMs – Average miss penalty(P) < Limit
8
Policy-based Resource Provisioning
• Multi-level cache management
– Eviction
– Miss penalty = F(hit rates)
– Exclusive caching
9
Disk
Memory
L3
L2
L1
Booted
Frozen
VM
Suspended
VM
VM
VM Hierarchy (LXC) Memory Hierarchy
Policy-based Resource Provisioning
• Page replacement
– Temporal locality
– Reactive (demand-based) vs. Proactive (prefetching)
10
Main Memory
Swap space
Booted
Frozen
VM
Suspended
VM
VM
− VM eviction
− VM active duration
− Writeback
− Pinned page
• Demand-driven
• LRU, NRU, Second-chance, Clock, …
• Optimal policy
– Belady’s MIN optimal demand policy [Aho et al. ’71]
– Unknown for multi-state hierarchy[Gill et al. ’08]
Reactive Policies
Main Memory
Swap space
Booted
Frozen
VM
Suspended
VM
VM
11
Reactive Policies: Lower Bound
• Miss penalty ≥ Ti,0
• Total miss penalty(P, ω) ≥ Σ hi.Ti,0
• Lower bound on Σ hi.Ti,0 – Lower bound on miss penalty
12
VM
Booted
Inactive i Bi
VM
Ti,0
Proactive Policies • Prefetching
• Further reduce #page faults – Optimal: DPMIN [Trivedi et al. ‘76]
13
M Time
Main Memory
Swap space
Proactive Policy: Sliding Window
14
VM
VM
VM
Time
VM Request
B0 B1 B2
VM Idle
Tnext
B0
B1
B2
• Online implementation – Predict Tnext : next arrival per VM – e.g., using ARMA
Measuring Model Parameters
• Model input – Transition times (Ti,j) – State capacities (Bi)
• Experiments – Sensitivity analysis – Density analysis
• Example virtualization solution: LXC – Open source, Mainstream Linux, CCC, Dockr – States: booted, suspended, frozen [Menage et al. ’07]
• Experiment setup – Server machine: 24 cores 3.46 GHz, 128 GB RAM
15
B F S
Measuring Model Parameters
Suspended-to-booted vs. #Booted VMs
16
0
500
1000
1500
2000
2500
3000
t2 t1 t2 t1 t2 t1 t2 t1 t2 t1 t2 t1 t2 t1 t2 t1 t2 t1
Tim
e (m
s)
Number of Booted VMs
Transition to Booted
400350300250200150100500
Measuring Model Parameters
• Frozen-to-booted v/s #Booted VMs – Similar behavior with #frozen VMs
• Identify bottlenecks to LXC density
17
0
10
20
30
40
50
60
0 50 100 150 200 250 300 350 400
Tim
e (m
s)
Number of Booted VMs
Frozen-to-booted (t1)
Model Parameters for LXC
• Mean-value analysis
• Similar analysis for other virtualization solutions
• Stochastic-value analysis
18
How do different policies effect miss penalty? – Reactive vs. Lower bound vs. Proactive
19
Policy Evaluation
• Sample low duty cycle workload: personal servers – Topic of active research [Shakimov et al. ‘11, Elsmore et al. ‘12,
Ha et al. ’13, Singh et al. ‘13, Gupta et al. ’14]
– Request inter-arrivals and durations
• Machine generated requests vs. User-generated
– Periodic data uploads, VISs, cloud-offloading
20
Inter-‐arrival ,me Dura,on
Rela>vely fixed Rela>vely fixed
Stochas-c Rela>vely fixed
Stochas-c Stochas-c
Fixed Inter-arrivals + Fixed Duration
21
0.1
1
10
100
1000
250 350 450 550 650 750 850 950 1050
Av
g M
iss
Pen
alty
(in
ms)
VM density
Suspended-to-booted
Frozen-to-booted
Fixed Inter-arrivals + Fixed Duration
22
0.1
1
10
100
1000
250 350 450 550 650 750 850 950 1050
Av
g M
iss
Pen
alty
(in
ms)
VM density
Suspended-to-booted
Frozen-to-booted MinΦ
(ω) / |ω|
Fixed Inter-arrivals + Fixed Duration
23
0.1
1
10
100
1000
250 350 450 550 650 750 850 950 1050
Av
g M
iss
Pen
alty
(in
ms)
VM density
Suspended-to-booted
Frozen-to-booted MinΦ
(ω) / |ω|LRU
Fixed Inter-arrivals + Fixed Duration
24
0.1
1
10
100
1000
250 350 450 550 650 750 850 950 1050
Av
g M
iss
Pen
alty
(in
ms)
VM density
Suspended-to-booted
Frozen-to-booted MinΦ
(ω) / |ω|LRU
SlidingWindow+ARMASlidingWindow+Ground Truth
Stochastic Inter-arrivals + Stochastic durations
• Datasets: Newton et al. ‘13, Arlitt et al. ’96 25
0.1
1
10
100
1000
250 350 450 550 650 750
Av
g M
iss
Pen
alty
(in
ms)
VM density
Suspended-to-booted
Frozen-to-booted
Stochastic Inter-arrivals + Stochastic durations
• Datasets: Newton et al. ‘13, Arlitt et al. ’96 26
0.1
1
10
100
1000
250 350 450 550 650 750
Av
g M
iss
Pen
alty
(in
ms)
VM density
Suspended-to-booted
Frozen-to-booted MinΦ
(ω) / |ω|
Stochastic Inter-arrivals + Stochastic durations
• Datasets: Newton et al. ‘13, Arlitt et al. ’96 27
0.1
1
10
100
1000
250 350 450 550 650 750
Av
g M
iss
Pen
alty
(in
ms)
VM density
Suspended-to-booted
Frozen-to-booted MinΦ
(ω) / |ω|LRU
Stochastic Inter-arrivals + Stochastic durations
• Datasets: Newton et al. ‘13, Arlitt et al. ’96 28
0.1
1
10
100
1000
250 350 450 550 650 750
Av
g M
iss
Pen
alty
(in
ms)
VM density
Suspended-to-booted
Frozen-to-booted MinΦ
(ω) / |ω|LRU
SlidingWindow+Ground Truth
Stochastic Inter-arrivals + Stochastic durations
• Datasets: Newton et al. ‘13, Arlitt et al. ’96 29
0.1
1
10
100
1000
250 350 450 550 650 750
Av
g M
iss
Pen
alty
(in
ms)
VM density
Suspended-to-booted
Frozen-to-booted MinΦ
(ω) / |ω|LRU
SlidingWindow+Ground TruthSlidingWindow+ARMA
Policy Evaluation
• Proactive vs. Reactive
– If miss penalties are small => use proactive, else reactive
• Proactive works well for relatively predictable workloads – Upto 2.2× density, 1 ms miss penalty
30
Conclusion
• State-based VM consolidation improves VM density – Legacy compatible, can leverage transient idleness
• Imperative to keep miss penalty low – Optimize policies (control plane) and mechanisms
• Future work: lots! – VM heterogeneity—capacity, SLAs, .. – Workloads, bottlenecks, .. – Native integration of inactive states
31
top related