1
Joshua Reich, Oren Laadan, Eli Brosh, Alex Sherman, Vishal Misra,
Jason Nieh, and Dan Rubenstein
VMTorrent: Scalable P2P Virtual Machine Streaming
2
• VM: software implementation of computer
• Implementation stored in VM image
• VM runs on VMM– Virtualizes HW
– Accesses image
VM Basics
VMMVMImage
VM
3
Where is Image Stored?
VMMVMImage
VM
4
Traditionally: Local Storage
VMM VMLocalStorage
5
IaaS Cloud: on Network Storage
VMM VMVM
Image
NetworkStorage
6
Can Be Primary
VMM VMNFS/iSCSIVM
Image
NetworkStorage
e.g., OpenStack Glance Amazon EC2/S3
vSphere network storage
7
Or Secondary
VMMVM
Image
VMNetworkStorage Local
Storage
e.g., Amazon EC2/EBSvSphere local
storage
8
Either Way, No Problem Here
VMM VMVM
Image
NetworkStorage
9
Here?
VMImage
NetworkStorage
Bottleneck!
10
Lots of Unique VM Images
NetworkStorage
on EC2 alone
54784 unique images*
*http://thecloudmarket.com/stats#/totals , 06 Dec 2012
11
Unpredictable Demand
NetworkStorage
• Lots of customers
• Spot-pricing
• Cloud-bursting
12
Don’t Just Take My Word• “The challenge for IT teams will be
finding way to deal with the bandwidth strain during peak demand - for instance when hundreds or thousands of users log on to a virtual desktop at the start of the day - while staying within an acceptable budget” 1
• “scale limits are due to simultaneous loading rather than total number of nodes” 2
• Developer proposals to replace or supplement VM launch architecture for greater scalability 3
1. http://www.zdnet.com/why-so-many-businesses-arent-ready-for-virtual-desktops-7000008229/?s_cid=e539
2. http://www.openstack.org/blog/2011/12/openstack-deployments-abound-at-austin-meetup-129
3. https://blueprints.launchpad.net/nova/+spec/xenserver-bittorrent-images
13
Challenge: VM Launch in IaaS
• Minimize delay in VM execution
• Starting from time launch request arrives
• For lots of instances (scale!)
14
Naive Scaling Approaches
• Multicast – Setup, configuration, maintenance, etc.
1
– ACK implosion
– “multicast traffic saturated the CPU on [Etsy] core switches causing all of Etsy to be unreachable“ 2
1. [El-Sayed et al., 2003; Hosseini et al., 2007]2. http://codeascraft.etsy.com/2012/01/23/solr-bittorrent-index-replication
15
Naive Scaling Approaches
• P2P bulk data download (e.g., Bit-Torrent)– Files are big (waste bandwidth)
–Must wait until whole file available (waste time)
– Network primary? Must store GB image in RAM!
16
Both Miss Big Opportunity
VM image access • Sparse• Gradual
• Most of image doesn’t need to be transferred
• Can start w/ just a couple of blocks
17
VMTorrent Contributions
• Architecture
–Make (scalable) streaming possible: Decouple data delivery from presentation
–Make scalable streaming effective: Profile-based image streaming techniques
• Understanding / Validation
–Modeling for VM image streaming
– Prototype & evaluation not highly optimized
18
Talk
• Make (scalable) streaming possible: Decouple data delivery from presentation
• Make scalable streaming effective: Profile-based image streaming techniques
• VMTorrent Prototype & Evaluation
(Modeling along the way)
19
Decoupling Data Delivery from Presentation
(Making Streaming Possible)
20
Generic Virtualization Architecture
VM
VMM Host
Hardware
VM Imag
e
FS
• Virtual Machine Monitor virtualizes hardware
• Conducts I/O to image through file system
21
Cloud Virtualization Architecture
VM
VMM
Hardware
VM Imag
e
FS
• Either to download image
NetworkBacken
d
Network backend used
• Or to access via remote FS
22
CustomFS
VMTorrent Virtualization Architecture
VM
VMM
Hardware
VM Imag
e
FS
NetworkBacken
d
• Divide image into pieces• But provide appearance
of complete image to VMM
• Introduce custom file system
23
CustomFS
Decoupling Delivery from Presentation
VM
VMM
Hardware
NetworkBacken
d
0 1 23 4 56 7 8
VMM attempts to read piece 1Piece 1 is present, read completes
24
CustomFS
VM
VMM
Hardware
NetworkBacken
d
0 1 23 4 56 7 8
VMM attempts to read piece 0Piece 0 isn’t local, read stallsVMM waits for I/O to completeVM stalls
Decoupling Delivery from Presentation
25
CustomFS
VM
VMM
Hardware
NetworkBacken
d
0 1 23 4 56 7 8
FS requests piece from backendBackend requests from network
Decoupling Delivery from Presentation
26
VM
VMM
Hardware
NetworkBacken
d
0
Later, network delivers piece 0
CustomFS
1 23 4 56 7 8
0
Read completesCustom FS receives, updates piece
VMM resumes VM’s execution
Decoupling Delivery from Presentation
27
Decoupling Improves Performance
VM
VMM
Hardware
NetworkBacken
d
CustomFS
1 23 4 56 7 8
0
Primary StorageNo waiting for image download to complete
28
Decoupling Improves Performance
VM
VMM
Hardware
NetworkBacken
d
CustomFS
1 23 4 56 7 8
0
X
X
Secondary StorageNo more writes or re-reads over network w/ remote FS
29
But Doesn’t Scale
Assuming a single server,the time to download a single piece is
t = W + S / (rnet / n)
• W : wait time for first bit • rnet : network speed
• S : piece size• n : # of clients
Transfer time,each client getsrnet / n of server BW
30
Read Time Grows Linearly w/ n
Assuming a single server,the time to download a single piece is
t = W + n * S / rnet
• W : wait time for first bit • rnet : network speed
• S : piece size• n : # of clients
Transfer timelinear w/ n
31
This Scenario
VM
VMM
Hardware
NetworkBacken
d
CustomFS
1 23 4 56 7 8
0
csd
32
Alleviate network storage bottleneck
VM
VMM
Hardware
NetworkBacken
d
CustomFS
1 23 4 56 7 8
0
Decoupling Enables P2P Backend
Swarm
P2PManage
r
1 23 4 56 7 8
0
• Exchange pieces w/ swarmP2P copy must remain pristine
33
VM
VMM
Hardware
CustomFS
1 23 4 56 7 8
0
Space Efficient
Swarm
P2PManage
r
1 23 4 56 7 8
0
FS uses pointers to P2P imageFS does copy-on-write
6 7
34
VM
VMM
Hardware
CustomFS
1 23 4 56 7 8
0
Minimizing Stall Time
Swarm
P2PManage
r
1 23 4 56 7 8
0
Non-local piece accesses
6 7
4?
4?
4!
Trigger high priority requests
35
Now, the time to download a single piece is
t = W(d) + S / rnet
• W(d) : wait time for first bit as function of
• d : piece diversity• rnet : network speed
• S : piece size• n : # of peers
P2P Helps
Wait is function of diversity
Transfer timeindependent of n
36
High Diversity Swarm Efficiency
37
Low Diversity Little Benefit
Nothing to share
38
All peers request same pieces at same time
t = W(d) + S / rnet
Low piece diversity Long wait (gets worse as n grows) Long download times
P2P Helps, But Not Enough
39
VM
VMM
Hardware
CustomFS
1 23 4 56 7 8
0
This Scenario
Swarm
P2PManage
r
1 23 4 56 7 8
0
6 7
p2pd
40
Profile-based Image Streaming Techniques
(Making Streaming Effective)
41
How to Increase Diversity?
Need to fetch pieces that are
• Rare: not yet demanded by many peers
• Useful: likely to be used by some peer
Profiling
• Need useful pieces
• But only small % of VM image accessed
• We need to know which pieces accessed
• Also, when (need later for piece selection)
42
43
Build Profile
• One profile for each VM/workload
• Ran one or more times (even online)
• Use FS to track –Which pieces accessed
–When pieces accessed
• Entries w/ average appearance time, piece index, and frequency
44
Piece Selection
• Want pieces not yet demanded by many
• Don’t know piece distribution in swarm
• Guess others like self
• Gives estimate when pieces likely needed
45
Piece Selection Heuristic
• Randomly (rarest first) pick one of first k pieces in predicted playback window
• fetch w/ medium priority (demand wins)
46
Profile-based Prefetching
• Increases diversity
• Helps even w/ no peers (when ideal access exceeds network rate)
47
Profile-based window-randomized prefetch
t = W(d) + S / rnet
High piece diversity Short wait (shouldn’t grow much w/ n)
Quick piece download
Obtain Full P2P Benefit
48
VM
VMM
Hardware
CustomFS
1 23 4 56 7 8
0
Full VMTorrent Architecture
Swarm
P2P Manager
1 23 4 56 7 8
0
6 7
profile
p2pp
49
Prototype
50
VM
Hardware
CustomFS
1 23 4 56 7 8
0
VMTorrent Prototype
BT Swarm
P2P Manager
1 23 4 56 7 8
0
6 7
profile
Custom CUsing FUSE
Custom C++& Libtorrent
51
Evaluation Setup
52
Testbeds
• Emulab [White, et. al, 2002]
– Instances on 100 dedicated hardware nodes
– 100 Mbps LAN
• VICCI [Peterson, et. al, 2011]
– Instances on 64 vserver hardware node
slices
– 1 Gbps LAN
53
VMs
54
Workloads
• Short VDI-like tasks• Some cpu-intensive, some I/O
intensive
55
• Measured total runtime – Launch through shutdown
– (Easy to measure)
• Normalized against memory-cached execution– Ideal runtime for that set of hardware
– Allows easy cross-comparison • different VM/workload combinations
• Different hardware platforms
Assessment
56
Evaluation
57
100 Mbps Scaling
Starting to increase
58
Due to Decreased Diversity
# peers increases more demand requests to seed less opportunity to build diversity longer to reach max swarming efficiency + lower max
59
Due to Decreased Diversity
# peers increases more demand requests to seed less opportunity to build diversity longer to reach max swarming efficiency + lower max
60
Due to Decreased Diversity
# peers increases more demand requests to seed less opportunity to build diversity longer to reach max swarming efficiency + lower max
We optimized too much for single instance!
(choosing demand requests take precedence)
61
(Some) Future Work
• Piece selection for better diversity
• Improved profiling
• DC-specific optimizations
Current work orders of magnitude better than state-of-art
62
Demo(video omitted for space)
63
See Paper for More Details
• Modeling– Playback process dynamics – Buffering (for prefetch)– Full characterization of r incorporating
impact of centralized and distributed models on W
– Other elided details
• Plus–More architectural discussion!– Lots more experimental results!
64
Summary• Scalable VM launching needed
• VMTorrent addresses by– Decoupling data presentation from streaming
– Profile-based VM image streaming
• Straightforward techniques, implementation, no special optimizations for DC
• Performance much better than state-of-art– Hardware evaluation on multiple testbeds
– As predicted by modeling
