Download - ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network
![Page 1: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/1.jpg)
SHADOWSTREAM: PERFORMANCE EVALUATION AS A CAPABILITY IN PRODUCTION INTERNET LIVE STREAM NETWORK
ACM SIGCOMM 20122012.10.15
CING-YU CHU
![Page 2: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/2.jpg)
MOTIVATION• Live streaming is a major Internet
application today• Evaluation of live streaming
• Lab/testbed, simulation, modeling• Scalability• realism
• Live testing
![Page 3: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/3.jpg)
CHALLENGE• Protection
• Real views’ QoE• Masking failures from real viewers
• Orchestration• Orchestrating desired experimental
scenarios (e.g., flash-crowd)• Without disturbing QoE
![Page 4: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/4.jpg)
MODERN LIVE STREAMING• Complex hybrid systems
• Peer-to-peer network• Content delivery network
• BitTorrent-like• Tracker peers watching same channel
overlay network topology• Basic unit: pieces
![Page 5: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/5.jpg)
MODERN LIVE STREAMING• Modules
• P2P topology management• CDN management• Buffer and playpoint management• Rate allocation• Download/upload scheduling• Viewer-interfaces• Share bottleneck management• Flash-crowd admission control• Network-friendliness
![Page 6: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/6.jpg)
METRICS• Piece missing ratio
• Pieces not received by the playback deadline
• Channel supply ratio• Total bandwidth capacity (CDN+P2P) to
total streaming bandwidth demand
![Page 7: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/7.jpg)
MISLEADING RESULTS SMALL-SCALE• EmuLab: 60 clients vs. 600 clients• Supply ratio
• Small: 1.67• Large: 1.29
• Content bottleneck!
![Page 8: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/8.jpg)
MISLEADING RESULTS SMALL-SCALE• With connection limit
• CDN server’s neighbor connections are exhausted by those clients that join earlier
![Page 9: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/9.jpg)
MISLEADING RESULTS MISSING REALISTIC FEATURE• Network diversity
• Network connectivity• Amount of network resource• Network protocol implementation• Router policy• Background traffic
![Page 10: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/10.jpg)
MISLEADING RESULTS MISSING REALISTIC FEATURE• LAN-like network vs. ADSL-like network
• Hidden buffers• ADSL has larger buffer but limited upload
bandwidth
![Page 11: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/11.jpg)
SYSTEM ARCHITECTURE
![Page 12: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/12.jpg)
STREAMING MACHINE• self-complete set of algorithms to
download and upload pieces• Multiple streaming machines
• experiment (E)• Play buffer
![Page 13: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/13.jpg)
R+E TO MASK FAILURES• Another streaming machine
• For protection• repair (R)
![Page 14: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/14.jpg)
R+E TO MASK FAILURES• Virtual playpoint
• Introducing a slight delay• To hide the failure from real viewers
• R = rCDN• Dedicated CDN resources• Bottleneck
![Page 15: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/15.jpg)
R = PRODUCTION• Production streaming engine
• Fine-tuned algorithms (hybrid architecture)• Larger resource pool• More scalable protection• Serving clients before experiment starts
![Page 16: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/16.jpg)
PROBLEM OFR = PRODUCTION• Systematic bias
• Competition between experiment and production
• Protect QoE higher priority for production underestimate experiment
![Page 17: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/17.jpg)
PCE• R = P + C
• C: CDN (rCDN) with bounded resource• P: production• δ
![Page 18: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/18.jpg)
PCE• rCDN as a filter• It “lowers” the piece missing ratio curve
of experiment visible by production down by δ
![Page 19: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/19.jpg)
IMPLEMENTATION• Modular process for streaming machines• Sliding window to partition downloading
tasks
![Page 20: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/20.jpg)
STREAMING HYPERVISOR• Task window management: sets up
sliding window• Data distribution control: copies data
among streaming machines• Network resource control: bandwidth
scheduling among stream machines• Experiment transition
![Page 21: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/21.jpg)
STREAMING HYPERVISOR
![Page 22: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/22.jpg)
TASK WINDOW MANAGEMENT• Informs a streaming machine about the
pieces that it should download
![Page 23: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/23.jpg)
DATA DISTRIBUTION CONTROL
• Data store• Shared data store• Each streaming machine pointer
![Page 24: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/24.jpg)
NETWORK RESOURCE CONTROL
• Production bears higher priority• LED-BAT to perform bandwidth estimation
• Avoid hidden buffer network congestion
![Page 25: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/25.jpg)
EXPERIMENT ORCHESTRATION• Triggering• Arrival• Experiment Transition• Departure
![Page 26: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/26.jpg)
SPECIFICATION AND TRIGGERING• Testing behavior pattern
• Multiple classes• Each class
• Arrival rate function during interval• Duration function L
• Triggering condition
tstart
![Page 27: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/27.jpg)
ARRIVAL• Independent arrivals to achieve global
arrival pattern• Network-wide common parameters
• tstart, texp and λ(t)• Included in keep-alive message
![Page 28: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/28.jpg)
EXPERIMENT TRANSITION• Current t0, join at ae,i [t0, ae,i]• Connectivity Transition
• Production neighbor’s production (not in test)
• Production rejoins
![Page 29: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/29.jpg)
EXPERIMENT TRANSITION• Playbuffer State Transition
• Legacy removal
![Page 30: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/30.jpg)
DEPARTURE• Early departure
• Capturing client state snapshot• Using disconnection message• Substitution
• Arrival process again
• Only equal or more frequent than the real viewer departure pattern
![Page 31: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/31.jpg)
EVALUATION• Software Framework• Experimental Opportunities• Protection and Accuracy• Experiment Control• Deterministic Replay
![Page 32: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/32.jpg)
SOFTWARE FRAMEWORK• Compositional Run-time
• Block-based architecture
• Total ~8000 lines of code• Flexibility
![Page 33: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/33.jpg)
EXPERIMENTAL OPPORTUNITIES• Real traces from 2 living streaming
testing channel (impossible in testbed)• Flash-crowd• No client departs
![Page 34: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/34.jpg)
PROTECTION AND ACCURACY• EmuLab (weakness)
• Multiple experiment with same settings• 300 clients• δ ~ 4%• Buggy code!
![Page 35: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/35.jpg)
EXPERIMENT CONTROL• Trace-driven simulation• Accuracy of distributed arrivals
• Impact of clock synchronization• Up to 3 seconds
![Page 36: ShadowStream : performance Evaluation as a Capability in Production Internet Live Stream Network](https://reader036.vdocuments.us/reader036/viewer/2022062501/5681637d550346895dd45c53/html5/thumbnails/36.jpg)
DETERMINISTIC REPLAY• Minimize logged data• Hypervisor
• Protocol packet: whole payload• Data packet: only header