improving the user experience of multimedia streaming services in highly dynamic environments, frank...
DESCRIPTION
VERDIKT conference 2012TRANSCRIPT
![Page 1: Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO](https://reader034.vdocuments.us/reader034/viewer/2022051817/548e33d1b4795927058b49c2/html5/thumbnails/1.jpg)
Improving the user experience of mul4media streaming services in highly dynamic environments
Frank Eliassen ([email protected]) Verdikt conference, 26th April 2012
4/26/2012 ROMUS project @ VERDIKT 1
![Page 2: Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO](https://reader034.vdocuments.us/reader034/viewer/2022051817/548e33d1b4795927058b49c2/html5/thumbnails/2.jpg)
Many problems still hampering Internet live video streaming
• Increased heterogeneity of networks and terminals
• Variability in resource availability such as bandwidth, CPU, and joining and leaving of devices
• Challenge: How to provide each consumer with the best possible viewing experience when considering heterogeneity and variability, while maintaining efficiency and scalability
Car computer
Home PC
GSM /UMTS
The Internet
BT/ WLAN
4/26/2012 ROMUS project @ VERDIKT 2
![Page 3: Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO](https://reader034.vdocuments.us/reader034/viewer/2022051817/548e33d1b4795927058b49c2/html5/thumbnails/3.jpg)
ROMUS project: main objec4ve • To inves4gate and provide solu4ons for mul4media (video) streaming services to provide each consumer best possible experience in a highly dynamic environment – best possible quality (image quality, con4nuous playback)
– least possible visual distor4on
4/26/2012 ROMUS project @ VERDIKT 3
![Page 4: Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO](https://reader034.vdocuments.us/reader034/viewer/2022051817/548e33d1b4795927058b49c2/html5/thumbnails/4.jpg)
ROMUS – three main areas of results • Adapta4on and Robustness in Live P2P Streaming
– Chameleon: a novel adap4ve P2P streaming protocol targe4ng live video streaming (video “broadcasts”)
– S*r: A social network based P2P streaming solu4on to be[er handle peer dynamics in live video streaming
• Video quality assessment – Randomized Pair Comparison (R/PC), a novel test method for
subjec4ve video quality assessment – A set of guidelines to reduce visual distor4on in scalable video
streaming • Mul4core Processing to handle mul4media workloads
– Techniques for exploi4ng mul4core processing and graphical processing units on individual peer nodes to improve video quality
– P2G: a framework for distributed processing on computer nodes in a cluster suppor4ng mul4media workloads with so_ deadlines.
4/26/2012 ROMUS project @ VERDIKT 4
![Page 5: Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO](https://reader034.vdocuments.us/reader034/viewer/2022051817/548e33d1b4795927058b49c2/html5/thumbnails/5.jpg)
ROMUS team • Professor Frank Eliassen, University of Oslo (project leader) • Professor Carsten Griwodz, Simula Research Laboratory • Professor Pål Halvorsen, Simula Research Laboratory • Dr. Viktor S. Wold Eide, University of Oslo (post doc for 18
months) • Dr. Eli Gjørven, University of Oslo (post doc for 6 months) • Anh Tuan Nguyen, University of Oslo (PhD scholar) • Pengpeng Ni, Simula Research Laboratory (PhD scholar) • Håkon Stensland, Simula Research Laboratory (PhD scholar)
4/26/2012 ROMUS project @ VERDIKT 5
![Page 6: Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO](https://reader034.vdocuments.us/reader034/viewer/2022051817/548e33d1b4795927058b49c2/html5/thumbnails/6.jpg)
Adapta4on and Robustness in Live Peer-‐to-‐Peer Streaming
ROMUS project
4/26/2012 ROMUS project @ VERDIKT 6
![Page 7: Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO](https://reader034.vdocuments.us/reader034/viewer/2022051817/548e33d1b4795927058b49c2/html5/thumbnails/7.jpg)
Mo4va4on • Limita4ons of tradi4onal live P2P streaming systems – No differen4ated QoS: users must receive the same stream regardless of their bandwidth (high capacity users perceive the same low quality as average users)
– No con4nuous playback/black block images: with the current best-‐effort Internet and the peer dynamics, the streaming quality at each peer is easily impaired (when the available bandwidth at a peer drops below the streaming rate, it may suffer playback skips)
4/26/2012 ROMUS project @ VERDIKT 7
![Page 8: Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO](https://reader034.vdocuments.us/reader034/viewer/2022051817/548e33d1b4795927058b49c2/html5/thumbnails/8.jpg)
Main hypotheses
• Adaptable coding techniques (such as SVC) can bring significant benefits in terms of differen4ated QoS and con4nuous playback to live P2P streaming
• Network coding and social networking can improve the robustness of the P2P system with respect to network fluctua4ons and peer dynamics
• Quality-‐aware overlays can ensure high capacity peers will receive high quality video
4/26/2012 ROMUS project @ VERDIKT 8
![Page 9: Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO](https://reader034.vdocuments.us/reader034/viewer/2022051817/548e33d1b4795927058b49c2/html5/thumbnails/9.jpg)
Scalable Video Coding (H.264 AVC/SVC)
• A video coding technique: encodes a video into layers of quality
• Standardized in July 2007 by ITU-‐T (H.264)
• ~10% bitrate overhead and an indis4nguishable visual quality compared to H.264 AVC
4/26/2012 ROMUS project @ VERDIKT 9
![Page 10: Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO](https://reader034.vdocuments.us/reader034/viewer/2022051817/548e33d1b4795927058b49c2/html5/thumbnails/10.jpg)
Scalable Video Coding (source: h[p://www.hhi.fraunhofer.de)
The three scalability dimensions “Any” sub-stream can be extracted
4/26/2012 ROMUS project @ VERDIKT 10
![Page 11: Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO](https://reader034.vdocuments.us/reader034/viewer/2022051817/548e33d1b4795927058b49c2/html5/thumbnails/11.jpg)
Network Coding (Linear network coding)
• Instead of simply forwarding data, intermediate nodes may recombine several input packets into one or several output packets
• Perfect collabora4on – Poten4al throughput improvements – A high degree of robustness
4/26/2012 ROMUS project @ VERDIKT 11
![Page 12: Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO](https://reader034.vdocuments.us/reader034/viewer/2022051817/548e33d1b4795927058b49c2/html5/thumbnails/12.jpg)
Chameleon: a pull-‐based P2Pstreaming protocol
Chameleon’s architecture with key components
4/26/2012 ROMUS project @ VERDIKT 12
![Page 13: Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO](https://reader034.vdocuments.us/reader034/viewer/2022051817/548e33d1b4795927058b49c2/html5/thumbnails/13.jpg)
Evalua4on: Baseline • FABALAM: Y. Liu, W. Dou, and Z. Liu, “Layer Alloca4on
Algorithms in Layered Peer-‐to-‐Peer Streaming,” in Proc. of IFIP interna*onal conference on network and parallel compu*ng (NPC), Oct. 2004, pp. 167–174
• Commons – Pull-‐based P2P streaming protocol – Adaptability
• Differences
Chameleon FABALAM
H.264/SVC Synthe4c layered data
Network coding -‐ A layer is delivered from mul4ple senders
Approxima4on algorithm -‐ A layer is delivered from one sender
4/26/2012 ROMUS project @ VERDIKT 13
![Page 14: Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO](https://reader034.vdocuments.us/reader034/viewer/2022051817/548e33d1b4795927058b49c2/html5/thumbnails/14.jpg)
Evalua4on: on scalability
Chameleon is scalable and offers much lower skip rates and higher quality satisfaction
Chameleon vs. FABALAM: Skip rates Chameleon vs. FABALAM: Quality satisfaction
4/26/2012 ROMUS project @ VERDIKT 14
![Page 15: Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO](https://reader034.vdocuments.us/reader034/viewer/2022051817/548e33d1b4795927058b49c2/html5/thumbnails/15.jpg)
Evalua4on: on peer dynamics
Using weibull(k,2) for generation of different levels of peer dynamic Skip rates
Quality satisfaction
Chameleon can adapt well to peer dynamics to maintain low skip rates and high quality satisfaction
4/26/2012 ROMUS project @ VERDIKT 15
![Page 16: Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO](https://reader034.vdocuments.us/reader034/viewer/2022051817/548e33d1b4795927058b49c2/html5/thumbnails/16.jpg)
Video Quality Assessment
Flicker effects in Adap4ve Video Streaming
4/26/2012 ROMUS project @ VERDIKT 16
ROMUS project
![Page 17: Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO](https://reader034.vdocuments.us/reader034/viewer/2022051817/548e33d1b4795927058b49c2/html5/thumbnails/17.jpg)
Quality adapta4on mechanism in Chameleon at work
What are acceptable limits of quality fluctuations for the user? Can we provide guidelines for how to adapt to reduce visual distortion?
4/26/2012 ROMUS project @ VERDIKT 17
![Page 18: Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO](https://reader034.vdocuments.us/reader034/viewer/2022051817/548e33d1b4795927058b49c2/html5/thumbnails/18.jpg)
Media Performance Group
Visual perception of dynamically adaptive video (1)!Understanding and using limits of user perception and perceived quality"
Signal-‐to-‐noise ra4o (SNR) scaling Noise flicker
Resolu4on scaling Blur flicker
Frame rate scaling Judder
Three main types of visual artifacts"
![Page 19: Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO](https://reader034.vdocuments.us/reader034/viewer/2022051817/548e33d1b4795927058b49c2/html5/thumbnails/19.jpg)
Media Performance Group
Visual Perception of dynamically adaptive video (2)
Encoding Layers
Frames over Time
High Frequency
Low Frequency
Two main fluctuation factors"
Amplitude" Frequency"
Field study"• mobile devices, free seating, resolution 480x320@30fps, no sunlight,
lounge chairs"Experiment design"• repeated measures, single-stimulus, randomized block design"• blocking by flicker type and amplitude level"• baselines for highest and lowest quality without quality fluctuations"
![Page 20: Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO](https://reader034.vdocuments.us/reader034/viewer/2022051817/548e33d1b4795927058b49c2/html5/thumbnails/20.jpg)
Media Performance Group
Visual Perception of dynamically adaptive video (3)!
Three influential factors"
Amplitude"Most dominant effect"Flicker is almost undetectable at amplitudes < 8QP and almost always detectable for larger amplitudes"
Frequency"Major effect"Acceptance thresholds compared to constant low quality video:"worse when above 1 Hz,"often better when below 0.5 Hz""
Content"Minor effect"But: content can influence"flicker perception;"low interaction for noise flicker and stronger for blur flicker""
![Page 21: Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO](https://reader034.vdocuments.us/reader034/viewer/2022051817/548e33d1b4795927058b49c2/html5/thumbnails/21.jpg)
P2G: Parallel Processing Graphs
A Framework for Distributed Real-‐Time Processing of Mul*media Data
4/26/2012 ROMUS project @ VERDIKT 21
ROMUS project
![Page 22: Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO](https://reader034.vdocuments.us/reader034/viewer/2022051817/548e33d1b4795927058b49c2/html5/thumbnails/22.jpg)
Mo4va4on • The poten4al to use mul4core processing and new heterogeneous technology (e.g., GPUs) to handle mul4media workloads on individual peer nodes to meet new demands
• Challenge: difficult to program because of heterogeneous architectures (e.g., data parallelism vs. thread parallelism)
• Exis4ng frameworks have all some short-‐comings; do not meet all requirements of mul4media data processing
4/26/2012 ROMUS project @ VERDIKT 22
![Page 23: Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO](https://reader034.vdocuments.us/reader034/viewer/2022051817/548e33d1b4795927058b49c2/html5/thumbnails/23.jpg)
P2G main features • The P2G framework allows developers to
express continuous multimedia workloads with fine grained parallelism in a single language
• The runtime decides itself how a program should be partitioned, and which execution node should execute them
• Open source at http://p2gproject.org 4/26/2012 ROMUS project @ VERDIKT 23
![Page 24: Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO](https://reader034.vdocuments.us/reader034/viewer/2022051817/548e33d1b4795927058b49c2/html5/thumbnails/24.jpg)
Mo4on JPEG Experiment
• Continuous workload, CIF resolution • DCT consumes most of the encoding time.
4/26/2012 ROMUS project @ VERDIKT 24
![Page 25: Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO](https://reader034.vdocuments.us/reader034/viewer/2022051817/548e33d1b4795927058b49c2/html5/thumbnails/25.jpg)
Mo4on JPEG Results
21
11
8 7 7
6 6 5
0
2
4
6
8
10
12
14
16
18
20
22
1 2 3 4 5 6 7 8
4-‐way Core i7 30
15
10
8
6 5 5 5
0
2
4
6
8
10
12
14
16
18
20
22
24
26
28
30
1 2 3 4 5 6 7 8
8-‐way Opteron
Threads Threads
Tim
e (s
)
Tim
e (s
)
4/26/2012 ROMUS project @ VERDIKT 25
![Page 26: Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO](https://reader034.vdocuments.us/reader034/viewer/2022051817/548e33d1b4795927058b49c2/html5/thumbnails/26.jpg)
Conclusions (1) • The development of a simulation model to simulate and
evaluate novel adaptive and robust live P2P video streaming solutions is essentially important for long-term research in the field
• It is essential to find solutions to important limitations of live P2P streaming technologies. Our basic solutions and findings could be inherited by other initiatives to build a more practical protocol taking other network metrics into account.
• Tools and guidelines for how to design real time video streaming infrastructure adopting SVC techniques is crucial for being able to provide the best possible user experience with minimal visual distortion.
4/26/2012 ROMUS project @ VERDIKT 26
![Page 27: Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO](https://reader034.vdocuments.us/reader034/viewer/2022051817/548e33d1b4795927058b49c2/html5/thumbnails/27.jpg)
Conclusions (2) • Our results indicate that quality adaptation can
outperform constant low quality, but frequency and amplitude, as well as switching patterns are relevant
• Multicore processor scheduling to handle multimedia workloads on individual peer nodes will be important in the future and may improve the multimedia experience of the user even further.
• Our results shows the feasibility of providing a programming framework for automatic parallel, real-time processing of multimedia workloads exploiting heterogeneous multicore processors.
4/26/2012 ROMUS project @ VERDIKT 27
![Page 28: Improving the user experience of multimedia streaming services in highly dynamic environments, Frank Eliassen, UiO](https://reader034.vdocuments.us/reader034/viewer/2022051817/548e33d1b4795927058b49c2/html5/thumbnails/28.jpg)
Thank you!
Q&A
4/26/2012 ROMUS project @ VERDIKT 28