Quality of Experience for Inter-Destination Media Synchronization

Christian Timmerer

Alpen-Adria-Universität Klagenfurt (AAU) w Faculty of Technical Sciences (TEWI) w Department of Information Technology (ITEC) w Multimedia Communication (MMC) w Sensory Experience Lab (SELab)

@timse7 w http://blog.timmerer.com w http://selab.itec.aau.at/ mailto:christian.timmerer@itec.uni-klu.ac.at

September  20,  2014  

Acknowledgments.  This  work  was  supported  in  part  by  the  EC  in  the  context  of  the  SocialSensor  (FP7-­‐ICT-­‐287975)  and  QUALINET  (COST  IC  1003)  projects  and  partly  performed  in  the  Lakeside  Labs  research  cluster  at  AAU.  

Outline

•  Introduction •  Quality of Experience •  Inter-Destination Media Synchronization •  IDMS Building Blocks •  Adaptive Media Playout & QoE Impact •  Conclusions

Sep.  20,  2014   QoMEX  2014,  Singapore   2  

Multimedia is Predominant on the Internet

•  Real-time entertainment –  Streaming video and

audio –  More than 50% of Internet

traffic at peak periods •  Popular services

–  YouTube (13.2%), Netflix (34.2%), Amazon Video (1.9%), Hulu (1.7%)

–  All delivered over-the-top (OTT)

–  MPEG Dynamic Adaptive Streaming over HTTP

Sep.  20,  2014   QoMEX  2014,  Singapore   3  

Global  Internet  Phenomena  Report:  1H  2014    

Quality of Experience •  Quality of Experience

–  “… is the degree of delight or annoyance of the user of an application or service…”

–  Factors influencing / features of QoE may lead to application-specific definitions

•  Subjective quality assessments –  Laboratory [ITU-T B.500 / P.910] –  Crowdsourcing with special platforms or social

networks •  QoE of DASH-based services

–  Startup delay (low) –  Buffer underrun / stalls (zero) –  Quality switches (low) and media throughput (high)

Sep.  20,  2014   QoMEX  2014,  Singapore   4  

P.  Le  Callet,  S.  Möller  and  A.  Perkis,  eds.,  “Qualinet  White  Paper  on  Defini_ons  of  Quality  of  Experience  (2012)”,  European  Network  on  Quality  of  Experience  in  Mul6media  Systems  and  Services  (COST  Ac6on  IC  1003),  Lausanne,  Switzerland,  Version  1.2,  March  2013."  

Inter-Destination Media Synchronization •  IDMS == the playout of media streams at two or more geographically distributed

locations in a time synchronized manner [draft-ietf-avtcore-idms-13] •  Use case: two friends watching football using a online platform and

communicating via real-time communication channel –  A: Goooooaaaaalllll! Have you seen this? –  B: Whoaaat? No, here they’re still preparing for the free kick, thanks for the spoiler, dude!

Sep.  20,  2014   QoMEX  2014,  Singapore   5  

Live  Sport  Events    -­‐  Olympic  Games    -­‐  NFL  Superbowl    -­‐  FIFA  Worldcup    -­‐  …  

Satellite  Broadcast  

(Semi-­‐)Professional  &  

Amateur  Content  AcquisiPon  

Consumer  

Consumer  

Consumer  

Heterogeneous  Broadband  Networks  (wired,  wireless)  

Geographically  distributed  consumers  accessing  the  same  content  while  communicaPng  among  them.  

Synchron

ized

 Playout  

Online  Community  

C.  Timmerer,  B.  Rainer,  "The  Social  Mul_media  Experience,"  IEEE  Computer,  vol.  47,  no.  3,  pp.  67-­‐69,  Mar.,  2014    

IDMS Building Blocks

•  Building blocks –  Session management –  Identify the synchronization point and threshold

of asynchronism –  Signal timing and control information among the

participating entities –  Adapt the media playout to establish or restore

synchronism •  IDMS schemes –  Server/client (aka master/slave, MS) –  Synchronization maestro scheme (SMS) –  Distributed control scheme (DCS)

Sep.  20,  2014   QoMEX  2014,  Singapore   6  

Adaptive Media Playout •  Initially introduced for compensating the impact

of error prone communication channels on the smoothness of the multimedia playout to avoid buffer under-/overruns

•  Static, simple, naïve approach –  Skip/pause content sections –  Easy to implement, non-negligible QoE impact

•  Dynamic Adaptive Media Playout (AMP) –  Dynamically increase/decrease the playout rate for

certain content sections –  Find appropriate content sections where the media

playout rate can be modified without significant impact on the QoE

Sep.  20,  2014   QoMEX  2014,  Singapore   7  

Demo

Sep.  20,  2014   QoMEX  2014,  Singapore   8  

QoE for IDMS •  Increasing/decreasing media playout rate ➪perceptual distortion in audio and/or video

•  Select appropriate metrics, e.g.: –  Audio: the spectral energy of an audio frame –  Video: the average length of motion vectors

between two consecutive frames •  How do metrics correlate with QoE? Find out… ➪Subjective quality assessments (w/ crowdsourcing)

•  Define a utility model and incorporate into the media client to carry out the IDMS

Sep.  20,  2014   QoMEX  2014,  Singapore   9  

B.  Rainer,  C.  Timmerer,  "A  Quality  of  Experience  Model  for  Adap_ve  Media  Playout”,  In  Proceedings  of  QoMEX  2014,  Singapore,  Sep  2014.    B.  Rainer,  C.  Timmerer,  "Self-­‐Organized  Inter-­‐Des_na_on  Mul_media  Synchroniza_on  for  Adap_ve  Media  Streaming”,  accepted  for  publica6on  in  ACM  Mul6media  2014,  Orlando,  Florida,  Nov.  2014.  

Conclusions

•  QUALINET white paper on QoE definitions – Generally agreed definition of QoE –  Factors influencing / features of QoE

•  App-specific Quality of Experience –  Identify those QoE factors/features – Derive a utility/QoE model – Validate through subjective tests

•  IDMS is an interesting application area for a broad range of QoE topics –  Everyone is invited – get involved in and

excited about IDMS!

Sep.  20,  2014   QoMEX  2014,  Singapore   10  

Thank you for your attention

... questions, comments, etc. are welcome …

Priv.-Doz. Dipl.-Ing. Dr. Christian Timmerer Associate Professor

Klagenfurt University, Department of Information Technology (ITEC) Universitätsstrasse 65-67, A-9020 Klagenfurt, AUSTRIA

christian.timmerer@itec.uni-klu.ac.at http://blog.timmerer.com/

Tel: +43/463/2700 3621 Fax: +43/463/2700 3699 © Copyright: Christian Timmerer

11  Sep.  20,  2014   QoMEX  2014,  Singapore