adaptive media streaming: the role of standards

AdaptiveMediaStreaming:TheRoleofStandardsDynamicAdaptiveStreamingoverHTTP

Priv.-Doz.Dr.ChristianTimmerer[Ack:AliC.Begen,MediaMelon,Inc.,OzyeginUniversity]

Alpen-Adria-UniversitätKlagenfurt(AAU)w FacultyofTechnicalSciences (TEWI) w DepartmentofInformationTechnology(ITEC)wMultimediaCommunication(MMC)w SensoryExperience Lab(SELab)

http://blog.timmerer.com w http://selab.itec.aau.at/ w http://dash.itec.aau.at w [email protected](CIO)atbitmovinGmbH

http://www.bitmovin.comw [email protected]

Tutorial@QoMEX 2016,June2016

http://www.slideshare.net/christian.timmerer

ImportanceofMultimediaDelivery• Multimediaispredominanton

theInternet• Real-timeentertainment

– Streamingvideoandaudio– Morethan70%ofInternettraffic

atpeakperiods• Popularservices

– YouTube(17.85%),Netflix(37.05%),AmazonVideo(3.11%),Hulu (2.58%)

– Alldeliveredover-the-top (OTT)

June2016 QoMEX2016Tutorial 2GlobalInternetPhenomenaReport:Dec2015

OpenDigitalMediaValueChain

June2016 QoMEX2016Tutorial 3

CreateContent

AggregateMonetize

DistributeContent

ConsumeContent

AnyContent AnyStorefront AnyNetwork AnyDevice

CDNsMediaProtocols

InternetTransport

DRMEncoding

EncapsulationDynamicAds

Clients

HappyUser

WhatisStreaming?


Streamingistransmissionofacontinuouscontentfromaservertoaclientanditssimultaneousconsumptionbytheclient

TwoMainCharacteristics1. Clientconsumption ratemaybelimitedbyreal-timeconstraintsasopposed tojust

bandwidth availability2. Servertransmissionrate(looselyortightly)matchestoclientconsumptionrate

CommonAnnoyancesinStreaming• Wrongformat• Wrongprotocol• Pluginrequirements• DRMissues• Longstart-updelay• Poorquality• Frequentstalls• Qualityoscillations• Noseekingfeatures


Over-The-Top– AdaptiveMediaStreaming• Inanutshell…


Adaptation logic is within the client, not normatively

specified by the standard, subject to research and

development

Multi-BitrateEncodingandRepresentationSwitching


Contents ontheWebServer

MovieA– 200Kbps

MovieA– 400Kbps

MovieA– 1.2Mbps

MovieA– 2.2Mbps

...

...

MovieK– 200Kbps

MovieK– 500Kbps

MovieK– 1.1Mbps

MovieK– 1.8Mbps

.. .

.. .

Time(s)

Startquickly

Keep requestingImprove quality

Loss/congestiondetection

Revampquality

...

...Segments

AdaptiveStreamingoverHTTP


…………

HTTP GETs

ClientBuffer

MediaPlayer

HTTPServer

FormatsandStandards• Adobe

– HTTPDynamicStreaming (HDS)– Almostdead

• Apple– HTTPLiveStreaming (HLS)– RequiredforiOS

• Microsoft– SmoothStreaming– Almostdead, too!Ornot?

• MPEGDynamicAdaptiveStreamingoverHTTP(DASH)– Supported byNetflix,YouTube,Bitmovin,etc.


Source: http://xkcd.com/927/

ScopeofDASH:whatisspecified?


Media Presentation on HTTP Server

DASH-enabled ClientMedia Presentation Description

.

.

.

Segment

…

.

.

.Segment

…

.

.

.

Segment

…

.

.

.Segment

…

…

Segments located by HTTP-URLs

DASH Control Engine

HTTP/1.1 HTTP Client

MPD Parser

Media Engine

On-time HTTP requests to segments

DASHDataModel


MPD

Period id=1start=0s

Period id=3start=300s



AdaptationSet0subtitleturkish

AdaptationSet2audioenglish

Adaptation Set1BaseURL=http://abr.rocks.com/

Representation2Rate=1Mbps


Representation1Rate=500Kbps

Representation 3Rate=2Mbps

Resolution =720p

SegmentInfoDuration=10s

Template:3/$Number$.mp4

Segment Access

InitializationSegmenthttp://abr.rocks.com/3/0.mp4

MediaSegment1start=0s

http://abr.rocks.com/3/1.mp4

MediaSegment2start=10s

http://abr.rocks.com/3/2.mp4

AdaptationSet3audiogerman

AdaptationSet1video



Selectionofcomponents/tracks

Well-definedmediaformat

Selectionofrepresentations

Splicingofarbitrarycontentlikeads

Chunkswithaddressesandtiming


type=static typically,for on demand content

Base URL of the segments

Subtitles

Audio adaptation set with different representations (bw)

Video adaptation set with different representations (bw)

Different codecs (profiles)

Segment URL constructed with template and base URL

http://www.dash-player.com/demo/


OStandard,WhereArtThou?WhatisaStandard?http://en.wikipedia.org/wiki/Standards

Atechnicalstandardisanestablishednormorrequirement.Itisusuallyaformaldocumentthatestablishes uniform engineeringortechnicalcriteria,methods,processesandpractices.

è I N T E R O P E R A B I L I T Y

TypesofStandard• Openstandards

– ITU-T,W3C,IETF,MPEG,Linux,…• Closedstandards

– MSMediaPlayer,…

• Dejurestandards– ANSI,ISO,DIN,…

• Defactostandards– MSWindows,OSX,Unix,Linux,…

• Mandatorystandards– Health,radiation,…

• Voluntarystandards– JPEG,…


AdaptiveStreamingContentWorkflow


Source Transcoding Encapsulation Encryption OriginServer

HelperDistribution

Client

Linear:MulticastVoD:FTP,RTMP,HTTP,etc.

UnicastHTTP(PUSH),FTP,etc.

HTTPGETsmallobjects

Singlehighest-bitratestream Multiplestreamsat

targetbitrates

Multiplestreamsattargetencapsulationformats

Largevideo/virtualfilesandmanifests

AdaptiveStreamingContentWorkflowSimplified


StandardDeliveryInfrastructure(CDN)Source Transcoding Encapsulation Encryption

Multiplestreams:video[bitrate(32000-20000000),profile(baseline,main,high),preset(standard,professional,premium),height(128-7680),width(96-4320),framerate(1-120),codec(h264,hevc)],audio:[bitrate(8000-256000),samplerate(0,8000,11025,12000,16000,22050,24000,32000,44100,48000,64000,88200,96000)]

Singlehighest-bitratestream:HTTP,FTP,RTMP;mp4,ts;AVC,AAC,Subtitles

Multiplestreamsattargetencapsulationformats:DASH(MPD+mp4),HLS(m3u8,ts)

MultiplestreamswithmultipleDRMformats:MPEG-CENC,Widewine,PlayReady,PrimeTime,Fairplay

Player

HeterogeneousClientse.g.BitmovinHTML5AdaptivePlayerDASH,HLS,HTML5,MSE,EME

QoEforDASHServices• DifferentapplicationdomainshavedifferentQoE

requirements– NeedtoprovidespecializationsofthegeneralQoE

definition– Takeintoaccountrequirements formulatedbymeansof

influence factorsandfeaturesofQoE• QoEinfluencefactorsforDASH

– Initial/start-updelay(low)– Bufferunderruns, stalls,freezes(zero)– Qualityswitches(low)– Mediathroughput (high)– …


!

http://www.bitcodin.com/

http://www.dash-player.com/

QoEEvaluationforDASH-basedServices• Testsequence

– Manydatasetsavailable– AdoptedBigBuckBunny&DASHed itwithbitcodin

• Players– bitdash– Proprietarysolutions(smooth,HLS,HDS)– YouTube,dash.js,DASH-JS– …andcompareitwithtendifferentadaptationalgorithms

• Objectiveevaluation– Commontestsetupusingnetworkemulation&bandwidthshaping– Predefinedbandwidthtrajectory(orrealnetworktraces)

• Subjectiveevaluation– Lab[ITU-TB.500/P.910]vs.

crowdsourcingwithspecialplatformsorsocialnetworks

June2016 QoMEX 2016Tutorial 19

CrowdsourcedQoEEvaluation• QualityofExperience …

– MeanOpinionScore[0..100]– [otherobjectivemetrics:

start-uptime,throughput,stalls]• …Web-basedAdaptiveHTTPStreamingClients…

– HTML5+MSE: DASH-JS (dash.itec.aau.at), dash.js (DASH-IF,v1.1.2),YouTube• …Real-WorldEnvironments…

– DASH-JS,dash.js hostedatITEC/AAU (~10Gbit/s)– YouTubehostedatGoogledatacenters– Content:TearsofSteel @144p(250kbit/s),240p(380kbit/s),360p(740kbit/s),480p(1308kbit/s),and

720p(2300kbit/s);segment size:2s– Usersaccess contentovertheopenInternet (i.e.,real-worldenvironment)

• …Crowdsourcing– CampaignatMicroworker platform(othersalsopossible:Mechanical Turk,socialnetworks) limitedto

Europe,USA/Canada,India– ScreeningTechniques: Browserfingerprinting,stimuluspresentationtime, QoEratingsandpre-

questionnaire

June2016 QoMEX2016Tutorial 20B. Rainer, C. Timmerer, “Quality of Experience of Web-based Adaptive HTTP Streaming Clients in Real-World Environments using Crowdsourcing”, Proceedings of International Workshop on VideoNext: Design, Quality and Deployment of Adaptive Video Streaming, Sydney, Australia, Dec. 2014.

MOS andAverageBitrate

• 288microworkers,33screened(Fingerprinting: 20,presentationtime:6,QoEratingsandpre- questionnaire: 7),175maleand80female,majority(80%)isagedbetween18and37


StartupTimeandNumberofStalls


ResultsSummary• DASH-JS

– Highstart-uptime– Lownumberofstalls– Goodthroughput,QoE

• dash.js– Lowstart-uptime– High#stalls– Lowthroughput– LowQoE

• YouTube– Lowstart-uptime– Lownumberofstalls– Bestthroughput,QoE


Now,10differentadaptationlogics…


Adaptationlogicswell-known inresearchliterature

Predefinedbandwidthtrajectoryandtestsetup

Differentsegmentsizes,RTTs,HTTP/2,etc.

C. Timmerer, M. Maeiro, B. Rainer, “Which Adaption Logic? An Objective and Subjective Performance Evaluation of HTTP-based Adaptive Media Streaming Systems”, arXiv cs.MM, June 2016, http://arxiv.org/abs/1606.00341.

DASH-JSvs.bitdash


C. Timmerer, D. Weinberger, C. Mueller, and S. Lederer, “Ultra-High-Definition-Quality of Experience with MPEG-DASH”, Proceedings of the Broadcast Engineering Conference (BEC), NAB2015, Las Vegas, NV, USA, April 2015.

ObjectiveEvaluations


Stalls(lowerisbetter)AverageBitrate(higherisbetter)

Stallsarereallybad…


Conviva:ViewerExperienceReport.2014

Conclusions(1)• MPEG-DASHdefinesformatsonly

– MediaPresentationDescription (MPD)– Segment format:mp4, ts

• MPEG-DASHisnot– System,protocol,presentation,codec,interactivity,DRM,clientspecification– Otherstandardsrequired foracompleteecosystem:e.g.,DASH-IF,WAVE,

HMTL5,MSE,EME• DoweneedMPEG-DASH?(foradaptivemediastreaming)

– Notnecessarily:e.g.,WebM +VPx +manifest&controlend-to-end– Requiredtoaddressheterogeneous environments

• Roleofstandardssometimesoverrated butoftenunderestimated


Conclusions(2)• QoEforDASH-basedservices(aruleofthumb)

– Startupdelay(low[butlivevs.on-demand&shortvs.long-tailcontent])

– Bufferunderrun /stalls(zero)– Qualityswitches(low)andmediathroughput(high)– Energy- andcost-awareness(dataplan)

• NogeneralapplicableQoEmodelforDASH– (Too)manyfactorsinfluencing/featuresofQoEforDASH-based

services– Methodologyforreproducibleresearchisinplaceandwellestablished– Ampleresearchopportunities


MainQoEfactorsforDASH

➡ ComeupwithourownQoEfactoranddesign, conduct,analyzeasmall-scaleexperiment