building media streaming apps and sites without plug-ins using mpeg-dash
Post on 25-Feb-2016
89 Views
Preview:
DESCRIPTION
TRANSCRIPT
Building media streaming apps and sites without plug-ins using MPEG-DASHDaniel SchneiderSenior Lead Program Manager, Microsoft Anthony ParkVice President, Engineering - Netflix 3-089
Key questions this talk aims to answer• What is MPEG-DASH and how does it help you to deliver
a better video experience to your customers at a lower cost?
• How does Windows enable video services to build MPEG-DASH clients?
• Which tools are available to quickly get started developing an MPEG-DASH service on Windows?
Agenda• Introduction• The MPEG-DASH Standard• W3C Media Source Extensions• W3C Encrypted Media Extensions• Implementing an MPEG-DASH service in IE or a
Windows Store app using JavaScript and HTML• Summary
Why standardize video APIs in browsers?• Customers expect to be able to watch Netflix on
any operating system or browser• Common playback APIs reduce the burden of
developers to have different implementations on different platforms
• Custom adaptive streaming algorithms can be written once in JavaScript and leveraged everywhere
Demo: Netflix in IE11 Previewon Windows 8.1 Preview
The Future of Netflix on Windows• HTML5 Video in the Web Browser
• Supports multi-tasking scenarios, multi-tab, etc.
• Windows Store App• Touch-first design, great for tablet and touch-screen hardware• Provides deep integration with platform features (share contracts, search
integration, etc.)• Rich app experience (ex: semantic zoom)• Dolby Digital 5.1 audio support
Moving from Plugins to HTML5 video• Improved battery/CPU efficiency
• GPU-acceleration means less work for the CPU• Full-quality 1080p now possible within the browser on low to mid-range PCs
• New browsers shying away from plug-in support• Internet Explorer in the new Windows UI, as an example, doesn’t support plug-ins
• No more installing plug-ins!• Removes a hurdle for new customers• HTML5 video just works, no plug-ins necessary
0306
09
Relevant standards and proposalsISO/IEC 23009-1 “Dynamic Adaptive Streaming over HTTP (DASH)”- Specifies manifest and media segment formats ISO/IEC 14496-12 “Part 12: ISO Base Media File Format, Edition 4” (aka MP4 container)- Specifies data structures for packaging media ISO/IEC 23001-7 “Common Encryption in ISO Base Media File Format”- Specifies how to package protected content for
consumption by multiple DRM systemsW3C Proposal “Media Source Extensions”- Specifies an HTML extension API enabling the
implementation of adaptive streaming in the browserW3C Proposal “Encrypted Media Extensions”- Specifies an HTML extension API enabling access to
DRM implementations on the system from the browser
MSE and EME
Trident / IE
Web app or Windows Store app
using JavaScript and HTML
MFDRM
Manifest
- 480,- 720- 1080
Formats
Client
The MPEG-DASH standard• DASH stands for “Dynamic Adaptive
Streaming over HTTP”• Specifies an adaptive media-streaming
model• Control of the presentation lies exclusively with the client• Uses HTTP and standard web servers
• Specifies formats for • Media Segments (media files)• Media Presentation Description (manifest)
• Set of restrictions on the offered Media Presentation (MPD & Segments)
• Windows MSE implementation supports implementation of ISOBMFF based profiles:
• On-Demand Profile• Segments are addressed as byte-range
requests• Suitable for VOD only
• Live Profile• Segments can be addressed by index number
or time stamp• Can be used for Live and VOD delivery
DASH Profiles
Full Profile
ISO Base media file format Main (MP4)
MPEG-2 TS
MainISO
BMFF On
Demand
ISO BMFF Live
MPEG-2 TS Simpl
e
Typical DASH ISOBMFF (MP4) file
Mime Type (ftyp)
Movie Metad
ata (moov)
Segment
Index (sidx)
Movie Frag #1 (moof)
Media Data #1
(mdat)
Movie Frag #2
(moof)
Media Data #2
(mdat)....
Initialization Segment
IndexSegment
(not for live)
Media (or Data)Segment
Media (or Data)Segment
Typical DASH MP4 file
Mime Type (ftyp)
Movie Metad
ata (moov)
Segment
Index (sidx)
Movie Frag #1 (moof)
Media Data #1
(mdat)
Movie Frag #2
(moof)
Media Data #2
(mdat)....
PT: 0PT: 3PT: 6Index
Segment(not for live)
Typical DASH MP4 file
Mime Type (ftyp)
Movie Metad
ata (moov)
Segment
Index (sidx)
Movie Frag #1 (moof)
Media Data #1
(mdat)
Movie Frag #2
(moof)
Media Data #2
(mdat)....
Initialization Segment
IndexSegment
(not for live)
Media (or Data)Segment
Media (or Data)Segment
DASH Manifest (MPD) high-level structure (live profile)
<MPD profiles="urn:mpeg:dash:profile:isoff-live:2011“><Period>
<ContentProtection schemeIdUri="urn:mpeg:dash…" value="cenc"/><AdaptationSet mimeType=“video/mp4”>
<SegmentTemplatemedia=“Level($Bandwidth$)/Fragment($Time$)initialization="Level($Bandwidth$)/Fragments(init)><SegmentTimeline>
<S d=3 r=430 /></SegmentTimeline>
</SegmentTemplate><Representation bandwidth="6000000" width="1920" height="1080“/><Representation bandwidth="2962000" width="1280" height="720“/><Representation bandwidth="1427000" width="856" height="480“/>
</AdaptationSet><AdaptationSet mimeType=“audio/mp4”>
<SegmentTemplate …<Representation bandwidth=“160000”/><Representation bandwidth=“64000”/>
Manifest
- 480,- 720- 108
0
Media Presentation Description Live.XML
DASH Manifest (MPD) high-level structure (on-demand profile)
Manifest
- 480,- 720- 108
0
<MPD profiles="urn:mpeg:dash:profile:isoff-on-demand:2011“/><Period>
<ContentProtection schemeIdUri="urn:mpeg:da…" value="cenc"/><AdaptationSet mimeType=“video/mp4”>
<Representation bandwidth="6000000" width="1920" height="1080“>
<BaseURL>movie-high.mp4</BaseURL><SegmentBase indexRange=“804-1123”>
<Initialization range=“0-803”></SegmentBase>
</Representation><Representation bandwidth="2962000" width="1280" height="720“>
<BaseURL>movie-medium.mp4</BaseURL><SegmentBase …
</Representation><Representation bandwidth="1427000" width=“856" height="480“>
<BaseURL>movie-low.mp4</BaseURL><SegmentBase …
</Representation></AdaptationSet>
…
Media Presentation Description OD.XML
W3C Media Source Extensions
- W3C proposal extending the HTML standard- Defines an API to control media getting to the media element- Allows a web app to push init and media fragments into the
video/audio tag- The main objects are MediaSource and SourceBuffer
W3C Media Source Extensions
<video> MediaSource
SourceBuffer
audio
SourceBuffer
video
src sourceBuffers
JavaScript AppJavaScript mediaSource <video>
sourceBuffer Video
new MediaSource()
video.src = URL.creatObjectURL(mediaSource)
sourceopen !event
addSourceBuffer($video/mp4 ')
'updateend' event
'loadedmetadata' event
appendBuffer(initSegment_video_quality1)
HTTP Server
XHR to get initSegment
Setup
1. Create MediaSource and set it as src to the video tag
2. Add source buffers for audio and video
3. Download and append init segments to each of them
JavaScript AppJavaScript mediaSource <video> sourceBuffer Video
appendBuffer(dataSegment1_video_quality1)
updateend !event
canplay !event
play()
appendBuffer(dataSegment2_video_quality1)
updateend !event
remove(sourceBufferVideo.buffered.start(0), video.currentTime - MAX_BUFFER)
updateend !event
HTTP Server
XHR to get dataSegment
XHR to get dataSegment
Playback
1. Download and append data segment to each source buffer
2. Keep downloading and appending as content plays
3. Also keep removing to control your buffer
JavaScript AppJavaScript mediaSource <video> sourceBuffer Video
appendBuffer(dataSegment3_video_quality2)
updateend !event
'updateend' event
appendBuffer(initSegment_video_quality2)
HTTP Server
XHR to get initSegment
XHR to get dataSegment
Switch streams
1. Download and append init segment for new quality level
2. Download and append data segments at the new quality level
JavaScript AppJavaScript mediaSource <video> sourceBuffer Video
appendBuffer(dataSegment10_video_quality2)
updateend !event
'seeking' event
video.currentTime = a new time position (mapped to dataSegment10)
map video.currentTime to dataSegment10
'seeked' event
HTTP Server
XHR to get dataSegment
Seek
1. currentTime on video tag being set to new position
2. Map currentTime to data segments and download these
3. Append data segments for the new position
Demo: Simple streaming client using Media Source Extensionshttp://ie.microsoft.com/testdrive
W3C Encrypted Media Extensions
- W3C proposal extending the HTML standard- Defines an API that allows the use of a DRM system in a web
app entirely in JavaScript- The main objects are MediaKeys and MediaKeySession- The entry points to the API are prefixed with MS
W3C Encrypted Media Extensions
<video> MSMediaKeys MediaKeySession
msKeys
CloudLicense Server JavaScript mediaKeys mediaKeySession <video>
new MSMediaKeys("com.microsoft.playready")
'msneedKey' event w/ initData
'mskeymessage' event w/ key request + default URL + session ID
License Challenge
License Response
update(licenseResponse)
'mskeyadded' event
msSetMediaKeys(mediaKeys)
createSession(type, initData,...)
Content Playback
'canplay' event
play()
Protected Content1. In response to
msneedKey event, create MSMediaKeys object and set it on the video tag
2. Create a MediaKeySession using init data from msneedkey event
3. In resonse to mskeymessage event acquire license from server and update mediaKeySession with response
Implementation of MSE and EME in Windows• MSE and EME can be used in IE and “Windows Store app using
JavaScript and HTML”• Through MSE and EME, Windows can process/play DASH
segments• Only ISO Base Media File Format (no MPEG2-TS)• Supported codecs: – In IE: H264, AAC– In HTML store apps: also Dolby Digital+
• PlayReady key system is pre-installed on all Windows versions with media capabilities and can be invoked via: • new MSMediaKeys(‘com.microsoft.playready’)
Implementing your MPEG-DASH streaming service in Internet Explorer or in a Windows Store app using JavaScript and HTML
Getting started: what you need 1. DASH sample app • (e.g. DASH.js)
2. A tool to inspect MP4 files • (e.g. MP4 Explorer)
3. Packaging tool/packaging pipeline • (e.g. MP4 box, Windows Azure Media Services)
4. Delivery Server • (e.g. IIS, Windows Azure Media Services)
• The DASH Industry Forum’s reference player.
• A JavaScript implementation of a DASH client in the browser using MSE and EME
Player: http://dashif.org/reference/players/javascript/index.html Source code: https://github.com/Dash-Industry-Forum/dash.js
DASH.JS
http://www.windowsazure.com/en-us/pricing/free-trial/
http://www.windowsazure.com/en-us/pricing/member-offers/msdn-benefits/
Begin today!
Things to note• When fetching segments / acquiring license via
XHR• Observe Cross Origin Resource Sharing (CORS)• Host the player page on an http (not https) URL
• For power efficiency• To avoid unnecessary disk caching, set
xhr.responseType = ‘ms-stream’ or xhr.msCaching = ‘disabled’• make use of the HTML Fullscreen API
document.getElementById(“video”).msRequestFullscreen();
• Monitor download speed and playback quality to determine when to change bitrate
partial interface HTMLVideoElement { readonly attribute VideoPlaybackQuality videoPlaybackQuality;};
interface VideoPlaybackQuality { readonly attribute DOMHighResTimeStamp creationTime; readonly attribute unsigned long totalVideoFrames; readonly attribute unsigned long droppedVideoFrames; readonly attribute unsigned long corruptedVideoFrames; readonly attribute double playbackJitter;};
Monitoring playback quality
Attributes likely to change
Summary• MPEG-DASH provides standardization in the area of adaptive
streaming• One format going to multiple clients
• In IE and Windows Store apps using JavaScript and HTML, the MSE and EME APIs can be leveraged• Use the same code across app and site• Take full advantage of the hardware• Plugin-free in the browser
• End-to-end tools are available today to kick-start development of a DASH service
ResourcesDASH Specification (ISO 23009-1): • http://standards.iso.org/ittf/PubliclyAvailableStandards/index.html Media Source Extensions Specification: • https://dvcs.w3.org/hg/html-media/raw-file/tip/media-source/media-source.html Encrypted Media Extensions Specification: • https://dvcs.w3.org/hg/html-media/raw-file/tip/encrypted-media/encrypted-media.html ISO Base Media File Format Specification (ISO 14496-12:2008/FDAM-3):• http://standards.iso.org/ittf/PubliclyAvailableStandards/index.html Common Encryption Specification (ISO 23001-7): • http://
www.3gpp.org/ftp/Inbox/LSs_from_external_bodies/ISO_IEC_JTC1_SG29_WG11/29n12313.zip
DASH.js: Reference Implementation of DASH in the browser by the DASH-IF.• https://github.com/Dash-Industry-Forum/dash.js/wiki• http://dashif.org/reference/players/javascript/0.2.3/index.htmlAdaptive Streaming SDK + Player Framework• http://playerframework.codeplex.com/
Evaluate this session
Scan this QR code to evaluate this session and be automatically entered in a drawing to win a prize!
© 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
top related