south san francisco conference center - webrtc world
TRANSCRIPT
![Page 1: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/1.jpg)
![Page 2: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/2.jpg)
November 27-29, 2012
South San Francisco Conference Center
Open Standards: Codecs, Issues and Challenges
Dan Burnett, Moderator
Director of Standards
Voxeo
November 28, 2012
![Page 3: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/3.jpg)
Session Participants
• Justin Uberti
– Software Engineer (and WebRTC team tech lead)
• Christian Hoene
– CEO
– Symonics GmBH
![Page 4: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/4.jpg)
November 27-29, 2012
South San Francisco Conference Center
Justin Uberti
WebRTC Team Tech Lead
![Page 5: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/5.jpg)
Open issues and challenges
• Some significant topics from this year
– Microsoft CU-RTCWEB proposal
– JSEP details
– Handling of multiple media streams
– Multiplexing of multiple media types
– SRTP keying mechanisms
![Page 6: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/6.jpg)
Microsoft CU-RTCWEB proposal
• Microsoft made W3C proposal as an alternative to PeerConnection
– Effectively a “low-level” API
– Not compatible with current WebRTC API
– Not SDP-based
– Provided more direct control over media functionality
– Raised several good points about needed functionality
![Page 7: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/7.jpg)
JSEP Details
• Several parts of PeerConnection behavior were underspecified
– What states are possible, and what are the valid transitions?
– What SDP does createOffer/Answer generate?
– What SDP does setLocal/setRemote accept?
– How do we surface ICE information?
– How can we report better errors?
![Page 8: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/8.jpg)
Handling of multiple media streams
• General lack of agreement on exactly what a "media stream" is: – An m= line?
– A MediaStream?
– A MediaStreamTrack?
– An SSRC?
• And how they should be signaled: – As individual SSRCs in an m= line?
– As individual m= lines?
– Via RTP/RTCP?
• Significant issue when dealing with dozens of media streams
![Page 9: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/9.jpg)
Multiplexing of multiple media types • Currently, each m= line is sent over a separate ICE
"connection"
• Real-world data indicates that multiplexing over a single connection would yield better results
• Several challenges here:
– What does this look like at the RTP level?
– How does this work in SDP?
– How do we do QoS with this approach?
• Relevant to the prior question as well
![Page 10: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/10.jpg)
SRTP keying mechanisms
• Two options:
– SDES-SRTP, somewhat widely deployed, but less secure
– DTLS-SRTP, newer and less deployed, but more robust; also slightly longer call setup
• What requirements should be put on DTLS?
– MUST vs SHOULD?
– Can we easily gateway to SDES?
– Can we address the latency concerns?
![Page 11: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/11.jpg)
Questions?
![Page 12: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/12.jpg)
November 27-29, 2012
South San Francisco Conference Center
SYMONICS GMBH
Christian Hoene
CEO
![Page 13: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/13.jpg)
Talk on Codecs for WebRTC
• Speech and Audio – G.711, AMR-WB,
– AAC-eLD
– Opus, eVC
• Video – VP8, H.264s
– VPnext, H.265
• Selecting MTI Codecs – Audio
– Video
• Patents – License Fees
– Risks
– Wars
![Page 14: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/14.jpg)
Soundcheck and Video-Test
![Page 15: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/15.jpg)
IPR and Codecs
• Codecs are complex algorithms – Compress and decompress multimedia signal
– Time consuming to developed
– Non-trival
• Codecs are well patentable – Technical problem
– Global consensus on patent-eligiblity
• Most codecs are protected by IPRs – Most codec patents require paying license fees
![Page 16: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/16.jpg)
Are IPR royalty-free good?
Pro
• Manufactors of „boxes“ (phones, GWs, software) – Cisco, Apple, Nokia, Ericsson …
• Payment for development work
done – Paid for R&D
• IPR fees add to a non-free
product – Technical products and patents
• Deployment only to paying
customers – Mobile phone model
Contra
• Providers of services and open source software – Google, Mozilla …
• Money/fame is earned by other
means – Paid for ads with donations or
• Free software and patents would change that – Software product and patents
• Global, wide-scale deployment
easy as everbody can use it – Internet
![Page 17: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/17.jpg)
Speech Coding
• Reference (CD Quality)
• Analog Telephone (60s-70s)
• ISDN – G.711 (80s)
• Cellular Phone (90s)
• 2000s: VoIP is not better
• Skype Silk (24kHz, 24kb/s)
![Page 18: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/18.jpg)
Audio Coding
• Reference
• GSM (FR, 8 kbits)
• Opus (stereo, 64 kHz, 64 kbits)
© Symonics GmbH 2012 18
![Page 19: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/19.jpg)
IETF Opus Speech and Audio Codec
• Opus consists of two coding algorithms – Based on CELT and Silk
• Opus supports three different operational modes – Silk only – Hybrid (Silk and CELT) – CELT only
• IETF RFC 6716: Audio Codec call Opus – 6-512kbps – 2.5 to 60ms – Dynamic changeable modes – Support for concealment of time adjustment – by JM. Valin, K. Vos, T. Terriberry – September 2012
• Status – Mandatory for WebRTC – Royality free? (Qualcomm has filed a IPR statement)
© Symonics GmbH 2012 19
![Page 20: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/20.jpg)
IPR Risks = Costs (and angst)
• Codec may covered by unknown IPR
• IPR fees are subject of negotiation
• Patent usage may be forbitten
Substantial risks for codec users
IPR Risk Management
• Cross licensing, patent pools
• Careful codec development
• Ask for IP insurance
• Munich Re, 10 M€ cover, Premium 300k€, Propability 1:40
![Page 21: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/21.jpg)
Comparision Features vs. Costs
AMR-WB
AAC-LD
G.711Opus
EVC (2014)
Cost
s (z
ero
to
hig
h)
Features (few to many)
![Page 22: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/22.jpg)
VP8
• Some specs
![Page 23: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/23.jpg)
H.264
• Some specs
![Page 24: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/24.jpg)
Video Codec for WebRTC
• Status: Unsettled
• Many different proposals, no consensus – draft-alvestrand-rtcweb-vp8-00
• Google In favor of VP8 – reasonable draft – draft-burman-rtcweb-h264-proposal-00
• Ericsson, Nokia, MS, RIM: H.264/AVC Constrained Baseline Profile Level 1.2 – reasonable draft
– draft-dbenham-webrtc-videomti-00
• Cisco, Apple: In favor of H264/AVC – propaganda – draft-marjou-rtcweb-video-codec-00
• France Telecom: In favor of H264/AVC –short
• Interviews: Many different opinions
![Page 25: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/25.jpg)
Comparision
• Todo: Two videos side by side
![Page 26: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/26.jpg)
Technical Comparision
H.264
• Quality vs. Power vs. Bandwidth
• Well deployed (since many years)
• Interop (Media gateways)
• No of implementations
• Hardware support
• Rather equal
Vp8
• Quality vs. Power vs. Bandwidth
• Rather new
• Few implementation
• Hardware support just coming
![Page 27: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/27.jpg)
Development perspectives
H265
• What ever codec is selected now, other codecs will be used in (near) future.
VPnext
![Page 28: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/28.jpg)
Royalties and IPR Risks
H.264
• long tradition – MPEG LA. Cost for both codec
and content
– Up to 100.000 users free
– Internet usages are free
– but Motorola vs. Microsoft case (Seattle)
– 0,20 cent per license with caps
VP8
• : So far it is royalty free – No costs
– But MPEG LA is searching for patents in VP8
• (anti trust case is going on).
![Page 29: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/29.jpg)
Patent War: Important court cases
• Microsoft vs. Motorola
– On H.264
• Google vs. MPEG LA
– Anti Trust in respect to VP8 pool
![Page 30: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/30.jpg)
Reasonable Outcome
• Google and MPEG LA agree on
– Google joins MPEG H264 pool
– MPEG LA opens VP8 pool
– Some H264/AVC profile become royalty free
• H.264 is selected for WebRTC
![Page 31: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/31.jpg)
Alternative Outcome
• No agreement
– Law suits will continue
• Google will provide VP8 insurance
– VP8 will be used in WebRTC
![Page 32: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/32.jpg)
Summary
• For speech and audio codecs: Opus is the best choise
• Opus will dominate the market
– Despite lack of mobile and multi-channel support
• As for video codecs, we have to wait for legal courts to decise
– If all behave reasonable, H.264 will be chosen
• Whatever the decision, in a few year other video codecs will be used…
![Page 33: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/33.jpg)
Questions?
![Page 34: South San Francisco Conference Center - WebRTC World](https://reader031.vdocuments.us/reader031/viewer/2022020702/61fb106b2e268c58cd59bad2/html5/thumbnails/34.jpg)
November 27-29, 2012
South San Francisco Conference Center
Thank You
Dan Burnett, Moderator
Voxeo
dburnett at voxeo dot com