网易nrtc支持webrtc的工业级实践 - huodongjia.com
TRANSCRIPT
![Page 1: 网易NRTC支持WebRTC的工业级实践 - Huodongjia.com](https://reader030.vdocuments.us/reader030/viewer/2022020916/61a3ab04c84b4c2c2d294c23/html5/thumbnails/1.jpg)
网易NRTC支持WebRTC的工业级实践
2017年10月21日
赵加雨Rain
![Page 2: 网易NRTC支持WebRTC的工业级实践 - Huodongjia.com](https://reader030.vdocuments.us/reader030/viewer/2022020916/61a3ab04c84b4c2c2d294c23/html5/thumbnails/2.jpg)
NRTC
• NetEase RTC• 工业级的功能完整的音视频技术方案
![Page 3: 网易NRTC支持WebRTC的工业级实践 - Huodongjia.com](https://reader030.vdocuments.us/reader030/viewer/2022020916/61a3ab04c84b4c2c2d294c23/html5/thumbnails/3.jpg)
NRTC MCU NRTC BMS
录制 Mixer
NRTC LVS NCDN
推流SDK NRTC PlayerNRTC SDK
UDP TCP RTMP/FLV/HLS
混屏 转码 录制
KCP
SIPGW 录制
第三方推流
第三方播放器
存储
NRTC Architecture
![Page 4: 网易NRTC支持WebRTC的工业级实践 - Huodongjia.com](https://reader030.vdocuments.us/reader030/viewer/2022020916/61a3ab04c84b4c2c2d294c23/html5/thumbnails/4.jpg)
• 实时音视频通话• 直播• 互动直播• 点播• 短视频
NRTC Capabilities
![Page 5: 网易NRTC支持WebRTC的工业级实践 - Huodongjia.com](https://reader030.vdocuments.us/reader030/viewer/2022020916/61a3ab04c84b4c2c2d294c23/html5/thumbnails/5.jpg)
• 信令:SDP, JSEP, SIP, Jingle, ROAP• 传输:RTP, RTCP, DTLS, RTMP, FLV, HLS• P2P:ICE, STUN, TURN, NAT
• 网络:UDP, TCP• 音频:Opus, G711, AAC, Speex, 3A• 视频:H264, VP8
• QoS:FEC, NACK, BWE• Server:SFU, MCU• 端:Capture, Render, 各种适配
音视频技术栈
![Page 6: 网易NRTC支持WebRTC的工业级实践 - Huodongjia.com](https://reader030.vdocuments.us/reader030/viewer/2022020916/61a3ab04c84b4c2c2d294c23/html5/thumbnails/6.jpg)
• The specification in browser• Web Real-Time Communication• A specification that is being standardized by W3C and IETF• Enables web browsers with audio, video and sharing
capabilities via simple JavaScript APIs• Zero install• Peer 2 peer, also capable for conference• Interoperability with existing voice and video systems
• An open source project• Contributed by Google• In C++ and it is cross platform• From Google’s acquisition of Global IP Solutions• A fully implemented VoIP client framework
WebRTC introduction
![Page 7: 网易NRTC支持WebRTC的工业级实践 - Huodongjia.com](https://reader030.vdocuments.us/reader030/viewer/2022020916/61a3ab04c84b4c2c2d294c23/html5/thumbnails/7.jpg)
WebRTC Architecture
WebRTC Javascript API
WebRTC Native API (C++)
Session Management (SDP)
Voice Engine
Voice Codecs
NetEQ for voice
AEC, AGC, ANR
Video Engine
Video Codec
Jitter Buffer
Image Enhancements
Transport
SRTP
Multiplexing
ConnectivityICE, STUN, TURN
Your App
AudioCapture/Render
VideoCapture/Render
Network I/O
![Page 8: 网易NRTC支持WebRTC的工业级实践 - Huodongjia.com](https://reader030.vdocuments.us/reader030/viewer/2022020916/61a3ab04c84b4c2c2d294c23/html5/thumbnails/8.jpg)
• JavaScript APIs, run in browser• Signaling not defined• No SFU/MCU• Follow standards• 依赖browser的实现
WebRTC limitations
![Page 9: 网易NRTC支持WebRTC的工业级实践 - Huodongjia.com](https://reader030.vdocuments.us/reader030/viewer/2022020916/61a3ab04c84b4c2c2d294c23/html5/thumbnails/9.jpg)
• JavaScript APIs• libwebrtc• Compatible with WebRTC
使用WebRTC
![Page 10: 网易NRTC支持WebRTC的工业级实践 - Huodongjia.com](https://reader030.vdocuments.us/reader030/viewer/2022020916/61a3ab04c84b4c2c2d294c23/html5/thumbnails/10.jpg)
• NRTC早于WebRTC• NRTC是VoIP的完整方案, WebRTC ≈ NRTC SDK• NRTC的实现更灵活• NRTC是工业级的实现
NRTC vs WebRTC
![Page 11: 网易NRTC支持WebRTC的工业级实践 - Huodongjia.com](https://reader030.vdocuments.us/reader030/viewer/2022020916/61a3ab04c84b4c2c2d294c23/html5/thumbnails/11.jpg)
NRTC MCU
NRTC SDK
WebRTC Gateway
Web SDK
SRTPNPDU over UDP
NPDU over UDP
NRTC with WebRTC
![Page 12: 网易NRTC支持WebRTC的工业级实践 - Huodongjia.com](https://reader030.vdocuments.us/reader030/viewer/2022020916/61a3ab04c84b4c2c2d294c23/html5/thumbnails/12.jpg)
WebRTC Gateway
WebRTC Gateway
信令
媒体
Web
wss://
SRTP
iOS
AOS
PC
NRTCMCU
SRTP
ICE
WebSocket
SDP/ICE
包转封装
![Page 13: 网易NRTC支持WebRTC的工业级实践 - Huodongjia.com](https://reader030.vdocuments.us/reader030/viewer/2022020916/61a3ab04c84b4c2c2d294c23/html5/thumbnails/13.jpg)
• Browser的兼容性• Lite ICE• RTCP feedbacks• Reliable connections• Congestion Control
工业化
![Page 14: 网易NRTC支持WebRTC的工业级实践 - Huodongjia.com](https://reader030.vdocuments.us/reader030/viewer/2022020916/61a3ab04c84b4c2c2d294c23/html5/thumbnails/14.jpg)
• browser兼容性 - adapter.js• Video resolution
• MediaStream lifecycle• getUserMedia call success, no media
Browser的各种坑
![Page 15: 网易NRTC支持WebRTC的工业级实践 - Huodongjia.com](https://reader030.vdocuments.us/reader030/viewer/2022020916/61a3ab04c84b4c2c2d294c23/html5/thumbnails/15.jpg)
• NAT• STUN - RFC 5389
• TURN – RFC 5766• ICE – RFC 5245• TCP
ICE
![Page 16: 网易NRTC支持WebRTC的工业级实践 - Huodongjia.com](https://reader030.vdocuments.us/reader030/viewer/2022020916/61a3ab04c84b4c2c2d294c23/html5/thumbnails/16.jpg)
• 当通信一方是Server时,有公网IP• Host candidates only• Full peer发起连通检查,2步连通检查
Lite ICE
![Page 17: 网易NRTC支持WebRTC的工业级实践 - Huodongjia.com](https://reader030.vdocuments.us/reader030/viewer/2022020916/61a3ab04c84b4c2c2d294c23/html5/thumbnails/17.jpg)
网络监测
• WebSocket disconnect event• RTCPeerConnection disconnect events
• oniceconnectionstatechange• onsignalingstatechange
• keepalive over signaling channel
![Page 18: 网易NRTC支持WebRTC的工业级实践 - Huodongjia.com](https://reader030.vdocuments.us/reader030/viewer/2022020916/61a3ab04c84b4c2c2d294c23/html5/thumbnails/18.jpg)
• Start over• Detach stream, 销毁现有连接等• 信令连接、鉴权、媒体连接
• ICE restart
断开重连
pc.createOffer({iceRestart: true}).then(function(offer) {
return pc.setLocalDescription(offer);})
![Page 19: 网易NRTC支持WebRTC的工业级实践 - Huodongjia.com](https://reader030.vdocuments.us/reader030/viewer/2022020916/61a3ab04c84b4c2c2d294c23/html5/thumbnails/19.jpg)
Multiplexing and bundle
![Page 20: 网易NRTC支持WebRTC的工业级实践 - Huodongjia.com](https://reader030.vdocuments.us/reader030/viewer/2022020916/61a3ab04c84b4c2c2d294c23/html5/thumbnails/20.jpg)
• 请求关键帧
FIR and PLI
![Page 21: 网易NRTC支持WebRTC的工业级实践 - Huodongjia.com](https://reader030.vdocuments.us/reader030/viewer/2022020916/61a3ab04c84b4c2c2d294c23/html5/thumbnails/21.jpg)
GCC
![Page 22: 网易NRTC支持WebRTC的工业级实践 - Huodongjia.com](https://reader030.vdocuments.us/reader030/viewer/2022020916/61a3ab04c84b4c2c2d294c23/html5/thumbnails/22.jpg)
• 接收端最大接收码率估测• SDP:
• a=rtcp-fb:107 goog-remb
REMB
![Page 23: 网易NRTC支持WebRTC的工业级实践 - Huodongjia.com](https://reader030.vdocuments.us/reader030/viewer/2022020916/61a3ab04c84b4c2c2d294c23/html5/thumbnails/23.jpg)
• Delay-based controller• Transport cc
• Loss-based controller• RTCP SR/RR
GCC feedbacks
![Page 24: 网易NRTC支持WebRTC的工业级实践 - Huodongjia.com](https://reader030.vdocuments.us/reader030/viewer/2022020916/61a3ab04c84b4c2c2d294c23/html5/thumbnails/24.jpg)
• RTCP feedback, RFC 4585• Bi-direction retransmission• SDP:
• m=video 1234 RTP/SAVPF 107• a=rtpmap:107 H264/90000• a=rtcp-fb:107 nack• a=rtcp-fb:107 nack pli
NACK
![Page 25: 网易NRTC支持WebRTC的工业级实践 - Huodongjia.com](https://reader030.vdocuments.us/reader030/viewer/2022020916/61a3ab04c84b4c2c2d294c23/html5/thumbnails/25.jpg)
一个SDP例子v=0o=nrtc 0 2 IN IP4 127.0.0.1a=group:BUNDLE audio videoa=ice-ufrag:193055970146817/378951a=ice-pwd:nfklsam=audio 1 RTP/SAVPF 111c=IN IP4 0.0.0.0a=rtcp:1 IN IP4 0.0.0.0a=sendrecva=mid:audioa=rtcp-muxa=rtpmap:111 opus/48000/2a=fmtp:111 maxplaybackrate=16000; sprop-maxcapturerate=16000;a=fmtp:111 minptime=60;useinbandfec=1a=setup:passivea=maxptime:60a=ptime:60m=video 1 RTP/SAVPF 107c=IN IP4 0.0.0.0a=rtcp:1 IN IP4 0.0.0.0a=sendrecva=mid:videoa=rtcp-muxa=rtpmap:107 H264/90000a=rtcp-fb:107 ccm fira=rtcp-fb:107 nacka=rtcp-fb:107 nack plia=rtcp-fb:107 goog-remba=rtcp-fb:107 transport-cca=fmtp:107 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01fa=extmap:5 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01a=setup:passive
![Page 26: 网易NRTC支持WebRTC的工业级实践 - Huodongjia.com](https://reader030.vdocuments.us/reader030/viewer/2022020916/61a3ab04c84b4c2c2d294c23/html5/thumbnails/26.jpg)
Thank you