webrtc - · pdf filewebrtc: real-time communica2ons on the web dan burne9, phd ... –...

0

WebRTC:Real-TimeCommunica2onsontheWeb

DanBurne9,PhDh9p://standardsplay.comh9p://allthingsrtc.org

AlanJohnston,PhDRowanUniversity

WebRTC

IETF100SingaporeNovember2017

WebRTC:AJointStandardsEffort•  InternetEngineeringTaskForce(IETF)andWorldWideWeb

ConsorFum(W3C)areworkingtogetheronWebRTC•  IETF

–  Protocols–“bitsonwire”–  MainprotocolsarealreadyRFCs,butmanyextensionsin

progress–  RTCWEB(Real-TimeCommunicaFonsontheWeb)Working

Groupisthemainfocus,butotherWGsinvolvedaswell•  MMUSICWG(ICEandSDPextensions)•  TRAMWG(STUNandTURNextensions)•  AVTCOREWG(RTPextensions)•  RMCATWG(RTPMediaCongesFonAvoidance)

•  W3C–  APIs–usedbyJavaScriptcodeinHTML5–  WEBRTCWGandMediaCaptureTFaremaingroups

1IETF100SingaporeNovember2017

WebRTCProtocols

IP

UDP

WebSocket

SRTP SDP

TURNSTUN

ICE

TCPTLS

TransportLayer

NetworkLayer

ApplicaFonLayer

DTLS

HTTP

UDPDTLS

SCTP


TheBrowserRTCFuncFon•  NewReal-TimeCommunicaFon(RTC)FuncFonbuilt-intobrowsers

•  Contains–  Audioandvideocodecs–  AbilitytonegoFatepeer-

to-peerconnecFons–  EchocancellaFon,packet

lossconcealment•  InChrome,Firefox,Microso^EDGE,andSafaritoday

3

HTTPorWebSockets

On-the-wireprotocols

RTCAPIsOtherAPIs

NaFveOSServices

WebServer

JavaScript/HTML/CSS

WebBrowser

BrowserRTC

FuncFon

(Signaling)

(MediaorData)

HTTPorWebSockets

SignalingServer


BenefitsofWebRTC

ForDeveloper •  Streamlineddevelopment–

oneplaaorm•  NATtraversalonlyuses

expensiverelayswhennootherchoice

•  Advancedvoiceandvideocodecswithoutlicensing

ForUser•  Nodownloadorinstall–

easytouse•  Allcommunicaton

encrypted–private•  Reliablesession

establishment–“justworks”•  Excellentvoiceandvideo

quality•  Manymorechoicesforreal-

FmecommunicaFon


WebRTCTriangle

•  BothbrowsersrunningthesamewebapplicaFonfromwebserver

•  PeerConnecFonestablishedbetweenthemwiththehelpofthewebserver

WebServer(ApplicaFon)

BrowserM(RunningHTML5ApplicaFon

fromWebServer)

BrowserL(RunningHTML5ApplicaFon

fromWebServer)

PeerConnecFon(Audio,Video,and/orData)


WebRTCTriangle

•  BothbrowsersrunningthesamewebapplicaFonfromwebserver

•  PeerConnecFonestablishedbetweenthemwiththehelpofthewebserver("Signaling")

WebServer(ApplicaFon&Signaling)


fromWebServer)

BrowserL(RunningHTML5ApplicaFon

fromWebServer)

PeerConnecFon(Audio,Video,and/orData)


"Signaling"

WebRTCTrapezoid

•  SimilartoSIPTrapezoid•  WebServerscommunicateusingSIPorJingleorproprietary

WebServerA(ApplicaFonA)

PeerConnecFon(Audioand/orVideo)

WebServerB(ApplicaFonB)

SIPorJingle


fromWebServerA)

BrowserT(RunningHTML5ApplicaFon

fromWebServerB)


NATTraversalandSTUN

Peer-to-PeerMediawithWebRTC

WebServer

Internet

Router

BrowserL

HomeWiFiRouter

BrowserT

BrowserM

CoffeeShopWiFiRouter

BrowserD

9

Mediacangodirectlybetweenbrowsers!


WebServer

Internet

RouterwithNAT

BrowserL

HomeWiFiwithNAT

BrowserT

BrowserM

MostbrowsersarebehindNATsontheInternet,whichcomplicatestheestablishmentofpeer-to-peermediasessions.

CoffeeShopWiFiwith

NAT

BrowserD

NATComplicatesPeer-to-PeerMedia


Internet

HomeWiFiwithNAT

BrowserT192.168.0.6

BrowserM192.168.0.5

“Outside”PublicIPAddress203.0.113.4

“Inside”PrivateIPAddresses192.168.x.x

NAT–NetworkAddressTranslator

11

ANATmapsan“inside”addresstoan“outside”addressallowingmulFplehoststosharethesameIPaddress.


WebServer

Internet

RouterwithNAT

BrowserL

HomeWiFiwithNAT

203.0.113.4

BrowserT

BrowserM192.168.0.5

CoffeeShopWiFiwith

NAT

BrowserD

TURNServer198.51.100.2

BrowsersendsSTUNtestpackettoSTUNservertolearnitspublicIPaddress(addressoftheNAT).

STUNServer198.51.100.9

STUNSessionTraversalUFliFesforNAT


NoauthenFcaFonofSTUN!

WebServer

Internet

RouterwithNAT

BrowserL

HomeWiFiwithNAT

BrowserT

BrowserM

CoffeeShopWiFiwith

NAT

BrowserD

TURNServerasaMediaRelay

Insomecases,holepunchingfails,andaTURNMediaRelayonthepublicInternetmustbeused.

STUNServer

TURN–TraversalUsingRelayaroundNAT


What'sNewforWebDevelopers

TwoAPIpieces

•  Localmediacapture– Camera,Microphone,<video>element,screen/window

•  PeerconnecFon– Mediaanddatabetweentwoclientbrowsers(orarbitrarydevicesusingnaFveAPI)

IETF100SingaporeNovember2017 15

WebRTCusageinbrief

Set Up Peer Connection

Obtain Local Media

Exchange Offer/Answer

Attach Media or Data

Getmoremedia

Allmediaadded

PeerConnecFonestablished

Apachmoremediaordata

Readyforcall


WebRTCusageinbrief

•  getUserMedia() –  Audioand/orvideo–  Constraints–  Userpermissions

•  Browsermustaskbeforeallowingapagetoaccessmicrophoneorcamera

•  MediaStream •  MediaStreamTrack


Obtain Local Media



Getmoremedia

Allmediaadded



Readyforcall


WebRTCusageinbrief

•  RTCPeerConnection –  Directmedia–  Betweentwopeers–  ICEprocessing–  SDPprocessing–  DTMFsupport–  Datachannels–  IdenFtyverificaFon–  StaFsFcsreporFng


Obtain Local Media



Getmoremedia

Allmediaadded



Readyforcall


WebRTCusageinbrief

•  addTrack() –  Doesn'tchangemediastate!

•  removeTrack() –  Dipo!

•  createDataChannel() –  Dependsontransport


Obtain Local Media



Getmoremedia

Allmediaadded



Readyforcall


WebRTCusageinbrief

•  createOffer(), createAnswer()

•  setLocalDescription(), setRemoteDescription()

•  Offer/answerexchangeneededforthistowork


Obtain Local Media



Getmoremedia

Allmediaadded



Readyforcall


WebRTCusageinbrief

SetUpPeerConnecFon

ObtainLocalMedia

ExchangeSession

DescripFons

ApachMediaorData

Getmoremedia


SetUpSignalingChannel


LocalMedia

Mediacaptureanduse

•  navigator. mediaDevices.getUserMedia() – Requestcamera,microphoneaccess– Permissioncheckrequiredforhpppage

•  <video>.srcObject = <mediastream>– Directassignment– Worksfor<audio>aswell

•  new MediaStream() – Canbuildfromtracksinotherstreams


BrowserMediaModel:SourcesandSinks

•  Sourcescanbe:–  staFc:files,RTSP,<canvas>,etc–  dynamic:

•  local:webcam,microphone-needgetUserMediatoaccess

•  remote:RTCPeerConnecFonorstreamingmedia

•  Sinksare<img>,<video>,and<audio>tags–  Theycanbesized,whichcancausescalingdependingontheconstraintsonthesource

•  RTCPeerConnecFoncanbebothasourceandasink


BrowserPromptsforPermission


Local:TracksandStreams

•  AMediaStreamTrack –  isahandletoonereal-Fmesourceofmediaofonetype(audio/video/depth)

•  AMediaStream – representsacollecFonofzeroormoreMediaStreamTracks,ofthesameordifferentmediatypes

–  indicatesthatthecollectedtracksaretobekeptsynchronizedtothebestabilityofthebrowser


PeerConnecFons

RTCPeerConnecFonAPIs

•  Trackstuff– addTrack(),removeTrack()– onaddtrack,onremovetrack

•  Offer/answerstuff– createOffer(),createAnswer()– setLocalDescripFon(),setRemoteDescripFon()

•  Muchmore


SDPOffer/Answer

SessionDescripFonProtocol(SDP)

•  UsedbybrowserandWebRTCAPIstodescribemediasession– RTPmediaflows,candidatetransportaddresses,codecinformaFon,mediakeyinginformaFon

•  SDPiswidelyusedinSIPVoIPandvideosystems•  BrowseruseofSDPisbasedonOffer/Answer– DefinedinJSEP-JavaScriptSessionEstablishmentProtocol

– NegoFaFonproceedsaccordingtoOffer/AnswerStateMachineinnextslides


SDPOffer/AnswerinRTCPeerConnecFon

•  PeerConnecFonAPIstreattheprotocol-levelconfig(SDP)asablob

•  createOffer(),createAnswer() –  generateSDP

•  setLocalDescription(),setRemoteDescription() –  tellthebrowserwhichSDPtouse


Offer/AnswerStateMachine

32

stable

have-remote-offer

have-local-offer


createOffer()


33

stable

have-remote-offer

have-local-offer

setLocalDescripFon(localOffer)


createOffer()


34

stable

have-remote-offer

have-local-offer


setRemoteDescripFon(remoteOffer)


SDPoffersentoversignalingchanneltootherbrowser

createOffer()

createAnswer()


35

stable

have-remote-offer

have-local-offer

setLocalDescripFon(localAnswer)





createOffer()

createAnswer()


36

stable

have-remote-offer

have-local-offer

setLocalDescripFon(localAnswer)

setRemoteDescripFon(remoteAnswer)




SDPanswersentoversignalingchanneltootherbrowser


createOffer()

createAnswer()

SDPExtensionsinWebRTC•  AnumberofnewSDPextensionsdevelopedforWebRTC.Theyinclude:– BUNDLE–awaytosignalthatasetofm=medialines(e.g.audioandvideo)shouldbemulFplexedoverthesametransportaddressandport•  a=group:BUNDLE audio video

– MSID–awaytosignaltheMediaStreamIDusedinJavaScriptinSDP•  a=msid

– Source-SpecificApributes–properFesofasource•  a=ssrc


SDPOffer/AnswerUsesSignaling

(notstandardized)

WebRTCSignalingApproaches•  Signalingisrequiredforexchangeofcandidatetransport

addressesandSDPblobs•  ManyopFons–choiceisuptowebdeveloper


Example:WebRTCSignalingusingSIP

40

WebServer

SRTPMedia

SIPProxy/RegistrarServer

BrowserM(runningJavaScriptSIPUA)

BrowserT(runningJavaScriptSIPUA)

HTTP(HTML5/CSS/JavaScript)

WebSocket(SIP)

WebSocket(SIP)

HTTP(HTML5/CSS/JavaScript)

•  BrowserrunsaSIPUserAgentbyrunningJavaScriptfromWebServer•  SRTPmediaconnecFonusesWebRTCAPIs•  DetailsinRFC7118thatdefinesSIPtransportoverWebSockets


DataChannel

DataChannelProtocols

•  Datachannelprovidesanon-mediachannelbetweenbrowsers•  StreamControlTransportProtocol(SCTP)providesreliability,

congesFoncontrol,andmessagedelivery•  MulFplexedoversameportsasRTPmedia

42

DTLS

DataChannelData

ProvidescongesFonandflowcontrol

Providessecurity(confidenFality)

ProvidestransportthroughNAT(a^erICEholepunching)

Internet

UDP

DTLS

SCTP


Datachannel

•  RTCPeerConnecFon.createDataChannel()– send()methodforasyncmessagesending– onmessagehandlerforasyncmessagereceipt

•  BidirecFonalbydefault•  CansendstringsorArrayBuffertypes


Otherdetails

MediaCodecs

45

•  Audiomandatory-to-implement:–  Opus(RFC6716):NarrowbandtowidebandInternetaudiocodecforspeechandmusic

–  G.711(RFC3551):PCMaudioencodingforPSTNinterworkingandbackwardscompaFbilitywithVoIPsystems

–  TelephoneEvents(RFC4733):TransportofDualToneMulFFrequency(DTMF)tones

•  Videomandatory-to-implement:–  H.264(RFC6184):Commonvideocodec(requireslicensing)

–  VP8(RFC6386):Opensourcevideocodec


Security

•  MediaisalwaysencryptedinWebRTC•  SecureRTP(SRTP)isusedtoassureconfidenFalityandauthenFcaFonofRTPandRTCPpackets


Privacy

•  SRTP,DTLSguaranteeintegrityofcontentbutnottheendpoint

•  NewIdenFtyProxyproposedinWebRTC– dra^-iea-rtcweb-security– Allowsuseofthird-partyidenFtyservice(e.g.,FacebookConnect)

– BrowsersignsSRTPkeysusingmaterialfromidenFtyservice,verifiedatotherendusingidenFtyservice


NewLow-levelControls

V

A

HowitReallyWorks


AV

AA

V

AV

V

AV

Audiom-line

Audiom-line

Videom-line

Videom-line

Stream1 Stream1

Stream2 Stream2

SDPm-line:bidirecFonalSRTPstream

Browser1 Browser2

Senders/Receivers

•  WebRTC1.0recentlyadded:–  RTCRtpSenders–AhandletoanoutboundRTPstream(one-halfofanm-line)

–  RTCRtpReceivers–AhandletoaninboundRTPstream(theotherhalfofanm-line)

•  ForeverytracksentorreceivedoveraPeerConnecFonthereisanassociatedsenderorreceiver.ThesegivedirectaccesstoandcontrolovertherelevantRTPstreams.


V

A

HowitReallyWorks


AV

AA

V

AV

V

AV

Audiom-line

Audiom-line

Videom-line

Videom-line

Stream1 Stream1

Stream2 Stream2

Senders

Senders

Receivers

Receivers


Transceivers

•  Butthatwasn'tenough,because–  attheSDPleveleverysenderispairedwithareceiver,evenifonlyonehasatrackthatisacFvelyusingit

•  EntertheRTCRtpTransceiver.Ateachendpoint –  Thereispreciselyonetransceiverforeachm-line.–  Thereisasenderandareceiverforeachtransceiver.–  Themidofthem-lineisthemid(mediaid)ofthetransceiver.ItisuniqueperPeerConnecFon.


V

A

HowitReallyWorks


AV

AA

V

AV

V

AV

Audiom-line

Audiom-line

Videom-line

Videom-line

Stream1 Stream1

Stream2 Stream2

Senders

Senders

Receivers

Receivers

Foreachside,eachofthesehasaTransceiver


StatusofWebRTCAPIs•  JavaScriptAPIsarebeingstandardizedbyW3C•  TwomainspecificaFons–  “WebRTC1.0:Real-FmeCommunicaFonBetweenBrowsers”(akaPeerConnecFon)•  CandidateRecommendaFon:hpp://www.w3.org/TR/webrtc•  Coreisstable,justcleaningupedgecasesnow

–  “MediaCaptureandStreams”(akagetUserMedia)•  CandidateRecommendaFon:

– hpps://www.w3.org/TR/mediacapture-streams/•  Ohsoclose...

•  NeedimplementaFonexperienceatthispoint54IETF100SingaporeNovember2017

StatusofWebRTCProtocols

•  Tworeferencestocheckforthis:–  PrimaryisCullenJennings'WebRTCDependenciesdra^dra^-jennings-rtcweb-deps

–  RTCWEBdocumentscanbefollowedasusualhpps://tools.iea.org/wg/rtcweb/


ToolsandServices•  TohelpWebRTCprogramming –  Libraries,codesnippets

•  EasyRTC,SimpleWebRTC.js,CoTurn(hpps://github.com/coturn/coturn)

–  SIPsignaling•  sipML5,JsSIP,reSIProcate

•  TohelpWebRTCdeployment–  Hostedsignalingservices

•  PubNub,FireBase–  HostedSTUNandTURNservers

•  XirSys,Twilio–  HostedSFUand/ornetworkopFmizaFon

•  Jitsi,SwitchRTC,Agora.io


Higher-levelAPIs

•  Alternate(higher-level)APIs– Twilio,TokBox– APIDaze– PeerJS,RTCMulFConnecFon


Greatonlineresources

•  InformaFonalsites–  hpp://webrtc.org–  http://html5rocks.com/en/tutorials/webrtc/basics–  http://webrtchacks.com–  hpps://webrtcstandards.info

•  Games/demos/apps–  hpp://www.cubeslam.com–  hpp://shinydemos.com/facekat–  hpp://sharefest.me(github.com/Peer5/Sharefest)


Whataboutdeployments?•  Well-knownservices

–  FacebookChat–  AmazonMayday(andnowChime)–  GoogleHangouts,Duo

•  Plaaormembeddings–  EveryUCplaaormtoday–  Andmosttelecomproviders

•  Free(mium)commservices–  Crowdcast.io–  GoToMeeFngFree(previouslyHu.p)–  vLine–  Talky.io–  Appear.in–  Gruveo–  Room–  Rabbit–  GetARoom.io–  UberConference


webrtc - · pdf filewebrtc: real-time communica2ons on the web dan burne9, phd ... –...

Documents