webrtc and mobile integration
TRANSCRIPT
WebRTCandMobileIntegration
GiorgioNatili
Aboutmee-mail:[email protected]
twitter:@giorgionatiliblog:webplatform.io(comingsoon!)
community:codeinvaders.net
Agenda• WhatisWebRTC
• WebRTCandModernBrowsers• NinjaBuildandGYPfiles
• CompileWebRTCforAndroidandiOS• LetMobileandtheWebtalkeachother
WhatisWebRTC
Whatisit?• WebRTCisafree,openprojectthatenableswebbrowserswithReal-TimeCommunications(RTC)capabilitiesviaJavascriptAPIs
• TheWebRTCNativeCodepackageismeantforbrowserdevelopersthatwanttointegrateWebRTC
• WebRTCdefinesawayforbrowserstoimplementapplicationslikevideochatinawaythatisbothinteroperablewithother
clients,anddoesnotrequiretheuseofaplugin
MainResponsibilities
• Getreal-timestreamingaudioandvideoordata• Communicatereal-timestreamingaudio,videoordata
• Exchangesessioncontrolmessagesandmediainformation(notinthestandard)
WebRTCEnvironment
• Clients(browsers,desktop,mobile,etc.)• Servers(stun,ice,etc.)
• Gateway(i.e.PSTN)• Internet
NativeC++API• RTPStack,handlethenetworkstackfortheRTPprotocol• STUN/ICE,handlethepeerconnectionsusingSTUNorICE• SessionManagement,signallingmanagertosetupcalls
• VoiceEngine,audiomediachainmanager(card->network)• Audiocodecs,suchasiSAC/iLBC/Opus
• VideoEngine,videomediachainmanager(cam->network)NativeAPI
NativeAPI
WebRTCTriangleThemostcommonscenarioislikelytobewherebothbrowsersarerunningthesameWebRTCwebapplication,downloadedfromthe
samewebpage
• Twowebbrowsersrunningthesameapplication• Oneserverforsignalingandnegotiation
IsaServerStillRequired?
• SignallingServer(SIP,XMPP,WebSocket,Socket.io,XHR,etc.)• ICEServers(STUN,TURN)
• MediaServers(Asterisk,etc.)
Stun&Turn• STUN(SessionTraversalUtilitiesforNAT)isastandardizedset
ofmethodsandanetworkprotocoltoallowanendhosttodiscoveritspublicIPaddressifitislocatedbehindaNAT
• TURN(TraversalUsingRelaysaroundNAT)isaprotocolthatallowsforaclientbehindanetworkaddresstranslator(NAT)orfirewalltoreceiveincomingdataoverTCPorUDPconnections
WhatisSignalling?• It'sthemechanismbywhichpeerssendcontrolmessagesto
eachothertoestablishingacommunicationprotocol• Signallingprotocolsareawaytocoordinateandcontrolthe
communicationbetweenpeers
• Sessioncontrolmessages(communicationinitialization&co.)• Networkconfiguration(e.g.computer'sIPaddressandport)
• Mediacapabilities(e.g.codecsandresolutions)
SignallingProcess• Thelocalpeeropensoneormorereal-timeports(RTP)
• Thelocalpeerthenhastolearnoftheportsthatitsremotepeerhasopened
• Aprocessofdiscoveryisusedtofindalocalandremoteportpair(acandidatepair)thatcanexchangeUDPpackets
• Finally,areal-timetransportisestablishedonthepairofports
WhatisSDP?SessionDescriptionProtocol(SDP):Aprotocolthatisusedto
announcesessions,managesessioninvitations,andperformothertypesofinitiationtasksformultimediasessions
SDPFlow
SDPFlow
CompleteProcess
CompleteProcess
WebRTCinModernBrowsers
TheAPIDon'tconfuseWebRTCAPIandthethe<device>API!!!
WebRTCAPIisavailablein:
• Chrome24+• Firefox22+• Opera12+
• Andonmobile!!!
InternetExplorerWhat'saboutourgoodfriendIE?
• It'sunclearhowandwhenthesupportwillbeavailable• SofaryoucanuseChromeFrametosupportWebRTC(moreinfo
abouttheChromeFrameextensionareavailableontheofficialwebpageoftheprojectdevelopers.google.com/chrome/chrome-
frame/)• Microsoftisgoingtoworkonaproprietarystandard(weird?)
WebRTCproand
WebRTCproandcons
• ThereareseveralgoodreasonstostarttouseWebRTCespeciallyifcomparedwithtodaymarketwheremostofthe
componentsareproprietary• Nolicencesorotherfeesareneededtostartwithit
• Theenduserdoesn'thavetodownloadandinstalladditionalsoftware
• IntegrationisperformedusingstandardAPIaccessedbyJavaScriptSupport
SupportThesupportispartialandthedocumentationisveryfragmented
KeyFeaturesThekeyfeaturesoftheWebRTCAPIare:
• MediaStreams(akagetUserMedia),accesstoandcontrolcontroloftheusercameraandmicrophone
• PeerConnection,negotiateandconnectclientsinordertoallowdirectcommunication
• DataChannels,peertopeerdataexchange
AccessMicand
AccessMicandCamera
InordertoaccesstheusercameraandaudioisenoughtousethegetUserMediamethodofthenavigatorobject
navigator.getUserMedia({video:true,audio:true},onStreamReady,onStreamError);
• TheonStreamReadyhandlerreceiveaMediaStreamobject• TheonStreamErrorhandlerreceiveaNavigatorUserMediaError
objectwithacode
DemoOpenasupportedbrowserandfollowthislinkhttps://webrtc-
demos.appspot.com/html/gum2.html
TheMediaStream
TheMediaStreamObject
AMediaStreamobjectismadeupby:
• thestop();method• TwoMediaStreamTrackListstoredintheaudioTracksand
videoTracksproperties(actuallyoneisforthevideoandonefortheaudio)
• TheMediaStreamTrackobjectstoredinthelistsdefinethepropertieskind(e.g."video")andlabel(e.g."FaceTimeHDCamera
(Built-in))
PlaywithSoundsOpenhttp://webcamtoy.com/andenjoy!AddAudiotoaVideoTrack
• Requestaccesstothemedia
navigator.webkitGetUserMedia({audio:true},onStreamReady);
varac=newwebkitAudioContext();varmediaStreamSource=ac.createMediaStreamSource(stream);
mediaStreamSource.connect(ac.destination);
• Requestaccesstothemedia
• CreateanewprefixAudioContextandcreatenewAudioNodefromthestream
• Connecttothestreamusingthedestination
What'sJustHappened?
Youcanaccesssoundsanddeliverthemtotheenduser,checkitouthttp://soundstep.com/blog/experiments/jsdetection/
(^_^)
RTCPeerConnectionThesignallingisinitiatedbyaRTCPeerConnectionobject
• ThemethodscreateOffer,setLocalDescription,createAnswerandsetRemoteDescriptionareexecutedatagiventime
• InordertoconnectpeersyourelyonICE• ICEisaframeworkforconnectingpeers,suchastwovideochatclients;initially,ICEtriestoconnectpeersdirectly,withthelowest
possiblelatency,viaUDP• Inthisprocess,STUNservershaveasingletask:toenableapeer
behindaNATtofindoutitspublicaddressandport
RTCPeerConnectionDemo
Let'sseeasimpledemotoclarifyRTCPeerConnectionobject
NinjaBuild
Whatisit?• Itisabuildsystemwrittenwiththespecificgoalofimproving
theedit-compilecycletime• ItbehavesverysimilartoMake
• Ninjaisincludedindepot_tools(aswellasgyp)sojustinstallthetools
Philosophy• Buildsystemsgetslowwhentheyneedtomakedecisions,Ninja
it'sveryfastwhendoingdecisionsandevaluation• Itcontainsthebarestfunctionalitynecessarytodescribe
arbitrarydependencygraphs• Itisintendedtobeusedwithaseparateprogramgeneratingits
inputfiles
DesignGoals
ninja-Cout/Debug-j10AppRTCDemo
• Veryfast• Verylittle
• Verysimple
InstallandRun
InstallandRunNinja
gitclonehttps://chromium.googlesource.com/chromium/tools/depot_tools.git
$exportPATH="$PATH":`pwd`/depot_tools
• Fetchdepot_tools:
• Adddepot_toolstoyourPATH:
• Windows(Cygwinandnon-Cygwin):
Thesame!!!
GYP(generateyourproject)
Whatisit?• AtoolfortheChromiumprojectthatgeneratesnativeVisual
Studio,XcodeandSCons(SoftwareConstructionTooland/ormakebuildfilesfromaplatform-independentinputformat
• Averyreadableformattohandlecomplexconfigurationsanddependencies
FileStructure{'targets':[{'target_name':'foo','type':'executable','sources':['linux_specific.cc',],'conditions':[['OS!="linux"',{'sources!':[#Linux-only;excludeonotherplatforms.
WebRTCforAndroid
GettheSourceCode
Runthefollowingcommandinanemptyfolder:
$gclientconfighttp://webrtc.googlecode.com/svn/trunk$gclientsync--force
Openthefile.gclientlocatedinthefolderyoudidthesyncandaddtheiosandmactargets:
BuildtheProjectforiOS
• Createanewfileandnameitmakeall-ios.sh• Changethepermissionschmod+xmakeall-ios.sh
• Openthemakeall-ios.shfile(local)• Runitandsearchfortheall.xcodeprojfile• OpenitwithXcodeandbuildeverything
CreateaProjectFollowthisstepshttps://github.com/GiorgioNatili/TLC-Native-iOS
(comingsoon!)
Runthedemo!
Q&A
Thanks!