It’s all coming togetherAndreas Pehrson, Oslo WebRTC, 2015-08-20

Andreas Pehrson

WebRTC Competency Center

Working for Telenor Digital

Working with Mozilla

Working in Taipei

@pehrsons

appear.in/pehrsons

MediaStreams are everywhere!

Producers• GetUserMedia

• RTCPeerConnection

• WebAudio

• VideoWorker (early prototype)• CaptureStream (<audio>,

<video> and <canvas>)

Consumers• RTCPeerConnection

• MediaRecorder

• WebAudio

• VideoWorker (early prototype)

����

���

�

Audio Processing with WebAudioPeerConnection to WebAudio● Works in Firefox

● Does not work in Chrome

● Example: WebRTC -> Speech Analyser -> Speakers

WebAudio to PeerConnection● Recently fixed in Firefox

● Works in Chrome

● Example: Microphone -> Voice Filter -> WebRTC

● DEMO Time!

Video Processing with VideoWorker

● Mozilla-developed proof-of-concept, Project “FoxEye”

● Workers registered as Monitors and Processors

● Where should the standardization work live? - Discussed in W3C now

● Funny Hats are coming to the Web!

● DEMO Time!

Canvas.captureStream()Sample Use Cases:

● Game streaming / recording

● Collaborative Whiteboard

● Video Compositing

○ Forging several video streams into one

○ Wait, isn’t that an MCU?

● DEMO Time!

Standard appear.in use caseMesh network - everyone connects to everyone

Example - 5 Peers● Each client encodes and decodes 4 streams = 20 streams total

● Too much for mobile devices

JS-Based Client Side MCU!● Video Compositing in 2D Canvas, streamed with captureStream()

● Audio Mixing in WebAudio

● Clients encode and decode 1 stream each - excellent case for mobile

● Implemented in appear.in in less than a week

● Firefox running Headless in Amazon’s cloud

● DEMO Time!

Thank you