nubomedia: the cloud infrastructure for webrtc and ims multimedia real-time communications

Nubomedia: the cloud infrastructure for WebRTC and IMS multimedia real-time

communications

Luis Lopez

lulop@kurento.org

WebRTC infrastructures

http://www.kurento.org2

Peer-to-Peer WebRTC Application (without media infrastructure)

WebRTC video stream

WebRTC Application based on media infrastructure

media infrastructure

Function of WebRTC media servers

http://www.kurento.org3

Transcoding media server

VP8 H.264

MCU/SFU media server

Recording media server

Evolution of media servers

http://www.kurento.org

4

What common WebRTC Media Servers do:• Transcoding• MCU• Recording

What future MediaServers will do:• Flexible processing• Augmented reality• Blending• Mixing• Analyzing• Etc.

Media ishere

Media goesthere

TranscodingMCU/SFURecording

Current media servers

Media ishere

Rich Mediagoes there

Future media servers

Media Events

Transcoding, MCU/SFU,Recording,

Enrich,Augment, Analyze,

Combine,Transform,Adapt, …

ContextContent

Commands

Why is this important

WebRTCMaking

callsContent and

entertainment

IoTM2MP2M

Multimedia

Making callsWARNING! Overcrowded

AdvertisingBroadcasting

GamingeLearning

Blue Ocean

Smart citiesEmergencies

SecurityBlue Ocean

Kurento Media Server

http://www.kurento.org6

Sink

SRC Sink

SRC

SRC

Sink

Sink

Media Element

• Provides a specific media

functionality

› Send/receive media

› Process media

› Transform media

• Exchange media through

› Sources

› Sinks

Media pipeline

• Chain of media elements

implementing the desired media

logic.

• The Media API provides the

capability of creating media

pipelines by joining media

elements of the toolbox

Media Element

Sin

k

SRC

Creating applications basing on Kurento Media Server

http://www.kurento.org7

Protocols&Codecs

ComputerVision

AugmentedReality

Mul sensoryMul media

MediaRepository

IPTVIntegra on

GroupCommunica ons

Sink%

H pEndpoint

Sink%

SRC%

RtpEndpoint

SRC$

DataChannelEndpointSink%

SRC%

WebRtcEndpoint

Sink%

SRC%

BarCodeReader

Sink%

SRC%

PointerTracker

Sink%

SRC%

FaceDetector

Sink%

SRC%

FaceOverlay

Sink%

SRC%

ChromaFilter

Sink%

SRC%

HeartRateBlender

Sink%

RecorderEndpoint

SRC$

PlayerEndpoint

Sink%

CdnUpload

Sink%

IPTVConnector Mixer

Sink

SRC Sink

SRC

Sink

SRC

Sink

Sink

Application 2

Sink

SRC

Sink

SRC

SinkSink

SRC

Sink

Application 3

Sink

SRC

SRC

Sink

SRC Sink

SinkSRC

Application 1

Toolbox of media elements

Beyond media servers: WebRTC clouds and the problem of scalability

http://www.kurento.org8

WebRTC Application based on media infrastructure

WebRTC Cloud

Cloud models for WebRTC infrastructures

http://www.kurento.org9

PaaSIaaS SaaSProvider- Computing resources

Developer- Installation- Administration- Security- Application logic

Provider- Development API

Developer- Application logic

Provider- Service

Developer- Nothing to do

WebRTC PaaS APIs: Requirements

• Requirements of WebRTC PaaS APIs– Functional requirements

• Media transport– Media endpoint– Media replication– Media routing

• Media persistence– Media storage– Media recovery

• Media processing– Transcoding– Analysis– Augmentation

– Non-functional requirements• Security• Dependability• Scalability

http://www.kurento.org10

WebRTC multimedia session

• RFC 5117– A multimedia session is an association among a group of participants

engaged in the communication via one or more RTP Sessions.

• Characterized by– Communication topology

• Graph of multimedia flows

– Multimedia processing• Function of each edge of the graph of media flows

http://www.kurento.org11

TransportTranscode

RecordAnalyze

Etc.

Scalability of RTC multimedia services

http://www.kurento.org12

Number of concurrent sessions

Nu

mb

er

of

use

rs p

er

sess

ion

WebRTC PaaS Phone system

TV Broadcasting

CDNs

Media Serverbased Services

“RTC calls”

“Home-madeRTC”

“Broadcasting Non-RTC”

“Beyond RTC calls”

Scalability of RTC multimedia services

http://www.kurento.org13

Number of concurrent sessions

Nu

mb

er

of

use

rs p

er

sess

ion

WebRTC PaaS Phone system

TV Broadcasting

CDNs

Media Serverbased Services

“RTC calls”

“Home-madeRTC”

“Broadcasting Non-RTC”

“Beyond RTC calls”

The scalability problem in “call” clouds

CallCallCall Call CallCall

CallCallCall Call CallCall

CallCallCall Call CallCall

Anatomy of WebRTC PaaS for call models: Flat Architecture

http://www.kurento.org15

Load

Balan

cer

Fun

ction

IaaS Cloud Manager

Ap

plicatio

n Se

rver

Fun

ction

Bro

ker Fu

nctio

n

Me

dia Se

rver

Fun

ction

Cloud Orchestrator

Cloud functions: IaaSmanager

http://www.kurento.org16

IaaS Cloud Manager

CloudOrchestrator

• Function– Provides APIs for IaaS

management• Images

• Instances

• Storage

• Metrics

• Security

• Etc.

• Requires– Physical infrastructure

IaaS Cloud Manager

CloudOrchestrator

ImageManagement

ComputingManagement

StorageManagement

NetworkingManagement

SecurityManagement

Metrics andKPIs

Infrastructure as a Service APIs

Load

%Balan

cer%

Func.on%

IaaS%Cloud%Manager%

Applica.

on%Se

rver%

%Func.on%

Broker%Fu

nc.on%

Media%Se

rver%

Func.on%

Cloud%Orchestrator%

Cloud functions: CloudOrchestrator

http://www.kurento.org17

IaaS Cloud Manager

CloudOrchestrator

• Function– Lifecycle management of the

platform• It acquires virtual resources and

allocate them to the specific services

– Runtime management with autoscaling• It scales out new service instances

in situations of peak load• It scales in service instances

whenever they are not required any longer

• Requires– Autoscaling rules

• Ex. If average load is over 60% add two new instances

ImageManagement

ComputingManagement

StorageManagement

NetworkingManagement

SecurityManagement

Metrics andKPIs

Infrastructure as a Service APIs

Load

%Balan

cer%

Func.on%

IaaS%Cloud%Manager%

Applica.

on%Se

rver%

%Func.on%

Broker%Fu

nc.on%

Media%Se

rver%

Func.on%

Cloud%Orchestrator%

Media Server Function

http://www.kurento.org18

SendReceiveAnalyze

AugmentEnrich

TransformTranscode

RecordProcess

Replicate

Media Server Instance

ApplicationServer

Instance

Media ControlProtocol

• Function– Provides media

capabilities• WebRTC transport

• Recording

• Transcoding

• Etc.

• Requires– Control Protocol

– Media Protocols

– Media Codecs

Media Server

Instance

Load

%Balan

cer%

Func.on%

IaaS%Cloud%Manager%

Applica.

on%Se

rver%

%Func.on%

Broker%Fu

nc.on%

Media%Se

rver%

Func.on%

Cloud%Orchestrator%

Broker Function

http://www.kurento.org19

Broker Function

ApplicationServer

Instance

Media Server

Instance

Media Server

Instance

Media Server

Instance

Media Server

Instance

• Function– Assigns “call” to specific media

server instances• Give me a media server

instance to take care of this call

– “call” are never split among media servers

• Requires– Scheduling policy

• Round robing• Random• Less load• Etc.

– Registration of MSis• All media server instances need

to be known by the broker

Call Call

Load

%Balan

cer%

Func.on%

IaaS%Cloud%Manager%

Applica.

on%Se

rver%

%Func.on%

Broker%Fu

nc.on%

Media%Se

rver%

Func.on%

Cloud%Orchestrator%

Application Server

http://www.kurento.org20

Application Server Instance

Signaling frontend

Security logic

API logic

Client request • Function– Signaling

• Send/receive of signaling messages

– Security logic• Authentication, Authorization,

Accounting

– PaaS API logic• Control of media server functions

for providing API semantics

• Requires– Signaling protocol

implementation• SIP, JSON, etc.

– Security rules• ACLs, CAP, etc.

– Specific logic• Media server dependent

Load

%Balan

cer%

Func.on%

IaaS%Cloud%Manager%

Applica.

on%Se

rver%

%Func.on%

Broker%Fu

nc.on%

Media%Se

rver%

Func.on%

Cloud%Orchestrator%

Load balancer

• Function– Distributes client requests

among available AS instances

– Usually stateful

• Requires– Balancing policy

• Round robin• Random• Less load• Etc.

• Scaling needs– Low

http://www.kurento.org21

Load Balancer Function

Client App.

ApplicationServer

Instance

Client App. Client App.

ApplicationServer

Instance

ApplicationServer

Instance

Load

%Balan

cer%

Func.on%

IaaS%Cloud%Manager%

Applica.

on%Se

rver%

%Func.on%

Broker%Fu

nc.on%

Media%Se

rver%

Func.on%

Cloud%Orchestrator%

The flat Nubomedia implementation

http://www.kurento.org22

Load

Balan

cer

Fun

ction

IaaS Cloud Manager

Ap

plicatio

n Se

rver

Fun

ction

Bro

ker Fu

nctio

n

Me

dia Se

rver

Fun

ction

Cloud Orchestrator

Instance Lifecycle

http://www.kurento.org 23

OpenStack Swift

OpenStack Glance

Media ServerImage

ApplicationServer Image

BrokerImage

Load BalancerImage

Heat OrchestrationTemplate

OpenStack Nova

ScalabilityGroup

Open Stack HeatPacker

AutoscalingRules

Launch configurations

ScalabilityGroup

ScalabilityGroup

ScalabilityGroup

Load BalancerInstance

ApplicationServer

Instance

ApplicationServer

Instance

BrokerInstance

Media Server

Instance

Media Server

Instance

Media Server

Instance

Scalability of RTC multimedia services

http://www.kurento.org24

Number of concurrent sessions

Nu

mb

er

of

use

rs p

er

sess

ion

WebRTC PaaS Phone system

TV Broadcasting

CDNs

Media Serverbased Services

“RTC calls”

“Home-madeRTC”

“Broadcasting Non-RTC”

“Beyond RTC calls”

Beyond calls: convergence of broadcasting and phone-like services

http://www.kurento.org25

Users in call Monitoring users

Users can dynamically change their role

The scalability problem in “beyond call” clouds

http://www.kurento.org26

Media stream

On

e to

MA

NY

Anatomy of WebRTC PaaS for call models: Hierarchical Architecture

http://www.kurento.org27

Load

Balan

cer

Fun

ction

IaaS Cloud Manager

Ap

plicatio

n Se

rver

Fun

ction

Bro

ker Fu

nctio

n

Me

dia Se

rver

Fun

ction

Cloud Orchestrator

Media Server Function

http://www.kurento.org28

Media Server

Instance

Media Server

Instance

Media Server

Instance

Media Server

Instance

Media Server

Instance

Media Server

Instance

• Function– Provides elastics media

capabilities• Strong dependencies

among media server instances

• Media servers connect following a specific topology

• Requires– Glue mechanism among

media server instances

Load

%Balan

cer%

Func.on%

IaaS%Cloud%Manager%

Applica.

on%Se

rver%

%Func.on%

Broker%Fu

nc.on%

Media%Se

rver%

Func.on%

Cloud%Orchestrator%

The elastic media server

http://www.kurento.org29

• Elasticity– On the number of media

pipelines• Number of concurrent

sessions

– On the number of elements per media pipeline• Number of concurrent users

per session

– Media Pipeline• Distributed media pipeline

• Rigidity – The media element is a

monolithic (non distributed) entity

Sink

SRC

SRC

Sink

Sink

SRC

Sink

SRC

SRC

Sink

SRC

Sink

SRC

Sink

Load

%Balan

cer%

Func.on%

IaaS%Cloud%Manager%

Applica.

on%Se

rver%

%Func.on%

Broker%Fu

nc.on%

Media%Se

rver%

Func.on%

Cloud%Orchestrator%

Broker Function

http://www.kurento.org30

Broker Function

ApplicationServer

Instance • Function– Assigns “call legs” to specific

media server instances• Give me a media server instance to

take care of this call

– “call” are split among different media server instances

• Requires– Scheduling policy

• Topology aware• Network aware• SLA aware• Etc.

– Very complex problem• Leg adding may require additional

media server instances• Churn is very complex to manage

Media Server

Instance

Media Server

Instance

Media Server

Instance

Media Server

Instance

Media Server

Instance

Load

%Balan

cer%

Func.on%

IaaS%Cloud%Manager%

Applica.

on%Se

rver%

%Func.on%

Broker%Fu

nc.on%

Media%Se

rver%

Func.on%

Cloud%Orchestrator%

Hierarchical Nubomedia implementation:Work in progress

http://www.kurento.org3131

Load

%Balan

cer%

Func.on%

IaaS%Cloud%Manager%Applica.

on%Se

rver%

%Func.on%

Broker%Fu

nc.on%

Media%Se

rver%

Func.on%

Cloud%Orchestrator%

Ownimplementation

Lessons learnt

• CAP theorem– You cannot be available and consistent at the same time

• Always chose availability

• Managing failures is tricky– Beware of poison messages– Logs need to be centralized

• Video streams are not like water streams– FIRs & PLIs will break bandwidth

• …. and the battery

• Scaling out is simpler than scaling in– Move or wait?

• Managing video quality with limited BW is THE PROBLEM– Simulcast Vs Transcoding

http://www.kurento.org32

Nubomedia: the elastic RTC multimedia infrastructure

http://www.kurento.org33

Thanks

Partners

Luis Lopez

lulop@kurento.org