video streaming over the internet

Multimedia & Comm. LabMultimedia & Comm. Lab

Video Streaming over the Internet

98/11/25

정승훈shjeong@mmlab.snu.ac.kr

2Multimedia & Comm. LabMultimedia & Comm. Lab

Contents

Introduction

Architecture of Adaptive Video Streaming

Congestion Detection

Rate Adjustment Schemes

Examples

Conclusion

3Multimedia & Comm. LabMultimedia & Comm. Lab

Introduction

Video Streaming Problems Network

Protocol Issues Congestion control

Receiver capability receiver’s video playback capability

Server capability contention for CPU and I/O resources

QoS SNR Displayed Frame Rate

4Multimedia & Comm. LabMultimedia & Comm. Lab

Introduction

Assumptions Clients have heterogeneous network capacity and

processing power Large numbers of clients may access the server

simultaneously Low startup playback latency The server maintains a large number of different video

streams The server and clients are connected through the Internet

where the dominant competing traffic is TCP-based.

5Multimedia & Comm. LabMultimedia & Comm. Lab

Introduction

Goals well-behaved and TCP-friendly streaming utilize a fair share of bandwidth maximize the overall quality of delivered stream minimize the storage requirement at the server and the

client minimize the playback delay minimize processing requirements at the server

6Multimedia & Comm. LabMultimedia & Comm. Lab

General Architecture

Internet ClientEncoder rate

controller

Videosource

Buffer

Server

Feedback Information

Q factor

7Multimedia & Comm. LabMultimedia & Comm. Lab

System Model

Internet

Video Source

ratecontroller

Feedbackcollection

IP

UDP

buffer

Synch. control

IP

UDP

buffer

DeMux / Monitor

V buffer A buffer

V decoder A decoder

Server

Client

TCP TCP

8Multimedia & Comm. LabMultimedia & Comm. Lab

Congestion Detection

Congestion Factors Packet receiving rate Packet Loss Ratio Packet End-to-end Delay Packet Interarrival Jitter

ACK / NACK based

Using packet receiving rate

Receiver-oriented

Server-oriented

9Multimedia & Comm. LabMultimedia & Comm. Lab

Congestion Detection

ACK / NACK based Columbia Univ. - Dynamic Rate Shaping using TCP congestion control client only sends ACK or NACK server detects network congestion from ACK/NACK

Using Receiving Rate Bamba - IBM ‘97 Receiver monitors packet receiving buffer calculates packet receiving rate sends receiving rate to the server

10Multimedia & Comm. LabMultimedia & Comm. Lab

Congestion Detection

Receiver-oriented Receiver monitors receiving buffer collecting congestion factors determines network congestion degree sends congestion degree to the server server regulate sending rate in accordance with congestion

degree

11Multimedia & Comm. LabMultimedia & Comm. Lab

Congestion Detection

Server-oriented ATC, INRIA RTP/RTCP based receiver collects Feedback information

highest sequence number received the number of packet lost packet interarrival jitter timestamps

sends Feedback info. by using RTCP packet. server determines network congestion.

12Multimedia & Comm. LabMultimedia & Comm. Lab

Rate Adjustment

Encoder-level rate shaping regulates quantization factor

Frame dropping

Macroblock filtering

Quantization filtering discarding DCT coefficients

Layered Coding

Playback dilation

13Multimedia & Comm. LabMultimedia & Comm. Lab

Encoder-level rate shaping

Model INRIA (Bolot ‘98) real-time encoder adjusts the maximum output rate of the encoder linear increase / multiplicative decrease Congestion if loss rate > 5% NoCongestion if loss rate < 2%

if Congestionmax_rate = max(max_rate/GAIN, MIN_RATE)

else if NoCongestionmax_rate = min(max_rate+INC, MAX_RATE)

14Multimedia & Comm. LabMultimedia & Comm. Lab

Frame Dropping

Model StreamWorks, OGIST server determines congestion degree server adjusts frame rate by frame dropping

frame rate send pattern2.5 I I5.0 I P I P10.0 I P P P I P P P15.0 I PB P PB I PB P PB20.0 I BP BP BP BI BP BP BP B30.0 IBBPBBPBBPBBIBBPBBPBBPBB

15Multimedia & Comm. LabMultimedia & Comm. Lab

Macroblock Filtering

block dropping

Macroblock 중 일부를 주기적으로 제거 Client 는 인접 Macroblock 으로 대치 .

Feature-oriented block dropping

16Multimedia & Comm. LabMultimedia & Comm. Lab

Quantization Filtering

SNR adaptation video quality 를 떨어뜨려 Data 크기를 줄인다 .

DCT coefficient 를 인위적으로 제거 DCT coefficient 를 표현하는 비트 수를 줄임 .

17Multimedia & Comm. LabMultimedia & Comm. Lab

Layered Coding

Off-line transcoding Video source 를 video quality 에 따라 여러 계층으로 나누어

Encoding. Frame dropping, Macroblock filtering, Quantization filtering

모두 적용 .

서버는 congestion degree 또는 요구되는 QoS 에 맞는 Layer 를 동적으로 선택하여 전송 .

Architecture Tech. Co. Adaptive Video Streaming - LCN ‘97

University of Southern California ‘98

18Multimedia & Comm. LabMultimedia & Comm. Lab

Playback Dilation

Client 의 Playback rate 를 줄이는 기법 Receive buffer 가 underflow threshold 를 넘는 경우 client 의 system capability 가 떨어질 경우 .

Packet buffering delay 를 인위적으로 늘려서 Client의 버퍼를 일정한 크기로 유지 .

19Multimedia & Comm. LabMultimedia & Comm. Lab

Examples

Berkeley Continuous Media Player ‘92 software feedback mechanism playback dilation

OGIST ‘97 user specification of presentation quality

frame rate, simple QoS Frame dropping

StreamWorks Session 설정시 frame rate 결정 Frame dropping

20Multimedia & Comm. LabMultimedia & Comm. Lab

Examples

Architecture Tech. Co. Adaptive Video Streaming QoS Adaptation

Server, Network, Client Status Report (Feedback Information) Layered Coding

Off-line Video Transcoding Adaptation table

21Multimedia & Comm. LabMultimedia & Comm. Lab

Adaptation Table

AdaptationID

SNRAdaptation

TemporalDilation (fps)

RequiredThroughput (Mbps)

Adaptation 1 8bits, 64 coeff, I/P/B 30 2.0

Adaptation 2 7bits, 64 coeff, I/P/B 30 1.0

Adaptation 3 7bits, 64 coeff, I/P 25 0.6

Adaptation 4 7bits, 24 coeff, I/P 20 0.4

Adaptation 5 7bits, 24 coeff, I 20 0.25

22Multimedia & Comm. LabMultimedia & Comm. Lab

Further Issues

Congestion Prediction ?

Multicast Issues Feedback Explosion problem heterogeneity problem

Video Streaming over DiffServ

Error propagation of lost packet Early packet discarding

23Multimedia & Comm. LabMultimedia & Comm. Lab

Conclusion

Video Streaming over the Internet Congestion Detection Rate Adjustment Related Works

Further Issues