error resilient video coding - sharif

Error Resilient Video Coding

Instructor: Hamid R. Rabiee

Spring 2012

Digital Media Lab - Sharif University of Technology2

Outlines

Problems of video streaming

Error concealment

Error resilient video coding

Error-induced problems

Multiple description video coding

What Is Problem?

Internet: best effort service

No guarantees for:

Bandwidth

Loss rates

Goal: reliable delivery of high quality video over the Internet


Problem Solution

Bandwidth Rate control

Loss Error control

Rate Control

Two main steps:

Estimation of the available bandwidth

Matching the video rate to available bandwidth

It can be perform in both sides.


Sender ReceiverDynamic

Bandwidth

Error Control

Forward Error Correction (FEC)

Retransmission

Error-resilient video coding

Error concealment


Source Coding (Current lecture)

Channel Coding

Error Types

Type Example

Few bit errors and packet loss Videophone over PSTN

Packet loss up to 30% Video streaming over the Internet

Burst Error Wireless video


Error Concealment: Motivation

Problem

Lost information caused by transmission errors.

Goal

Estimating lost information in order to conceal (no correct) the error.

Observation

High amount of temporal/spatial correlation in video.

General Approach

Performing interpolation/regression to estimate the lost information based on

correctly received neighboring ones.


Error Concealment: Approaches

There are three approaches with respect to considered correlation:

Spatial

Temporal

Spatio-Temporal


Spatial Error Concealment

Estimate the values of

the lost pixels f*

based on the correctly

received neighboring

pixels y.

Main Assumption:

Spatial Smoothness


Motion-Compensated Temporal Error Concealment

Estimate lost block as

motion-compensated

block from prior frame

Main Assumption:

Temporal Smoothness


Error Concealment: Challenges

Usually lose more than a single

macroblock

E.g. lose a row of macroblocks or an

entire frame

Low correlation in spatial domain

for some videos

Low correlation in temporal domain

for some others.


Error Resilient Coding

Two basic error-induced problems:

Loss of bitstream synchronization

Decoder: what bits correspond to what parameters?

Error in Huffman codeword

Incorrect state and error propagation

Different states in encoder and decoder

Design the compression algorithm in the way that generates bitstreams

resilient to channel errors.


Loss of Bitstream Synchronization

Decoder does not know what bits correspond to what parameters

In variable length coding (VLC), single bit error can result in significant

subsequent loss

Fixed length code does not have this problem but suffer from low

compression ratio.

However, VLC is widely used.



First Solution

Use resync markers

Use a marker codeword different from all codewords.

Restart decoding after marker.

Location of Markers:

Strategic locations such as beginning of frame, slice, …..

Periodic locations



Second Solution:

Reversible variable length coding (RVLC):

Conventional VLC codes are only decodable in forward direction

RVLC codes provide the ability of decoding also in backward direction

If an error is detected, jump to the next resync position and start decoding

backwards, enabling partial recovery of data.



Third Solution:

Data Partitioning

Bits near the resync markers are more likely to be correct.

Place most important information immediately after the resync

marker.


Summary: Loss of Bitstream Synchronization

Solutions:

Resync markers at strategic locations

Resync markers at periodic locations

Reversible VLC’s (RVLC’s)

Data partitioning


Incorrect state and error propagation

Incorrect state and error propagation at decoder:

An error causes the reconstructed frame (state) at the decoder to be

incorrect

Decoder’s state is different from encoder’s, leading to mismatched

predictions and often significant error propagation.


Incorrect state and error propagation: Solution

Limit effect of error propagation: for example reinitialize prediction loop.

How?

All I-frames

Usage of intra-coded frames degrades compression ratio.

Periodic I-frames

This approach leads to existence of group of pictures (GOP).

Partial intra-coding of frames

Individual macroblocks are intra-coded

Periodic

Preemptive


Intra-Coding Approaches

Advantages

Limits the effect of error propagation

Disadvantages

High bit rate required for intra coding limits its use

Optimal tradeoff between on intra/inter mode, depends on accurate knowledge

of channel characteristics


Scalable Video Coding

Codes video into a base layer and one or more enhancement layers

Temporal

I and P frames in base layer and B frames in enhancement layer.

Spatial

Down-sampling and up-sampling in encoder and decoder

respectively.

SNR

Combination of coarse and fine quantization

Best match for networks which support different qualities of service


Multiple Description Video Coding

Code video into a number of descriptions, each of roughly equal

importance

Properties:

Receiving either bitstream leads to good quality video

Receiving both bitstreams leads to highest quality video


Single Description versus Multiple-State Approach

Single description coding:

Multiple description coding:


Acknowledgement

This lecture is based on the lecture provided by John G. Apostolopoulos.


error resilient video coding - sharif

Documents