Feature-Based Intra-/InterCoding Mode Selection for H.264/AVC

C. Kim and C.-C. Jay Kuo

CSVT, April 2007

Outline

Introduction Overview of Proposed Algorithm Feature Selection Feature Space Partitioning Coding Mode Prediction Experimental Results

Introduction

Inter/Intra Mode Decision in H.264 Skip mode, direct mode, intra modes, and inter

modes Full mode decision

Testing all possible modes and then choosing the best mode with smallest cost

Fast algorithms Selection of optimal inter-prediction mode Selection of optimal intra-prediction mode Binary decision of intra/inter mode

Overview of Proposed Algorithm

Motion activity

f1, Residual of intra prediction

f0, Residual of inter prediction

MB

Risk-Free

Risk-Tolerable

Risk-Intolerable

Choose min(f0,f1)

Compute risk- minimizing mode

Full mode decision

Feature Selection (1/4)

Intra mode feature Calculate SATD for 5 modes

DC, vertical, horizontal, diagonal down-left, and diagonal down-right

Let f1 or fIntra be the SATD of the MB of the chosen modes

Feature Selection (2/4)

Inter mode feature MV is obtained by

MVFAST + Two more candidates Residual of every visited point is remembered

in the memory Search points of a MB < 512

Let f0 or fInter be SATD of MB residual of the chosen MV

(i,j)

(i-1,j-1)

nn-1

Feature Selection (3/4)

Motion activity classification

Motion activity, decision error, and skipped frames Decision metric

df = f1 – f0 Intra (Inter): df < 0 (df > 0)

Decision error probability P(df<0inter)+P(df>0intra)

22yx vvMV

Feature Selection (4/4)

Motion activity, RD cost difference dc, and feature difference df dc = (D1 +1R1) - (D0+ 0R0) Positive (Negative) if inter (intra) is better

Low motion High motionmedium motion

Best intra mode Best inter mode

Feature Space Partitioning

The 3-D feature space is partitioned into three regions (off-line)

Lp: normalized RD cost between the best mode and the wrongly selected mode

pp

pp

p

LLR

LLLR

LLR

MVffF*

eintolerabl

*freetolerable

freefree

10 |]|,,[

Inter mode featureIntra mode feature

Motion activity

Threshold

TT

TT

DR

RDRRLp

)ˆ()ˆ(

Feature Space Partitioning

Let every cell has about equal training data |MV|

f0

f1

Feature Space Partitioning

Getting training data from Akiyo, Hall Monitor, Foreman, Coastguard, Stefan, Table Tennis, and Mobile.

Coding Mode Prediction (1/4)

Risk-Free region Distribution of f0 and f1 in a given motion

classbased on feature difference

Risk free

Coding Mode Prediction (2/4)

Risk-tolerable/-intolerable region

Risk-tolerable and Risk-intolerable

Coding Mode Prediction (3/4)

Risk-tolerable region Risk function

1

0

1

0

)|~(~

i jjiij mmPCR

),~( jiij mmPC

)(

~

j

ij

mP

C

i

dFFfFmP j)()|(

)~()~|( iij mPmmP

1

0

)()(i

ii

dFFfFR

1

0

)|()(j

jiji FmPCFFor simplicity, let stands for cost instead of R

mj is the best modem0: intram1: inter

The chosen mode

Cost of deciding ~mi under mj

Coding Mode Prediction (4/4)

Risk-minimizing mode selection Mode selection rule

)|()|( 0001010 FmPCFmPC )|()|( 0101111 FmPCFmPC 0 0

Likelihood ratio1.Parametric2.Semi-parametric3.Nonparametric

Experimental Results (1/6)

Environments JM7.3a 32 x 32 motion search range Fast full search with 5 reference frames No B-frame QP= {10, 16, 22, 28, 34} 5 skipped frames

Experimental Results (2/6)

QCIF Table Tennis

Experimental Results (3/6)

QCIF Table Tennis

Computation complexity Saving time

Experimental Results (4/6)

QCIF Foreman 5 skipped frames

0 skipped frames

Experimental Results (5/6)

QCIF Stefan

Experimental Results (6/6)