Flexible Fast Block Matching Flexible Fast Block Matching Algorithm Design based on Algorithm Design based on

Complexity-Distortion Complexity-Distortion OptimizationOptimization

Pol Lin Tai, Chii Tung Liu, Shih Yu Huang*, Jia Shung Wang

Department of Computer ScienceNational Tsing Hua University, Taiwan, R.O.C.

*Department of Information ManagementMing Chuan University, Taiwan, R.O.C.

OutlineOutline

Introduction - Block Motion Estimation

Traditional Fast Block Motion Estimation

Adaptive Complexity-Distortion Block Motion Estimation

Experimental Results Conclusions

Full-Search Block Full-Search Block MatchingMatching

Searcharea

Current block

motion vector = ),(minarg),(

vuMAEvu

N

i

N

jPvjPuibjia

NNvuMAE

1 1),(),(

1),(

vector MAE

(-3,-3) 50(-2,-3) 70

(-1,-3) 45

(-2,2) 0

(3,3) 120

Frame n-1 Frame n

x

y

0 1 2 3-1-2-31

2

3

-1

-2

-3

Motion vectorMotion vector

Three-Step Search

Fast Block Matching Fast Block Matching AlgorithmAlgorithm

4

2

x xxxx

x xx

1

motion vector = Pvu,P- and ),,(minarg),(

vuMAEvu

Current frame

1616

Referenced frame

Search area

Motion vector

Current block

fast block matching- three-step search, four-step search- new three-step search reduce check point (u,v)

Problems of the Problems of the Traditional BMTraditional BM

(1) Fixed Computational Complexity

10log(check point)

400500600700800900

10001100

4 9 14 19 24 29

FSBMTSSNew-TSSFSS

MSE

Problems of the Traditional Problems of the Traditional BM (cont.)BM (cont.)

(2) Complexity-Distortion Optimization

254=100

:1+9+1+25 = 36

Block 1 Block 2 block3 Block 4

The search path for TSS

50

50

50

50

60

40

40

40

30

30

30

30

80

70

60

50

Initial mv (0,0)

Step one(8 check points)

Step two(8 check points)

Step three(8 check points)

Complexity-Distortion Complexity-Distortion Optimization BMOptimization BM

10log(check point)

400500600700800900

10001100

4 9 14 19 24 29

CDOBM

FSBM

TSS

New-TSS

FSS

MSE

flexible computational complexity better complexity-distortion performance

Complexity-Distortion Complexity-Distortion OptimizationOptimization

50

50

50

50

60

40

40

40

30

30

30

30

80

70

60

50

ComplexityTarget Complexity subject to

min

i iDistortion

In Rate-Distortion Optimization

trellis system- dynamic programming Viterbi algorithm

In Complexity-Distortion Optimization

MSE Benefit ? Non-Reusable Heuristic search algorithm

0 20

How to Define MSE How to Define MSE benefitbenefit

In each checking step, select the block that could achieve the maximum improved distortion benefit to apply searching procedure

Block 1 Block 2 block3 Block 4

50

50

50

50

60

40

34

30

30

20

20

20

80

65

60

50

Initial mv (0,0)

Step one(8 check points)

Step two(8 check points)

Step three(8 check points)

1 23

4

Predictive Complexity-Distortion Predictive Complexity-Distortion Benefit ListBenefit List

300

Block1

500

Block2

200

Block3

400

Block4

100

Block5

MSE:

300

Block1

500

Block2

200

Block3

400

Block4

100

Block5

benefit:

Assign benefit

Sort500

Block2

400

Block4

300

Block1

200

Block3

100

Block5

Predictive Complexity-Distortion Benefit List

Updating PCDB ListUpdating PCDB List

Select Block2 to check candidate blocks (MSEcheck=250)

500

Block2

400

Block4

300

Block1

200

Block3

100

Block5

PCDB List

Update PCDB List

400

Block4

350

Block2

300

Block1

200

Block3

100

Block5

Benefit2= 0.7Benefit2

MSEcheck<MSE2 MSEcheck>MSE2

Update PCDB List

400

Block4 Block1

250

Block2

200

Block3

100

Block5

Benefit2=MSEcheck

300

Flexible Fast Block Matching Flexible Fast Block Matching Algorithm DesignAlgorithm Design

Select the first blockfrom PDCB list

Checking candidate blocks

Yes

No

End

InitializePCDB list

StartUpdating PCDB list Total complexity >

target complexity ?

Checking Candidate Motion Checking Candidate Motion VectorVector

Select Block2 to check candidate motion vector

500

Block2

400

Block4

300

Block1

200

Block3

100

Block5

PCDB List

Search patternSearch patternFSBM 3-Step

Experimental ResultsExperimental Results

QCIF: “Miss America”, “Suzie” SIF: “Football” full-search block matching, 3-step

search, new 3-step search, 4-step search

frame rate: 15Hz and 7.5Hz search area: SIF (-15,15), QCIF (-7,7) is set as 0.8

Experimental Results Experimental Results (cont.)(cont.)

The complexity-distortion performance comparison for the QCIF "Miss America" with (a) frame rate 15Hz

7

9

11

13

15

17

19

0 10 20 30 40 50

flexible-FSBM

flexible-TSS

flexible-New-TSS

flexible-FSSFSBM

TSSNew-TSSFSS

Complexity (10ln(check point))

Distortion (MSE)

Experimental Results Experimental Results (cont.)(cont.)

The complexity-distortion performance comparison for the QCIF "Miss America" with (a) frame rate 7.5Hz

11

16

21

26

31

36

41

0 10 20 30 40 50

flexible-FSBM

flexible-TSS

flexible-New-TSS

flexible-FSS

FSBMTSS

New-TSSFSS

Complexity (10ln(check point))

Distortion (MSE)

Experimental Results Experimental Results (cont.)(cont.)

video frame rate Low High Low High

15 FSBM New-TSS TSS FSS7.5 FSS New-TSS TSS FSS

15 FSBM New-TSS TSS FSS7.5 FSS New-TSS TSS FSS

15 FSS New-TSS FSBM FSS7.5 TSS New-TSS FSBM FSS

Football

The Best Algorithm The Worst Algorithm

Miss America

Suzie

complexity

The performance summary of the flexible BMAs

ConclusionsConclusions

Discuss how to utilize the limited computational complexity of the block matching algorithm to achieve the maximum quality of the compensated image under a target computational complexity

Propose predictive complexity-distortion benefit list technique

The flexible block matching algorithm design not only improves the efficiency of the traditional BMAs, but also provides a flexible motion estimation tool that allows user to terminate it at any computational complexity