1

Low-complexity mode decision for MVC

Liquan Shen, Zhi Liu, Ping An, Ran Ma and

Zhaoyang Zhang CSVT 2010

2

Outline

• Introduction• Early SKIP mode decision• Adaptively early termination• Fast mode size decision• Selectively intra coding in inter frame• Overall algorithm • Experiment results

3

Introduction

• Two frames of the same time instant from different views correspond to the same content with similar video characteristic.(prediction mode, RDcost, motion vector)

• SKIP mode and RDcost correlation among inter-views are not fully studied

• Early SKIP mode decision• Adaptively early termination• Fast mode size decision• Selectively intra coding in inter frame

View 0 View 1 View 2

4

Early SKIP mode decision

• Observation – Many MBs finally end up with being decided as SKIP mode

because they belong to background or motionless object.

• Previous method– If SKIP mode has an RDcost less than a predefined threshold,

the variable size ME and DE process are skipped.– How to find a reasonable threshold is a serious problem

• This approach method– Coding mode among inter-views are similar.– Utilizing DE to find corresponding MBs among different views,

coding information can be effectively shared and reused from the coded view channel.

5

Early SKIP mode decision

• Idea– When the corresponding MB in the frame of the same time

instant from the neighbor view is with SKIP mode, the MB tends to have SKIP mode as well.

GDV

6

Early SKIP mode decision

• In this work, coding information is derived from the forward inter-view reference frame.

• GDV is not exactly disparity between the current MB and the corresponding one, the mode of the corresponding MB and its eight neighboring MBs are used to estimate mode characteristics of a MB.

• A MB after quantization with all zero coefficient (All-zero block) is preferred to be coded with SKIP mode.– Once the SAD between a block and its matching block is

smaller than a threshold, the current MB is early decided as SKIP mode.

7

Early SKIP mode decision

• “I” the MBs whose corresponding MB and its eight neighboring MBs in the previously coded view are all coded with SKIP mode.

• “II” remaining MBs in the current frame

36%13.6%

8

Adaptive early termination

• Idea – The majority of best prediction modes after mode decision are

with large sizes such as 16x16, 16x8, 8x16.

• Threshold-based early termination

– N: 9

– When the RDcost of current MB is smaller than the threshold early termination

9

Adaptive early termination

• The accuracy of ET() : R/M x 100%– M: total number of early terminated– R: the number of MBs having the same prediction mode as full

mode decision among these M MBs

𝛽

𝑏𝑖𝑡𝑟𝑎𝑡

𝑒

𝑎𝑐𝑐𝑢𝑟𝑎

𝑐𝑦

< 0.2%

96% 𝛽=0.5

10

Fast inter mode size decision

• Idea– Large size are always chosen for MBs in the homogeneous

region, and small sizes are always chosen for MBs with active motion or rich texture.

– From the mode maps of encoded frames, learn the motion or texture characteristics of the video frame.

• Mode Complexity (MC) parameter is defined in [9]

– : mode-weight factor assigned base on the complexity of each mode• 16x16: 1, 16x8 and 8x16: 2, 8x8: 3, intra: 4

[9] L. Shen, T. Yan, Z. Liu; Z. Zhang; P. An; Lei Yang, Fast mode decision for multiview video coding, in Proc. IEEE ICIP, Nov. 2009, pp. 2593-2596.

11

Fast inter mode size decision

– : 1, : 2

12

Fast inter mode size decision

• Fast sub-mode decision – Based on the information from the corresponding 8x8 block

and its eight neighboring blocks in the previously coded view.• One of them with 4x4 complex sub-mode

– perform all sub-mode

• One of them with 8x4 or 4x8 normal sub-mode – perform 8x8, 8x4, 4x8

• Otherwise– perform only 8x8

13

Selective intra prediction in inter frame

• The intra mode are enabled when a MB satisfies one of the following two conditions:– One MB among the corresponding MB and its eight neighboring

MBs in the previously coded view is coded with intra modes.– RDcost value of inter prediction for current MB is larger than

the maximum of RDcost from these nine corresponding MBs.

Enableskip

14

Overall algorithm

• Step1: start mode decision for a MB

• Step2: locate the corresponding MB and its eight neighboring MBs from the previously coded view by GDV

• Step3: derive the prediction mode and RDcost of these MBs

15

Overall algorithm

• Step4: if the corresponding MB and its eight neighboring MBs are all coded with SKIP mode or the SAD value of SKIP mode for the current MB satisfied the AZB condition, perform early SKIP mode decision

• Step5: compute Thr for RDcost based on Eq.1 and MC based on Eq.2

• Step6: test inter16x16 and get the minimal RDcost. If RDcost is smaller than Thr, go to Step12; else if the current MB is simple mode, go to Step10; else go to Step8 (Early termination&Fast inter mode size decision)

16

Overall algorithm

• Step7: test inter16x8 and get the minimal RDcost. If the minimal RDcost is smaller than Thr, goto Step12; else if the current MB is normal mode, go to Step10; else go to Step9(Early termination&Fast inter mode size decision)

• Step8: test inter8x16 and get the minimal RDcost. If the minimal RDcost is smaller than Thr, goto Step12; else if the current MB is normal mode, go to Step10; else go to Step9(Early termination&Fast inter mode size decision)

• Step9: for each 8x8, when one of the corresponding block and its eight neighboring blocks – With 4x4, test all sub-mode– With 4x8 or 8x4, test inter8x8, 8x4, 4x8– Otherwise, test inter8x8

if RDcost is smaller than Thr, go to Step12; else go to Step10 (Early termination)

17

Overall algorithm

• Step10: check the conditions to skip intra prediciton– If RDcost of inter prediction for current MB is smaller than the

maximum of RDcosts from the corresponding MB and its eight neighboring MBs and these MBs are all coded with inter modes, skip intra prediction and go to Step 12; else go to Step 11.

• Step11: perform intra prediction

• Step12: determine the best prediction mode. Go to Step1 and proceed with next MB

18

Experiment results

• JMVM 6.0• GOP 16• Search range 64• QP varies from 24 to 36• Enable motion skip mode• Compare to JMVM & the fast inter frame prediction algorithm

for MVC (FIFPMVC) [7] & the selective DE and variable size ME algorithm (SDEME) [8] & fast mode decision algorithm (FMD) [9]

• Three views of test sequences are chosen for simulation. The second view is used for the implementation of the proposed algorithm

[7] X. M. Li, D. B. Zhao, X. Y. Ji, Q. Wang, W. Gao, A fast inter frame prediction algorithm for multi-view video coding, in Proc. IEEE ICIP, Sep. 2007, pp. 417-420,. [8] L. Shen, Z. Liu, S. Liu, Z. Zhang, and P. An, Selective disparity estimation and variable size motion estimation based on motion homogeneity for multi-view coding, IEEE Trans. Broadcast., vol. 55, no. 4, pp. 761-766, Dec. 2009. [9] L. Shen, T. Yan, Z. Liu; Z. Zhang; P. An; Lei Yang, Fast mode decision for multiview video coding, in Proc. IEEE ICIP, Nov. 2009, pp. 2593-2596.

19

Individual performance results

20

Individual performance results

21

Individual performance results

22

Combine results

23

Combine results