Implementation and Study of Unified Loop Filter in H.264

EE 5359 Multimedia ProcessingSpring 2012

Guidance : Prof K R Rao

Pavan Kumar Reddy Gajjala1000769393

Pavankumar.gajjala@mavs.uta.edu

Project Objective• The project is based on implementation and study of unified

loop filter for video coding(H.264)[1], which suppresses the quantization noise optimally and improves the objective and subjective qualities of the reconstructed picture simultaneously.[2]

• Unified loop filter unifies nonlinear enhancement filter and linear restoration filter within the classical optimization framework of least mean square error.[2]

• The motivation to unify de-blocking filtering and Wiener filtering into one filtering framework is that, if the number of the sources causing information loss is reduced, the capability of picture restoration can be further improved.[2]

Overview of H.264

• H.264 or MPEG-4 part 10: AVC [12] is the next generation video codec developed by MPEG of ISO/IEC and VCEG of ITU-T, together known as the JVT (Joint Video Team). The H.264/MPEG-4 AVC standard, like previous standards, is based on motion compensated transform coding method.

• H.264 also uses hybrid block based video compression techniques such as transformation for reduction of spatial correlation, quantization for bit-rate control, motion compensated prediction for reduction of temporal correlation and entropy coding for reduction in statistical correlationFig 1.1 shows the H.264 encoder block diagram and Fig 1.2

shows the H.264 decoder block diagram.

Fig. 1.1: H.264 encoder block diagram [12]

Fig. 1.2: H.264 decoder block diagram [12]

H.264/AVC profiles

• H.264 standard [1]is defined with a large variety of coding tools. This is done to make sure that standard caters to all classes of applications.

• However, not all tools are required for a particular application. So, the coding tools are segregated into different groups called profiles.

The basic profiles defined in the standard are shown in Fig. 2

Fig. 2: Profile structure in H.264 [12]

Unified Loop Filter

• The unified loop filter [2] unifies nonlinear enhancement filter (for removing blocking and ringing artifacts) and linear restoration filter (for improving coding efficiency) within the classical optimization framework of least mean square error (LMSE).

• Nonlinear similarity-ordered statistics filter is concatenated with linear spatially ordered statistics filter, a.k.a. Wiener filter, to form the unified loop filter [2].

H.264 encoder block diagram with unified loop filter is given in Fig 3.

Fig 3: H.264 Encoder block diagram with unified loop Fig 3: H.264 Encoder block diagram with unified loop filter.[2]

Fig 3: H.264 Encoder block diagram with unified loop filter.[2]

Unified loop filter Design

• For each nonlinear and linear group, unified loop filter should be carefully designed to meet different quantization error characteristics.

• Enhancement pixels of luma component :The nonlinear part consists of one 12-tap diamond filter, whereas the linear part consists of four kinds of taps (1-tap, 13-tap, 25-tap, and 41-tap) diamond filters with quadrant symmetry, as shown in Fig. 4(a). [2]

• Restoration pixels of luma component: The nonlinear part consists of one 8-tap cross filter, whereas the linear part consists of three kinds of taps (13-tap, 25-tap, and 41-tap) diamond filters with central point symmetry, as shown in Fig. 4(b). [2]

Fig. 4(a): Construction of classification-based unified loop filter for enhancement pixels of luma component [2].

Fig. 4(b): Construction of classification-based unified loop filter for restoration pixels of luma component [2].

• Enhancement and restoration pixels of chroma component: The nonlinear part consists of one 4-tap diamond filter, whereas the linear part consists of two kinds of taps (1-tap and 13-tap) diamond filters with quadrant symmetry, as shown in Fig. 5(a) and Fig. 5(b).

Fig. 5: Construction of classification-based unified loop filter for (a) enhancement pixels and (b) restoration pixels of chroma component [2].

• Since the unified loop filter for each group has different combinations of nonlinear part and linear part, the best combination for each group is decided by RDO selection on a frame basis,[2]

• Where D is the distortion between the filtered frame and the original frame, and R is the number of bits for the filter side information which includes filter tap type and filter coefficient quantization bits.

ACRONYMS AND ABBREVIATIONS

• ALF: Adaptive Loop Filter• AVC: Advanced Video Coding• CABAC: Context-based Adaptive Binary Arithmetic Coding• CAVLC: Context-based Adaptive Variable Length Coding• DLF: De-blocking Loop Filter• DPB: Decoded Picture Buffer• DVB: Digital Video Broadcasting• FMO: Flexible Macro block Ordering• GOP: Group of Pictures• ISO: International Standards Organization• ITU: International Telecommunication Union• JVT: Joint Video Team

• LMSE: Least Mean Square Error• MC: Motion Compensation• MDCT: Modified Discrete Cosine Transform• ME: Motion Estimation• MPEG: Moving Picture Experts Group• PSNR: Peak Signal to Noise Ratio• RDO: Rate Distortion Optimization• SI: Switching I• SP: Switching P • SSIM: Structural Similarity Index Metric• ULF: Unified Loop Filter• VCEG: Video Coding Experts Group

References• [1] Kwon Soon-kak, A Tamhankar and K R Rao, “Overview of H.264/MPEG-4 part

10”, 4th EURASIP Conference on Video/Image Processing and Multimedia Communications, vol. 1, pp. 1-51, July 2003.

• [2] Y. Liu, “Unified Loop Filter for Video Compression”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 20, no. 10, pp. 1378 – 1382, Oct. 2010.

• [3] T. Wedi “Adaptive interpolation filter for motion compensated prediction”, IEEE International Conference on Image Processing (ICIP2002), New York, USA, vol. 2, pp. II-509 - II-512, Sept. 2002.

• [4] A. Bovik , T. Huang and D. Munson, “A generalization of median filtering using linear combinations of order statistics”, IEEE Transactions on Acoustics, Speech and Signal Processing, vol. 31,no. 6, pp. 1342 – 1350, Dec. 1983.

• [5] Y. Liu and Y. Huo “Unified loop filter for high-performance video coding”, IEEE International conference on Multimedia and Expo (ICME), pp. 1271-1276, July 2010.

• [6] S. Wittmann and T. Wedi , “Transmission of post-filter hints for video coding schemes”, IEEE International Conference on Image Processing (ICIP), vol. 1, pp. 81-84, Sept. 16 2007-Oct 19 2007.

• [7]P. List, A. Joch, J. Lainema, G. Bjontegaard and M. Karczewicz, “Adaptive de blocking filter”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, no. 7, pp. 614-619, July 2003.

• [8] C. Qian, Z. Yunfei, Y. Peng, L. Xiaoan, J. Sole, X. Qian, E. Francois and W. Dapeng,

“Classified quad tree-based adaptive loop filter”, 2011 IEEE International Conference on Multimedia and Expo (ICME), pp. 1-6, 15 July 2011.

• [9] A. Bovik, T. Huang and D. Munson , “Nonlinear filtering using linear combinations of order statistics”, IEEE International Conference on Acoustics, Speech, and Signal Processing, pp. 2067-2070, May 1982.

• [10] I. E. Richardson, “The H.264 advance video compression standard”, 2nd Edition, Wiley 2010.

• [11] H.264/AVC JM reference software. Website: http://iphome.hhi.de/suehring/tml/download

• [12]T. Wiegand, “Overview of the H.264/AVC video coding standard”, IEEE Trans. on Circuits and Systems for Video Technology, vol. 13, pp. 560-576, July 2003.

• [13] C. Tomasi and R. Manduchi, “Bilateral filtering for gray and color images”, Computer Vision, Sixth International Conference, pp. 839-846, Jan 1998.

• [14] Y. Chiu and L. Xu, “Adaptive (Wiener) filter for video compression”, ITU-T SG16 Contribution, C437, Geneva, April 2008.

• [15] Z. Wang et al, “Image quality assessment: from error visibility to structural similarity”, IEEE transactions on Image processing, vol. 13, pp 600-612, April 2004.

• [16] X. Wang, “Recursive algorithms for linear LMSE estimators under uncertain observations”, IEEE Transactions on Automatic control, vol. 29, pp 853-854, Sept. 1984.

•

Thank You