high efficiency video codec

High Efficiency Video Coding HEVC/H.265

Tejus Adiga M Department of Electronics and Communication, NMAMIT, Nitte.

Presented By:

A New Standard for Video Coding• Video is going richer, all growing the bit rate

• Frame size– 4K to 8K Ultra HD• Temporal resolution - 24 FPS to 240 FPS• Color resolution, bit depth - 8bpc to 12 bpc• Stereo and 3D Video Technology

• Process intensive Algorithms in consumer electronics is now affordable• Devices for Recording/Displaying HD to ultra HD video is affordable• Multi Threaded multi Core CPUs and hardware Accelerators• Hardware Video Decoders

• Necessary video data rate grows faster than feasible network transport capacities• Better Video compression standard needed.

• New ITU-T standard H265 addressing needs of an efficient video coder of future• Same Visual quality – 50% reduction in bitrate than H264/MPEG4 part 10.• Better visual quality – Same bitrate as of H264/MPEG4 part 10.

May 3, 2023 Department of Electronics and Communications, NMAMIT, Nitte.2

May 3, 2023 Department of Electronics and Communications, NMAMIT, Nitte. 3

Block Diagram

Fig 1: Block diagram of HEVC encoder

CABAC: Context Adaptive Binary Arithmetic Coding.

CTU: Coding Tree Unit. 64x64 Luma block (Y) and two 32x32 Chroma block (U and V) as in YUV 420 format.

SAO: Sample Adaptive offset: Non linear amplitude mapping to better reconstruct original pixel amplitudes.

May 3, 2023 Department of Electronics and Communications, NMAMIT, Nitte.

4

Coding Modes• Intra Frame Coding (I Frames)

• CTU to encode is predicted from previously encoded and decoded CTUs of upper

rows and left column of the same frame.

• Inter Frame Coding (P and B frame)

• Coded using Motion Estimation and Motion Compensation or Skip mode.• P Frame: Coded with reference to Previously encoded and decoded frames.

• B Frame: Coded with reference to Previous and Future encoded and decoded

frames.

May 3, 2023

Block Partitioning• Frame is partitioned into Non Overlapping 64x64 Coding

Tree Units (CTU)• A CTU consists of 64x64 Luma Coding Tree Block (CTB)

and two 32 x 32 Chroma CTB for U and V components. • Every CTB is further split into one or multiple Coding Unit

(CU) in Quad tree structure.• Every CU has an associated Prediction Unit (PU) and

Transform Unit (TU).• Prediction Unit (PU) is partitioning to perform Intra/Inter

prediction. It has root in CU and can be further split down to 4x4 block including rectangle blocks like 8x4, 16x8 etc.

• Transform Unit (TU) is partitioning to perform Transform and Entropy coding of Prediction Residual blocks down to 4x4 size.

Department of Electronics and Communications, NMAMIT, Nitte.

CTULuma (Y)

CTB CTB

CTB

CB

CB

CB

CB

CB

CB

CB

CB

CBCB

5


Intra Frame Coding• Transform Unit size can be from 4x4 upto 32x32• 33 Directional Angular prediction with Planar and DC prediction

modes are used to predict a TB• A Prediction Block is formed with previously decoded boundary

samples from spatially neighboring TBs.• A PB with minimum prediction error is selected using the error

metric like SAD, MAD, SSE, MSE

• Prediction Residual block is found by

• Integer DCT is applied to Residual block with Butterfly computation.

• Integer DCT of size 4x4, 8x8, 16x16 and 32x32 computed with fast Separable Butterfly computation.

• Alternatively for only 4x4 residual block DST can be used as it better fit the statistical property that the residual amplitudes tend to increase as the distance from the boundary samples that are used forprediction becomes larger.


Transform

4x4 transform

8x8 transform


Transform


Scalar Quantization and Scaling• Integer DCT introduces Non Linearity.• DCT Matrix is split into two integer

matrix with appropriate scaling and combined with Quantization to form Mf

• Inverse Operation

. .

Quantization and Scaling

Inverse Quantization and Scaling


Scalar Quantization and Scaling• Transformed Residual coefficients are quantized with Scalar

Quantization.• Quantization Parameter (QP):

• Based on constraints like target bitrate, quality a value in between 0 to 51 need to be specified termed as Quantization Parameter.

• For every 6 QP value increase the Qstep doubles.• Lower the QP, higher the video quality but less compression

• Quantization is done by multiplying Mf with Residual block.

• Inverse Quantization at decoder side can be done by

• The selection of the coefficient scanning order depends on the directionalities of the intra picture prediction.

• Scanning methods• Diagonal upright scanning: Default• Vertical Scanning: Used when the prediction direction is close to horizontal• Horizontal Scanning : Used when the prediction direction is close to vertical

• Scanned Coefficients are Entropy Coded using Context Adaptive Binary Arithmetic Coding (CABAC) and formatted in bit stream format as specified by H265 NAL Unit Syntax


Entropy Coding

• Current frame is encoded using previously decoded frame as reference.• Motion Estimation and Motion

Compensation is carried out at CB level to get he best match of current CB in the reference frame in the search window.• Offset (Δx, Δy) between current CB

and motion estimated CB referred to as Motion Vector.


Intra Picture Coding

• Difference between current CB and motion estimated CB is computed to get the Prediction Block which is Transformed, quantized and entropy coded along with Motion Vectors.• Various Fast motion estimation algorithms are proposed to speedup the

motion estimation process like Three Step, Logarithmic, Hexagonal, Diamond, Small Diamond etc.• Optimizations:

• Motion Vector can be predicted spatially and temporally.• Motion Vector Merge: Average of previously encoded motion vectors of current frame.• Skip Mode: MVs are copied from MVs of some other CB.


Encoding of motion Estimated Parameters

• Due to block processing of CBs there occurs discontinuities in adjacent blocks.

• To prevent excessive filtering filter are applied to PBs greater than 4x4 blocks.

• Filter parameters β and tc are derived from QP.

• Types of filtering:• Strong Filter: High variation in pixel values at boundary.P0, P1, P2 and Q0, Q1, Q2 are filtered with (1, 2, 2, 2, 1)/8, (1, 1, 1, 1)/4, and (2, 3, 1, 1, 1)/8• Weak Filter: Slowly varying pixel values at boundary.P0 and Q0 are filtered with (8, 19, −1, 9, −3)/32• No Filter: Skip filtering for intra frame 4x4 block, inter

frame blocks whose MV derived from same reference frame.


In Loop De-Blocking Filter


15

Comparison of HEVC with H.264

Block Partitioning in H264 and HEVCSubjective quality comparison of H264 and HEVC


16

• High degree of parallelism in decoding can take advantage of SIMD instruction sets like MMX, SSE, SSE2, 3D Now etc in Software Decoding.• As decoding operation is modular wrt CTUs it is better suited to run on

Graphics processors• Dedicated VLSI Decoders can take advantage of modularity of HEVC to

accelerate decoding.• Can provide better video quality for Real time video communication

applications.• Efficient storage of Video.

Advantages


17

• Better suited for parallel processing.• HEVC will accelerate Video services on Mobile devices.• Competitors like VP9 from Google and Vorbis from XIPH.org provide

royalty free codecs that is slightly inferior to HEVC.• HEVC Specification is not yet completely finalized. Opportunity to

improve efficiency further.

Conclusion and Future

1. Overview of the High Efficiency Video Coding (HEVC) Standard, G. J. Sullivan, J. R. Ohm, W. J. Han, and T. Wiegand, IEEE Transactions on Circuits and Systems for Video Technology, Vol 22, no 12, Jan 2013.

2. HEVC Complexity and Implementation Analysis, Frank Bossen, Benjamin Bross, Karsten Suhring, and David Flynn, IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, Jan 2013.

3. Comparison of the Coding Efficiency of Video Coding Standard – Including High Efficient Video Coding (HEVC), J.R. Ohm, G. J. Sullivan, H. Schwarz, T. K. Tan, T. Wiegand, IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, Jan 2013.

4. HEVC Deblocking Filter, Andrey Norkin, Gisle Bjøntegaard, Arild Fuldseth, Matthias Narroschke, Masaru Ikeda, Kenneth Andersson, Minhua Zhou, and Geert Van der Auwera, IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, Jan 2013.

5. Overview of H264/AVC video coding standard, T. Wiegand, G. J. Sullivan, G. Bjontegaard, and A. Luthra, IEEE Transactions on Circuits and Systems for Video Technology, vol 13, no 7, July 2003.

6. Overview of the scalable video coding extension of the H.264/AVC standard, H. Schwarz, D. Marpe, and T. Wiegand, IEEE Trans. Circuits Systems for Video Technology, vol. 17, no. 9, pp. 1103–1120, Sep. 2007.

7. HEVC: The New Gold Standard for Video Compression, Mahsa T. Pourazad, Colin Doutre, Maryam Azimi, and Panos Nasiopoulos, IEEE Consumer Electronics Magazine, July 2012.

8. White Paper: 4x4 Transformation and Quantization in H264 AVC, Iain Richardson.

9. A High-Throughput VLSI Architecture for Deblocking Filter in HEVC, Weiwei Shen, Qing Shang, Sha Shen, Yibo Fan, Xiaoyang Zeng, IEEE Trans. Circuits Systems for Video Technology, Vol 22, no 12, Jan 2013.

10. CE10: Core transform design for HEVC, A. Fuldseth, G. Bjøntegaard, M. Budagavi and V. Sze, JCTVC-G495, November 2011

11. ITU-T Recommendation H265, High efficiency video coding.


References