H.264 OverviewBroadcast television and home entertainment

have been revolutionized by the advent of

digital TV and DVD-video. These applications

and many more were made possible by the

standardization of video compression technol-

ogy. The recent standard in the MPEG (Moving

Pictures Expert Group) series, MPEG-4, is

enabling a new generation of internet-based

video applications whilst the ITU-T H.263

standard for video compression is now widely

used in video conferencing systems.

H.264/AVC (aka MPEG-4 Part 10), the result

of the collaboration between the ISO/IEC

MPEG group and the ITU-T Video Coding

Experts Group (VCEG), is the latest video

coding standard. The goals of this standard-

ization effort were: enhanced compression

efficiency; network friendly video representa-

tion for interactive applications (video tele-

phony) and non-interactive applications

(broadcast applications, storage media appli-

cations, etc). H.264/AVC provides gains in

compression efficiency up to 50% over a wide

range of bit rates and video resolutions com-

pared to previous standards. Compared to

previous standards, the decoder complexity

is about four times that of MPEG-2 and two

times that of MPEG-4 visual simple profile.

Relative to prior video coding standards,

H.264/AVC introduces the following changes

� In order to reduce the blocking artifacts, an

adaptive loop filter is used in the prediction

loop to reduce blocking artifacts.

� A prediction scheme called intra prediction

is used that exploits spatial redundancy. In

this scheme, data from previously processed

macroblocks is used to predict the data for

the current macroblock in the current

encoding frame.

� Previous video coding standards use an 8x8

real discrete cosine transform (DCT) to exploit

the spatial redundancy in the 8x8 block of

image data. In H.264/AVC, a smaller 4x4

integer DCT is used which significantly

reduces ringing artifacts associated with

the transform.

� Various block sizes from 16x16 to 4x4 are

allowed to perform motion compensation

prediction.

� Previous video coding standards used a

maximum of half-pixel accuracy for motion

estimation.

� Inter prediction mode allows multiple

reference frames for block-based motion

compensation prediction.

� Context-adaptive variable length coding

(CAVLC) and context-adaptive binary

arithmetic coding (CABAC) are used for

entropy encoding/decoding which improves

compression by 10% compared to previous

schemes.

H.264 Baseline Profile Video DecoderCompressed H.264 bit-stream data is avail-

able on slice-by-slice basis whereas a slice is

usually a group of macro blocks processed

in raster scan order. Two slice types are sup-

ported in H.264 baseline profile. In an I-slice,

all macro blocks are encoded in intra mode.

In a P-slice, some macro blocks are predicted

using a motion compensated prediction with

one reference frame among the set of reference

frames and some macro blocks are encoded in

intra mode. H.264 decoder processes the data

on a macro block by macro block basis. For

every macro block depending on its charac-

teristics, it will be constructed by the predicted

part of the macro block and the residual

(error) part, which is coded using CAVLC.

Figure 1 shows the overall block diagram of

an H.264 baseline profile video decoder.

H.264 bit-stream passes through the “slice

header parsing” block. This block extracts

the information about each slice. In H.264

video coding, each macro block is categorized

as either coded or skipped. If the macro block

is skipped, then the macro block is completely

reconstructed using the inter prediction

module. In this case, the residual information

is zero. If the macro block is coded, then based

on the prediction mode it passes through the

“Intra 4x4 prediction” block or “Intra 16x16

prediction” block or “Inter prediction” block.

H.264 LibraryDetailed Reference for Verification and Design Exploration

SliceHeaderParsing

H.264ElementaryBit Stream

MacroBlock

Parsing

Scale &Inverse

Transform

Sub MBParsing

Skip MB

Prediction Path

Residual Path

InterPrediction

Intra 4x4Prediction

DeblockingFilter

Intra 16x16Prediction

CAVLCDecoding

Figure 1: Block diagram of H.264 baseline profile video decoder.

The output macro block is reconstructed

using the prediction output from the predic-

tion module and the residual output from the

“scale and transform” module. Once all the

macro blocks in a frame are reconstructed,

de-blocking filter will be applied for the

entire frame.

The “macro block parsing module” parses

the information related to the macro block,

such as prediction type, number of blocks

coded in a macro block, partition type,

motion vectors, etc. The “sub macro block”

parsing module parses the information if the

macro block is split into sub macro blocks of

one of the sizes 8x8, 8x4, 4x8, and 4x4 when

the macro block is coded as inter macro

block. If the macro block is not split into sub

macro blocks, all the three prediction types

(Intra16x16, Intra4x4, or Inter) can be used.

In inter prediction module, the motion

compensated predicted blocks are predicted

from the previous frames, which are already

decoded.

Intra prediction means that the samples of a

macro block are predicted by using the already

transmitted macro blocks of the same image.

In H.264/AVC, two different types of intra

prediction modes are available for coding

luminance component of the macro block.

The first type is called INTRA_4x4 mode and

the second type is called INTRA_16x16

mode. In INTRA_4x4 prediction mode, each

macro block of size 16x16 is divided into

small blocks of size 4x4 and prediction is

carried out individually for each sub-block

using one of the nine prediction modes avail-

able. In INTRA_16x16 prediction mode, the

prediction is carried out at macro block level

using one of the four prediction modes

available. Intra prediction for chrominance

components of a macro blocks is similar

to the INTRA_16x16 prediction of the lumi-

nance component.

H.264/AVC baseline profile video decoder is

using CAVLC entropy coding method to

decode the encoded quantized residual

transform coefficients. In CAVLC module,

the number of non-zero quantized transform

coefficients, the actual size and the position

of each coefficient are decoded separately.

The tables used for decoding these parameters

are adaptively changed depending on the

previously decoded syntax elements. After

decoding, the coefficients are inverse zigzag

scanned and form a 4x4 blocks which are

given to “scale and inverse transform module”.

In “scale and inverse transform module”,

inverse quantization and inverse transforma-

tion are done on the decoded coefficients

and form a residual data suitable for inverse

prediction. Three different types of transforms

are used in H.264 standard. The first type is

4x4 inverse integer discrete cosine transform

(DCT), which is used to form the residual

blocks of both luminance and chrominance

blocks. Second type is a 4x4 inverse Hadamard

transform, which is used to form the DC

coefficients of the 16 luminance blocks of the

INTRA_16x16 macro blocks. Third transform

is a 2x2 inverse Hadamard transform, which

is used to form the DC coefficients of the

chrominance blocks.

The 4x4 block transform and motion com-

pensation prediction can be the source of

blocking artifacts in the decoded image.

Filtering the block edges will improve the

final visual quality of decoded image. H.264

standard is using in-loop deblocking filter to

remove the blocking artifacts.

CoWare Signal Processing Designer Library ofH.264 Baseline Profile Video DecoderThe Signal Processing Designer library of

H.264 baseline profile video decoder system,

developed by iDeaWorks, is shown in Figure 2.

The decoder takes H.264 baseline profile

elementary bit stream from a source as NAL

units and decodes the compressed video

streams and displays and dumps the decoded

frames. Frame width, frame height, and NAL

unit size can be set using global parameters.

There are nine major blocks in the top level

hierarchy of H.264 video decoder, which are:

Figure 2.: CoWare Signal Processing Designer View of H.264 baseline decoder operation

2H.264 Library

3

Source Block

This block converts H.264 baseline profile

elementary bit stream into fixed sized frame

NAL units. Each frame NAL unit contains all

NAL units of a single frame. The block takes

H.264 bit stream file name and frame NAL

unit size as parameters. It gives frame NAL

unit as a vector of size specified by the input

parameter and end of sequence (EOS) flag

as outputs.

Parser Block

This module is used to parse the following

set of information

� Control data required for doing intra and

inter prediction

� Motion compensation data required for

quarter pixel interpolation

� Quantized residual transform coefficients

� Control parameters for performing scale

and transform

� Control data required for applying the

deblocking filter on the decoded image

� Control data required for display re-ordering

The parser works at frame level.

Frame to Macroblock Conversion Block

This is a multirate block, which converts

frame level parameters to macroblock level

parameters. The parser block drives this

block. The output rate is ((Frame Width x

Frame Height)/256) times the input rate.

Macroblock Process Block

The macroblock process block consists of

four major modules, which are

� Scale and transform module

� Prediction module

� Deblocking module

� Intra feedback data module

This block operates at macroblock level.

Frame Construct Block

This is a multirate block, in which the input

is coming in as MBs and the output is going

out as frame. This block takes vectors of size

532 bytes and forms a single frame. There

are three modules in this block, which are

� Polymorphic vector-to-scalars

conversion block

� Polymorphic scalars-to-vector

conversion block

� Pur filter data block

Display Reorder Block

This block is used to reorder the decoded

YUV display data and is also used to crop the

output YUV data according to cropping

parameters. This block takes decoded frame

data, display re-order control data, and end

of sequence flag + cropping parameters as

inputs and gives display frame data and a

flag indicating whether the display frame

data is valid or not.

Sink Block

This block is used to dump the decoded

YUV data into an output file. This block

takes display frame data, flag that indicates

the validity of frame data, control data for

cropping, and end of sequence flag as inputs.

The output file name can be set using para-

meters of sink block.

H.264 Library Use ModelsThe H.264 Baseline Video Decoder was devel-

oped as a optimized, architecture neutral ref-

erence standard for users incorporating

advanced video decoding in embedded or

stand-alone products. It consists of a fully

decomposed software C-based algorithm

encapsulated within CoWare Signal

Processing Designer. This environment

insures users can use this product as a refer-

ence tool for FPGA, ASIC, SoC, or embedded

IC implementations. The algorithms have

been decomposed down to the ‘leaf ‘or lowest

common functions. Parallelism and concur-

rency is fully exposed in the CoWare Signal

Processing Designer H.264 Design Library

and CoWare Signal Processing Designer

H.264 Source Library.

The algorithm model meets these industry

standards.

� ITU-T H.264 standard OR

� ISO/IEC MPEG-4 Part 10 standard

Included with the library are reference test

benches consisting of standard media video

feeds capable of exercising the algorithm and

any implementation of the reference library.

The decoder library is qualified with the JVT

standards compliance video streams.

The following packages are available:

� CoWare Signal Processing Designer H.264

Reference Library

–H.264 models as partitioned verification

reference

–Reference media streams

–Value: Validated, partitioned executable

reference inside CoWare flow for either

customer H.264 algorithm optimization

� CoWare Signal Processing Designer H.264

Design Library

–Same as reference library, but down to very

low level of granularity

–Reference module test benches

–Value: Verification and detailed reference

for the design and optimization of sub

functions

–Documentation: Detailed Algorithm docu-

mentation representing, Control, dataflow

and Memory requirements.

–Test Vectors: the scaling transform, predic-

tion, and de-blocking filter modules have a

full ten frames of test vectors for QCIF, CIF,

and HD 720p resolutions.

� CoWare Signal Processing Designer H.264

Source Library

–Requires H.264 Design Library

–Source code license agreement

–Specific jump-start service package included

–All leaf level blocks as source code

–Value: working C-code for embedded imple-

mentation or starting point for detailed

RTL design

Customer FocusCoWare provides a complete range of training,

support, design methodology consulting, and

integration services. Technical support

requests are handled directly by experienced

design engineers who are fully familiar with

the application of CoWare tools and method-

ologies to real-world designs. Training courses

are available at CoWare offices or at the

customer site and can be tailored to meet the

specific needs of the design team.

Sales OfficesCoWare has sales offices in the U.S., Europe,

Asia Pacific and Japan. For a complete

listing with contact information visit

www.CoWare.com. For technical or sales

information call 1-888-CoWare8 or email

info@CoWare.com.

4H.264 Library

CoWare and the CoWare logo are registered trademarks of CoWare, Inc.All other marks are the property of their respective owners. ©2006 CoWare, Inc.

The ESL Design Leader

CoWare, Inc.Corporate Headquarters1732 N. First Street

San Jose, CA 95112

Main: 408-436-4720

Fax: 408-436-4740

www.CoWare.com