Download - H.264 Library
H.264 OverviewBroadcast television and home entertainment
have been revolutionized by the advent of
digital TV and DVD-video. These applications
and many more were made possible by the
standardization of video compression technol-
ogy. The recent standard in the MPEG (Moving
Pictures Expert Group) series, MPEG-4, is
enabling a new generation of internet-based
video applications whilst the ITU-T H.263
standard for video compression is now widely
used in video conferencing systems.
H.264/AVC (aka MPEG-4 Part 10), the result
of the collaboration between the ISO/IEC
MPEG group and the ITU-T Video Coding
Experts Group (VCEG), is the latest video
coding standard. The goals of this standard-
ization effort were: enhanced compression
efficiency; network friendly video representa-
tion for interactive applications (video tele-
phony) and non-interactive applications
(broadcast applications, storage media appli-
cations, etc). H.264/AVC provides gains in
compression efficiency up to 50% over a wide
range of bit rates and video resolutions com-
pared to previous standards. Compared to
previous standards, the decoder complexity
is about four times that of MPEG-2 and two
times that of MPEG-4 visual simple profile.
Relative to prior video coding standards,
H.264/AVC introduces the following changes
� In order to reduce the blocking artifacts, an
adaptive loop filter is used in the prediction
loop to reduce blocking artifacts.
� A prediction scheme called intra prediction
is used that exploits spatial redundancy. In
this scheme, data from previously processed
macroblocks is used to predict the data for
the current macroblock in the current
encoding frame.
� Previous video coding standards use an 8x8
real discrete cosine transform (DCT) to exploit
the spatial redundancy in the 8x8 block of
image data. In H.264/AVC, a smaller 4x4
integer DCT is used which significantly
reduces ringing artifacts associated with
the transform.
� Various block sizes from 16x16 to 4x4 are
allowed to perform motion compensation
prediction.
� Previous video coding standards used a
maximum of half-pixel accuracy for motion
estimation.
� Inter prediction mode allows multiple
reference frames for block-based motion
compensation prediction.
� Context-adaptive variable length coding
(CAVLC) and context-adaptive binary
arithmetic coding (CABAC) are used for
entropy encoding/decoding which improves
compression by 10% compared to previous
schemes.
H.264 Baseline Profile Video DecoderCompressed H.264 bit-stream data is avail-
able on slice-by-slice basis whereas a slice is
usually a group of macro blocks processed
in raster scan order. Two slice types are sup-
ported in H.264 baseline profile. In an I-slice,
all macro blocks are encoded in intra mode.
In a P-slice, some macro blocks are predicted
using a motion compensated prediction with
one reference frame among the set of reference
frames and some macro blocks are encoded in
intra mode. H.264 decoder processes the data
on a macro block by macro block basis. For
every macro block depending on its charac-
teristics, it will be constructed by the predicted
part of the macro block and the residual
(error) part, which is coded using CAVLC.
Figure 1 shows the overall block diagram of
an H.264 baseline profile video decoder.
H.264 bit-stream passes through the “slice
header parsing” block. This block extracts
the information about each slice. In H.264
video coding, each macro block is categorized
as either coded or skipped. If the macro block
is skipped, then the macro block is completely
reconstructed using the inter prediction
module. In this case, the residual information
is zero. If the macro block is coded, then based
on the prediction mode it passes through the
“Intra 4x4 prediction” block or “Intra 16x16
prediction” block or “Inter prediction” block.
H.264 LibraryDetailed Reference for Verification and Design Exploration
SliceHeaderParsing
H.264ElementaryBit Stream
MacroBlock
Parsing
Scale &Inverse
Transform
Sub MBParsing
Skip MB
Prediction Path
Residual Path
InterPrediction
Intra 4x4Prediction
DeblockingFilter
Intra 16x16Prediction
CAVLCDecoding
Figure 1: Block diagram of H.264 baseline profile video decoder.
The output macro block is reconstructed
using the prediction output from the predic-
tion module and the residual output from the
“scale and transform” module. Once all the
macro blocks in a frame are reconstructed,
de-blocking filter will be applied for the
entire frame.
The “macro block parsing module” parses
the information related to the macro block,
such as prediction type, number of blocks
coded in a macro block, partition type,
motion vectors, etc. The “sub macro block”
parsing module parses the information if the
macro block is split into sub macro blocks of
one of the sizes 8x8, 8x4, 4x8, and 4x4 when
the macro block is coded as inter macro
block. If the macro block is not split into sub
macro blocks, all the three prediction types
(Intra16x16, Intra4x4, or Inter) can be used.
In inter prediction module, the motion
compensated predicted blocks are predicted
from the previous frames, which are already
decoded.
Intra prediction means that the samples of a
macro block are predicted by using the already
transmitted macro blocks of the same image.
In H.264/AVC, two different types of intra
prediction modes are available for coding
luminance component of the macro block.
The first type is called INTRA_4x4 mode and
the second type is called INTRA_16x16
mode. In INTRA_4x4 prediction mode, each
macro block of size 16x16 is divided into
small blocks of size 4x4 and prediction is
carried out individually for each sub-block
using one of the nine prediction modes avail-
able. In INTRA_16x16 prediction mode, the
prediction is carried out at macro block level
using one of the four prediction modes
available. Intra prediction for chrominance
components of a macro blocks is similar
to the INTRA_16x16 prediction of the lumi-
nance component.
H.264/AVC baseline profile video decoder is
using CAVLC entropy coding method to
decode the encoded quantized residual
transform coefficients. In CAVLC module,
the number of non-zero quantized transform
coefficients, the actual size and the position
of each coefficient are decoded separately.
The tables used for decoding these parameters
are adaptively changed depending on the
previously decoded syntax elements. After
decoding, the coefficients are inverse zigzag
scanned and form a 4x4 blocks which are
given to “scale and inverse transform module”.
In “scale and inverse transform module”,
inverse quantization and inverse transforma-
tion are done on the decoded coefficients
and form a residual data suitable for inverse
prediction. Three different types of transforms
are used in H.264 standard. The first type is
4x4 inverse integer discrete cosine transform
(DCT), which is used to form the residual
blocks of both luminance and chrominance
blocks. Second type is a 4x4 inverse Hadamard
transform, which is used to form the DC
coefficients of the 16 luminance blocks of the
INTRA_16x16 macro blocks. Third transform
is a 2x2 inverse Hadamard transform, which
is used to form the DC coefficients of the
chrominance blocks.
The 4x4 block transform and motion com-
pensation prediction can be the source of
blocking artifacts in the decoded image.
Filtering the block edges will improve the
final visual quality of decoded image. H.264
standard is using in-loop deblocking filter to
remove the blocking artifacts.
CoWare Signal Processing Designer Library ofH.264 Baseline Profile Video DecoderThe Signal Processing Designer library of
H.264 baseline profile video decoder system,
developed by iDeaWorks, is shown in Figure 2.
The decoder takes H.264 baseline profile
elementary bit stream from a source as NAL
units and decodes the compressed video
streams and displays and dumps the decoded
frames. Frame width, frame height, and NAL
unit size can be set using global parameters.
There are nine major blocks in the top level
hierarchy of H.264 video decoder, which are:
Figure 2.: CoWare Signal Processing Designer View of H.264 baseline decoder operation
2H.264 Library
3
Source Block
This block converts H.264 baseline profile
elementary bit stream into fixed sized frame
NAL units. Each frame NAL unit contains all
NAL units of a single frame. The block takes
H.264 bit stream file name and frame NAL
unit size as parameters. It gives frame NAL
unit as a vector of size specified by the input
parameter and end of sequence (EOS) flag
as outputs.
Parser Block
This module is used to parse the following
set of information
� Control data required for doing intra and
inter prediction
� Motion compensation data required for
quarter pixel interpolation
� Quantized residual transform coefficients
� Control parameters for performing scale
and transform
� Control data required for applying the
deblocking filter on the decoded image
� Control data required for display re-ordering
The parser works at frame level.
Frame to Macroblock Conversion Block
This is a multirate block, which converts
frame level parameters to macroblock level
parameters. The parser block drives this
block. The output rate is ((Frame Width x
Frame Height)/256) times the input rate.
Macroblock Process Block
The macroblock process block consists of
four major modules, which are
� Scale and transform module
� Prediction module
� Deblocking module
� Intra feedback data module
This block operates at macroblock level.
Frame Construct Block
This is a multirate block, in which the input
is coming in as MBs and the output is going
out as frame. This block takes vectors of size
532 bytes and forms a single frame. There
are three modules in this block, which are
� Polymorphic vector-to-scalars
conversion block
� Polymorphic scalars-to-vector
conversion block
� Pur filter data block
Display Reorder Block
This block is used to reorder the decoded
YUV display data and is also used to crop the
output YUV data according to cropping
parameters. This block takes decoded frame
data, display re-order control data, and end
of sequence flag + cropping parameters as
inputs and gives display frame data and a
flag indicating whether the display frame
data is valid or not.
Sink Block
This block is used to dump the decoded
YUV data into an output file. This block
takes display frame data, flag that indicates
the validity of frame data, control data for
cropping, and end of sequence flag as inputs.
The output file name can be set using para-
meters of sink block.
H.264 Library Use ModelsThe H.264 Baseline Video Decoder was devel-
oped as a optimized, architecture neutral ref-
erence standard for users incorporating
advanced video decoding in embedded or
stand-alone products. It consists of a fully
decomposed software C-based algorithm
encapsulated within CoWare Signal
Processing Designer. This environment
insures users can use this product as a refer-
ence tool for FPGA, ASIC, SoC, or embedded
IC implementations. The algorithms have
been decomposed down to the ‘leaf ‘or lowest
common functions. Parallelism and concur-
rency is fully exposed in the CoWare Signal
Processing Designer H.264 Design Library
and CoWare Signal Processing Designer
H.264 Source Library.
The algorithm model meets these industry
standards.
� ITU-T H.264 standard OR
� ISO/IEC MPEG-4 Part 10 standard
Included with the library are reference test
benches consisting of standard media video
feeds capable of exercising the algorithm and
any implementation of the reference library.
The decoder library is qualified with the JVT
standards compliance video streams.
The following packages are available:
� CoWare Signal Processing Designer H.264
Reference Library
–H.264 models as partitioned verification
reference
–Reference media streams
–Value: Validated, partitioned executable
reference inside CoWare flow for either
customer H.264 algorithm optimization
� CoWare Signal Processing Designer H.264
Design Library
–Same as reference library, but down to very
low level of granularity
–Reference module test benches
–Value: Verification and detailed reference
for the design and optimization of sub
functions
–Documentation: Detailed Algorithm docu-
mentation representing, Control, dataflow
and Memory requirements.
–Test Vectors: the scaling transform, predic-
tion, and de-blocking filter modules have a
full ten frames of test vectors for QCIF, CIF,
and HD 720p resolutions.
� CoWare Signal Processing Designer H.264
Source Library
–Requires H.264 Design Library
–Source code license agreement
–Specific jump-start service package included
–All leaf level blocks as source code
–Value: working C-code for embedded imple-
mentation or starting point for detailed
RTL design
Customer FocusCoWare provides a complete range of training,
support, design methodology consulting, and
integration services. Technical support
requests are handled directly by experienced
design engineers who are fully familiar with
the application of CoWare tools and method-
ologies to real-world designs. Training courses
are available at CoWare offices or at the
customer site and can be tailored to meet the
specific needs of the design team.
Sales OfficesCoWare has sales offices in the U.S., Europe,
Asia Pacific and Japan. For a complete
listing with contact information visit
www.CoWare.com. For technical or sales
information call 1-888-CoWare8 or email
4H.264 Library
CoWare and the CoWare logo are registered trademarks of CoWare, Inc.All other marks are the property of their respective owners. ©2006 CoWare, Inc.
The ESL Design Leader
CoWare, Inc.Corporate Headquarters1732 N. First Street
San Jose, CA 95112
Main: 408-436-4720
Fax: 408-436-4740
www.CoWare.com