NTHU H.264/AVC Video Encoder & Decoder

Youn-Long LinDepartment of Computer Science

National Tsing Hua UniversityHsin-Chu, TAIWAN 300

ylin@cs.nthu.edu.tw

2007/02/10IC-DFN, Las Vegas

YLLIN NTHU-CS 2

Evolution of Video Coding Standards

ITU-TStandards

1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004

ISOStandards

Joint ITU-T/ISOStandards

MPEG-1MPEG-1

MPEG-4MPEG-4

H.261 H.263H.263+

H.263+

H.262/MPEG-2

H.262/MPEG-2 H.264

H.264

VCD MP3

DVD/DTV

DV/IPCam

YLLIN NTHU-CS 3

Get More (Quality) for Less (Bit-rate)

MPEG2

H.264

YLLIN NTHU-CS 4

Ref: G. Sullivan & T. Wiegand, “Video Compression—From Concepts to the H.264/AVC Standard”, Proceedings of the IEEE, Vol 93, No.1, Jan 2005

1) No MC; 2) Adding Skip mode to for

m a CR coder. 3) Allow only zero-valued

MVs.4) Allow integer-pel MC. 5) Allow half-pel MC6) Allowing 4-MV; 7) Allowing quarter-pel MC.

Effectiveness of basic techniques

YLLIN NTHU-CS 5

Features of Video Coding Standards

Standard MPEG-1 MPEG-2 MPEG-4 H.264

MB size 16*16 16*16(frame) 16*16 16*16

Block size 8*8 8*8 16*16, 8*8 16*16, 16*8, 8*16, 8*8, 8*4, 4*8, 4*4

Transform DCT DCT DCT/ Wavelet 4*4 int transform

Entropy coding VLC VLC VLC VLC, CAVLC and CABAC

ME, MC Yes Yes Yes 41 MVs per MB

Pixel accuracy ½ pel ½ pel ¼ pel ¼ pel

Reference frames One frame One frame One frame Multiple (5) frames

Picture type I, P, B I, P, B I, P, B I, P, B

Transmission rate Up to 1.5 Mbps

2-15 Mbps 64kbps~2Mbps 64kbps ~ 150Mbps

YLLIN NTHU-CS 6

Competing Standards

• H.264 Advanced Video Coding (H.264/AVC)– Also Called MPEG-4 Part 10

• WMV/VC-1 (MicroSoft)• Chinese AVS (Audio Video Coding Standard)

YLLIN NTHU-CS 7

H.264/AVC Profiles

Weightedprediction

B slice

I slice

P slice

CAVLC

Slicegroup

ASO

Redundant Slice

SP, SIslice

Datapartition

Interlace

CABAC

8x8transform

Quantizationmatrix

ColorSampling

8/10/12 bitsampling

Extendedprofile

Mainprofile

Baselineprofile

FREext(High)profile

YLLIN NTHU-CS 8

Global UniChip Multimedia SOC Platform

CPUAccelerator

(FPGA)USB(PHY)

Daughter Board

ROM/Flash Memory

SRAMSDRAM

VIC USB 2.0 Staticmemory

SDRAM Controller(4-CH)

JPEGCodec DMA SRAM PWM WDT TIMER

APBBridge Capture

DisplayController

DAI SSI SD SM UART GPIO 12C

Audio CodecI2S

Flash memory with SSI

Flash Card Button LED Video-InCCIR601 TV/LCD

High-Speed Bus

Peripheral Bus

FPGA

YLLIN NTHU-CS 9

AHB1

AHB2

SDC

ARM926EJS

Slave

SlaveSDRAM

TVMaster

TimerSlave

SD CardSlave

H264Master

Slave

LM

UARTSlave

SDRAM

SDC

Slave

Slave

H.264/AVC Decoder System Diagram

YLLIN NTHU-CS 10

H.264/AVC Decoder Architecture

IQ/IDCT ResidualSRAM

CABADMBinfoSRAM

CoeffSRAM

MC

Intrapred

PredSRAM

PicRec

reconstructSRAM

unfilterSRAM

MVSRAM

Ref idxSRAM

DF

ParaSRAM

Parser

DECODERCAVLD

SDRAMInput/Ref./Display

Frame

AHB

DisplayStorage DeviceCPU

SD Card

YLLIN NTHU-CS 12

H.264 Decoder

control register

slave wrapper

MFU

master wrapper 1

SDCarbiter 1 arbiter 2

VLC & TV OUT DF & MC

master wrapper 2

AHB B

AHB A

LM

AMBA interface

YLLIN NTHU-CS 13

System descriptio

nCompilation

Software image

FPGA Verify

System Integrate

User Spec.

System configuration

System.h

API

HW lib.HDL IPs

Acceleration

NTHU Design Flow

FPGA prototyping

Area & Timing & Power

evaluation

Embedded Software

Co-Sim

Yes

No

HW IP Synthesizer

Evaluation

Platform spec.

Software spec. in C & Acceleration

specify

HW/SW co-simulation

Accelerator.v System.v Parameterized ISS

System generation

Acceleration

Pin assignment & Hardware compilation

Integration

Hardware image

Platform model

SW lib.C models,

drivers

Performance constraint

YLLIN NTHU-CS 14

Demo Video

YLLIN NTHU-CS 16

Encoder Demo – Video Capturing

NTHU-PKU Collaboration

Porting NTHU H.264/AVC Encoder and Decoder to PKU SOC Platform

YLLIN NTHU-CS 22

Participants

YLLIN NTHU-CS 23

YLLIN NTHU-CS 24

Job Function

YLLIN NTHU-CS 25

Schedule

YLLIN NTHU-CS 26

Schedule

YLLIN NTHU-CS 27

System Overview

YLLIN NTHU-CS 28

FPGA Prototyping

YLLIN NTHU-CS 29

Results

YLLIN NTHU-CS 30

Future Systems

YLLIN NTHU-CS 31

High-Performance QFHD

YLLIN NTHU-CS 32

3840x2160 - SuperHDTV

SuperHDTV (3840x2160 QFHD)

YLLIN NTHU-CS 33

Summary

• H.264/AVC Encoder & Decoder IP– Two Platforms: UniChip and PKU– Very High Performance (Cycles per MB)

• Future Directions– Super High Resolution (4X HDTV, 16X HDTV)– Low Power Design– Enhanced Platform and Design Flow– Chip Implementation