h.264 nal and rtp

H.264 NAL & RTP

www.dsp-ip.comFast Forward Your Development

H264 Layer Structure

Video Coding Layer

Data PartitioningControl Data

Macroblock

www.dsp-ip.comFast Forward Your Development

Data Partitioning

Network Abstraction Layer

H.320 H.324 H.323/IP MPEG2

Slice/Partition

H264 & NAL

• Motivation

▫ Many delivery methods are based on packet based networks

▫ Its better to do the packetization inside the encoder where all coding information than in other separate modules

• Architecture: NAL units as the transport entity

www.dsp-ip.comFast Forward Your Development

• Architecture: NAL units as the transport entity

▫ NAL units may be mapped into a bit stream▫ NAL units are self-contained-independently

decodable▫ The decoding process assumes NAL units are in

decoding order

Network Abstraction Layer (NAL)

• H.264 encoder is composed of two layers:

• VCL - Video Coding Layer – unit which translates the video information into a stream of bits

• NAL - Network Abstraction Layer (NAL). Which maps and packetize the VCL biststream into unitsprior to transmission or storage

• Each NAL unit contains:

www.dsp-ip.comFast Forward Your Development

• Each NAL unit contains:

▫ Payload – RBSP (Raw Byte Sequence Payload), which contains set of data corresponding to coded video data or header information

▫ NAL Unit Header – which contains the NAL header

NAL

• The coded video sequence is represented by a sequence of NAL units that can be transmitted over a packet-based network or a bitstream transmission link or stored in a file

• There are two NAL Types

▫ VCL Units – NAL units which represents encoded

www.dsp-ip.comFast Forward Your Development

▫ VCL Units – NAL units which represents encoded video data

▫ Non-VCL Units – Parameters sets

NAL Unit Header

NAL unit header- 1 byte consisting of:• Forbidden_bit(1 bit) may be used to signal that a NAL unit is

corrupt

• nal_storage_idc(2 bit): signal relative importance, and if the picture is stored in the reference picture buffer.

NAL unit header NAL unit payload

www.dsp-ip.comFast Forward Your Development

picture is stored in the reference picture buffer.

• nal_unit_type(5 bit): signal 1 of 10 different NAL unit types:▫ Coded slice (regular VCL data)

▫ Coded data partition. (DPA,DPB,DPC)

▫ Instantaneous decoder refresh (IDR)

▫ Supplemental enhancement information (SEI)

▫ Sequence and picture parameter sets (SPS,PPS)

▫ Picture delimiter (PD) and filler data (FD)

RBSP (NAL Payload) Types• Parameter Set – global parameters for a sequence

includes: Resolution, video format, macroblock, allocation map

• Supplemental Enhancement Information

• Picture Delimiter – boundary between video pictures

• Coded slice – header and data for a slice, this unit contains actual coded video data

www.dsp-ip.comFast Forward Your Development

contains actual coded video data

• Data Partition A,B or C – Data Partitioned slice layer data (A – header data for all MBs in the slice, B – intra coded data, C – inter coded data)

• End of sequence

• End of stream

• Filler data

RTP payload format for H.264

• Based on IETF RFC 3984, February 2005

• Described how to use H.264 NAL inside RTP with proper packetization Employs the native NAL (Network Abstraction Layer) interface, based on NAL units (NALUs)

• NALU – byte string of variable length that contains syntax elements of a certain class

• NALU header – defines the information within the NAL

www.dsp-ip.comFast Forward Your Development

• NALU header – defines the information within the NAL Unit (Corrupted, Type etc)

• There are two basic methods for RTP packetization of NAL units:

▫ Non fragmented NAL units

▫ Fragmented NAL units

RTP Payload for H.264 NAL

• The most common method is to configure the encoder to output one NAL unit for each RTP packet. Each NAL unit is ~1.4KB

www.dsp-ip.comFast Forward Your Development

• Fragment a large NAL unit (Frame) into many RTP Packets. The difference is in the RTP Header information

RTP and H.264

• RTP Packetization of NAL allows both aggregation of many NAL units into one RTP Packet and fragmentation of one NAL units into many RTP packets

www.dsp-ip.comFast Forward Your Development

DSP-IP Contact information

Download slides at: Download slides at: www.dspwww.dsp--ip.comip.com

Course materials & lecture requestCourse materials & lecture request

Adi YakovAdi YakovTraining ManagerTraining Manager

adi@dspadi@dsp--ip.comip.com

++972972--99--86519338651933

www.dsp-ip.comFast Forward Your Development

Projects development services:Projects development services:

++972972--99--86519338651933

wwwwww..dspdsp--ipip..comcomMailMail :: info@dspinfo@dsp--ipip..comcomPhonePhone:: ++972972--99--88509568850956,,FaxFax :: ++972972--5050-- 89629108962910

Alona AshkenaziAlona Ashkenazi

Development Development ServicesServices

alona@dspalona@dsp--ip.comip.com

++972972--99--88509568850956