overview of fine granularity scalability in mpeg-4 video standard
DESCRIPTION
Overview of Fine Granularity Scalability in MPEG-4 Video Standard. Weiping Li Presented by : Brian Eriksson. Introduction - Problem. Tradition System Video encoded close to channel capacity Decoded using all of the bits Internet Video System Channel capacity changes - PowerPoint PPT PresentationTRANSCRIPT
Overview of Fine Granularity Scalability in MPEG-4 Video
Standard
Weiping Li
Presented by : Brian Eriksson
Introduction - Problem
• Tradition System– Video encoded close to channel capacity– Decoded using all of the bits
• Internet Video System– Channel capacity changes– Partially decodable within a bit range
Introduction – Video Coding Performance
Low HighChannel Bitrate
Received Quality
Bad
Good
= Distortion-Rate Curve
= Dialup Rate
= Broadband Rate
= Layered Coding
= Desired Coding
Low HighChannel Bitrate
Received Quality
Bad
Good
Previous Layered Coding Methods
= Layered Coding
MPEG-2 Overview
• I-Frame – Intracoded Picture– Contains all frame information
• P-Frame – Predictive-Coded Picture– Uses previous frame and motion
information to reconstruct frame
• B-Frame – Bidirectional-Coded Picture– Uses previous frame, next frame, and
motion information to reconstruct the frame
Temporal Scalability
•Video is encoded into two layers with equal resolution but using different prediction.
•Base layer has lower frame rate
Base Enh. Enh. Base Enh. Enh. Base Enh. Enh.
Base X X Base X X Base X X
With Enhancement Layer : Frame Rate = 30 fps
Without Enhancement Layer : Frame Rate = 10 fps
MPEG-2 SNR Scalability
Enhancement Layer Stream
Base Layer Stream
+ IDCT
Video Output
•Motion prediction includes enhancement information
•Efficiency dependent on two factors
•Encoder uses enhancement layer (drift can occur)
•Decoder receives enhancement layer
Motion Compensation
+
Spatial Scalability
IDCTEnhancement Bitstream
Base Bitstream (Motion Compensated)
Upsampler
+Enhancement Layer Video
Base Layer Video
•Same frame rate, different resolutions
•Base frame = 128x128, enhancement = 256x256
•Enhancement Layer not in prediction loop
Low HighChannel Bitrate
Received Quality
Bad
Good
MPEG-4 Fine Granularity Scalability Technique
= Desired Coding
= Distortion-Rate Curve
Bitplane Coding Technique
Given vector X:
X = [-12, -53, 62, -7, 31,…,180,-43,…,5]
•Convert to signed magnitude
•Separate into sign bit and absolute values
Bitplane Coding Technique
Sign(X) = [0, 0, 1, 0, 1,…,1,0,…,1]
Abs(X) = [12, 53, 62, 7,31,…,180,43,…,5]
•To find the number of bitplanes needed, find the maximum value of abs(X)
•Max(abs(X)) = 180 = 10110100b
•8 bitplanes
•Sign bit plane
•1 bitplane
Bitplane Coding Technique
c
Sign(X) = [0, 0, 1, 0, 1,…,1,0,…,1]
X = [-12, -53, 62, -7, 31,…,180,-43,…,5]
0,1,0,1,1,…,0,1,…,10,0,1,1,1,…,0,1,…,01,1,1,1,1,…,1,0,…,11,0,1,0,1,…,0,1,…,00,1,1,0,1,…,1,0,…,00,1,1,0,0,…,1,1,…,00,0,0,0,0,…,0,0,…,00,0,0,0,0,…,1,0,…,00,0,1,0,1,…,1,0,…,1
Run Level Encoding
• Symbol : (RUN, EOP)– RUN = Number of Consecutive Zeros before a 1– EOP = 0 if there are more ones– EOP = 1 if the rest of the line are zeros.
• Example:– {1,0,1,0,0,0,1,0,0,0,…}
• = (0,0),(1,0),(3,1)– {0,0,0,1,0,0,0,…}
• = (3,1)
MPEG-4 Bitplane FGS Technique
• Base layer reaches lower bound of bit-range• Divide image into 8x8 DCT blocks• Divide blocks into Y,U,V color components• Use bitplane run-level coding to
encode/decode
Fine Granularity Scalability Encoder
+-
DCT Bitplane Encoding
Original Signal
Base Layer Signal (Motion Compensated)
Enhancement Layer Signal
Fine Granularity Scalability Decoder
Bitplane Decoding
IDCT +
Base Layer Video (Motion Compensated)
Enhancement Layer Video
Truncated Bitplane
c
Number of Truncated Layers = 0
Received X = [-12, -53, 62, -7,31,…,180,-43,…,-5]
Original X = [-12, -53, 62, -7,31,…,180,-43,…,-5]
0,1,0,1,1,…,0,1,…,10,0,1,1,1,…,0,1,…,01,1,1,1,1,…,1,0,…,11,0,1,0,1,…,0,1,…,00,1,1,0,1,…,1,0,…,00,1,1,0,0,…,1,1,…,00,0,0,0,0,…,0,0,…,00,0,0,0,0,…,1,0,…,00,0,1,0,1,…,1,0,…,1
c
Number of Truncated Layers = 1
Received X = [-12, -52, 62, -6,30,…,180,-42,…,-4]
Original X = [-12, -53, 62, -7,31,…,180,-43,…,-5]
0,0,0,0,0,…,0,0,…,00,0,1,1,1,…,0,1,…,01,1,1,1,1,…,1,0,…,11,0,1,0,1,…,0,1,…,00,1,1,0,1,…,1,0,…,00,1,1,0,0,…,1,1,…,00,0,0,0,0,…,0,0,…,00,0,0,0,0,…,1,0,…,00,0,1,0,1,…,1,0,…,1
c
Number of Truncated Layers = 2
Received X = [-12, -52, 60, -4,28,…,180,-40,…,-4]
Original X = [-12, -53, 62, -7,31,…,180,-43,…,-5]
0,0,0,0,0,…,0,0,…,00,0,0,0,0,…,0,0,…,01,1,1,1,1,…,1,0,…,11,0,1,0,1,…,0,1,…,00,1,1,0,1,…,1,0,…,00,1,1,0,0,…,1,1,…,00,0,0,0,0,…,0,0,…,00,0,0,0,0,…,1,0,…,00,0,1,0,1,…,1,0,…,1
c
Number of Truncated Layers = 3
Received X = [-8, -48, 56, -0,24,…,176,-40,…,-0]
Original X = [-12, -53, 62, -7,31,…,180,-43,…,-5]
0,0,0,0,0,…,0,0,…,00,0,0,0,0,…,0,0,…,00,0,0,0,0,…,0,0,…,01,0,1,0,1,…,0,1,…,00,1,1,0,1,…,1,0,…,00,1,1,0,0,…,1,1,…,00,0,0,0,0,…,0,0,…,00,0,0,0,0,…,1,0,…,00,0,1,0,1,…,1,0,…,1
Advanced Bitplane Techniques
Frequency Weighting in Bitplane
Normal Bitplane
DCT Index
Frequency Weighted Bitplane
DCT Index
Selectively Enhanced Bitplane
•Detect visually significant area
•Shift upward in bitplane
•More likely to be included in truncated bitstream
Other Methods
•Error Resilience in Bitplane•Random burst errors in the bitstream
•Resynchronization markers are used to resynch.
•Temporal Scalability in Bitplane•Uses FGS to encode/decode the entire temporal enhancement frame
Simulation - DCT
Simulation - DWT
Selectively Enhanced / Weighted
Bitplane Setup
Simulation – Enhanced Weighted Bitplane
1 Layer Removed2 Layers Removed3 Layers Removed4 Layers Removed5 Layers Removed6 Layers Removed
Simulation – Li
Conclusions
• FGS Bitplane method allows for quality parallel to distortion-rate curve
• A wavelet-based approach may yield better results
• Fairly simple implementation
Questions?