compression synopsis h264-h265
TRANSCRIPT
19 October 2016 ALL DATA IS COMPANY CONFIDENTIAL Sheet 1
Compression Synopsish.264 vs. h.265
By
Paul HightowerCEO, Instrumentation Technology Systems
Presented to
Optical Systems Group
Fall 2016At
NASA Armstrong Flight Research Center
19 October 2016 ALL DATA IS COMPANY CONFIDENTIAL Sheet 2
30 Years of Compression
Each major system has added new compression tools
JPEG
MPEG 1
MPEG 2/H.262
MPEG 4H.261
H.264
H.265
1986 1994 2000 2013
H.263
• Targeted @ video conferencing• Introduced I &P frames
• Targeted @ HD distribution• variable block size• Deblocking filters• 8 modes motion vectors• Context-adaptive binary
arithmetic coding
• Targeted @ SD/ED distribution• Introduced motion vectors• Bidirectional estimation• ½ pixel motion vectors• Intraframe prediction• More frame formats
• Targeted @ UHD distribution• 4K and 8K at up to 120 FPS• Larger more flexible block sizes
using Coding Tree Units• Larger transform size• 33 modes motion vectors• Merge/Skip vector modes
• Targeted at single images• Wavelet transform
19 October 2016 ALL DATA IS COMPANY CONFIDENTIAL Sheet 3
Why Compression?
Content delivery
Television & Cable have narrow bands available to deliver content
• Compression is an enabler
• Allocations are in the 10s of Mb/sec
Internet Streaming
• Premium speed limit is 500 Mb/s
Wi-Fi
• 802.11ac max is 1,300 Mb/sec
Content Storage
After Theater Movie Sales
• DVD holds 4.7 GB
• Blu-ray holds 50 GB
• Hard Drives 1000-4000 GB now
• Cinema industries stores 10s of PB/day!
Format Raw Data Rate
SD/30 270 Mb/s
720/60 1,485 Mb/s
1080/60 2,970 Mb/s
2160/60 (4K) 11,880 Mb/s
Format Single Frame Size 2 Hours
SD/30 1.1 MB 250 GB
720/60 3.1 MB 1,400 GB
1080/60 6.2 MB 2,700 GB
2160/60 (4K) 24.8 MB 10,700 GB
19 October 2016 ALL DATA IS COMPANY CONFIDENTIAL Sheet 4
Challenges of Compression
A 2 hour movie on a Blu-ray disk
Transport over GigE Ethernet; ONLY 1 Channel, not shared with other traffic
Transport via 100Mb/s Wi-Fi link Not shared with other traffic
Format Minimum Compression Ratio
SD >6:1
720/60 >30:1
1080/60 >50:1
2160/60 >200:1
Format Minimum Compression Ratio
SD none
720/60 2:1
1080/60 4:1
2160/60 >14:1
Format Minimum Compression Ratio
SD 3:1
720/60 >15:1
1080/60 >30:1
2160/60 >120:1
19 October 2016 ALL DATA IS COMPANY CONFIDENTIAL Sheet 5
Challenges of Compression
Objectives
Reduce Frame Size (bytes) as much as possible
• Take advantage of redundancies in the image data of a frame (spatial)
• Take advantage of image areas that are same frame to frame (temporal)
• Take advantage of information that can be derived across a group of frames (prediction)
• Create standard algorithms that analyze and transform image data into compact nuggets
Enable decoder to render visually good imagery
• Largely a subjective measure
• Take advantage of eye sensitivities to changes in brightness, color and bandwidth
• Different criteria than still imagery
Manage Latency
• The greater the compression:
The more time it takes to crunch it
The more time it takes to re-render the images
The more buffering may be needed at the decoder end
19 October 2016 ALL DATA IS COMPANY CONFIDENTIAL Sheet 6
Latency
Introduction of I & P frames (MPEG) introduces interframe dependencies
Latency is dynamic and is continuously variable
• Latency varies with the number of dependent frames, scene complexity, buffer sizes,transport bandwidth, hardware and software from source to destination
Image Quality
Unlike data compression, image compression takes advantage of human perception
• Temporal and spatial image characteristics replace image areas (macroblocks)
• Encoded Pixels are replaced with transform coefficients.
• Decode Filters, motion prediction, pixel interpolation reduce the perceived errors
Bit-depth may be compromised
• Many encoders change sampling from 4:2:2 to 4:2:0 effectively reducing “shades”resolution of pixels and shift color
Can vary with the encoder, transport activity, rendering horsepower
Challenges of Compression
19 October 2016 ALL DATA IS COMPANY CONFIDENTIAL Sheet 7
Image Quality
Image quality is largely based on human perception
Noise perceived
Sharpness perceived
Blur
Color Space
Bandwidth sensitivities and limits
No one metric stands alone
“beauty is in the eye of the beholder”
Encode/Decode results may differ from encoder vendor to encoder vendor
There are many settings that vary with the camera, transport, view needs, lightingand content
MPEG & H.xxx specifications define the bit streams and data structures (Decoding)
Encoding is left to the developer
End-to-End quality is not controlled, only output given the same input
19 October 2016 ALL DATA IS COMPANY CONFIDENTIAL Sheet 8
Image Quality
Common Metrics of Image Quality
Peak Signal to Noise Ratio (PSNR)
• Basic, but flawed
• PSNR values do not correlate well with perceived picture quality due to the complex,highly non-linear behavior of the human visual system.( source: https://sonnati.wordpress.com/2014/06/20/h265-part-i-technical-overview/)
Source: http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/VELDHUIZEN/node18.html
All images have the same PSNR
19 October 2016 ALL DATA IS COMPANY CONFIDENTIAL Sheet 9
Image Quality
Structural Similarity (SSIM) : Math
• Compares groups of original pixels with thepost-decoded pixels in the same image area.
• This objective analysis compares favorably withMOS
Mean Opinion Score (MOS): Observation
• Subjective measure
• Uses standard clips
• Uses standard viewing environments
• Human subjects score what they see comparedto raw clips.
19 October 2016 ALL DATA IS COMPANY CONFIDENTIAL Sheet 10
H.265
Standard Managed by
ITU-T Video Coding Experts Group (VCEG)
ISO/IEC Moving Picture Experts Group (MPEG)
First joint HEVC standard released Jan 2013
Design Goals
50% reduction of bit rate compared to H.264 at the same perceptual image quality
Support transport of 4K video and beyond (more pixels)
Support high dynamic range color (more shades representable)
Support higher frame rates (120 and up)
Increase data loss resilience
Enabling Technologies
Parallel processing
• Needed for encoding (image analysis) and decoding (re-rendering)
New modulation techniques increasing the bits/Hz ratio
• QAM, quadrature modulation increases the bits/Hz in the transport channel
19 October 2016 ALL DATA IS COMPANY CONFIDENTIAL Sheet 11
What is common H.264-H.265
Split frames into Coding Tree Units (macro blocks)
This is essentially the starting point for an I frame
Transform (DCT), scaling & quantization
Spatial motion prediction (Intrapicture) & differences compared
• Prediction of block data derived from within the I (reference) frame alone
The two elements are transformed (DCT), quantized and scaled
Entropy Encoding; Context adaptive binary arithmetic coding (CABAC)
• Lossless data compression (the final step)
The result is put in to the bit stream
19 October 2016 ALL DATA IS COMPANY CONFIDENTIAL Sheet 12
What is common H.264-H.265
Temporal Encoding
Motion prediction of elements across a group of frames (GOP) from the I frame
• Results in differential frames
• B and P frames rely on data from the I frame
P frames (predictive) and can be used for prediction blocks in B frames
B frames process predictions based on data before and after the current frame.
• The resultant data for B and P frames is very small
• Most of the average compression comes from P and B frames
• P & B frames may be lost if the reference I frame is corrupted
19 October 2016 ALL DATA IS COMPANY CONFIDENTIAL Sheet 13
What is different H.264-H.265
Larger macroblocks
More macroblock types
Spatial prediction; 9 modes > 35 modes
Temporal prediction; H.265 adds rectangular blocks
Transform sizes; H.264 max 16x16; H.265 adds 32x32, 64x64 plus non-square forms
Larger block sizes are more coding efficient
19 October 2016 ALL DATA IS COMPANY CONFIDENTIAL Sheet 14
Is H.265 Better?
H.265 is not MORE compression; it is smarter
Increased flexibility
More fine control of prediction vectors
Better pixel interpolation
Finer deblocking
19 October 2016 ALL DATA IS COMPANY CONFIDENTIAL Sheet 15
Is H.265 Better?
19 October 2016 ALL DATA IS COMPANY CONFIDENTIAL Sheet 16
Is H.265 Better?
A range of scenes with similar PSNR and MOS compared to H.264 scenarios
Bit rates are better, but vary with scene complexity
Note a ±25% bit rate variance with the same encoder/decoder and hardware
19 October 2016 ALL DATA IS COMPANY CONFIDENTIAL Sheet 17
Is H.265 Better?
A 2 hour movie on a Blu-ray disk
Transport over GigE Ethernet UDP with exclusive use of the network
Transport via 100Mb/s Wi-Fi link Exclusive use at full bandwidth
Format Minimum Compression Ratio
H.264 H.265
SD >6:1 3:1
720/60 >30:1 14:1
1080/60 >50:1 28:1
2160/60 >200:1 111:1
Format Minimum Compression Ratio
H.264 H.265
SD none None
720/60 2:1 None
1080/60 4:1 2:1
2160/60 >14:1 8:1
Format Minimum Compression Ratio
H.264 H.265
SD 3:1 2:1
720/60 >15:1 8:1
1080/60 >30:1 16:1
2160/60 >120:1 62:1
19 October 2016 ALL DATA IS COMPANY CONFIDENTIAL Sheet 18
Conclusions
H.265 expands on legacy of tools
Macro blocks
Spatial and Temporal prediction
Motion vectors
DCT and Entropy
H.265 is not MORE compression; Smarter Processing
More tools, more analytical choices
More Computing Power Required
Parallel processing a must
Real-time (30 to 60 fps) does not change with more work to do encoding/decoding
Encoding can be longer even with more capable hardware
Compression goal met at 48-50% for same image quality
Latency?
Positive Side Effect:
At current transport bandwidths available 720p/60 and 1080p/60 should survivewith better image fidelity than with H.264
19 October 2016 ALL DATA IS COMPANY CONFIDENTIAL Sheet 19
Concerns
Greater compression may mean more sensitivity to bit errors in transport
Encoders will vary from source to source
Image quality needs to be scored
H.265 Encoder/Decoder sets must come on line with metadata support
Two H.264 Encoder/Decoders have been tested for Metadata by ITS
Delta Digital 4480E and 9600 (decoder)
Haivision Makito X
19 October 2016 ALL DATA IS COMPANY CONFIDENTIAL Sheet 20
Concerns
Decode side is specified
Encode side is up to the supplier
Encoders must be evaluated for image quality
Image quality for engineering test is different than moviegoers
• More details are important
• Critical frames are often most different frames
Latency
For the same channel width is latency greater or lesser?
Does latency vary more?
19 October 2016 ALL DATA IS COMPANY CONFIDENTIAL Sheet 21
Bibliography of Resources
Overview of the High Efficiency Video Coding (HEVC) Standard Gary J. Sullivan, Fellow, IEEE, Jens-Rainer Ohm, Member, IEEE, Woo-Jin Han, Member, IEEE, and Thomas Wiegand, Fellow, IEEE;
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 22, NO. 12, DECEMBER 2012
Design and Implementation of Next Generation Video Coding Systems (H.265/HEVC Tutorial) Vivienne Sze ([email protected]); Madhukar Budagavi ([email protected])
ISCAS Tutorial 2014
Video Coding Basics Yao Wang, Polytechnic University, Brooklyn, NY11201, [email protected], 2003
“Intra Prediction in HEVC,”High Efficiency Video Coding (HEVC): Algorithms and Architectures Springer, 2014. J. Lainema, W.---J. Han,