icme 2016 - high efficiency video coding - coding tools and specification: hevc v3 and coming...
TRANSCRIPT
![Page 1: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/1.jpg)
High Efficiency Video Coding – Coding Tools andSpecification: HEVC V3 and Coming Developments
Mathias Wien
Institut für NachrichtentechnikRWTH Aachen University
ICME2016
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 1 / 235
![Page 2: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/2.jpg)
Outline Part I
1 Video Coding Systems
2 Structure of a Video Sequence
3 Specification
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 2 / 235
![Page 3: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/3.jpg)
Outline Part II
4 Coding Structures
5 Reference Pictures
6 High-Level Syntax
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 3 / 235
![Page 4: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/4.jpg)
Outline Part III
7 Intra Prediction
8 Inter Prediction
9 Residual Coding
10 Loop Filtering
11 Entropy Coding
12 Profiles in HEVC Edition 1
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 4 / 235
![Page 5: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/5.jpg)
Outline Part IV
13 Range Extensions
14 Annex F: Common specifications for multi-layer extensions
15 Annex G: Multiview High Efficiency Video Coding
16 Annex H: Scalable High Efficiency Video Coding
17 Annex I: 3D High Efficiency Video Coding
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 5 / 235
![Page 6: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/6.jpg)
Outline Part V
18 Screen Content Coding
19 Free Viewpoint Television
20 Wide Color Gamut / High Dynamic Range Coding
21 Future Video Coding
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 6 / 235
![Page 7: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/7.jpg)
Outline Part VI
22 Summary and Outlook
23 Books and Tools
24 Resources
25 Acronyms
26 References
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 7 / 235
![Page 8: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/8.jpg)
Part I
Introduction
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 8 / 235
![Page 9: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/9.jpg)
Outline
1 Video Coding Systems
2 Structure of a Video Sequence
3 Specification
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 9 / 235
![Page 10: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/10.jpg)
Video Coding Systems
source pre-processing encoding
transmission
decodingpost-processingdisplay
Generalized overview of the processing chainVarious realizations of the chain
Communication (e. g. video conferencing)Broadcast (e. g. TV, streaming)Storage (DVD, Blu-Ray, . . . )
Transcoding may be part of transmission
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 10 / 235
![Page 11: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/11.jpg)
Hybrid Coding Scheme: Encoder
Decoder
CTB
input picture+−
TR+Q
TB
iTR+iQ TB
+
Intra
PB
EntropyCod-ing
bitstream
Deblk. Slice
LoopFilter
Slicerec. picture
Inter
PB
Buffer n pics
ME
PB
CTB – Coding Tree BlockME – Motion EstimationPB – Prediction BlockQ – QuantizationTB – Transform BlockTR – Transform
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 11 / 235
![Page 12: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/12.jpg)
Outline
1 Video Coding Systems
2 Structure of a Video SequenceRepresentation of Color
3 Specification
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 12 / 235
![Page 13: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/13.jpg)
Structure of a Video Sequence
Sequence of pictures successively captured or rendered
Progressive and interlaced formats
Picture rate measured in pictures per second, unit Hertz (Hz)Minimum picture rate at 24 Hz for fluent motion [1]
Standard Definition TV at 50/60 Hz interlacedHigh Definition (HD) video at 50/60 Hz progressiveUltra HD (UHD) video up to 120 HzUp to 300 Hz considered
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 13 / 235
![Page 14: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/14.jpg)
Chroma Sub-Sampling
Human visual system less sensitive to color than to structure and texture⇒ full resolution luma, lower resolution chroma
Chroma sub-sampling types commonly specified by relation betweennumber of luma an chroma samples
YCbCr Y :X1:X2
With Y : number of luma pixels
Sub-sampling format of chroma components specified by X1 and X2
X1: horizontal sub-sampling
X2 = 0: vertical sub-sampling identical to horizontal sub-sampling
X2 = X1: no vertical sub-sampling
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 14 / 235
![Page 15: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/15.jpg)
Chroma Sub-Sampling
Example:Assumed location of chroma samples relative to luma samples in HEVC
luma
chroma
(a) YCbCr 4:2:0 (b) YCbCr 4:2:2
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 15 / 235
![Page 16: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/16.jpg)
Outline
1 Video Coding Systems
2 Structure of a Video Sequence
3 SpecificationSpecification Scope
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 16 / 235
![Page 17: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/17.jpg)
Standardization
Driving factor for standardization: InteroperabilityDesire to enable devices and applications from different manufacturersand sources to interoperateDefinition of requirements
Application needsExtensibility options
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 17 / 235
![Page 18: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/18.jpg)
Specification Scope
source pre-processing encoding
transmission
decodingpost-processingdisplay
Parsing process
Bitstream syntax
Decoding process
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 18 / 235
![Page 19: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/19.jpg)
Part II
HEVC Coding Structures and High-LevelSyntax
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 19 / 235
![Page 20: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/20.jpg)
Outline
4 Coding StructuresRandom Access PointsLeading and Trailing PicturesTemporal Sub-Layer AccessBlocks, Units, and SlicesTilesBlock Types
5 Reference Pictures
6 High-Level Syntax
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 20 / 235
![Page 21: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/21.jpg)
Picture Types: Intra Random Access PointsInstantaneous Decoder Refresh (IDR)
Start of new CVS, reset of decoding process, DPB emptied
Only reference to pictures following IDR in coding order
No reference ’across’ IDR picture
Clean Random Access (CRA)
DPB left intact if within coded video sequence
Pictures after CRA in coding and output order without reference topictures before CRA
If used as starting point: Conformingly decodable pictures following CRAin output order
Broken Link Access (BLA)
Renaming of CRA after splicing operation
Indicates removal of pictures containing broken / unavailable references
CVS = Coded Video Sequence, DPB = Decoded Picture Buffer
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 21 / 235
![Page 22: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/22.jpg)
Picture Types: Leading and Trailing Pictures
Leading Pictures
Precede associated IRAP in output order
Follow associated IRAP in coding order
Random Access Decodable Leading Picture (RADL)
Correctly decodable if decoding starts at corresponding IRAP
No reference to pictures prior to IRAP in coding and output order
Random Access Skipped Leading Picture (RASL)
May contain references to unavailable pictures if decoding starts at IRAP
Shall not be output if associated IRAP is a BLA
Trailing Pictures
Follow the associated IRAP in coding and output order
Follow all leading pictures associated to the IRAP in coding order
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 22 / 235
![Page 23: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/23.jpg)
Picture Types: Leading and Trailing Pictures – Example
2 70 1 3 4 5 6 8 9output ord.
coding ord. 01 2 34 56 7 9 8
pic. type RADL RADL IDR TRAIL TRAIL RADL RADL CRA TRAIL TRAIL
NUT 7 6 19 0 1 7 6 21 0 1
(a)
2 70 1 3 4 5 6 8 9 10output ord.
coding ord. 01 2 34 56 7 89
pic. type RADL RADL IDR TRAIL TRAIL RASL RASL CRA TRAIL TRAIL
NUT 7 6 19 0 1 9 8 21 0 1
(b)
NUT: NAL unit type, NAL = Network Abstraction Layer
Coding structure for demonstration purpose only
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 23 / 235
![Page 24: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/24.jpg)
Picture Types: Leading and Trailing Pictures – Splicing
SEQUENCE A
2 0 1 4 3 7 5 6 9 8 12 10 11 14 13output ord.
coding ord. 0 1 2 3 4 5 6 7 8 9
pic. type IDR RADL RADL TRAIL TRAIL CRA RASL RASL TRAIL TRAIL
NUT 19 7 6 1 0 21 9 8 1 0
SEQUENCE B
0 1 2 3 × × 5 4 8 6 7 10 9 11output ord.
coding ord. 0 1 2 3 4 5 6 7 8 9
pic. type IDR TRAIL TRAIL BLA RASL RASL TRAIL TRAIL CRA RASL
NUT 19 1 0 16 9 8 1 0 21 9
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 24 / 235
![Page 25: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/25.jpg)
Picture Types: Temporal Sub-Layer Access
Switching temporal layers
Temporal nesting: at any picture to higher or lower tidGeneral: switching of temporal layer only at tid = 0
Temporal sub-layer access: additional option
Temporal Sub-Layer Access (TSA)
Switch to any higher tid at TSA picture
No reference to higher tid by TSA picture
Stepwise Temporal Sub-Layer Access (STSA)
Switch to tid of STSA picture possible
Switch to higher temporal layers not possible
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 25 / 235
![Page 26: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/26.jpg)
Spatial Coding StructuresBlocks and Units
Block: Square or rectangular area in a color component arrayUnit: Collocated blocks of the (three) color components, associatedsyntax elements and prediction data (e. g. motion vectors)
Picture partitioningCoding Tree Blocks / Coding Tree Units (CTBs / CTUs)Each CTU in exactly one slice segmentIndependent slice segment: Full header, independently decodableDependent slice segment: very short header, relies on correspondingindependent slice, inherits CABAC state
Slice typesI-slice: Intra prediction onlyP-slice: Intra prediction and motion compensation with one referencepicture listB-slice: Intra prediction and motion compensation with one or tworeference picture lists
CABAC: Context-based Adaptive Binary Arithmetic Coding
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 26 / 235
![Page 27: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/27.jpg)
Wavefront Parallel Processing (WPP)
Storage of CABAC states for synchronization
Two CTUs offset per row (availability of top-left CTU)
Entry points coded in the slice segment header
CTU CTU CTU CTU CTU CTU0 1 2 3 4 5
CTU CTU CTU CTUNc Nc+1 Nc+2 Nc+3
CTU CTU2Nc 2Nc+1
decoder1
decoder2
decoder3
slice seg.header
CTU CTU CTU CTU CTU CTU0 1 Nc Nc+1 2Nc 2Nc+1
· · · · · · · · ·
ep0 ep1 ep2
Bitstream
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 27 / 235
![Page 28: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/28.jpg)
Tiles
Change scanning order of CTBs in picture
Slices in tiles, or tiles in slices
Reset of prediction and entropy coding→ parallel processing(entry points like WPP)
Slice 1
Slice 2
Slice 3
Slice 4
Slice 5
Slice 6
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 28 / 235
![Page 29: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/29.jpg)
Tiles
Change scanning order of CTBs in picture
Slices in tiles, or tiles in slices
Reset of prediction and entropy coding→ parallel processing(entry points like WPP)
Slice 1
Slice 2
Slice 3
Slice 4
Slice 5
Slice 6
Tile 1 Tile 2 Tile 3
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 28 / 235
![Page 30: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/30.jpg)
Tiles
Change scanning order of CTBs in picture
Slices in tiles, or tiles in slices
Reset of prediction and entropy coding→ parallel processing(entry points like WPP)
Slice 1
Slice 2 Slice 3 Slice 4
Slice 5
Tile 1 Tile 2 Tile 3
Tile 4 Tile 5 Tile 6
Tile 7 Tile 8 Tile 9
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 28 / 235
![Page 31: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/31.jpg)
Coding Tree Blocks and Coding Blocks (CBs)
Quadtree partitioning of CTB into CBs
If picture size not integer multiple of CTB size:Implicit CTB partitioning to meet picture size (must be multiple of 8×8pixels)
0
1 2
3 4
5 67
8 9
10 11
12 13
14 1516
17 18
19 2021
0
1 2
3 4 5 6
7 8 9 10 11
12 13 14 15
16
17 18 19 20
21
(a) (b)
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 29 / 235
![Page 32: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/32.jpg)
Prediction Blocks (PBs) and Transform Blocks (TBs)
Prediction block partitioning of a 2N×2N CB
INTER
2N×2N 2N×N N×2N N×N
2N×nU 2N×nD nL×2N nR×2N
INTRA
2N×2N
4×4
Transform block partitioning of a CB
Quadtree partitioning of CB→ Residual Quad Tree (RQT)
Transform size 4×4 to 32×32
TB size 4×4 to 64×64
PB boundaries inside TBs allowed
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 30 / 235
![Page 33: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/33.jpg)
Outline
4 Coding Structures
5 Reference PicturesReference Picture SetReference Picture List
6 High-Level Syntax
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 31 / 235
![Page 34: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/34.jpg)
Reference Picture Sets (RPS)Reference Picture Set
Set of previously decoded pictures
To be used as reference for inter prediction
Identified by POC value
Picture marking
Use in current or following pictures
Unused for reference (can be removed from DPB)
Construction
Short-term before (B)
Short-term after (A)
Long-term
spec further reading: [4]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 32 / 235
![Page 35: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/35.jpg)
Short-Term RPS – Example
POC
RPS
0 8B0 *B1B2B3
0 842 61 3 5 7-1-2-3-4-5-6-7-8
1 9B0 A0*B1
2 10B0 A1A0*B1
3B0 A2A1A0*
4B1 A1A0B0 *
5B2 A0B0B1 *
6B1 A1B0 A0*
7B2 A0B1 B0 *
0 842 61 3 5 7-1-2-3-4-5-6-7-8
RPS of random access configuration from the JCT-VCcommon testing conditions JCTVC-K1100 [8]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 33 / 235
![Page 36: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/36.jpg)
Reference Picture List (RPL)
Reference picture lists constructed from available RPSSize of RPL signaled in PPS or slice segment header
One list in P-slices (List0)
Two lists in B-slices (List0, List1)
ConstructionList0
Short-term beforeShort-term afterLong-term
List1Short-term afterShort-term beforeLong-term
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 34 / 235
![Page 37: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/37.jpg)
Outline
4 Coding Structures
5 Reference Pictures
6 High-Level SyntaxNetwork Abstraction LayerParameter SetsPicture Order CountHypothetical Reference DecoderSupplemental Enhancement InformationVideo Usability Information
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 35 / 235
![Page 38: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/38.jpg)
Network Abstraction Layer (NAL)
Coded Video Sequence (CVS)
Starts with a ’new’ IRAP (associated RASLs to be discarded)
One or more CVSs in a bitstream→ Coded Video Sequence Group (CVSG)
Network Abstraction Layer
Encapsulation of coded video sequence for transport and storageVideo coding layer (VCL) NAL units
All video data, i. e.Slices with CTUs, PUs, TUs
Non-VCL NAL unitsParameter setsSupplemental enhancement information. . .
ScHaWaWe12 [9]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 36 / 235
![Page 39: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/39.jpg)
NAL Unit Structure
| | | | | | | | |
. . .
byte 0 1 2 . . . . . . NU−1
1 0 0 0 0 0
NALU header RBSP
SODBRBSP stop bitMSB LSB
RBSP: Raw byte sequence payloadSequence of bytes comprising the coded NAL unit payloadRBSP stop bit (=’1’) plus zero bytes for byte alignment
SODB: String of data bitsConcatenation of bits in the RBSP bytes from MSB to LSBAll bits needed for the decoding processOnly the bits needed for the decoding process
MSB: Most significant bitLSB: Least significant bit
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 37 / 235
![Page 40: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/40.jpg)
NAL Unit Header (2 bytes)
“0” NAL unit type NUH layer id temporal id
byte 1 byte 2
NAL unit type: characterize content of VCL/ non-VCL NAL unit
NAL unit header (NUH) layer id: equal to 0 for HEVC Version 1, for use inscalable extension (spatial layers, quality layers), multi-view extension(view id)
temporal id ( tid): temporal sub-layer identifier
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 38 / 235
![Page 41: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/41.jpg)
Access Units
startAUD
(prefix)SEI
CSSsuffixSEI
EOS EOBend
Access Unit (AU)Set of all NAL units associated to exactly one pictureAll NAL units of AU share output time of included pictureNot depicted
Parameter setsOther NAL units types
Decoding UnitsDecoder operation on sub-picture levelOperation of the HRD on sub-picture basis(HRD = Hypothetical Reference Decoder, see below)Enables sub-picture output before complete decoding of full picture
AUD Access unit delimiterSEI Supplemental enhancement informationEOS End of sequenceEOB End of bitstream
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 39 / 235
![Page 42: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/42.jpg)
Parameter Sets
Hierarchical structure
Separation of information for different hierarchy levels
Highest-priority information
In-band or out-of-band transmission
Available parameter setsVideo parameter set (VPS)Sequence parameter set (SPS)Picture parameter set (PPS)
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 40 / 235
![Page 43: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/43.jpg)
Parameter Sets
VPS1
VPS2
SPS1
SPS2
SPS3
PPS1
PPS2
PPS3
PPS4
PPS5
slice headers in coded video seq. A
slice headers in coded video seq. B
slice headers in coded video seq. M
slice headers in coded video seq. N
...
...
bitstream I
bitstream II
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 41 / 235
![Page 44: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/44.jpg)
Coded Video Sequence Group
CVS1
CVS2
CVSG
VPS1
SPS1 PPS2 SSH lid = 1
SPS1 PPS1 SSH lid = 0
SPS3 PPS4 SSH lid = 1
SPS3 PPS3 SSH lid = 0
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 42 / 235
![Page 45: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/45.jpg)
Video Parameter Set
Introduced for handling of multi-layer bitstreamsGeneral information, activated for all layers
Coded video sequence(s), included layers, available operation points
Highest layer parameter set: sets global constraints
HEVC Version 1: not needed, copy of SPS information
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 43 / 235
![Page 46: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/46.jpg)
Sequence Parameter Set
Activated for the coded video sequence
Profile, tier, and level
Usage of tools
Video usability information
. . .
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 44 / 235
![Page 47: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/47.jpg)
Picture Parameter Set
Activated per picture
May change from picture to picture (only one per picture)Tool configuration
CABACQuantizers,Loop filtersTiles. . .
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 45 / 235
![Page 48: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/48.jpg)
Slice Segment Header
Activates the chain of parameter sets (once per picture)
Slice segment header must refer PPS with identical content (but notnecessarily identical PPS id)
Includes picture order count
Includes all information needed to decode the independent slice segmentand associated dependent slice segments
Entry points for wavefront parallel decoding / tiles
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 46 / 235
![Page 49: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/49.jpg)
Picture Order Count (POC)
POC-2
POC-1
POC
POC+1
Identifier of picture in the DPB
Indicates output order of the pictures, strictly increasing
POC of IDR pictures always 0: start of new coded video sequence
Used for derivation of picture distance in inter prediction, scaling ofmotion vectors
Constant POC delta not necessary
VPS: indication of POC relation to picture time difference possible
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 47 / 235
![Page 50: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/50.jpg)
Hypothetical Reference Decoder (HRD)
hypotheticalstream
scheduler(HSS)
codedpicturebuffer(CPB)
decodingprocess(instan-
taneous)
decodedpicturebuffer(DPB)
outputcropping
Encoder: control of buffer states
Introduction of timing
Conformance testing
Parameter signalling in VPS or SPS
Sub-picture operation possibleAU operation→ DU operation (new in HEVC)
Specified in HEVC Annex C spec , further reading: [10]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 48 / 235
![Page 51: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/51.jpg)
HRD: Decoded Picture Buffer
hypotheticalstream
scheduler(HSS)
codedpicturebuffer(CPB)
decodingprocess(instan-
taneous)
decodedpicturebuffer(DPB)
outputcropping
Reconstructed picturesReference pictures (short-term / long-term)Pictures to be displayed
Picture output timing: Delay relative to CPB removal timeNote: Prevention of picture output possible
Signalled in the bitstreamIndicated by BLA picture type
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 49 / 235
![Page 52: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/52.jpg)
Supplemental Enhancement Information (SEI)
SEI message content
Not required for the decoding process
Potentially useful for the decoding process
Attached and integrated into bitstream, sent with coded video sequence
Error recovery, testing for integrity, . . .
Prefix SEI: Cannot occur after the NAL units of the coded picture
Suffix SEI: Cannot occur before the NAL units of the coded pictures
Concept of SEI from H.264 |AVC, payload types aligned for inherited SEImessages
Specified in Annex D spec
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 50 / 235
![Page 53: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/53.jpg)
Video Usability Information VUI
Not required by the decoding process
Information on the interpretation of the decoded video sequence
Signaling in-band (as part of SPS) or out-of-band
Parameters
Geometric relations→ sample aspect ratio (SAR), overscan indication
Video signal type and color information
Frame / field indication
Default display window (suggested area within the cropping window)
HRD/ Timing information (including clock tick for picture rate)
Restrictions: Tiles, motion vectors, RPL, spatial segmentation, maximumbyte cost (picture / CTU)
Specified in Annex E spec
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 51 / 235
![Page 54: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/54.jpg)
Part III
HEVC Coding Tools
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 52 / 235
![Page 55: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/55.jpg)
Outline
7 Intra PredictionIntra Prediction ModesIntra Coding Example
8 Inter Prediction
9 Residual Coding
10 Loop Filtering
11 Entropy Coding
12 Profiles in HEVC Edition 1
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 53 / 235
![Page 56: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/56.jpg)
Intra Prediction
Decoder
CTB
input picture+−
TR+Q
TB
iTR+iQ TB
+
Intra
PB
EntropyCod-ing
bitstream
Deblk. Slice
LoopFilter
Slicerec. picture
Inter
PB
Buffer n pics
ME
PB
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 54 / 235
![Page 57: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/57.jpg)
Intra Prediction Modes
0 : Planar1 : DC
1817
16
15
14
13
12
11109
8
7
6
5
4
32
3433
3231
3029 28 27 26 25 24 23
2221
2019
Planar prediction: mode 0DC intra prediction: mode 1Numbering from diagonal-up to diagonal-downHorizontal: mode 10, vertical: mode 26
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 55 / 235
![Page 58: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/58.jpg)
Intra Prediction Block size
Intra prediction mode coded per CU
Prediction block size derived from residual quadtree
Boundary samples of neighboring block used for prediction
Efficient representation
Local update of prediction source
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 56 / 235
![Page 59: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/59.jpg)
Intra Coding Example
DC
DC
DC
DC
DC
P
P DC
P P
DC
P
P
DC
P
P
P
DC
P
P P
DC
original
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 57 / 235
![Page 60: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/60.jpg)
Intra Coding Example
DC
DC
DC
DC
DC
P
P DC
P P
DC
P
P
DC
P
P
P
DC
P
P P
DC
prediction
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 58 / 235
![Page 61: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/61.jpg)
Intra Coding Example
DC
DC
DC
DC
DC
P
P DC
P P
DC
P
P
DC
P
P
P
DC
P
P P
DC
residual
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 59 / 235
![Page 62: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/62.jpg)
Outline
7 Intra Prediction
8 Inter PredictionMotion Compensated PredictionMotion Vector RepresentationInter Coding Examples
9 Residual Coding
10 Loop Filtering
11 Entropy Coding
12 Profiles in HEVC Edition 1
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 60 / 235
![Page 63: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/63.jpg)
Inter Prediction
Decoder
CTB
input picture+−
TR+Q
TB
iTR+iQ TB
+
Intra
PB
EntropyCod-ing
bitstream
Deblk. Slice
LoopFilter
Slicerec. picture
Inter
PB
Buffer n pics
ME
PB
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 61 / 235
![Page 64: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/64.jpg)
Motion Compensated Prediction
POC-2
POC-1
POC
POC+1
Prediction from reference picture listsUni-prediction
P-slices only with List0, B-slices with List0 or List1Minimum PB size 8×4 or 4×8
Bi-prediction, only in B-slicesOne predictor from List0, one predictor from List1Minimum PB size 8×8
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 62 / 235
![Page 65: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/65.jpg)
Motion Vector Representation
Merge mode
Motion vector (MV) derived from candidate set (spatial and temporalneighborhood)
Merge mode candidate index coded
No motion vector difference encoded
Advanced motion vector prediction
Predictor derived from candidate set (spatial and temporal neighborhood)
Predictor index coded
Motion vector difference encoded
HeOuBrMaBi12 [12]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 63 / 235
![Page 66: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/66.jpg)
Merging Motion Vectors
Merge candidate list
List with up to five different motion merge candidates (list length indicatedin the slice segment header)
Unavailable candidates are ignored
List filled to specified length→ Improved loss robustness:Available list length independent of derivation process
Additional combination of reference picture list 0 and 1 candidates forB-slices
Merge mode granularity
PU grid size configured in the PPS
PUs below grid size share merge candidate list
Adjustable computational complexity
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 64 / 235
![Page 67: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/67.jpg)
Merging Motion Vectors – Spatial Candidates
A0
A1
B0B1
B2
(xP,yP)
PU
(xP,yP) (xP,yP)
(a) Merge candidates (b) 4×8 PUs
Processing order: A1,B1,B0,A0,B2
Candidates A1,B1 only if not right or bottom PU in CU with two partitions,respectively
Maximum four spatial merge candidates
Reduced line buffer storage requirement: shifted neighbor locations forblocks of width 4
spec
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 65 / 235
![Page 68: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/68.jpg)
Merging Motion Vectors – Temporal Candidates
collocated region inreference picture
C0
C1n−5 n−4 n−3 n−2 n−1 n
current picture
POC
td tb
Location C1 if C0 unavailable, or PU at bottom right CTU boundary(complexity)
Applicable reference picture selected on slice basisB-slices: Reference picture list 0 or 1 indicated by flag
Reference index of candidate always set to 0 (error resilience, complexity)
MV scaling according to POC distance
mvPU =tbtd·mvcol
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 66 / 235
![Page 69: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/69.jpg)
Predictive Motion Vector Coding
A0
A1
B0B1B2
(xP,yP)
PU
collocated region in
reference picture
C0
C1
Advanced motion vector prediction
Reference index and motion vector difference coded!
Selection of predictor by flag (only two options)Derivation process for each reference picture list
Locations from spatial neighborhood as shown for merge modeCandidate MVs scaled based on POC differenceCandidate A first of A0,A1, candidate B first of B0,B1,B2
Optional: Additional temporal candidate C if A or B not available
spec
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 67 / 235
![Page 70: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/70.jpg)
Motion Vector Signaling: Skip spec
CU Skip
Indication via flag at beginning of CU syntax
Based on Merge mode
Only merge index signalled
No further syntax elements for CU→ Residual inferred to be zero→ Very cheap coding mode
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 68 / 235
![Page 71: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/71.jpg)
Inter Coding Examples
Legend: List 0 = diagonal-down hatch, List 1 = diagonal-up hatch, ref.idx=0: gray, ref.idx=1, dark gray
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 69 / 235
![Page 72: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/72.jpg)
Inter Coding Examples
(a) mvL0 (b) mvL1
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 70 / 235
![Page 73: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/73.jpg)
Inter Coding Examples
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 71 / 235
![Page 74: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/74.jpg)
Inter Coding Examples
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 71 / 235
![Page 75: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/75.jpg)
Outline
7 Intra Prediction
8 Inter Prediction
9 Residual CodingCore TransformsQuantizationCoded Representation of Transform BlocksSpecial Modes
10 Loop Filtering
11 Entropy Coding
12 Profiles in HEVC Edition 1
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 72 / 235
![Page 76: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/76.jpg)
Residual Coding
Decoder
CTB
input picture+−
TR+Q
TB
iTR+iQ TB
+
Intra
PB
EntropyCod-ing
bitstream
Deblk. Slice
LoopFilter
Slicerec. picture
Inter
PB
Buffer n pics
ME
PB
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 73 / 235
![Page 77: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/77.jpg)
Core Transforms
Transform block sizes 4×4, 8×8, 16×16, and 32×32
Integer approximations of the DCT-II transform matrix
Additionally, integer approximation of the DST-VI transform matrix
’Single-norm’ design per transform block size→ simple quantizerimplementation
Not all perfectly orthogonal, leakage below normalization threshold
BuFuBjSzSa13 [13]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 74 / 235
![Page 78: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/78.jpg)
Quantizer Step Size. . . . . .
nq
x-4 -3 -2 -1 1 2 3 40
∆q
Quantizer step size ∆q derived from quantization parameter QP
Logarithmic relation of quantizer step sizes
Double step size every 6 QP
∆q( QP + 1) =6√
2 ·∆q( QP)
Definition: ∆q = 1 for QP = 4, thereby
∆q,0 ∈{
2−46 ,2−
36 ,2−
26 ,2−
16 ,1,2
16
}Quantizer step sizes for QP > 5
∆q( QP) = ∆q,0( QPmod6) ·2bQP6 c
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 75 / 235
![Page 79: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/79.jpg)
Quantizer Step Size
Quantizer rangeQP = 0, . . . ,51
Resulting quantizer step sizes
0.630≤∆q ≤ 228.1
Covering value range of an 8 bit input signal
Higher input bit depth:Extension towards finer quantizationRange extended by 6 QP steps per additional bit
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 76 / 235
![Page 80: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/80.jpg)
Transform Sub-Blocks (TSBs)
last significant coefficient (xlsc,ylsc)
16
16
4
4transform sub-block
Scan of transform sub-blocks
Last significant coefficient position
Determination of transform sub-blocks with non-zero coefficients
Coded sub-block flag
spec
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 77 / 235
![Page 81: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/81.jpg)
Transform Sub-Block Scanning
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
0 1 2 3
4 5 6 7
8 9 10 11
12 13 14 15
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
0
1
2
3
0
2
1
3
diag.,vert.
horiz.
(a) 4×4 diagonal (b) 4×4 horizontal (c) 4×4 vertical (d) 8×8 TSB scans
Partitioning of transform block into 4×4 transform sub-blocks (TSBs)
Scan direction selectable for 4×4 and 8×8 blocks, diagonal otherwise
Scan direction in TSB depending on (intra) prediction modeLevel distribution: ’trailing ones’ expected towards higher frequencies
Scan used in inverse directionStart with expected ’1’ values
Level: quantized transform coefficient level nq
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 78 / 235
![Page 82: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/82.jpg)
Special Modes
Transform skip, applicable to 4×4 TUs
Omit transform step
Inverse quantization operation
Additional bit shift to compensate missing transform
Transform and quantization bypass
No inverse transform, no quantizer scaling
Perfect reconstruction of residual (useful e. g. for graphics content)
PCM mode
Direct encoding of pixel levels, lossless representation option for CU
Configurable bit depth (SPS)
Maximum PCM block size 32×32
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 79 / 235
![Page 83: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/83.jpg)
Outline
7 Intra Prediction
8 Inter Prediction
9 Residual Coding
10 Loop FilteringDeblocking FilterSample Adaptive Offset
11 Entropy Coding
12 Profiles in HEVC Edition 1
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 80 / 235
![Page 84: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/84.jpg)
Loop Filtering
Decoder
CTB
input picture+−
TR+Q
TB
iTR+iQ TB
+
Intra
PB
EntropyCod-ing
bitstream
Deblk. Slice
LoopFilter
Slicerec. picture
Inter
PB
Buffer n pics
ME
PB
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 81 / 235
![Page 85: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/85.jpg)
Deblocking Filter
Design considerations
Reduction of visible blocking artifacts induced from block-wise processing
Computational complexity
Parallel processing
Deblocking processing
Operation on slice / tile basis
Slice-based control of deblocking filter configuration possiblePicture-based or block-based implementations possible
Line buffer for block-based processing
[14]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 82 / 235
![Page 86: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/86.jpg)
Deblocking Filter Neighborhood
vertical edges horizontal edges
q0,A1p0,A1
q0,A2p0,A2
q0,A3p0,A3
q0,A4p0,A4
q0,B1p0,B1 q0,B2p0,B2
q0,C1p0,C1
q0,C2p0,C2
q0,C3p0,C3
q0,A1
p0,A1
q0,A2
p0,A2
q0,A3
p0,A3
q0,A4
p0,A4
q0,B1
p0,B1
q0,B2
p0,B2
q0,B3
p0,B3
q0,B4
p0,B4
q0,C
p0,C
(a) (b)
Neighborhood determination
q0,p0: reference samples in current block and neighboring block
Filtering only on 8×8 sample grid!
Edge sections of 4 sample length
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 83 / 235
![Page 87: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/87.jpg)
Boundary Strength
Boundary strength parameter bS
Neighboring blocks determined by q0,p0
Possible boundary parameter strength values: 0,1,2
Determination of bS
At least one intra coded block: bS = 2
Transform block boundary and non-zero coefficients: bS = 1Motion information conditions: bS = 1
Different reference pictures or different number of motion vectorsSame number of MVs and |mvx ,p−mvx ,q| ≥ 1 or |mvy ,p−mvy ,q| ≥ 1(full-sample resolution)
Otherwise: bS = 0
spec
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 84 / 235
![Page 88: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/88.jpg)
Deblocking Filter Operation
Deblocking filter operationOperation on a 4-sample edge basis
Luma: Deblocking if bs ≥ 1Chroma: Deblocking if bs = 2
First vertical filtering, then horizontal filtering
Independent operation on 8×8 block grid→ parallel processing!
spec
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 85 / 235
![Page 89: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/89.jpg)
Deblocking Filter Example
(a) Original (b) Reconstruction with deblocking
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 86 / 235
![Page 90: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/90.jpg)
Deblocking Filter Example
(c) Structure, deblocked samples (d) Reconstruction without deblocking
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 87 / 235
![Page 91: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/91.jpg)
Deblocking Filter Example
(e) Normal deblocking (f) Strong deblocking
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 88 / 235
![Page 92: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/92.jpg)
Sample Adaptive Offset (SAO)
New filter type in ITU-T / MPEG video coding specificationsLocal processing of samples
Depending on local neighborhood (edge offset), orDepending on sample value (band offset)
Operation independent of processed samples→ parallel processing
Local filter parameter adaptation
Four different offset values available (plus SAO off)
Dedicated SAO parameters for each Y, Cb, Cr component
Common SAO mode for chroma components
further reading: [15]spec
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 89 / 235
![Page 93: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/93.jpg)
SAO: Edge Offset
pcp0 p1 pc
p1
p0
pc
p1
p0
pc
p1
p0
(a) horizontal (b) vertical (c) diagonal-down (d) diagonal-up
Neighborhood
Two samples from 8-connected neighborhood considered
Direction of neighborhood signaled on CTU basis
Relation of sample values determines edge offset index ie
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 90 / 235
![Page 94: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/94.jpg)
SAO: Edge Offset
p0 pc p1 p0 pc p1 p0 pc p1 p0 pc p1 p0 pc p1
(a) ie = 0 (b) ie = 1 (c) ie = 2
p0 pc p1 p0 pc p1 p0 pc p1
(d) ie = 4 (e) ie = 3
Offset categories
Edge offset index identifies category
Smoothing only, direction of offset predefined
Sign of SAO offsets not signaled
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 91 / 235
![Page 95: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/95.jpg)
SAO: Band Offset
low band transition high band
transition band position io
0 8 16 24 32 40 48 56 64 72 80 88 96 104 112 120 128 136 144 152 160 168 176 184 192 200 208 216 224 232 240 248
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
255
intensity value (example: bit depth 8bit)
Correction of sample intensity values for four transition bands
Signaling of transition band position
Offset values for transition bands freely configurable
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 92 / 235
![Page 96: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/96.jpg)
SAO Filter Example
(a) Reconstructed region (b) Samples with band offset active
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 93 / 235
![Page 97: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/97.jpg)
SAO Filter Example
(c) Reconstructed region (d) Samples with edge offset active
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 94 / 235
![Page 98: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/98.jpg)
SAO Filter Example
CTBs with SAO edge (lines) and band offset (gray squares)
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 95 / 235
![Page 99: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/99.jpg)
Outline
7 Intra Prediction
8 Inter Prediction
9 Residual Coding
10 Loop Filtering
11 Entropy CodingFixed Length and Variable Length CodingContext-Based Adaptive Binary Arithmetic Coding
12 Profiles in HEVC Edition 1
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 96 / 235
![Page 100: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/100.jpg)
Entropy Coding
Decoder
CTB
input picture+−
TR+Q
TB
iTR+iQ TB
+
Intra
PB
EntropyCod-ing
bitstream
Deblk. Slice
LoopFilter
Slicerec. picture
Inter
PB
Buffer n pics
ME
PB
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 97 / 235
![Page 101: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/101.jpg)
Entropy Coding
Fixed length and variable length codes (FLC, VLC)
High-level syntax
Parameter sets, slice segment header
SEI messages
Fixed-length codes, Exp-Golomb codes
Arithmetic coding
Slice level, CTUs
Context-based adaptive coding
Bypass coding (complexity, throughput)
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 98 / 235
![Page 102: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/102.jpg)
Variable Length and Arithmetic Coding
slice seg.header
NALUheader
CTU CTU CTU CTU0 NC−1 NC NS−1
· · · · · ·
FLC FLC,VLC CABAC CABACba
ep0
VCL NAL Unit
FLC, VLC for header information
CABAC for CTUs
Byte alignment in case of multiple tiles, or with wavefront parallelprocessing (not present otherwise)
CABAC = Context-based Adaptive Binary Arithmetic Codingba = byte alignment
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 99 / 235
![Page 103: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/103.jpg)
Context-Based Adaptive Binary Arithmetic Coding – CABAC
Binary Arithmetic Coder
syntaxele-
ment
BinarizerContextModeler
AdaptiveEngine
BypassEngine
bitstream
binstring
binary value
binvalue coded bits
bin value coded bits
context update
BinarizationContext model selectionBinary arithmetic codingOptimized binarization design, reduced number of non-bypass binscompared to H.264 |AVC
MaScWi03 [16]SzBu12 [17]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 100 / 235
![Page 104: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/104.jpg)
Outline
7 Intra Prediction
8 Inter Prediction
9 Residual Coding
10 Loop Filtering
11 Entropy Coding
12 Profiles in HEVC Edition 1ProfilesTiers and Levels
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 101 / 235
![Page 105: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/105.jpg)
Profiles, Tiers, and Levels in HEVC Edition 1
Main Still Picture
Main
Main 10
Annex A
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 102 / 235
![Page 106: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/106.jpg)
HEVC Edition 1 ProfilesMain Profile
YCbCr 4:2:0 8 bit video only
CTB block sizes 16×16, 32×32, and 64×64
Either tiles or wavefront parallel processing
Minimum tile size 256×64 pixels
Maximum number of CABAC coded bits in N×N CTU: 53 braw
with braw = (N2 + N/2) ·Bd for YCbCr 4:2:0, bit depth Bd
Main 10 Profile
Additional support of bit depths of 9 and 10 bit
Main Still Picture Profile
Like Main, but only one picture in the bitstream
No timing constraints for the decoder
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 103 / 235
![Page 107: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/107.jpg)
Tiers and Levels
Profiles
Defined tool (sub-)set of specification
Tools determined from application space
Tiers and levels
Levels: Restrictions on parameters which determine decoding andbuffering capabilities (13 levels defined)
Tiers: Grouping of level limits for different application spaces (currently:Main Tier for consumer, High Tier for professional applications)
Decoder capable of decoding Profile@Tier/Level must be able to decodeall lower levels of same and lower tier
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 104 / 235
![Page 108: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/108.jpg)
Extensions of HEVC
Range extensions (HEVC V.2/Ed. 2, 10/2014) V.x=ITU-Version, Ed.y=ISO/ICE Edition
Extended color formats (4:2:2, 4:4:4)
Extended bit depth
Scalable extensions(HEVC V.2/Ed. 2, 10/2014)
Simple, multiloop approach, no modifications on tool-level
Supports spatial, SNR scalability
Multi-view (HEVC V.2/Ed. 2, 10/2014)3D extensions (HEVC V.3/Ed. 3, 04/2015)
Stereo / multi-view coding
Multi-view with depth coding
Screen Content Coding (V.4/Ed. 3, 2016)
Dedicated tools for this content type
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 105 / 235
![Page 109: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/109.jpg)
Part IV
Range and Multi-Layer Extensions
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 106 / 235
![Page 110: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/110.jpg)
Outline
13 Range ExtensionsExtended Color FormatsRExt ToolsRExt ProfilesRExt Levels
14 Annex F: Common specifications for multi-layer extensions
15 Annex G: Multiview High Efficiency Video Coding
16 Annex H: Scalable High Efficiency Video Coding
17 Annex I: 3D High Efficiency Video Coding
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 107 / 235
![Page 111: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/111.jpg)
Range Extensions Applications
Professional Requirements
Highest quality, very high bitrates
Increased chroma resolution
Increased precision: higher bit depth
Applications
Professional: contribution, distribution
Medical applications
. . .
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 108 / 235
![Page 112: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/112.jpg)
4:2:2 Residual Quadtree
Y Cb Cr
YCbCr 4:2:2
Chroma shares vertical luma resolution, half horizontal resolution
Residual quadtree kept for the three components
Chroma includes two stacked N2×
N2 transform blocks with N×N luma
transform block
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 109 / 235
![Page 113: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/113.jpg)
4:4:4 Residual Quadtree
Y Cb Cr
YCbCr or GBR or YZX 4:4:4
Chroma shares vertical and horizontal luma resolutionTwo option, depending on separate_colour_plane_flag
0: Residual quadtree kept for the three components1: Each color component treated as independent monochrome plane
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 110 / 235
![Page 114: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/114.jpg)
RExt Tools: Transform Skip Rotation
0 1 2 3
4 5 6 7
8 9 A B
C D E F
=⇒180◦
F E D C
B A 9 8
7 6 5 4
3 2 1 0
Rotation of residual in transform block
Signaled in SPS
Applicable to intra 4×4 transform blocks using transform skip
Only if transform and quantization bypass enabled
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 111 / 235
![Page 115: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/115.jpg)
RExt Tools: Implicit/Explicit Residual DPCM Coding
Implicit Residual DPCM Coding
Usage signaled in SPS
Activated only for blocks using horizontal and vertical intra prediction
Transform skip or transform and quantization bypass to be enabled
Explicit Residual DPCM Coding
Inter blocks with transform bypass
Usage and direction explicitly signaled
Example: horizontal DPCM
r(x ,y)+ = r(x−1,y); for x = 1 . . .bS−1, y = 0 . . .bS−1
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 112 / 235
![Page 116: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/116.jpg)
RExt Tools: Inter-Component Residual Prediction
Tool for 4:4:4 content
Prediction of chroma component residual block from luma residual block
RC = R′C + α ·RY
with α ∈ {0,±1,±2,±4,±8}Additional decorrelation of residual signal
Gains on RGB content reported [18]
Activated in PPS
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 113 / 235
![Page 117: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/117.jpg)
RExt Profiles
Main
Intra
Main4:4:4Intra
Main4:4:4
10 Intra
Main4:4:4
12 Intra
Main4:4:4
16 Intra
High Throughput4:4:4
16 Intra
Monochrome
Monochrome16
Monochrome 12Main
4:2:2 Intra
Main4:2:2
10 Intra
Main10 Intra
Main10
Main12 Intra
Main4:2:2
12 Intra
Main12
MainStill Picture
Main4:4:4
Still Picture
Main4:4:416
Still Picture
Main4:4:4
Main4:4:410
Main4:4:412
4:2:2Main4:2:210
Main4:2:212
Range extensions profiles: general profile idc = 4High throughput profiles: general profile idc = 5
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 114 / 235
![Page 118: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/118.jpg)
RExt Levels
Level limits specified in Version 1 unsufficient for RExt profilesIntroduction of scaling factors
HbrFactor: High bitrate factorControlled by low bit rate flag spec
Values can be 1, 2, or 12, 24 for High Throughput profiles
CpbVclFactor / CpbNalFactorScaled by HbrFactor to apply for table A.4 spec
FormatCapabilityFactorScaling factor for number of bytes in an Access Unit
MinCrScaleFactorScaling factor for minimum compression ratio
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 115 / 235
![Page 119: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/119.jpg)
RExt Performance
Comparison of HEVC RExt in HM 16.2 relative to AVC FRExt in JM 18.6
Lossy coding [BD-rate %] Lossless coding, rate saving [%]
Figure from FlMaNaNgRoShSoXu16 [19]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 116 / 235
![Page 120: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/120.jpg)
Outline
13 Range Extensions
14 Annex F: Common specifications for multi-layer extensionsCommon Specification StructureParameter SetsBitstream SubsetsSEI messages
15 Annex G: Multiview High Efficiency Video Coding
16 Annex H: Scalable High Efficiency Video Coding
17 Annex I: 3D High Efficiency Video Coding
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 117 / 235
![Page 121: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/121.jpg)
Common Specification Structure for Multilayer Video
Scalable and multiview coding share concept of layersScalable layers: different picture resolutions, different reconstruction qualityMultiview layers: different views on the same scene
Specification approach in HEVC extensionsUnify common parts of extensions in a joint specification annexOnly separate tools in separate annexes
Chosen extension structureAnnex F: “Common syntax, semantics and decoding processes formultilayer video coding extensions”Annex G: “Multiview coding”Annex H: “Syntax, semantics and decoding processes for scalableextension”Annex I: “3D High Efficiency Video Coding”
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 118 / 235
![Page 122: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/122.jpg)
Annex F: Main Features
High level syntax
Introduction of layer identification and layer reference mechanims
Specification of layer structure and layer dependencies
Usability and scalability information
Extended ConceptsIntroduction of auxiliary layers (not for output)
DepthAlpha-plane
Option for non-HEVC baselayer (H.264 |AVC)
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 119 / 235
![Page 123: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/123.jpg)
Video Parameter Set
VPS1
SPS2 PPS2 SSH lid = 1
SPS1 PPS1 SSH lid = 0
Specification of available layers in the bitstreamScalable layersMultiple viewsAuxiliary pictures (e. g. depth or alpha planes)
Specification of layer dependenciesPrediction relationsLayer sets
Available operation points, output layer setsAvailable profiles and levels
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 120 / 235
![Page 124: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/124.jpg)
NAL Unit Header: Layer ID
“0” NAL unit type NUH layer id temporal id
byte 1 byte 2
Each layer has unique layer identifier lidScalability types defined in VPS
Indication of texture or depth layerView order index for multi-view coding voidx
Spatial / quality scalability→ dependency identifier didAuxiliary pictures (not used for coding of primary picture)
Auxiliary picture identifier aidNo prediction between pictures with different value of aid
Meaning of the NUH bits in identifier configurableCombination of different scalability types possible→ splitting flagAdditional identifier for each layer encoded in VPSOutput-flags: control of presentation of layers
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 121 / 235
![Page 125: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/125.jpg)
VPS Extension: Layer Dependencies
target layer
direct reference layer
indirect reference layer
poc-1
lid = 2
lid = 1
lid = 0
lid = 1
lid = 0
poc
lid = 2
lid = 1
lid = 0
poc+1
lid = 2
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 122 / 235
![Page 126: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/126.jpg)
Layer Dependencies – Adaptation in Network
Strict lid hierarchy
Layer hierarchy→ enable scalability
Simple MANEs: achieve sub-bistream extraction by cutting at some lid, tidcombination
Advanced MANES: analyze dependencies and make use of advancedknowledge
MANE = Media Aware Network Element
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 123 / 235
![Page 127: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/127.jpg)
VPS Extension: Layer Sets
Layer set
set of lid values which can be extracted from the bitstream for decoding(e. g. different spatial resolutions)
Output layer set (OLS)
Set of layers which are output if flag indicates OLS to be a target OLS
output_layer_flag: indicates if layer is to be output or notExample: Depth layer not output layer, but associated texture layer isouput layer
alt_output_layer_flag: top non-output layer is output if outputlayer not presentExample: Quality scalability, dropping of highest quality layer
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 124 / 235
![Page 128: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/128.jpg)
VPS: Layer Video Usability Information
Information on characteristics of layer (optional)
Cross layer picture type alignment
Cross layer IRAP alignment
Cross layer IDR alignment
Information on average bitrate, maximum bitrate
Indication of constant or variable picture rate
Hints for use of tiles, loop filters, WPP, cross layer tile alignment
Indication of restrictions on inter-layer prediction
. . .spec
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 125 / 235
![Page 129: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/129.jpg)
Sequence Parameter Set Extensions
Profile/Tier/Level information conveyed in VPS for non-baselayer SPS
Reference to layer representation format in VPS, index in SPSMultilayer extension
Constraint: Vertical component of inter-view MVs ≤ 56 luma samples⇒ Reduced memory access (within CTB-row above collocated CTB ininter-view reference picture)
spec
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 126 / 235
![Page 130: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/130.jpg)
Picture Parameter Set Extension
Specification of spatial relation of layersScaled reference layer offset
Offset of luma pixels collocated to corner luma pixels of reference layerregion
Reference region offsetOffset of reference layer region corner luma pixels to corner luma pixels ofreference layer picture
Resample phase informationSpatial phase shift between reference layer and current layer
spec
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 127 / 235
![Page 131: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/131.jpg)
PPS Extension: Color mapping table
Support of different color spaces, different layer bitdepths
Look-up table: map reference layer YCbCr values to target layer
Only increasing bitdepth between layers allowed
Figure from BoYeChRa15 [20]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 128 / 235
![Page 132: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/132.jpg)
Random Access Pictures and POC
IRAP pictures not necessarily aligned over layers
Different temporal resolution
Dependencies / independencies of layers
POC resetPOC reset indication
0: No reset1: Only MSB reset2: MSB and LSB reset3: More information signaled
RASL, RADL, pictures with tid > 0, discardable pictures:Identical POC reset handling for all pictures in AU requiredClean handling after potential bitstream extraction process
If at least IDR picture in AU: only reset types 1 or 2
If lid = 0 non-IDR: no reset types 1 or 2
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 129 / 235
![Page 133: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/133.jpg)
Slice Segment Header Extension
Extension at beginning of slice segment headerReserved bits before slice type syntax element (number from PPS)Version 1: number of reserved bits equal to 0 spec
Version 2/3: Number is set to max 2 spec
Decoders shall accept any value
Activation of inter-layer predictionNumber of inter-layer reference picturesIdentifiers of inter-layer pred ref pictures ( lid)
Slice segment header extension spec
POC modification information (if applicable)
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 130 / 235
![Page 134: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/134.jpg)
Independent Non-Baselayer Rewriting
Capability of bitstream rewriting spec
Indication of independent non-baselayer decodability of layer lid attemporal sub-layer tidProfile-level syntax structureFormal processing steps
Removal of all NAL units with lid 6=target lid and not SPS, PPS, EOBRemoval of SPS, PPS NAL units if not lid =target lid or lid = 0Removal of VPS NAL unitsRemoval of NAL units with tid >target tidRewriting of lid to lid = 0 for all NAL units→ Decodable HEVC V1 bitstream
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 131 / 235
![Page 135: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/135.jpg)
Annex F SEI messages
New SEI messages
plt Name Summary
160 Layers not present Indicate layers of VPS missing in bitstream [persists from SEI to changeSEI]
161 Inter-layer constrained tile sets Indicate decodability of tiles/tile sets[associated CLVS]162 Bitstream partition nesting Carries SEI messages applicable to defined OLS [persistence depending
on nested SEIs]163 Bitstream partition initial arrival time Initial arrival times for CPB operation [remainder of bitstream partition]164 Sub-bitstream property Bit rate information for a sub-bitstream [CVS containing SEI]165 Alpha channel information Control Alpha channel picture persistence [persistence specified by SEI]166 Overlay information Identify alpha and content of overlay pictures [persistence specified by
SEI]167 Temporal MV prediction constraints Indicate storage needs for MVs [persistence specified by SEI]168 Frame-field information How to display pictures [associated AU]
Annex D SEI messages
Syntax of previous SEI messages unchanged
Semantics partly adapted to meet requirements of application withmulti-layer bitstreams
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 132 / 235
![Page 136: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/136.jpg)
Outline
13 Range Extensions
14 Annex F: Common specifications for multi-layer extensions
15 Annex G: Multiview High Efficiency Video CodingMultiview Coding ConceptMultiview ProfilesMultiview SEI MEssagesMultiview Performance
16 Annex H: Scalable High Efficiency Video Coding
17 Annex I: 3D High Efficiency Video Coding
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 133 / 235
![Page 137: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/137.jpg)
Multiview Scenario
P1
picture plane C0 C2 cameras
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 134 / 235
![Page 138: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/138.jpg)
Multiview Scenario
P1
picture plane C0 C2 camerasC1
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 134 / 235
![Page 139: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/139.jpg)
Multiview Coding: View Arrangement
poc-2poc-1
poc
C1
C2
C0
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 135 / 235
![Page 140: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/140.jpg)
Multiview Concept: View-Id and View Order Index
vid = 1
vid = 2
vid = 0
voidx = 0
voidx = 1
voidx = 2
C1
C2
C0
Identification of views
Relative arrangement of views for presentation: view ID vid
Ordering of views for coding: view order index voidx
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 136 / 235
![Page 141: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/141.jpg)
MV-HEVC Layer Terminology
Base View, Independend ViewView order index voidx = 0, lid = 0Conforming to HEVC version 1Independent from other views
Dependent ViewsPredict from reference views
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 137 / 235
![Page 142: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/142.jpg)
Multiview Main Profile
Multiview Main Profile: general profile idc = 6
Multiview Profile based on Main profile
Scalability ID: only view ID or AUX pictures allowed
Aux picture may carry depth information(not used in decoding process for texture view)
Inter view MV vertical constraint flag active→ vertical MV component restricted
All view pictures identical size; no ref local offset allowed(all views aligned)
Number of ref layers (direct or indirect) ≤ 4
Number of pictures in DPB ≤ 8
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 138 / 235
![Page 143: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/143.jpg)
Annex G SEI Messages
plt Name Summary
176 3D reference displays information Information on recommended display [persistence specified by SEI]177 Depth representation information Information on depth and disparity ranges for auxiliary pictures178 Multiview scene information Minimum and maximum disparity in AU [associated CVS]179 Multiview acquisition information Information on aquisition environment [associated CVS]180 Multiview view position SEI message View position from left to right [associated CVS]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 139 / 235
![Page 144: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/144.jpg)
Multiview Performance
Results of subjective evaluation from MV-HEVC verification testJCT3V-N1001 [21]: Comparison of
MVC: AVC-based multi-view video coding (in which the non-base view iscoded using inter-view prediction)
Simulcast HEVC, in which each view is coded independently
MV-HEVC: HEVC-based multi-view video coding (in which the non-baseview is coded using inter-view prediction)
⇒ Gain of approx. 33 % compared to HEVC simulcast⇒ Gain of approx. 50 % compared to H.264 |AVC MVC
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 140 / 235
![Page 145: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/145.jpg)
Multiview Performance
Figure from JCT3V-N1001 [21]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 141 / 235
![Page 146: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/146.jpg)
Outline
13 Range Extensions
14 Annex F: Common specifications for multi-layer extensions
15 Annex G: Multiview High Efficiency Video Coding
16 Annex H: Scalable High Efficiency Video CodingSHVC: Coding ConceptSHVC ToolsScalable ProfilesSHVC Coding Performance
17 Annex I: 3D High Efficiency Video Coding
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 142 / 235
![Page 147: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/147.jpg)
SHVC Layer Terminology
Base layer (BL)Conforming to HEVC version 1Lowest available layer in bitstreamAlternative option: external base layer (H.264 |AVC)
Enhancement layers (EL)Hierarchical organizationDependent or independent from lower layersLower layers used for prediction in enhancement layer: reference layers
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 143 / 235
![Page 148: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/148.jpg)
SHVC Scalable Layers
target layer (EL2)
direct reference layer (EL1)
indirect reference layer (BL)
poc-1
lid = 2
lid = 1
lid = 0
lid = 1
lid = 0
poc
lid = 2
lid = 1
lid = 0
poc+1
lid = 2
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 144 / 235
![Page 149: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/149.jpg)
SHVC SpecificationApproach
Simple high-level syntax approach (“simple solution”)
No changes on CTU level
Inter-layer prediction via reference picture list
Multi-loop coding: Each layer with motion compensation, loop filter, . . .
H.264 |AVC- Scalable Video Coding (SVC) [22]
Single loop design
Motion compensation, loop filter only on target layer
Key picture concept: Limit drift if not all layers decoded
Claim: Performance only 10% below single layer codingIssues
High description complexityPerformance claim only fulfilled byvery well optimized encoders
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 145 / 235
![Page 150: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/150.jpg)
SHVC Tools: Resampling for Inter-Layer Prediction
Size and offset of reference layer picture to target layer picture definescaling for interpolation
Reference layer sample location derivation (16th-sample precision)
Interpolation operation
16 dedicated interpolation filters
DCTIF design: integrate with motion compensation interpolation filters
Phase alignment specified in VPS
(a) Center alignment (b) Top left alignment
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 146 / 235
![Page 151: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/151.jpg)
SHVC Tools: Resampling for Inter-Layer Prediction
−2 0 2 4−16
0
16
32
48
64Phase 0
−2 0 2 4−16
0
16
32
48
64Phase 1
−2 0 2 4−16
0
16
32
48
64Phase 2
−2 0 2 4−16
0
16
32
48
64Phase 3
−2 0 2 4−16
0
16
32
48
64Phase 4
−2 0 2 4−16
0
16
32
48
64Phase 5
−2 0 2 4−16
0
16
32
48
64Phase 6
−2 0 2 4−16
0
16
32
48
64Phase 7
−2 0 2 4−16
0
16
32
48
64Phase 8
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 147 / 235
![Page 152: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/152.jpg)
SHVC Tools: Coding Standard Scalability
Conventional scalable scheme
Base layer HEVC version 1
Enhancement layer with different spatial resolution, different quality
Coding standard scalability
Base layer coded with different standard (H.264 |AVC)
Enhancement layer using HEVC toolsApproach
Only use access to reconstructed samples (standard agnostic)Possible alternative: Specify means to access information regardingpartitioning, prediction modes, motion vectors (not further pursued)
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 148 / 235
![Page 153: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/153.jpg)
SHVC Tools: Coding Standard Scalability
poc-2
lid = 1
lid = 0
poc-1
lid = 1
lid = 0
poc
lid = 1
lid = 0
AVC BL
AVC BL
AVC BL
HEVC EL
HEVC EL
HEVC EL
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 149 / 235
![Page 154: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/154.jpg)
Scalable Main Profiles
Scalable Main Profile and Scalable Main 10 Profile: general profile idc = 7
Based on Main and Main 10 profiles, respectively.
Chroma format 4:2:0 for all layers
Maximum of 16 different representation formats in VPSs
Aux pictures allowed (not used in decoding process)
Layer ID lid → Dependency ID did
Number of ref layers (direct or indirect) ≤ 4
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 150 / 235
![Page 155: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/155.jpg)
SHVC Coding Performance
Results of subjective evaluation from SHVC verification testJCTVC-W1004 [23]: Comparison of SHVC vs. HEVC simulcast
Figure from JCTVC-W1004 [23]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 151 / 235
![Page 156: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/156.jpg)
Outline
13 Range Extensions
14 Annex F: Common specifications for multi-layer extensions
15 Annex G: Multiview High Efficiency Video Coding
16 Annex H: Scalable High Efficiency Video Coding
17 Annex I: 3D High Efficiency Video Coding3D Video Coding Concept3D Coding Tools3D SEI Messages3D Profiles3D-HEVC Coding Performance
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 152 / 235
![Page 157: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/157.jpg)
3D Video Coding
Multiple views with depth information
Video→ “texture view”
Depth information: represent object distance from camera
Coding of texture and depth
Ability to render additional virtual viewsExploit depth information for improved compression efficiency
Auxiliary picture carrying depthDecoding process changed for lid > 0 (not compatible to HEVC Version 1)Dedicated depth coding toolsTools exploiting the relation between texture and depth
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 153 / 235
![Page 158: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/158.jpg)
3D Coding Scenario with Depth
0s
z
s∆
P1
P2
f
u d1 d2
xd
zP1
zmax
zmin
d1
xd=
fzP1
;d2
s∆− xd=
fzP1
intercept theorem
d = d1 + d2 =f · xd
zP1
+f · (s∆− xd)
zP1
⇒ d =f · s∆zP1
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 154 / 235
![Page 159: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/159.jpg)
3D Coding Scenario with Depth
s∆1 s∆2
0s
z
P1
P2
zP1
zmax
zmin
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 154 / 235
![Page 160: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/160.jpg)
Depth Coding
World coordinate distance zP1 between camera system and object
Range between znear and zfar
Representation by depth value vd
Reciprocal relation between depth and distanceRepresentation by vd = 0 . . .vmax, e. g. vmax = 255 for 8 bit representationdepth array with same resolution as associated picture→ depth value for each pixel
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 155 / 235
![Page 161: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/161.jpg)
Depth Representation
0s
z
s∆
P1
f
u d1 d2
xd
zP1
zmax
zmin vd = 255
vd = 0
vd
Depth value vd:
zP1 =1
vd255 ·
(1
znear− 1
zfar
)+ 1
zfar
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 156 / 235
![Page 162: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/162.jpg)
3D Video Coding Concept
Extension of multiview approach by depth informationEnabling view synthesis at the receiver side
Generation of additional views as needed by the (N-view) displayAdaptation of the depth impression(e. g. ’correct’ stereo on large/small screens)
Figure by Fabian Jäger, RWTH internal report
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 157 / 235
![Page 163: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/163.jpg)
3D Video Coding Concept
Inter-component / -view dependencies→ improved predictionDisparity compensated predictionInter-view prediction of motionInter-view prediction of residualView synthesis prediction
Figure by Fabian Jäger, RWTH internal report
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 157 / 235
![Page 164: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/164.jpg)
Coding of Depth Information
Characteristics of depth map differ from texture
Homogeneous areas
Depth edges at (or close to) object boundaries
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 158 / 235
![Page 165: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/165.jpg)
Coding of Depth Information
Characteristics of depth map differ from texture
Homogeneous areas
Depth edges at (or close to) object boundaries
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 158 / 235
![Page 166: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/166.jpg)
Coding of Dependent Views
Coding tools
Disparity compensated prediction
Inter-view motion prediction
Advanced residual prediction
Illumination compensation
View synthesis prediction
Depth-based block partitioning
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 159 / 235
![Page 167: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/167.jpg)
Virtual Depth Estimate
Depth of reference view is availableDetermine ’virtual depth’ for current view
PU grid for depth view8×8 grid for texture viewMaximum depth value to derive disparity estimate
Use disparity estimate for prediction tools
Figure from JCT3V-K1003 [24]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 160 / 235
![Page 168: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/168.jpg)
Advanced Residual Prediction (ARP)
Determination of collocated block by estimated depth
Application of weighted residual as additional predictor for residual ofcurrent block
Prediction from temporal reference or reference view
Figure from JCT3V-K1003,JCT3V-C0049 [24, 25]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 161 / 235
![Page 169: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/169.jpg)
Illumination Compensation
Callibration of cameras my vary over views
Compensated for changed by linear model
Figure from JCT3V-K1003 [24]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 162 / 235
![Page 170: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/170.jpg)
View Synthesis Prediction (VSP)
Use estimated depth of reference view
Warp texture of reference view to current view
Figure from JCT3V-K1003 [24]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 163 / 235
![Page 171: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/171.jpg)
Depth-Based Block Partitioning
Two motion partitions
Derive texture segmentation from depth information
Integration based on conventional motion information
Ja13 [26]Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 164 / 235
![Page 172: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/172.jpg)
Coding of Depth Maps
Modifications to basic coding tools
Depth modelling
Segment-wise DC coding
Depth Loop-Up Table
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 165 / 235
![Page 173: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/173.jpg)
Depth Modelling:Wedgelets
Explicit coding of non-rectangular partitioning
Figure from JCT3V-K1003 [24]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 166 / 235
![Page 174: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/174.jpg)
Depth Modelling: Contour Partitions
Shape of partitions estimated from corresponding luma component
Figure from JCT3V-K1003 [24]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 167 / 235
![Page 175: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/175.jpg)
Depth: Segment-Wise DC Coding
DC-like structure often observed in depth componentCoding of Constant Partition Values (CPV)
Single value for conventional HEVC intra prediction modestwo values for segmented blocksPrediction from neighborhoodCoding using depth look-up table
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 168 / 235
![Page 176: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/176.jpg)
Depth Intra Coding
Horizontal or vertical prediction only
Prediction value from center depth pixel at boundary
No residual coded
Figure from JCT3V-K1003 [24]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 169 / 235
![Page 177: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/177.jpg)
Annex I SEI Messages
SEI messages of Annexes D, F, and G applyAdditional SEI message: Alternative depth information
Global View and Depth (GVD)Warp map
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 170 / 235
![Page 178: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/178.jpg)
3D Main Profile
3D Main Profile: general profile idc = 8
3D Profile based on Main profile
Aux picture to carry depth information
Scalability ID: only view ID or depth layer allowed
Inter view MV vertical constraint flag active→ vertical MV component restricted
All view pictures identical size; no ref local offset allowed(all views aligned)
Number of ref layers (direct or indirect) ≤ 4
Number of pictures in DPB ≤ 8
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 171 / 235
![Page 179: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/179.jpg)
3D-HEVC Coding Performance
Results of subjective evaluation from 3D-HEVC verification testJCT3V-M1001 [27]: Comparison of
MV-HEVC: anchor codec for multiview and depth without block-levelchanges to decoding process
3D-HEVC: enhanced compression of multiview and depth withmodifications to block-level decoding process for dependent texture views
⇒ Gain of approx. 20 % for 3D-HEVC compared to MV-HEVC
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 172 / 235
![Page 180: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/180.jpg)
3D-HEVC Coding Performance
Figure from JCT3V-M1001 [27]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 173 / 235
![Page 181: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/181.jpg)
3D-HEVC Coding Performance
Figure from JCT3V-M1001 [27]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 174 / 235
![Page 182: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/182.jpg)
Part V
Recent Extensions and Developments
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 175 / 235
![Page 183: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/183.jpg)
Outline
18 Screen Content CodingApplication ScenarioScreen Content Coding ToolsSCC ProfilesSCC Performance
19 Free Viewpoint Television
20 Wide Color Gamut / High Dynamic Range Coding
21 Future Video Coding
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 176 / 235
![Page 184: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/184.jpg)
Screen Content Coding (SCC)
Characteristics of screen content sequences differfrom camera captured video
Applications
Gaming
Remote desktops
. . .
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 177 / 235
![Page 185: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/185.jpg)
Screen Content Coding Tools
Additional set of tools on top of RExt profile
Intra block copy
Palette mode coding
Adaptive cross-component transformation
Adaptive motion vector resolution
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 178 / 235
![Page 186: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/186.jpg)
Intra Block Copy
Displaced prediction within the same picture
Displaced prediction of current PB/CB from available parts of picture
Harmonized with inter prediction under study, JCTVC-T1000 [28]
Figure from JCTVC-S1014 [29]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 179 / 235
![Page 187: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/187.jpg)
Palette mode
Pixel value prediction instead of ’regular’ prediction and transform
Palette predictor, size configurable in SPS
Horizontal or vertical traversal scan of block
Differential coding of new palette entries
Figure from JCTVC-S1014 [29]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 180 / 235
![Page 188: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/188.jpg)
Adaptive cross-component transformation
SCC content in 4:4:4 color format – redundancy among components
Selectable cross-component transform for decorrelation
Figure from JCTVC-S1014 [29]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 181 / 235
![Page 189: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/189.jpg)
Adaptive motion vector resolution
Motion in screen content sequences
Integer pixel displacements not unlikely (e. g. when moving windows on adesktop)
Quarter-sample motion vectors not needed in such casesSolution
Analysis of the observed motion in the coded picture at encoderIndication of quarter-sample or integer-sample motion vector precision
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 182 / 235
![Page 190: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/190.jpg)
SCC Development
Amendment for Screen Content Coding finalized, publication to follow
Tool specification integrated into main body of HEVC specification text⇒ Closer integration, usage with other extensions easier, like RExt
Seven dedicated profiles specified
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 183 / 235
![Page 191: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/191.jpg)
SCC Profiles
Screen Content Coding Profiles: general profile idc = 9
Screen-Extended Main and Screen-Extended Main 10 profiles
Screen-Extended Main 4:4:4 and Screen-Extended Main 4:4:4 10 profiles
Screen-Extended High Throughput 4:4:4, Screen-Extended HighThroughput 4:4:4 10, and Screen-Extended High Throughput 14 profiles
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 184 / 235
![Page 192: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/192.jpg)
SCC Performance
Comparison of HM-16.4 vs. HM-16.4+SCM-4.0 for SCC CTC sequences,lossy coding
Figure from JCTVC-U0051 [30]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 185 / 235
![Page 193: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/193.jpg)
SCC Performance
Comparison of HM-16.4 vs. HM-16.4+SCM-4.0 for SCC CTC sequences,lossy coding
Figure from JCTVC-U0051 [30]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 185 / 235
![Page 194: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/194.jpg)
SCC Performance
Comparison of HM-16.4 vs. HM-16.4+SCM-4.0 for SCC CTC sequences,lossy coding
Figure from JCTVC-U0051 [30]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 185 / 235
![Page 195: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/195.jpg)
Outline
18 Screen Content Coding
19 Free Viewpoint Television
20 Wide Color Gamut / High Dynamic Range Coding
21 Future Video Coding
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 186 / 235
![Page 196: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/196.jpg)
Free Viewpoint TelevisionExtension to 3D video coding
Restriction to co-planar camera arrangement relaxed
New application scenarios
Super-Multiview
Figure from N16128 [31]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 187 / 235
![Page 197: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/197.jpg)
Free Viewpoint TelevisionExtension to 3D video coding
Restriction to co-planar camera arrangement relaxed
New application scenarios
Free Navigation
Figure from N16128 [31]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 187 / 235
![Page 198: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/198.jpg)
Status of FTV Development
MPEG Ad-Hoc Group on FTV
June 2015: Call for Evidence on Free-Viewpoint Television:Super-Multiview and Free Navigation, N15348 [32]
February 2016: Results of the Call for Evidence on Free-ViewpointTelevision: Super-Multiview and Free Navigation, N16128 [31]
Outcome
1 proposal for Super-Multiview, 3 proposals for Free Navigation
Evaluation of results not yet conclusive
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 188 / 235
![Page 199: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/199.jpg)
Outline
18 Screen Content Coding
19 Free Viewpoint Television
20 Wide Color Gamut / High Dynamic Range Coding
21 Future Video Coding
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 189 / 235
![Page 200: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/200.jpg)
Wide Color Gamut / High Dynamic Range Coding
Colour spaceStandard Dynamic Range (SDR) video
Contrast approx. 1000 : 0ITU-R BT.709 colour space, [33]
High Dynamic Range (HDR) videoContrast approx. 1000000 : 0ITU-R BT.2020 colour space, [34]
Figure from N15084 [35]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 190 / 235
![Page 201: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/201.jpg)
Wide Color Gamut / High Dynamic Range Coding
Dynamic Range
Figure from N15084 [35]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 191 / 235
![Page 202: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/202.jpg)
Standardization Activities on HDR/WCG
ISO/IEC JTC 1/SC 29/WG 11 (MPEG)
02/2015: Requirements and Use Cases for HDR and WCG ContentCoding N15083 [36]
02/2015: Call for Evidence for HDR and WCG Video Coding N15084 [35]
06/2015: Test Results of Call for Evidence (CfE) for HDR and WCG VideoCoding N15350 [37]
10/2015: Assignment of HDR/WCG as work item to JCT-VC
JCT-VC
02/2016: Decision: no new profile, no new coding tools
02/2016: Technical report: Conversion and coding practices forHDR/WCG video (ISO/IEC TR 23008-14), N16063 [38]
06/2016: Conversion and Coding Practices for HDR/WCG Y’CbCr 4:2:0Video with PQ Transfer Characteristics, JCTVC-X1017 [39]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 192 / 235
![Page 203: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/203.jpg)
HDR/WCG Conversion Practices: Scope
Figure from JCTVC-X1017 [39]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 193 / 235
![Page 204: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/204.jpg)
Outline
18 Screen Content Coding
19 Free Viewpoint Television
20 Wide Color Gamut / High Dynamic Range Coding
21 Future Video Coding
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 194 / 235
![Page 205: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/205.jpg)
Future Video Coding Development
Standardization Requirements, e.g. MPEG N16359 [40]
50 % bitrate saving (30 % might suffice on important usecases)
Tentative target timeline: new standard by 2020
Variety of content (captured, computer generated, mixed, 3D, variousprojections (→ 360VR), multispectral, . . . )
Licensing model(s)?
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 195 / 235
![Page 206: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/206.jpg)
Joint Video Exploration Team (JVET)
Exploration activitiesFuture Video Coding AhG in MPEG
Exploration and requirements discussions, workshops
Coding Efficiency Improvements AhG in VCEGKey Technical Areas software, VCEG-AZ01 [41]
Foundation of JVET of MPEG and VCEG, similar to JCT-VC, Oct. 2015
Publicly available document sitehttp://phenix.int-evry.fr/jvet/
Publicly available software repository https://jvet.hhi.fraunhofer.de/svn/svn_HMJEMSoftware/
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 196 / 235
![Page 207: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/207.jpg)
JVET Status
Joint Exploration Model (JEM)JVET-A1001,JVET-B1001,JVET-C1001 [42, 43, 44]
Definition of Common Testing Conditions, JVET-B1010 [45]
Definition of Ad-Hoc Groups, JVET-B1000 [46]
Definition of Exploration Experiments, JVET-B1011 [47]
Collection of new test materialJVET-A1002,JVET-B1002,JVET-C1002 [48, 49, 50]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 197 / 235
![Page 208: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/208.jpg)
JEM: New Coding Tools – Block Partitioning
QTBT - Quadtree plus binary tree partitioning
Replacement of HEVC CU/PU/TU structure→ Adaptive block size transforms, Wi03 [51]
Maximum CTU size 256×256
Figure from JVET-C1001 [44]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 198 / 235
![Page 209: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/209.jpg)
JEM: New Coding Tools – Intra Coding
67 intra prediction modes, 65 directional modesIntra prediction mode coding: 6 MPM candidates4-tap interpolation filters for directional predictorsEnhanced boundary prediction, . . .
Figure from JVET-C1001 [44]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 199 / 235
![Page 210: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/210.jpg)
JEM: New Coding Tools – Inter Coding
Sub-PU level motion vector prediction→ Advanced temporal motion vector prediction (ATMVP)
Adaptive motion vector resolution (AMVR)
Overlapped block motion compensation (OBMC) at left and top blockboundary possible
Affine motion predition (MVs for 4×4 sub-blocks)
Local illumination compensation (LIC), Pattern-matched motion vectorderivation, bi-directional optical flow (BIO, pixel dense)
Figure from JVET-C1001 [44]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 200 / 235
![Page 211: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/211.jpg)
JEM: New Coding Tools – Transform Coding
More core transforms: additionalDST-VII, DCT-VIII, DST-I and DCT-V
Mode dependent non-separablesecondary transforms
Signal dependent transform (SDT)
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 201 / 235
![Page 212: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/212.jpg)
JEM: New Coding Tools – Loop Filters
Re-introduction of Adaptive Loop Filters (ALF)
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 202 / 235
![Page 213: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/213.jpg)
JEM: New Coding Tools – Entropy Coding
Modified context modeling for transform coefficients
Multi-hypothesis probability estimation with context-dependent updatingspeed
Adaptive initialization for context models
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 203 / 235
![Page 214: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/214.jpg)
JEM: Performance
Figure from JVET-C0001 [52]
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 204 / 235
![Page 215: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/215.jpg)
Part VI
Summary and Outlook
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 205 / 235
![Page 216: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/216.jpg)
Outline
22 Summary and OutlookSummaryVideo Coding Development: Outlook
23 Books and Tools
24 Resources
25 Acronyms
26 References
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 206 / 235
![Page 217: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/217.jpg)
Summary
Coding tools in HEVC version 1
Range extensions
Multilayer extensions- Multiview- Scalability- 3D
Screen Content coding
Free Viewpoint Television
High Dynamic Range and Wide Color Gamut
Future Video Coding - Status of JVET
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 207 / 235
![Page 218: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/218.jpg)
Video Coding Development: Outlook
Towards Immersive Media360 Virtual Reality
JVET AhG8: 360 video test conditions, JVET-C1000d [53]MPEG FTV AhG
LightfieldsMPEG AhG on lightfield compressionJoint AhG (JPEG and MPEG) for digital representations of light/soundfields for immersive media applications, WG1N72033 [54]
Left: 1st frame of BearAttacks sequence, Nokia. Right:https://s3.amazonaws.com/lytro-corp-assets/blog/camera_array.png
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 208 / 235
![Page 219: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/219.jpg)
Outline
22 Summary and Outlook
23 Books and Tools
24 Resources
25 Acronyms
26 References
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 209 / 235
![Page 220: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/220.jpg)
Further Reading
Books published
Wien, Mathias. High Efficiency Video Coding – Coding Tools andSpecification, Springer Berlin Heidelberg, 2015.
Sze, Vivienne, Budagavi, Madhukar, Sullivan, Gary J. (Eds.). HighEfficiency Video Coding (HEVC) – Algorithms and Architectures, SpringerInternational Publishing, 2014.
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 210 / 235
![Page 221: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/221.jpg)
YUView Player
YUV player supporting display of statistics informationhttps://github.com/IENT/YUView
Play back of raw YUV files, playlists
Comparison / side-by-side views of two files
Visualization of statistics in xml-format
HEVC bitstream decoding and mode visualization
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 211 / 235
![Page 222: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/222.jpg)
Outline
22 Summary and Outlook
23 Books and Tools
24 Resources
25 Acronyms
26 References
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 212 / 235
![Page 223: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/223.jpg)
HEVC Resources (1/2)
JCT-VC Mailing List
http://mailman.rwth-aachen.de/mailman/listinfo/jct-vc
JCT-VC Document Repository
http://phenix.it-sudparis.eu/jct/index.php
http://ftp3.itu.int/av-arch/jctvc-site/
JCT-VC HM Software Repository
SVN:https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware
Trac:https://hevc.hhi.fraunhofer.de/trac/hevc
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 213 / 235
![Page 224: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/224.jpg)
HEVC Resources (2/2)
JCT-3V Mailing List
http://mailman.rwth-aachen.de/mailman/listinfo/jct-3v
JCT-3V Document Repository
http://phenix.int-evry.fr/jct-3v/
http://wftp3.itu.int/av-arch/jct3v-site/
JCT-3V HM Software Repository
SVN:https://hevc.hhi.fraunhofer.de/svn/svn_3DVCSoftware/
Trac:https://hevc.hhi.fraunhofer.de/trac/3d-hevc
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 214 / 235
![Page 225: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/225.jpg)
JVET Resources
JVET Mailing List
http://mailman.rwth-aachen.de/mailman/listinfo/jvet
JVET Document Repository
http://phenix.int-evry.fr/jvet/
http://wftp3.itu.int/av-arch/jvet-site/
JVET JEM Software Repository
SVN:https://jvet.hhi.fraunhofer.de/svn/svn_HMJEMSoftware/
Trac:https://hevc.hhi.fraunhofer.de/trac/jem/
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 215 / 235
![Page 226: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/226.jpg)
Outline
22 Summary and Outlook
23 Books and Tools
24 Resources
25 Acronyms
26 References
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 216 / 235
![Page 227: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/227.jpg)
Acronyms3D-HEVC – 3D high efficiency video coding
AAC – Advanced audio codingAAP – Alternative Approval ProcessAHG – Ad-hoc group
AI – All intra (JCT-VC CTC)AIF∗ – Adaptive interpolation filterALF∗ – Adaptive loop filterAMD – Amendment (for ISO standard)AMP – Asymmetric motion partitioning
AMVP – Advanced motion vector predictionANG∗ – Angular intra predictionASO – Arbitrary slice ordering (H.264 |AVC)
AP – Aggregation packetAU – Access unit
AUD – Access unit delimiterAVC – Advanced video coding
BDSNR – Bjøntegaard Delta PSNRBD-rate – Bjøntegaard Delta rate
BL – Base layerBLA – Broken link accessBO – SAO band offset
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 217 / 235
![Page 228: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/228.jpg)
Acronyms IIBoG – Break-out group
BP – Bandpass [filter]BPB – Bitstream partition buffer
CABAC – Context adaptive binary arithmetic codingCAVLC – Context adaptive variable length coding
CB – Coding blockCBF∗ – Coded block flagCBP – Coded block pattern (H.264 |AVC)CBR – Constant bitrate
CD – Committee draftCD – Compact diskCE – Core experimentCfE – Call for evidenceCfP – Call for proposalsCIE – Commission internationale de l’éclairageCIF – Common interchange format 352×288
CPB – Coded picture bufferCRA – Clean random access
CRFB∗ – Compressed reference frame bufferCRT – Cathode ray tube
CS – Constraint set (in CfP)Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 218 / 235
![Page 229: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/229.jpg)
Acronyms IIICSS – Coded slice segment
CT – Collaborative teamCTB – Coding tree blockCTC – Common testing conditionsCTU – Coding tree unitCTX – CABAC context
CU – Coding unitCVS – Coded Video Sequence
CVSG – Coded Video Sequence GroupDAM – Draft amendment
DC – Direct currentDCT – Discrete cosine transformDST – Discrete sine transformDON – Decoding order numberDPB – Decoded picture buffer
DU – Decoding unitDCT – Discrete cosine transform
DCTIF – DCT interpolation filterDIF∗ – DCT interpolation filter, syn. DCTIFDIF∗ – Directional interpolation filterDIS – Draft international standard
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 219 / 235
![Page 230: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/230.jpg)
Acronyms IVDLP∗ – Decodable leading picture, syn. RADL
DMVD – Decoder side motion vector derivationDPB – Decoded picture buffer
DPCM – Differential Pulse Code ModulationDST – Discrete sine transform
DU – Decoding unitDUT∗ – Directional unified transformDVD – Digital Versatile Disk
EG – Exp-Golomb codeEL – Enhancement layerEO – SAO edge offset
EOB – End of bitstreamEOS – End of sequence
EOTF – Electro-optical transfer functionFCD – Final committee draft
FD – Filler dataFDAM – Final draft amendmentFDIS – Final draft international standardFLC – Fixed length code
FMO – Flexible macroblock ordering (H.264 |AVC)fps – Frames per second
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 220 / 235
![Page 231: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/231.jpg)
Acronyms VFRExt – Fidelity range extensions
FU – Fragmentation unitGBR – Green red blue, color format (see also RGB)GOP – Group of picturesGRD – Gradual Decoder Refresh (H.264 |AVC)
HBPS – Hypothetical bitstream partition schedulerHRD – Hypothetical reference decoder
HD – High definitionHDR – High dynamic range
HDTV – High definition televisionHEVC – High efficiency video coding
HM – HEVC test modelHP – Highpass [filter]
HRD – Hypothetical reference decoderHSS – Hypothetical stream schedulerHTM – 3D-HEVC test model
HVC∗ – High performance video coding (name of HEVC pre-project in MPEG)HVS – Human visual system
IBDI∗ – Internal bit-depth increaseIDCT – Inverse discrete cosine transform
IDR – Instantaneous decoder refreshMathias Wien (RWTH) HEVC V3 and Beyond ICME2016 221 / 235
![Page 232: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/232.jpg)
Acronyms VIIEC – International Electrotechnical Commission
IEEE – Institute of electrical and electronics engineersIRAP – Intra random access point, see also RAP
IS – International standardISDN – Integrated services digital network
ISO – International Standardization OrganizationITU – International telecommunication unionJCT – Joint collaborative team (of ISO and ITU)
JCT-VC – Joint collaborative team on video codingJCT-3V – Joint collaborative team on 3D video coding extension development
JEM – Joint exploration model (JVET test model)JM – Joint model (AVC test model)
JPEG – Joint photographic experts groupJTC – Joint technical committeeJVT – Joint video team
JVET – Joint video exploration teamKLT – Karhunen-Loève transformKTA – Key Technical Areas (H.264 based exploration software of VCEG)
LCTB∗ – Largest coded tree block, syn. CTBLCTB∗ – Largest coded tree unit, syn. CTU
LCU∗ – Larges coding unit, syn. CTUMathias Wien (RWTH) HEVC V3 and Beyond ICME2016 222 / 235
![Page 233: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/233.jpg)
Acronyms VIILD – Low delay (JCT-VC CTC)LP – Lowpass [filter]
LPS – Least probably symbolLSB – Least Significant Bit
MAC – Multiplexed analog componentsMBAFF – Macroblock adaptive frame/field coding (H.264 |AVC)
MC – Motion compensationMDDT∗ – Mode dependent directional transform
ME – Motion estimationMerge – Merge Mode (MV prediction)
MMCO – Memory management control operation (H.264 |AVC)MP3 – MPEG-2 audio layer III
MPEG – Moving picture experts groupMPM – Most probable modeMPS – Most probably symbol
MRST – Multiple RTP streams over a single media transportMRMT – Multiple RTP streams over multiple media transports
MSB – Most Significant BitMSE – Mean squared error
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 223 / 235
![Page 234: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/234.jpg)
Acronyms VIIIMV – Motion vector
MVC – Multiview video coding (H.264 |AVC)MVD – Motion vector difference
MV-HEVC – Multiview high efficiency video codingNAL – Network abstraction layer
MANE – Media aware network elementMB – Macroblock (H.264 |AVC)
NALU – NAL unitNB – National body (in ISO)
NGVC∗ – Next generation video coding (name of HEVC pre-project in VCEG)NTSC – National television systems committeeNUH – NAL unit headerNUT – NAL unit typePACI – Payload Content Information (packet)OLS – Output layer set
PAFF – Picture adaptive frame/field coding (H.264 |AVC)PAL – Phase alternating linePB – Prediction block
PCM – Pulse code modulationPDAM – Proposed draft amendment
POC – Picture order countMathias Wien (RWTH) HEVC V3 and Beyond ICME2016 224 / 235
![Page 235: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/235.jpg)
Acronyms IXPPS – Picture parameter set
PSNR – Peak signal to noise ratioPU – Prediction unit
QCIF – Quarter common intermediate format 176×144QHD – Quarter High Definition 960×540
QP – Quantization parameterRA – Random access (JCT-VC CTC)
RADL – Random access decodable leading pictureRAP – Random access point
RASL – Random access skipped leading pictureRBSP – Raw byte sequence payload
RD – Rate-distortionRDO – Rate-distortion optimization
RDOQ – Rate-distortion optimized quantizationRExt – HEVC Range extensionsRGB – Red green blue, color formatRPL – Reference picture listRPS – Reference picture setRQT – Residual quad-treeRTP – Real-time transport protocolSAD – Sum of absolute differences
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 225 / 235
![Page 236: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/236.jpg)
Acronyms XSAO – Sample adaptive offsetSAR – Sample aspect ratio
SATD – Sum of absolute transformed differencesSC – Sub-committee
SCC – Screen Content CodingSD – Standard definition (TV)
SDH – Sign data hidingSDP – Session description protocol
SECAM – Séquentiel couleur à mémoireSEI – Supplemental enhancement informationSG – Study group
SODB – String of data bitsSHVC – Scalable high efficiency video codingSODB – String of data bits
SOP – Structure of picturesSPS – Sequence parameter set
SRST – Single RTP stream over a single media transportSSD – Sum of squared differencesSSE – Sum of squared error
STSA – Stepwise temporal sub-layer accessSVC – Scalable video coding (H.264 |AVC)
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 226 / 235
![Page 237: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/237.jpg)
Acronyms XITB – Transform blockTE – Tool experiment
TFD∗ – Tagged for discard [picture], syn. RASLTSA – Temporal sub-layer access
TMuC – Test model under considerationTMVP – Temporal motion vector predictor
TR – Truncated Rice [binarization]TSA – Temporal sub-layer accessTSB – Transform sub-block
TSCI – Temporal scalability control informationTU – Transform unitTV – Television
UHD – Ultra High DefinitionVBR – Variable bitrate
VCEG – Visual coding experts groupVCL – Video coding layerVGA – Video Graphics Array 640×480VLC – Variable length codeVPS – Video parameter setVUI – Video usability information
WCG – Wide colour gamutWD – Working draftWG – Working group
WPP – Wavefront parallel processing
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 227 / 235
![Page 238: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/238.jpg)
Acronyms XIIXGA – Extended Graphics Array 1024×768XYZ – XYZ color space, also color format
YCbCr – Color format with luma and two chroma componentsYUV – XYZ color format
(Acronyms marked with ∗ are deprecated)
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 228 / 235
![Page 239: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/239.jpg)
Outline
22 Summary and Outlook
23 Books and Tools
24 Resources
25 Acronyms
26 References
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 229 / 235
![Page 240: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/240.jpg)
References I
[1] Charles Poynton. Digital Video and HD: Algorithms and Interfaces. Waltham, MA, USA: Morgan Kaufman Publishers, 2012.
[2] High efficiency video coding. ITU-T, Apr. 2013. URL: http://www.itu.int/rec/T-REC-H.265/en.
[3] Gary J. Sullivan et al. “Overview of the High Efficiency Video Coding (HEVC) Standard”. In: IEEE Transactions on Circuits andSystems for Video Technology 22.12 (Dec. 2012), pp. 1649–1668. DOI: 10.1109/TCSVT.2012.2221191.
[4] Rickard Sjöberg et al. “Overview of HEVC high-level syntax and reference picture management”. In: IEEE Transactions on Circuits andSystems for Video Technology 22.12 (Dec. 2012), pp. 1858–1870. DOI: 10.1109/TCSVT.2012.2223052.
[5] Advanced video coding for generic audiovisual services. ITU-T, Jan. 2012. URL:http://www.itu.int/rec/T-REC-H.264/en.
[6] Stephan Wenger. “H.264/AVC over IP”. In: IEEE Transactions on Circuits and Systems for Video Technology 13.7 (July 2003),pp. 645–656. DOI: 10.1109/TCSVT.2003.814966.
[7] Thomas Wiegand et al. “Overview of the H.264/AVC Video Coding Standard”. In: IEEE Transactions on Circuits and Systems for VideoTechnology 13.7 (July 2003), pp. 560–576.
[8] Frank Bossen. Common test conditions and software reference configurations. Doc. JCTVC-K1100. Shanghai, CN, 11th meeting: JointCollaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Oct. 2012.
[9] Thomas Schierl et al. “System Layer Integration of HEVC”. In: IEEE Transactions on Circuits and Systems for Video Technology 22.12(Dec. 2012), pp. 1871–1884. DOI: 10.1109/TCSVT.2012.2223054.
[10] Jordi Ribas-Corbera, Philip A. Chou, and Shankar Regunathan. “A Generalized Hypothetical Reference Decoder for H.264/AVC”. In:IEEE Transactions on Circuits and Systems for Video Technology 13.7 (July 2003), pp. 674–687. DOI:10.1109/TCSVT.2003.814965.
[11] Jani Lainema et al. “Intra Coding of the HEVC Standard”. In: IEEE Transactions on Circuits and Systems for Video Technology 22.12(Dec. 2012), pp. 1792–1801. DOI: 10.1109/TCSVT.2012.2221525.
[12] Philipp Helle et al. “Block Merging for Quadtree-based Partitioning in HEVC”. In: IEEE Transactions on Circuits and Systems for VideoTechnology 22.12 (Dec. 2012), pp. 1720–1731. DOI: 10.1109/TCSVT.2012.2223051.
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 230 / 235
![Page 241: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/241.jpg)
References II
[13] Madhukar Budagavi et al. “Core Transform Design in the High Efficiency Video Coding (HEVC) Standard”. In: IEEE Journal of SelectedTopics in Signal Processing tbd (2013), tbd. DOI: 10.1109/JSTSP.2013.2270429.
[14] Andrey Norkin et al. “HEVC Deblocking Filter”. In: IEEE Transactions on Circuits and Systems for Video Technology 22.12 (Dec. 2012),pp. 1746–1754. DOI: 10.1109/TCSVT.2012.2223053.
[15] Chih-Ming Fu et al. “Sample Adaptive Offset in the HEVC Standard”. In: IEEE Transactions on Circuits and Systems for VideoTechnology 22.12 (Dec. 2012), pp. 1755–1764. DOI: 10.1109/TCSVT.2012.2221529.
[16] Detlev Marpe, Heiko Schwarz, and Thomas Wiegand. “Context-Based Adaptive Binary Arithmetic Coding in the H.264/AVC VideoCompression Standard”. In: IEEE Transactions on Circuits and Systems for Video Technology 13.7 (July 2003), pp. 620–637. DOI:10.1109/TCSVT.2003.815173.
[17] Vivienne Sze and Madhukar Budagavi. “High Throughput CABAC Entropy Coding in HEVC”. In: IEEE Transactions on Circuits andSystems for Video Technology 22.12 (Dec. 2012), pp. 1778–1791. DOI: 10.1109/TCSVT.2012.2221526.
[18] Wei Pu et al. RCE1: Descriptions and Results for Experiments 1, 2, 3, and 4. Doc. JCTVC-O0202. Geneva, CH, 15th meeting: JointCollaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Oct. 2013.
[19] David Flynn et al. “Overview of the Range Extensions for the HEVC Standard: Tools, Profiles, and Performance”. In: IEEE Transactionson Circuits and Systems for Video Technology 26.1 (Jan. 2016), pp. 4–19. DOI: 10.1109/TCSVT.2015.2478707.
[20] Jill Boyce et al. “Overview of SHVC: Scalable Extensions of the High Efficiency Video Coding (HEVC) Standard”. In: IEEE Transactionson Circuits and Systems for Video Technology PP.99 (2015). DOI: 10.1109/TCSVT.2015.2461951.
[21] Vittrorio Baroncini, Karsten Mueller, and Shinya Shimizu. MV-HEVC Verification Test Report. Tech. rep. JCT3V-N1001. San Diego, CA,USA, 14th meeting, Feb. 2016. URL: http://phenix.int-evry.fr/jct-3v/doc_end_user/documents/14_San%20Diego/wg11/JCT3V-N1001-v2.zip.
[22] Heiko Schwarz, Detlev Marpe, and Thomas Wiegand. “Overview of the Scalable Video Coding Extension of the H.264/AVC Standard”.In: IEEE Transactions on Circuits and Systems for Video Technology 17.9 (Sept. 2007), pp. 1103–1120. DOI:10.1109/TCSVT.2007.905532.
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 231 / 235
![Page 242: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/242.jpg)
References III[23] Yan Ye, Vittrorio Baroncini, and Ye-Kui Wang. SHVC verification test report. Doc. JCTVC-W1004. San Diego, CA, USA, 23rd meeting:
Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Feb. 2016. URL: http://phenix.it-sudparis.eu/jct/doc_end_user/documents/23_San%20Diego/wg11/JCTVC-W1004-v1.zip(visited on 07/03/2017).
[24] Ying Chen et al. Test Model 11 of 3D-HEVC and MV-HEC. Tech. rep. JCT3V-K1003. Geneva, CH, 11th meeting, Feb. 2015.
[25] Li Zhang et al. 3D-CE4: Advanced residual prediction for multiview coding. Tech. rep. JCT3V-C0049. Geneva, CH, 3rd meeting, Jan.2013. URL:http://phenix.int-evry.fr/jct-3v/doc_end_user/documents/3_Geneva/wg11/JCT3V-C0049-v2.zip.
[26] Fabian Jäger. “Depth-based Block Partitioning for 3D Video Coding”. In: Proc. of International Picture Coding Symposium PCS ’13.San Jose, USA: IEEE, Piscataway, Dec. 2013.
[27] Vittrorio Baroncini, Karsten Mueller, and Shinya Shimizu. 3D-HEVC Verification Test Report. Tech. rep. JCT3V-M1001. Geneva, CH,13th meeting, Oct. 2015. URL: http://phenix.int-evry.fr/jct-3v/doc_end_user/documents/13_Geneva/wg11/JCT3V-M1001-v2.zip (visited on 07/03/2016).
[28] Gary J. Sullivan and Jens-Rainer Ohm. Meeting report of the 20th meeting of the Joint Collaborative Team on Video Coding (JCT-VC),Geneva, CH, 10–1x Feb. 2015. Doc. JCTVC-T1000. 20th meeting, Geneva, CH: Joint Collaborative Team on Video Coding (JCT-VC)of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Feb. 2015.
[29] Rajan Joshi et al. Screen content coding test model 3 (SCM 3). Doc. JCTVC-S1014. 19th meeting, Strasbourg, F: Joint CollaborativeTeam on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Oct. 2014.
[30] Bin Li, Jizheng Xu, and Gary Sullivan. Comparison of Compression Performance of HEVC Test Model 16.4 and HEVC Screen ContentCoding Extensions Test Model 4 with AVC High 4:4:4 Predictive profile. Doc. JCTVC-U0051. 21st meeting, Warsaw, PL: JointCollaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, June 2015.
[31] Vittorio Baroncini, Masayuki Tanimoto, and Olgierd Stankiewicz. Results of the Call for Evidence on Free-Viewpoint Television:Super-Multiview and Free Navigation. Tech. rep. N16128. San Diego, CA, USA, 114th meeting: ISO/IEC JTC1/SC29/WG11, Feb.2016. URL: http://phenix.it-sudparis.eu/mpeg/doc_end_user/documents/114_San%20Diego/wg11/w16128-v2-w16128.zip.
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 232 / 235
![Page 243: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/243.jpg)
References IV
[32] MPEG Requirements. Call for Evidence on Free-Viewpoint Television: Super-Multiview and Free Navigation. Tech. rep. N15348. 112thmeeting, Warsaw, Poland: ISO/IEC JTC1 SC29WG11 MPEG, June 2015. URL:http://mpeg.chiariglione.org/standards/exploration/free-viewpoint-television-ftv/call-evidence-free-viewpoint-television-super.
[33] BT.709: Parameter values for the HDTV standards for production and international programme exchange. ITU-R, Apr. 2004. URL:http://www.itu.int/rec/R-REC-BT.709/en.
[34] BT.2020: Parameter values for ultra-high definition television systems for production and international programme exchange. ITU-R,Oct. 2015. URL: http://www.itu.int/rec/R-REC-BT.2020/en.
[35] Ajay Luthra, Edouard Francois, and Walt (Eds.) Husak. Requirements and Use Cases for HDR and WCG Content Coding. Doc.N15084. Geneva, CH, 111th meeting: MPEG, Feb. 2015. URL:http://phenix.it-sudparis.eu/mpeg/doc_end_user/documents/111_Geneva/wg11/w15084.zip.
[36] Ajay Luthra, Edouard Francois, and Walt (Eds.) Husak. Call for Evidence (CfE) for HDR and WCG Video Coding. Doc. N15083.Geneva, CH, 111th meeting: MPEG, Feb. 2015. URL:http://phenix.it-sudparis.eu/mpeg/doc_end_user/documents/111_Geneva/wg11/w15083.zip.
[37] Requirements. Test Results of Call for Evidence (CfE) for HDR and WCG Video Coding. Doc. N15350. Warsaw, PL, 112th meeting:ISO/IEC JTC1/SC29/WG11, June 2015. URL:http://phenix.it-sudparis.eu/mpeg/doc_end_user/documents/112_Warsaw/wg11/w15350.zip (visitedon 07/03/2016).
[38] Jonatan Samuelsson. WD of ISO/IEC TR 23008-14 Conversion and coding practices for HDR/WCG video. Tech. rep. San Diego, CA,USA: ISO/IEC JTC1/SC29/WG11, Feb. 2016.
[39] Jonatan Samuelsson et al. Draft 2 of Conversion and Coding Practices for HDR/WCG YCbCr 4:2:0. Doc. JCTVC-X1017. 24th meeting,Geneva, CH: Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, May 2016.URL: http://phenix.it-sudparis.eu/jct/doc_end_user/documents/24_Geneva/wg11/JCTVC-X1017-v1.zip.
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 233 / 235
![Page 244: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/244.jpg)
References V[40] MPEG Requirements. Requirements for a Future Video Coding Standard v4. Tech. rep. N16359. Geneva, CH, 115th meeting: ISO/IEC
JTC1/SC29/WG11, June 2016. URL:http://phenix.it-sudparis.eu/mpeg/doc_end_user/documents/115_Geneva/wg11/w16359.zip.
[41] Marta Karaczewicz and Madhukar Budagavi. Report of AHG1 on Coding Efficiency Improvements. Doc. VCEG-AZ01. Warsaw, PL,52nd meeting: ITU-T SG16/Q6 VCEG, June 2015. URL:http://wftp3.itu.int/av-arch/video-site/1506_War/VCEG-AZ01.zip.
[42] Jianle Chen et al. Algorithm Description of Joint Exploration Test Model 1. Doc. JVET-A1001. Geneva, CH, 1st meeting: Joint VideoExploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Oct. 2015. URL:http://phenix.int-evry.fr/jvet/doc_end_user/documents/1_Geneva/wg11/JVET-A1001-v1.zip.
[43] Jianle Chen et al. Algorithm Description of Joint Exploration Test Model 2. Doc. JVET-B1001. San Diego, CA, USA, 2nd meeting: JointVideo Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Feb. 2016. URL: http://phenix.int-evry.fr/jvet/doc_end_user/documents/2_San%20Diego/wg11/JVET-B1001-v3.zip.
[44] Jianle Chen et al. Algorithm Description of Joint Exploration Test Model 3. Doc. JVET-C1001. Geneva, CH, 3rd meeting: Joint VideoExploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, May 2016. URL:http://phenix.int-evry.fr/jvet/doc_end_user/documents/3_Geneva/wg11/JVET-C1001-v3.zip.
[45] Xiang Li and Karsten Suehring. JVET common test conditions and software reference configurations. Doc. JVET-B1010. San Diego,CA, USA, 2nd meeting: Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Feb. 2016. URL:http://phenix.int-evry.fr/jvet/doc_end_user/documents/2_San%20Diego/wg11/JVET-B1010-v2.zip.
[46] Gary J. Sullivan and Jens-Rainer Ohm. Meeting report of the 2nd meeting of the Joint Video Exploration Team (JVET), San Diego, US,20–26 Feb. 2016. Doc. JVET-B1000. San Diego, CA, USA, 2nd meeting: Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3and ISO/IEC JTC 1/SC 29/WG 11, Feb. 2016. URL: http://phenix.int-evry.fr/jvet/doc_end_user/documents/2_San%20Diego/wg11/JVET-B1000-v1.zip.
[47] Elena Alshina et al. Description of Exploration Experiments on Coding Tools. Doc. JVET-B1011. San Diego, CA, USA, 2nd meeting:Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Feb. 2016. URL: http://phenix.int-evry.fr/jvet/doc_end_user/documents/2_San%20Diego/wg11/JVET-B1011-v2.zip.
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 234 / 235
![Page 245: ICME 2016 - High Efficiency Video Coding - Coding Tools and Specification: HEVC V3 and Coming Developments](https://reader030.vdocuments.us/reader030/viewer/2022021417/587d33561a28ab2a448b57f5/html5/thumbnails/245.jpg)
References VI
[48] Teruhiko Suzuki and Jianle Chen. Work plan for investigation of test sequence encoding. Doc. JVET-A1002. Geneva, CH, 1st meeting:Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Oct. 2015. URL:http://phenix.int-evry.fr/jvet/doc_end_user/documents/1_Geneva/wg11/JVET-A1002-v1.zip.
[49] Andrey Norkin et al. Call for test material for future video coding standardization. Doc. JVET-B1002. San Diego, CA, USA, 2nd meeting:Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Feb. 2016. URL: http://phenix.int-evry.fr/jvet/doc_end_user/documents/2_San%20Diego/wg11/JVET-B1002-v1.zip.
[50] Teruhiko Suzuki. Work plan for assessment of test material. Doc. JVET-C1002. Geneva, CH, 3rd meeting: Joint Video ExplorationTeam (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, May 2016. URL:http://phenix.int-evry.fr/jvet/doc_end_user/documents/3_Geneva/wg11/JVET-C1002-v2.zip.
[51] Mathias Wien. “Variable Block-Size Transforms for H.264/AVC”. In: IEEE Transactions on Circuits and Systems for Video Technology13.7 (July 2003), pp. 604–613. DOI: 10.1109/TCSVT.2003.815380.
[52] Marta Karaczewicz and Elena Alshina. JVET AHG report: Tool evaluation (AHG1). Doc. JVET-C0001. Geneva, CH, 3rd meeting: JointVideo Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, June 1, 2016. URL:http://phenix.int-evry.fr/jvet/doc_end_user/documents/3_Geneva/wg11/JVET-C0001-v1.zip.
[53] Gary J. Sullivan and Jens-Rainer Ohm. Meeting report of the 3rd meeting of the Joint Video Exploration Team (JVET), Geneva, CH, 26May – 1 June 2016. Doc. JVET-C1000. 3rd meeting, Geneva, CH: Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 andISO/IEC JTC 1/SC 29/WG 11, June 1, 2016. URL:http://wftp3.itu.int/av-arch/jvet-site/2016_05_C_Geneva/JVET-C_Notes_d7.doc.
[54] JAhG of MPEG and JPEG. Technical report of the joint ad hoc group for digital representations of light/sound fields for immersive mediaapplications. Doc. WG1N72033. Geneva, CH: ISO/IEC JTC1/SC29/WG1(JPEG) & WG11(MPEG), June 2016. URL: https://jpeg.org/downloads/jpegpleno/wg1n72033_20160603_report_jahg_light_sound_fields.pdf.
Mathias Wien (RWTH) HEVC V3 and Beyond ICME2016 235 / 235