Download - Rethinking Color for Next-Generation TV
![Page 1: Rethinking Color for Next-Generation TV](https://reader036.vdocuments.us/reader036/viewer/2022062323/5681594e550346895dc68bfa/html5/thumbnails/1.jpg)
Rethinking Color for Next-Generation TV
Craig Todd, Dolby [email protected] Sugawara, NHK, [email protected]
Seo Young Choi, Samsung, [email protected]
![Page 2: Rethinking Color for Next-Generation TV](https://reader036.vdocuments.us/reader036/viewer/2022062323/5681594e550346895dc68bfa/html5/thumbnails/2.jpg)
Parameters in draft UHDTV signal format
Parameter Value
Aspect Ratio 16x9
Pixel Raster 3840x2160 (4k), 7680x4320 (8k)
Pixel aspect ratio 1:1 (square pixels)
Scan Progressive
Frame Rate 120, 60, 60/1.001, 50, 30, 30/1.001, 25, 24, 24/1.001
Precision 10 bits 12 bits
Code for Reference Black 64 256
Code for Reference White 940 3760
Value for Reference Black (cd/m2) unspecified
Value for Reference White (cd/m2) unspecified
Non-linear coding Linear+Gamma 2.2 (same as Rec.709)
Annex 4 to Document ITU-R WP6C/564-E, 17 October 2011
![Page 3: Rethinking Color for Next-Generation TV](https://reader036.vdocuments.us/reader036/viewer/2022062323/5681594e550346895dc68bfa/html5/thumbnails/3.jpg)
PerceptionLimit
![Page 4: Rethinking Color for Next-Generation TV](https://reader036.vdocuments.us/reader036/viewer/2022062323/5681594e550346895dc68bfa/html5/thumbnails/4.jpg)
![Page 5: Rethinking Color for Next-Generation TV](https://reader036.vdocuments.us/reader036/viewer/2022062323/5681594e550346895dc68bfa/html5/thumbnails/5.jpg)
UHDTV Color Primaries
Chromaticity coordinates (CIE ,1931) x y Wavelength
Red primary (R) 0.708 0.292 630 nm
Green primary (G) 0.170 0.797 532 nm
Blue primary (B) 0.131 0.046 467 nm
Reference white (D65) 0.3127 0.3290
Picture information can be linearly indicated by the tristimulus values of RGB in the range of 0-1.The colorimetric values of the picture information can be determined based on the reference RGB primaries and the reference white.The color primaries are on the spectral locus.
![Page 6: Rethinking Color for Next-Generation TV](https://reader036.vdocuments.us/reader036/viewer/2022062323/5681594e550346895dc68bfa/html5/thumbnails/6.jpg)
Color gamut comparison in the CIE u′ v′ chromaticity diagram
![Page 7: Rethinking Color for Next-Generation TV](https://reader036.vdocuments.us/reader036/viewer/2022062323/5681594e550346895dc68bfa/html5/thumbnails/7.jpg)
Motivation for color difference encoding
• Convert R, G, B to Y , R′ ′-Y′, B′-Y′• Human perception is less sensitive to resolution in color
difference signals vs luma signals– High frequency cut-off at around 60 cpd in luma,
but at around 10-11 cpd in chrominance (R -Y , B -Y ) signals′ ′ ′ ′( cpd - cycle per 1 degree of visual angle)
• We can sub-sample R -Y′ ′, B -Y′ ′ to save bandwidth– In storage and interfacing– In compression
• Sub-sample R -Y′ ′, B -Y′ ′ by a factor of two in each dimension
• Resulting number of samples is ½ that of original RGB
![Page 8: Rethinking Color for Next-Generation TV](https://reader036.vdocuments.us/reader036/viewer/2022062323/5681594e550346895dc68bfa/html5/thumbnails/8.jpg)
Why luminance and chrominance signals are used ?(A hypothesized diagrammatic representation to describe color perception mechanism)
(http://www.nature.com/nrn/journal/v4/n7/fig_tab/nrn1138_F1.html)
• Zone Theory (G. E. Muller, "Ueber die Farbenempfindungen," Bd. 1 and 2. Psychol. Physiol. Sinnesorg., 1930, Erganzungsbd. 17, 1-430, 1930)
LMS cones in retina absorb light reaching eye
One achromatic and two opponent color signals in nerve fibers
De-correlating cone absorptions Providing efficient neural representation of color Taking advantage of low spatial acuity of the opponent colors for image storage and transmission
Sending these three signals to the lateral geniculate nucleus in brain
Transferring to the visual cortex in brain for interpretation
![Page 9: Rethinking Color for Next-Generation TV](https://reader036.vdocuments.us/reader036/viewer/2022062323/5681594e550346895dc68bfa/html5/thumbnails/9.jpg)
Barten's luminance contrast sensitivity as a function of
spatial frequency for different angular display sizes
Mullen's chromatic contrast sensitivity as a function of
spatial frequency for red-green and yellow-blue channels
(P. G. J. Barten, "Evaluation of Subjective Image Quality with the Square-Root Integral Method", Journal of the Optical Society of America 7: 2024)
(K. T. Mullen, "The Contrast Sensitivity of Human Color Vision to Red-Green and Blue-Yellow Chromatic Gratings” Journal of Physiology (London) 359: 381)
The background for down-sampling chrominance signals
• High frequency cut-off at around 10-11 cpd for the chromatic contrast sensitivity function but at around 60 cpd for the luminance contrast sensitivity function (※ cpd - cycle per 1 visual angle)
Red-Green
Yellow-Blue
![Page 10: Rethinking Color for Next-Generation TV](https://reader036.vdocuments.us/reader036/viewer/2022062323/5681594e550346895dc68bfa/html5/thumbnails/10.jpg)
The conventional way to create lumaNon-constant luminance
R
G
B0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
(γ=0.45)
R’
G’
B’
0.2627R’+ 0.6780G’+ 0.0593B’
Y’
LinearNon-Linear
Y’ is not a pure luma signal; it contains some chroma information because it was made from non-linear RGB signals. This is called “non-constant luminance”, or NCL
![Page 11: Rethinking Color for Next-Generation TV](https://reader036.vdocuments.us/reader036/viewer/2022062323/5681594e550346895dc68bfa/html5/thumbnails/11.jpg)
The conventional way to create Y’, Cb’, Cr’ Non-constant luminance
R
G
B 0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
(γ=0.45)
R’
G’
B’
0.2627R’+ 0.6780G’+ 0.0593B’
Y’
R’
B’
R’- Y’
B’- Y’
Y’
Cb’
Cr’
Linear Non-Linear
The Cb’ and Cr’ signals contain some luminance information. If they are sub-sampled, there will be some loss in luminance resolution
![Page 12: Rethinking Color for Next-Generation TV](https://reader036.vdocuments.us/reader036/viewer/2022062323/5681594e550346895dc68bfa/html5/thumbnails/12.jpg)
Constant Luminance
Y’ is now a pure luma signal; it contains no chroma information because it was made from linear RGB signals. This is called “constant luminance”, or “CL”. The color difference signals contain less luminance information and can be down sampled with much less effect on the luminance resolution. The Cb’, Cr’ signals can be created using slightly different schemes.
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
(γ=0.45)
R
G
B
0.2627R+ 0.6780G+ 0.0593B
Y Y’
R’
B’
R’- Y’
B’- Y’
R
B
Y’Cb
’Cr
’
Linear
![Page 13: Rethinking Color for Next-Generation TV](https://reader036.vdocuments.us/reader036/viewer/2022062323/5681594e550346895dc68bfa/html5/thumbnails/13.jpg)
Benefits of Constant Luminance Encoding
• Luminance resolution is less affected by down-sampling the chroma signals– detail and edge information is more accurately
maintained
• Lower correlation between Cr’, Cb’ and Y’• Better compression efficiency
– Chroma information survives compression better
![Page 14: Rethinking Color for Next-Generation TV](https://reader036.vdocuments.us/reader036/viewer/2022062323/5681594e550346895dc68bfa/html5/thumbnails/14.jpg)
For high luminance colors, correlation is significantly high between Y’ and Cb’ in the NCL
How much correlated in Y’-Cb’, Y’-Cr’, and Cb’-Cr’?
For most luminance values, correlation is significantly high between Y’ and Cr’ in the NCL→ non-orthogonal!
For high luminance colors, correlation is high between Cb’ and Cr’ in the NCL
Using uniformly sampled 4096 colors from the UHDTV gamut
Y’-Cr’
Y’-Cb’
Cb’-Cr’
![Page 15: Rethinking Color for Next-Generation TV](https://reader036.vdocuments.us/reader036/viewer/2022062323/5681594e550346895dc68bfa/html5/thumbnails/15.jpg)
The impact of the maintenance of lightness on image sharpness
• Significant lightness discrepancies in the
right-side four dark pixels after 4:1:0 sub-
sampling
Blurred boundary (white box)
Smeared black stripe line (green box)
(a) (b) (c)
(a) (b) (c)
48 45 23 7
46 41 17 6
(a) (b) (c)
49 42 23 11
47 39 20 6
(a) (b) (c)
45 39 28 22
43 37 25 19
Original 4:2:0 4:1:0
•The lightness values are almost maintained
in the right-side four dark pixels after 4:1:0
sub-sampling
Original sharp boundary (white box)
Apparent black strip line (green box)
(a) (b) (c)
Original image 4:2:0 4:1:0
(a) (b) (c)
48 45 23 7
46 41 17 6
(a) (b) (c)
48 44 23 8
46 41 17 6
(a) (b) (c)
48 44 23 7
46 41 18 6
Original 4:2:0 4:1:0
Constant Luminance Signals
Non-Constant Luminance Signals
![Page 16: Rethinking Color for Next-Generation TV](https://reader036.vdocuments.us/reader036/viewer/2022062323/5681594e550346895dc68bfa/html5/thumbnails/16.jpg)
Relative compression efficiency using HEVC (V. 4)
Test sequence having high colorfulnessTest sequence having high colorfulness
PSNR-lightness (L*) PSNR-color difference (ΔE*)
![Page 17: Rethinking Color for Next-Generation TV](https://reader036.vdocuments.us/reader036/viewer/2022062323/5681594e550346895dc68bfa/html5/thumbnails/17.jpg)
Downside of Constant Luminance
• It is a non-linear color space, i.e. color mixing is not along straight lines
+ =
Cyan + Red = Grey ?
RGB, R’G’B’, NCL Y’Cr’Cb’
CL Y’Cr’Cb’
= not quite Grey ?
![Page 18: Rethinking Color for Next-Generation TV](https://reader036.vdocuments.us/reader036/viewer/2022062323/5681594e550346895dc68bfa/html5/thumbnails/18.jpg)
Cross-fade between two pictures (addition of pictures in different domains)
R, G, B
R’, G’, B’ orNCL Y’, Cb’, Cr’
![Page 19: Rethinking Color for Next-Generation TV](https://reader036.vdocuments.us/reader036/viewer/2022062323/5681594e550346895dc68bfa/html5/thumbnails/19.jpg)
Cross-fade between two pictures
R’G’B’ orNCL Y’, Cb’, Cr’
CL Y’, Cb’, Cr’
![Page 20: Rethinking Color for Next-Generation TV](https://reader036.vdocuments.us/reader036/viewer/2022062323/5681594e550346895dc68bfa/html5/thumbnails/20.jpg)
[Mixing two images in RGB domain]
![Page 21: Rethinking Color for Next-Generation TV](https://reader036.vdocuments.us/reader036/viewer/2022062323/5681594e550346895dc68bfa/html5/thumbnails/21.jpg)
[Mixing two images in nonlinear R’G’B’ domain]
![Page 22: Rethinking Color for Next-Generation TV](https://reader036.vdocuments.us/reader036/viewer/2022062323/5681594e550346895dc68bfa/html5/thumbnails/22.jpg)
[Mixing two images in NCL-Y’Cb’Cr’ domain]
![Page 23: Rethinking Color for Next-Generation TV](https://reader036.vdocuments.us/reader036/viewer/2022062323/5681594e550346895dc68bfa/html5/thumbnails/23.jpg)
[Mixing two images in CL-Y’Cb’Cr’ domain]
![Page 24: Rethinking Color for Next-Generation TV](https://reader036.vdocuments.us/reader036/viewer/2022062323/5681594e550346895dc68bfa/html5/thumbnails/24.jpg)
Advantages of non-constant luminance
• Same results for color mixing between R’G’B’ and Y’Cr’Cb’
• We have been doing NCL for 60 years• Tried and true• No-risk involved, no surprises lurking• Why change something that works?
![Page 25: Rethinking Color for Next-Generation TV](https://reader036.vdocuments.us/reader036/viewer/2022062323/5681594e550346895dc68bfa/html5/thumbnails/25.jpg)
Conclusion• Open issues re UHDTV signal specification:
Terminology - do we call both 4k and 8k “Ultra HDTV”? Or should we use different words for each?
EOTF – don’t make the same mistake as Rec.709Color encoding – any reason to change? What is a suitable
threshold of benefit that a new method should reach to justify a change?
• If the UHDTV signal specification matters to you, get involved in the discussions and help drive this to the right conclusion.