basics of video courtesy of professor yao wang polytechnic university, brooklyn, ny11201...
TRANSCRIPT
![Page 1: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/1.jpg)
Basics of Video
Courtesy of Professor Yao WangPolytechnic University, Brooklyn, NY11201
![Page 2: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/2.jpg)
Adapted from Yao Wang, 2004
Video Basics
Outline
• Video capture– Photometric model– Geometric model
• Analog video– Progressive vs. interlaced rasters in analog TV system– Different color representations: YUV/YIQ
• Digital video– Sampling/quantization– Y’CbCr format
• Video display– Spatial/temporal/bit-depth resolution
![Page 3: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/3.jpg)
Adapted from Yao Wang, 2004
Video Basics 3
Photometric Model of Video Capture
Courtesy of Onur Guleryuz
![Page 4: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/4.jpg)
Adapted from Yao Wang, 2004
Video Basics 4
Geometric Model of Video Capture
Cameracenter
Imageplane
2-Dimage
3-Dpoint
The image of an object is reversed from its 3-D position. The object appears smaller when it is farther away.
Z
YFy
Z
XFx ,
![Page 5: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/5.jpg)
Adapted from Yao Wang, 2004
Video Basics 5
Implication of Models in Analog World
Miniature building Explosion from The Mummy Lighting in Filmmaking
![Page 6: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/6.jpg)
Adapted from Yao Wang, 2004
Video Basics 6
Progressive and Interlaced Raster Scans
Field 1 Field 2
Progressive Frame Interlaced Frame
Interlaced scan is developed to provide a trade-off between temporal and vertical resolution, for a given, fixed data rate (number of line/sec).
Horizontal retrace
Vertical retrace
![Page 7: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/7.jpg)
Adapted from Yao Wang, 2004
Video Basics 7
Color TV Broadcasting and Receiving
Lum ina nc e ,C hro m ina nc e ,Aud io M ultip le xing
M o d ula tio n
De -M o d ula tio n
De -M ultip le xing
YC 1C 2--->RG B
RG B--->
YC 1C 2
![Page 8: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/8.jpg)
Adapted from Yao Wang, 2004
Video Basics 8
Why not using RGB directly?
• R,G,B components are correlated– Transmitting R,G,B components separately is redundant– More efficient use of bandwidth is desired
• RGB->YC1C2 transformation– Decorrelating: Y,C1,C2 are uncorrelated– C1 and C2 require lower bandwidth– Y (luminance) component can be received by B/W TV sets
Color transformation is a compromised solution, but the ultimate one
![Page 9: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/9.jpg)
Adapted from Yao Wang, 2004
Video Basics 9
YIQ in NTSC
• I (in-phase): orange-to-cyan
• Q (quadrature): green-to-purple (human eye is less sensitive)– Q can be further bandlimited than I
• Phase=Arctan(Q/I) = hue, Magnitude=sqrt (I^2+Q^2) = saturation
• Hue is better retained than saturation
Recall: Quadrature amplitude modulation (QAM) in digital communication
![Page 10: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/10.jpg)
Color Image Y image
I image (orange-cyan) Q image (green-purple)
![Page 11: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/11.jpg)
Adapted from Yao Wang, 2004
Video Basics 11
I and Q on the color circle
I: orange-cyan
Q: green-purple
![Page 12: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/12.jpg)
Adapted from Yao Wang, 2004
Video Basics 12
Conversion between RGB and YIQ
Y = 0.299 R + 0.587 G + 0.114 BI = 0.596 R -0.275 G -0.321 BQ = 0.212 R -0.523 G + 0.311 B
R =1.0 Y + 0.956 I + 0.620 Q,G = 1.0 Y - 0.272 I -0.647 Q,B =1.0 Y -1.108 I + 1.700 Q.
• RGB -> YIQ
• YIQ -> RGB
![Page 13: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/13.jpg)
Adapted from Yao Wang, 2004
Video Basics 13
YUV in PAL
![Page 14: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/14.jpg)
Adapted from Yao Wang, 2004
Video Basics 14
YUV/RGB Conversion
Y = (( ( 66 * R + 129 * G + 25 * B + 128) >> 8) + 16)U = ( ( -38 * R - 74 * G + 112 * B + 128) >> 8) + 128V = ( ( 112 * R - 94 * G - 18 * B + 128) >> 8) + 128
Numerical approximations
![Page 15: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/15.jpg)
Adapted from Yao Wang, 2004
Video Basics 15
YIQ/YUV Comparison
![Page 16: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/16.jpg)
Adapted from Yao Wang, 2004
Video Basics 16
Different Color TV Systems
Parameters NTSC PAL SECAM
Field Rate (Hz) 59.95 (60) 50 50
Line Number/Frame 525 625 625
Line Rate (Line/s) 15,750 15,625 15,625
Color Coordinate YIQ YUV YDbDr
Luminance Bandwidth (MHz) 4.2 5.0/5.5 6.0
Chrominance Bandwidth (MHz) 1.5(I)/0.5(Q) 1.3(U,V) 1.0 (U,V)
Color Subcarrier (MHz) 3.58 4.43 4.25(Db),4.41(Dr)
Color Modulation QAM QAM FM
Audio Subcarrier 4.5 5.5/6.0 6.5
Total Bandwidth (MHz) 6.0 7.0/8.0 8.0
![Page 17: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/17.jpg)
Adapted from Yao Wang, 2004
Video Basics 17
Who uses what?
From http://www.stjarnhimlen.se/tv/tv.html#worldwide_0
![Page 18: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/18.jpg)
Adapted from Yao Wang, 2004
Video Basics 18
Digital Video
Taken from EE465: Image Acquisition
Sampling Quantization
![Page 19: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/19.jpg)
Adapted from Yao Wang, 2004
Video Basics 19
BT.601* Video Format
480
line
s
525
line
s
12 2p e l
16p e l
85 8 p e ls
72 0 p e ls
Ac tiveAre a
52 5/60 : 60 fie ld /s57
6 lin
es
625
line
s
86 4 p e ls
13 2p e l
12p e l
72 0 p e ls
Ac tiveAre a
62 5/50 : 50 fie ld /s
* BT.601 is formerly known as CCIR601
![Page 20: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/20.jpg)
Adapted from Yao Wang, 2004
Video Basics 20
RGB <--> Y’CbCr
Analog video
Digital video
![Page 21: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/21.jpg)
Adapted from Yao Wang, 2004
Video Basics 21
YUV vs. Y’CbCr
![Page 22: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/22.jpg)
Adapted from Yao Wang, 2004
Video Basics 22
Chrominance Subsampling Formats
4 :2 :0F o r ev er y 2x 2 Y P ix els
1 C b & 1 C r P ix el(S u b s am p l in g b y 2 :1 b o thh o r i zo n ta l l y an d v e r t i c al l y )
4:2:2 Fo r ev er y 2x 2 Y P ix e ls
2 C b & 2 C r P ix e l(S u b s am p lin g b y 2:1
h o r izo n tal l y o n ly )
4 :4:4 Fo r ev er y 2x 2 Y P ix e ls
4 C b & 4 C r P ix e l(N o s u b s am p l in g )
Y P ix el C b an d C r P ix el
4:1:1Fo r ev er y 4x 1 Y P ix e ls
1 C b & 1 C r P ix e l(S u b s am p lin g b y 4:1
h o r izo n tal ly o n ly )
![Page 23: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/23.jpg)
Adapted from Yao Wang, 2004
Video Basics 23
Digital Video Formats
Video Format Y Size Color Sampling
Frame Rate (Hz)
Raw Data Rate (Mbps)
HDTV Over air. cable, satellite, MPEG2 video, 20-45 Mbps SMPTE296M 1280x720 4:2:0 24P/30P/60P 265/332/664 SMPTE295M 1920x1080 4:2:0 24P/30P/60I 597/746/746 Video production, MPEG2, 15-50 Mbps BT.601 720x480/576 4:4:4 60I/50I 249 BT.601 720x480/576 4:2:2 60I/50I 166 High quality video distribution (DVD, SDTV), MPEG2, 4-10 Mbps BT.601 720x480/576 4:2:0 60I/50I 124 Intermediate quality video distribution (VCD, WWW), MPEG1, 1.5 Mbps SIF 352x240/288 4:2:0 30P/25P 30 Video conferencing over ISDN/Internet, H.261/H.263, 128-384 Kbps CIF 352x288 4:2:0 30P 37 Video telephony over wired/wireless modem, H.263, 20-64 Kbps QCIF 176x144 4:2:0 30P 9.1
![Page 24: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/24.jpg)
Adapted from Yao Wang, 2004
Video Basics 24
4:2:0 YUV Video
Y: 288-by-352
U: 144-by-176 V: 144-by-176
![Page 25: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/25.jpg)
Adapted from Yao Wang, 2004
Video Basics 25
Tricky Photometric Situations
Shadow causes problem to background extraction
Video enhancement
![Page 26: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/26.jpg)
Adapted from Yao Wang, 2004
Video Basics 26
Geometric Invariance
![Page 27: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/27.jpg)
Adapted from Yao Wang, 2004
Video Basics 27
Video Display
• High-end– If the resolution of display device is higher than that of video
sequence, what can we do?
– Tradeoff between quality and complexity
– Subjective evaluation of video quality
• Low-end– If the resolution of display device is lower than that of video
sequence, what can we do?
– What if the bit-depth resolution is lower? (e.g., display video on PDAs and portable DVDs)
It is the last and the least-researched component in visual communication systems
![Page 28: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/28.jpg)
Adapted from Yao Wang, 2004
Video Basics 28
Resolution, Resolution, Resolution
spatial
temporal
Bit-depth
1M 10M
30fps
300fps
8bpp
32bpp
![Page 29: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/29.jpg)
Adapted from Yao Wang, 2004
Video Basics 29
High Dynamic Range Imaging
Q: Can we generate a HDR image (16bpp) by a standard camera?A: Yes, adjust the exposure and fuse multiple LDR images together
![Page 30: Basics of Video Courtesy of Professor Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu](https://reader036.vdocuments.us/reader036/viewer/2022082310/551999675503464d068b48ee/html5/thumbnails/30.jpg)
Adapted from Yao Wang, 2004
Video Basics 30
HDR Display (after Toner Mapping)
Note that any commercial display devices we see these days are NOT HDR