video fundamentals september 9, 1999 lawrence a. rowe university of california, berkeley url: larry...

Video Fundamentals

September 9, 1999

Lawrence A. RoweUniversity of California, Berkeley

URL: http://www.BMRC.Berkeley.EDU/~larry

Copyright @1999, L.A. Rowe

Multimedia Systems and Applications 2

Outline

• History

• Broadcast System Elements

• Scanning and Interlace

• Color and Gamma

• Chroma Subsampling

• Representations


History

• 1839: Daguerreotype Cameras• 1893: Telephone Audio Broadcasting (Puskas)• 1895: Wireless Communication (Marconi,

Popov)• 1895: Film Presentation (Lumiere Brothers)• 1919: Radio Broadcasting (Holland, Canada)• 1934: US establishes FCC• 1935: TV Broadcasting (Germany, Britain)• 1941: US B&W TV

ADD: telephone invention, founding of ATT, tv inventions, first radio networks/stations


History (cont.)

• 1951: Videotape Recorder (Bing Crosby Enterprises)

• 1953: US Color TV (NTSC)• 1963: Geostationary Satellites• mid ‘70s: Fiber Optic Transmission - cable• 1985: FCC establishes ATSC - standard by

1993?• 1989: Analog HDTV Broadcasting (Japan)• 1996: ATSC Standard Adopted

ADD: first ntsc tv broadcasts, 1st cable system, consumer vcr, tivoli/replay device, direct tv, first hdtv broadcasts


Standards Groups

• ITU-T – ITU TelecommunicationsFormerly CCITT

• ITU-R – ITU RadiocommunicationsFormerly CCIR

• FCC• SMPTE - Society of Motion Picture and

Television Engineers… and many more!


Broadcast System

Tra

nsm

itte

r

Rece

iver

Goals:1. Efficient use of bandwidth2. High viewer perception of quality


Scanning and Interlace

• Transmission is continuous signalTransmitter captures images and encodesReceiver decodes for display

• Image traced out line-by-lineLeft-to-right top-down scanningvertical/horizontal blanking interval

• Signal is interlacedImproves perception of motion (?)Alternative is progressive scanning


Video Display Scanning

Amplitude

Time

Cathode

• Video composed of luma and chroma signals• Composite video combines luma and chroma• Component video sends signals separately


Camera Operation

• Camera has 1, 2, or 3 tubes for sampling• More tubes (CCD’s) and better lens produce

better pictures

Beam Splitter

Color Filters

En

cod

er

Camera Tubes

Zoom Lens Luma

Chroma

G

R

B


Direct View CRT

• Three guns (RGB) energize phosphorsVarying energy changes perceived intensityDifferent energies to different phosphors produces different colorsPhosphors decay so you have to refresh

• Different technologiesShadow mask (delta-gun dot mask)PIL slot maskSingle-gun (3 beams) aperture-grille (Trinitron)


Aperture-grille –vs- Shadow Mask

Shadow Mask Holes

Screen Phosphors

RG

B

RGB


Scanning Notation

• Lines/”frame rate” specificationNTSC 525/59.94PAL 625/50

• ATSC – everything is variable1080i 1920x1080 interlaced scanning 720p 1280x720 progressive scanning

• Why 59.94?Avoids interference problem between color and sound subcarrier


Interlaced Fields

vertical blanking

vertical blanking

Field 1

Field 2

Line 1 ---

Line 21 ---

Line 263 ---

Line 283 ---

Line 525 ---

1

484

485

...

485

24

...

485

484

23

Signal Format

Raster Format

ITU-R Rec. 601: 720x483


Scanning (525/59.94)

Active

Active

Active

squarepixel

601

4fsc

Total Size Active Area

780X525

858X525

910X525

644X483

720X483

757X483


Aspect Ratio/Refresh Rate

• Aspect ratioConventional TV is 4:3 (1.33)HDTV is 16:9 (2.11)Cinema uses 1.85:1 or 2.35:1

• Refresh RateNTSC is 60Hz (59.94Hz)PAL is 50HzCinema is 48Hz (but still only 24 fps)


NTSC Video

• 525 scan lines repeated 29.97 times per second (i.e.33.37 msec/frame)

• Interlaced scan lines divide frame into 2 fields each 262.5 lines (i.e.16.68 msec/field)

• 20 lines reserved for control information at the beginning of each fieldOnly 483 lines of visible dataLaserdisc and S-VHS display around 420 lines

(perception)Normal broadcast TV displays around 320 lines (“)

• Line lasts 63.6 usec(10.9 usec blanked)

(525-lines, 60-fields/sec)


PAL Video

• 625 scan lines repeated 25 times per second (i.e. 40 msec/frame)

• Interlaced scan lines divide frame into 2 fields each 312.5 lines (i.e. 20 msec/field)

• Approximately 20% more lines than NTSC

• NTSC vs. PAL roughly same bandwidth

(625-lines, 60-fields/sec)


Color Perception

• Color is perceived lightwave400nm to 700nm received at retinaHumans more sensitive to brightness than color

• Retina composed of cones and rodsCones respond to different frequencies (RGB)Rods measure brightness at low light levels (i.e., nightvision)

• CIE established standards for colorCIE XYZ, CIE xyY Linear RGB


Gamma

• CRT is inherently non-linearDisplay changes based on voltage driving it

• Human vision is also non-linearLightness sensation is a power function of intensity (y=xw)

• Serendipity happens…Non-linear CRT close to inverse human lightnessCoding intensity into a gamma corrected signal maximizes perceptual image

• Y -vs- Y’


Gamma in Video

• Camera performs gamma correction• Display imposes inverse power function

y = x 1/0.45

camera transmission display


Color Image Coding

• Image represented by 24 bit pixel (8 bpp)Each color value between 0 and 255

• Video uses non-linear codingUniform distribution of colors to codesRGB R’G’B’ (gamma corrected RGB)

• Video uses luminance/chromance R’G’B’ Y’CBCR

Luminance is Y (technically luma is Y’)Chromance is CBCR


Jargon

• CBCR actually color difference signalsCB is scaled version of (Y’-B’)

CR is scaled version of (Y’-R’)

• Confusing terms: YUV, YIQ, Y CBCR,…Scaled versions of <Y’, Y’-B’, Y’-R’>Green has highest contribution to luminance

• Luminance –vs- Luma (i.e., Y –vs- Y’)Y is linear luminanceY’ is gamma corrected luminance (aka luma)


Chroma Subsampling

• Chroma subsampling reduces data2 chroma/luma 16 bpp1 chroma/luma 12 bpp

• Notation is a:b:ca is luma samplesb is chroma samples per odd linec is chroma samples per even line

• Ex. 4:4:4, 4:2:2, 4:1:1, 4:2:0, …


Line SamplingY Y Y Y Y

4:4:4

CR/CB CR/CB CR/CB CR/CB CR/CBY Y Y Y Y

4:2:2

CR/CB CR/CB CR/CB

Y Y Y Y Y4:1:1

CR/CB

4:2:2 is referred to as broadcast quality4:1:1 is referred to as VHS quality4:2:0 is 2:1 down sampling in horizontal and vertical direction


4:2:0 Sampling

Luma sample

Chroma sample


Representations

• CompositeNTSC - 6MHz (4.2MHz video), 29.97 frames/secondPAL - 6-8MHz (4.2-6MHz video), 50 frames/second

• ComponentSeparation video (luma, chroma) - svhs, Hi8mmRGB, YUV, YIQ, …YCBCR - used for most compressed representations

• Separation video called “s-video”


Analog Video Representations

• NTSCY = 0.299R + 0.587G + 0.114B I = 0.596R - 0.275G - 0.321BQ = 0.212R - 0.523G + 0.311Bcomposite = Y + Icos(Fsc t) + Qsin(Fsc t)

• PALY = 0.299R + 0.587G + 0.114BU = 0.492(B-Y)Q = 0.877(R-Y)composite = Y + Usin(Fsc t) + Vcos(Fsc t)


Digitizing

• Analog TV is a continuous signal• Digital TV uses discrete numeric values

Signal is sampledSamples are quantizedSmall, discrete regions are digitized

• Image represented by pixel array


Image Sizes

QSIF(19Kp)

SIF (82Kp)

601 (300Kp)

SVGA (500Kp)

ATV (1Mp)

Workstation (1Mp)

HDTV (2Mp)

120

240

483

600

720

900

1080

160 352 720 800 1152 1280 1920


Workstation Images?

• [1 Mega Pixel] 1152 x 900 = 1,036,800 pixels

• [xvga] 1024 x 768 = 786,432 pixels

• [?] 1280 x 1024 = 1,310,720 pixels


Digital Video Representations

• Digital Composite Video(D2/D3,SMPTE 244M)142 Mb/s data rate, either parallel or serialSubsampled color signals 4:2:2

• Digital Component Video(D1/D5,SMPTE RP125)Maintain separate signals for luma and chroma270 Mb/s data rate, either parallel or serialSubsampled color signals 4:2:2

• Compressed Digital VideoMPEG, MJPEG, H.26x, DV, …


Digital Video Block Structure

• 4:2:2 YCBCR

16x16 macroblock8x8 pixel blocks8 bits/sample = 16 bits/pixel = 4Kbits/macroblock

• 4:1:1 YCBCR

3Kbits/macroblock12 bits/pixel

Y3

CB1

CB2 CR2

CR1Y1 Y2

Y4

macroblock

Y3

CB CR

Y1 Y2

Y4


What is Video Data Rate?

• Digital720x483 = 347,760 pixels/frame4:2:2 sampling gives 695,520 bytes/frame21 MB/sec (167 Mbs)4:4:4 sampling gives 250 Mbs

• ATV (MPEG MP@ML)1280x720 = 921,600 pixels/frame4:2:0 sampling gives 1,382,400 bytes/frame41 MB/sec (328 Mbs)

(Note: MPEG coded streams are 1.5-80 Mbs)


What is Video Data Rate (cont.)?

• ATSC (720P)720x1280 = 921,600 pixels per frame4:2:2 sampling = 1,843,200 bytes per frame24 fps = 44,236,800 bytes per second 44 MB/s = 354 Mbs

• ATSC (studio 1080I)1080x1920 = 2,073,600 pixels per frame4:4:4 sampling = 6,220,800 bytes per frame30 fps = 186,624,000 bytes per second187MB/s = 1.5 Gbs


Serial Digital Data Rates

• Serial digital is a video transport standardWidely used in broadcast and production studios

• Data ratesNTSC SDI approximately 200 MbsATSC SDI approximately 1.5 Gbs

• Routing SwitcherRemember typical facility might have a 256x256 routing switcherWhat is data rate across the switch?


Human Perception

•What is smooth motionDepends on source materialMost action is perceived as smooth at 24 fps

•Human most sensitiveLow frequenciesChanges in luminance and blue-orange axis

•Vision emphasizes edge detectionStrong bias to horizontal and verticle lines

•Visual masking by large luminance changes


Producing High Quality Video

• Need high quality cameraS-Video(SVHS, Hi8mm) better than composite3 chips better than 1 chipDigital better than analog

• Lights, lights, lights…Experiment with filters to change apparent colors

• Shoot scene from different angles and cut between them to create visual stimulation

• Study film/video techniquesLet person exit the scene without moving cameraKeep orientation of images correctChange scene/shot to reflect time change


Conclusions• NTSC/PAL were excellent standards that

lasted over 50 yearsTechnology has changed dramatically during this time

• Digital revolution is changing industryMain impact is development of ATSC standard

• Internet revolution is also chaning industryWebcasting –vs- mass market broadcastingWireless –vs- cable –vs- packet transmission

video fundamentals september 9, 1999 lawrence a. rowe university of california, berkeley url: larry...

Documents