digital video carnegie mellon © © copyright 2001 michael g. christel and alexander g. hauptmann...

Digital Video

CarnegieMellon© Copyright 2001 Michael G. Christel and Alexander G. Hauptmann

VideoVideo• Video comes from a camera, which records what it sees as a sequence of images Video comes from a camera, which records what it sees as a sequence of images

• Image frames comprise the videoImage frames comprise the video• Frame rate = presentation of successive framesFrame rate = presentation of successive frames• minimal image change between framesminimal image change between frames• Frequency of frames is measured in frames per second [fps].Frequency of frames is measured in frames per second [fps].

• Sequencing of still images creates the illusion of movementSequencing of still images creates the illusion of movement

> 16 fps is “smooth” > 16 fps is “smooth” Standards: 29.97 is NTSC, 24 for movies, 25 is PAL, 60 is HDTVStandards: 29.97 is NTSC, 24 for movies, 25 is PAL, 60 is HDTV

• Standard Definition Broadcast TV, NTSC, Standard Definition Broadcast TV, NTSC, • 15 bits/pixel of color depth, and 15 bits/pixel of color depth, and • 525 lines of resolution 525 lines of resolution • with 4:3 aspect ratio. with 4:3 aspect ratio. Scanning practices leave a smaller safe region.Scanning practices leave a smaller safe region.

• Display scan rate is different Display scan rate is different • monitor refresh ratemonitor refresh rate• 60 - 70 Hz (= 1/s)60 - 70 Hz (= 1/s)• Interlacing: half the scan lines at a time (-> flicker)Interlacing: half the scan lines at a time (-> flicker)


The Video Data FirehoseThe Video Data Firehose

• To play one To play one SECONDSECOND of uncompressed 16-bit color, 640 X 480 of uncompressed 16-bit color, 640 X 480 resolution, digital video requires approximately resolution, digital video requires approximately 18 MB18 MB of storage. of storage.

• One minute would require about One minute would require about 1 GB1 GB. .

• A CD-ROM can only hold about 600MB and A CD-ROM can only hold about 600MB and a single-speed (1x) player can only transfer 150KB per second. a single-speed (1x) player can only transfer 150KB per second.

Data storage and transfer problems increase proportionally with 24-bit Data storage and transfer problems increase proportionally with 24-bit color playback. color playback.

Without compression, digital video would not be possible with current Without compression, digital video would not be possible with current

storage technology.storage technology.


Storage/Transmission IssuesStorage/Transmission Issues

The storage/transmission requirements for video is The storage/transmission requirements for video is determined by: determined by: Video Source Data * Compression = Storage Video Source Data * Compression = Storage

• The amount of required storage is determined by The amount of required storage is determined by • how much and what type of video data is in the how much and what type of video data is in the

uncompressed signal and uncompressed signal and • how much the data can be compressed. how much the data can be compressed.

In other words, the original video source and the desired In other words, the original video source and the desired playback parameters dramatically affect the final playback parameters dramatically affect the final storage/transmission needs. storage/transmission needs.


Video CompressionVideo Compression

• The person recording video to be digitized can drastically affect the The person recording video to be digitized can drastically affect the later compression steps. later compression steps.

Video in which backgrounds are stable (or change slowly), for a Video in which backgrounds are stable (or change slowly), for a period of time will yield a high compression rate. period of time will yield a high compression rate.

Scenes in which only a person's face from the shoulders upward is Scenes in which only a person's face from the shoulders upward is captured against a solid background will result in excellent captured against a solid background will result in excellent compression. compression. • This type of video is often referred to as a 'This type of video is often referred to as a 'talking headtalking head'.'.


FilteringFiltering• A filtering step does not achieve compression, A filtering step does not achieve compression,

but may be necessary to minimize artifacts of compression. but may be necessary to minimize artifacts of compression.

• Filtering is a preprocessing step performed on video frame Filtering is a preprocessing step performed on video frame images before compression. Essentially it smoothes the images before compression. Essentially it smoothes the sharp edges in an image where a sudden shift in color or sharp edges in an image where a sudden shift in color or luminance has occurred. luminance has occurred.

• The smoothing is performed by averaging adjacent groups of The smoothing is performed by averaging adjacent groups of pixel values. pixel values. Without filtering, decompressed video exhibits aliasing Without filtering, decompressed video exhibits aliasing (jagged edges), and moiré patterns.(jagged edges), and moiré patterns.


Data Reduction through ScalingData Reduction through Scaling

• The easiest way to save memory is to store less, e.g. The easiest way to save memory is to store less, e.g. through size through size scalingscaling. Original digital video standards . Original digital video standards only stored a video window of 160 X 120 pixels. A only stored a video window of 160 X 120 pixels. A reduction of 1/16th the size of a 640 X 480 window. A reduction of 1/16th the size of a 640 X 480 window. A 320 X 240 digital video window size is currently about 320 X 240 digital video window size is currently about standard, yielding a 4 to 1 data reduction. standard, yielding a 4 to 1 data reduction.

• A further scaling application involves time instead of A further scaling application involves time instead of space. In temporal scaling the number of frames per space. In temporal scaling the number of frames per second (fps), is reduced from 30 to 24. If the fps is second (fps), is reduced from 30 to 24. If the fps is reduced below 24 the reduction becomes noticeable in reduced below 24 the reduction becomes noticeable in the form of jerky movement. the form of jerky movement.


Compression through TransformationCompression through Transformation

• CodecsCodecs (COmpression/DECompression algorithms) transform a (COmpression/DECompression algorithms) transform a two-dimensional spatial representation of an image into another two-dimensional spatial representation of an image into another dimension space (usually frequency).dimension space (usually frequency).

• Since most natural images are composed of low frequency Since most natural images are composed of low frequency information, the high frequency components can be discarded. information, the high frequency components can be discarded. • [What are high frequency components?][What are high frequency components?]• This results in a softer picture in terms of contrast. This results in a softer picture in terms of contrast.

• Most commonly, the frequency information is represented as 64 Most commonly, the frequency information is represented as 64 coefficients due to the underlying coefficients due to the underlying DCTDCT (Discrete Cosine (Discrete Cosine Transform), algorithm which operates upon 8 X 8 pixel grids. Low Transform), algorithm which operates upon 8 X 8 pixel grids. Low frequency terms occur in one corner of the grid, with high frequency terms occur in one corner of the grid, with high

frequency terms occurring in the opposite corner of the grid.frequency terms occurring in the opposite corner of the grid.


Compression through QuantizationCompression through Quantization

• The lossy The lossy quantizationquantization step of digital video uses fewer step of digital video uses fewer bits to represent larger quantities. The 64 frequency bits to represent larger quantities. The 64 frequency coefficients of the DCT transformation are treated as coefficients of the DCT transformation are treated as real numbers. These are quantified into 16 different real numbers. These are quantified into 16 different levels. The high frequency components (sparse in real-levels. The high frequency components (sparse in real-world images), are represented with only 0, 1 or 2 bits. world images), are represented with only 0, 1 or 2 bits. The zero mapped frequencies drop out and are lost. The zero mapped frequencies drop out and are lost.


Frame CompactionFrame Compaction

• The last step in compressing individual frames The last step in compressing individual frames ((intraframe compressionintraframe compression) is a sequence of three ) is a sequence of three standard text file compression schemes. Run-length standard text file compression schemes. Run-length encoding (RLE), Huffman coding, and arithmetic encoding (RLE), Huffman coding, and arithmetic coding. coding. • RLE replaces sequences of identical values with the RLE replaces sequences of identical values with the

number of times the value occurs followed by the value number of times the value occurs followed by the value (e.g., 11111000011111100000 ==>> (e.g., 11111000011111100000 ==>> 551144006611550). 0).

• Huffman coding replaces the most frequently occurring Huffman coding replaces the most frequently occurring values|strings with the smallest codes. values|strings with the smallest codes.

• Arithmetic coding, similar to Huffman coding, codes the Arithmetic coding, similar to Huffman coding, codes the commonly occurring values|strings using fractional bit commonly occurring values|strings using fractional bit codes. codes.


Interframe Compression (MPEG style)Interframe Compression (MPEG style)

• Interframe compressionInterframe compression takes advantage of minimal takes advantage of minimal changes from one frame to the next to achieve dramatic changes from one frame to the next to achieve dramatic compression. Instead of storing complete information compression. Instead of storing complete information about each frame only the difference information about each frame only the difference information between frames is stored. between frames is stored.

• MPEG stores three types of frames:MPEG stores three types of frames:• The first type The first type I-frameI-frame, stores all of the interframe , stores all of the interframe

compression information using no frame differencing. compression information using no frame differencing. • The second type The second type P-frameP-frame is a predicted frame two or four is a predicted frame two or four

frames in the future. This is compared with the frames in the future. This is compared with the corresponding actual future frame and the differences are corresponding actual future frame and the differences are stored (error signal).stored (error signal).

• The third type The third type B-framesB-frames, are bidirectional interpolative , are bidirectional interpolative predicted frames that fill in the jumped frames. predicted frames that fill in the jumped frames.


Streaming VideoStreaming Video

• Access disk fast enoughAccess disk fast enough

• RAIDsRAIDs• Don’t download everything firstDon’t download everything first

• Play as you start to downloadPlay as you start to download• Keep a buffer for variable network speedKeep a buffer for variable network speed

• equivalent to sampling a CD’s faster and filling a equivalent to sampling a CD’s faster and filling a bufferbuffer

• Drop frames/packets when you fall behind (not TCP)Drop frames/packets when you fall behind (not TCP)• Adjust the bandwidth dynamicallyAdjust the bandwidth dynamically

• need multiple encoding formatsneed multiple encoding formats• RTSP, QT, MS ASF, H.323 (video conferencing)RTSP, QT, MS ASF, H.323 (video conferencing)


WebcastingWebcasting

• LIVELIVE• Encode fast enoughEncode fast enough• Stream to multiple users connected at the same timeStream to multiple users connected at the same time

• Only time-synchronous viewingOnly time-synchronous viewing

Video Data RatesVideo Data RatesQualityQuality FormatFormat

(example)(example)

Transfer RateTransfer Rate Disk SpaceDisk Space

1 hour1 hour

Disk SpaceDisk Space

100,000 hours100,000 hours

NetcastingNetcasting VDOLiveVDOLive 0.06 Mbit/s0.06 Mbit/s 26.4MByte26.4MByte 2.6 TByte2.6 TByte

Preview (ISDN)Preview (ISDN) RealVideoRealVideo 0.1 Mbit/s0.1 Mbit/s 43.9 MByte43.9 MByte 4.4 TByte4.4 TByte

Preview (LAN)Preview (LAN) MPEG-1MPEG-1 1.5 Mbit/s1.5 Mbit/s 675 MByte675 MByte 67.6 TByte67.6 TByte

BroadcastBroadcast MPEG-2 MPEG-2 (MP @ ML)(MP @ ML)

8 Mbit/s8 Mbit/s 3.5 GByte3.5 GByte 350 TByte350 TByte

EditingEditing MPEG-2 MPEG-2 (4:2:2P@ML )(4:2:2P@ML )

DVCPro50DVCPro50

18 Mbit/s18 Mbit/s

50 Mbit/s50 Mbit/s

7.9 GByte7.9 GByte

22 GByte22 GByte

790 TByte790 TByte

2.2 PByte2.2 PByte

ArchiveArchive MJPEG MJPEG

LosslessLossless

100 Mbit/s100 Mbit/s 43.9 GByte43.9 GByte 4.4 PByte4.4 PByte

UncompressedUncompressed ITU-R ITU-R BT.601-5BT.601-5

270 Mbit/s270 Mbit/s 118.7 GByte118.7 GByte 11.9 PByte11.9 PByte


MPEG: Motion Picture Experts GroupMPEG: Motion Picture Experts Group

• MPEG-1 (1992)MPEG-1 (1992)• Compression for StorageCompression for Storage• 1.5Mbps1.5Mbps• Frame-based CompressionFrame-based Compression

• MPEG-2 (1994)MPEG-2 (1994)• Digital TVDigital TV• 6.0 Mbps6.0 Mbps• Frame-based CompressionFrame-based Compression

• MPEG-4 (1998)MPEG-4 (1998)• Multimedia Applications, digital TV, synthetic graphicsMultimedia Applications, digital TV, synthetic graphics• Lower bit rateLower bit rate• Object based compressionObject based compression

• MPEG-7MPEG-7• Multimedia Content Description Interface, XML-basedMultimedia Content Description Interface, XML-based

• MPEG-21MPEG-21• Digital identification, IP rights managementDigital identification, IP rights management

MPEG-1 MPEG-1 System LayerSystem Layer

• CCombines one or more data streams from the video and audio parts with ombines one or more data streams from the video and audio parts with timing information to form a single stream suited to digital storage or timing information to form a single stream suited to digital storage or transmission.transmission.


MPEG-1 MPEG-1 Video LayerVideo Layer

• a coded representation that can be used for compressing video sequences - both 625-a coded representation that can be used for compressing video sequences - both 625-line and 525-lines - to bitrates around 1.5 Mbit/s. line and 525-lines - to bitrates around 1.5 Mbit/s.

• Developed to operate from storage media offering a continuous transfer rate of about Developed to operate from storage media offering a continuous transfer rate of about 1.5 Mbit/s. 1.5 Mbit/s.

• Different techniques for video compression:Different techniques for video compression:• Select an appropriate spatial resolution for the signal. Use block-based motion Select an appropriate spatial resolution for the signal. Use block-based motion

compensation to reduce the temporal redundancy. Motion compensation is used compensation to reduce the temporal redundancy. Motion compensation is used for causal prediction of the current picture from a previous picture, for causal prediction of the current picture from a previous picture,

for non-causal prediction of the current picture from a future picture, for non-causal prediction of the current picture from a future picture, or for interpolative prediction from past and future pictures. or for interpolative prediction from past and future pictures.

• The difference signal, the prediction error, is further compressed using the discrete The difference signal, the prediction error, is further compressed using the discrete cosine transform (DCT) to remove spatial correlation and is then quantised. cosine transform (DCT) to remove spatial correlation and is then quantised.

• Finally, the motion vectors are combined with the DCT information, and coded using Finally, the motion vectors are combined with the DCT information, and coded using variable length codes.variable length codes.

• When storing differences MPEG actually compares a block of pixels (macroblock) and if When storing differences MPEG actually compares a block of pixels (macroblock) and if a difference is found it searches for the block in nearby regions. This can be used to a difference is found it searches for the block in nearby regions. This can be used to alleviate slight camera movement to stabilize an image. It is also used to efficiently alleviate slight camera movement to stabilize an image. It is also used to efficiently represent motion by storing the movement information (motion vector), for the block.represent motion by storing the movement information (motion vector), for the block.

MPEG-1 MPEG-1 Video LayerVideo Layer


MPEG-1MPEG-1

• I,B,P FramesI,B,P Frames

• Choice of audio encodingChoice of audio encoding

• Picture size, bitrate is variablePicture size, bitrate is variable

• No closed-captions, etc.No closed-captions, etc.

• Group of PicturesGroup of Pictures

• one I frame in every groupone I frame in every group

• 10-15 frames per group10-15 frames per group

• P depends only on I, B depends on both I and PP depends only on I, B depends on both I and P

• B and P are random within GoPB and P are random within GoP


MPEG-1 MPEG-1 Audio LayerAudio Layer

• Compress audio sequences in mono or stereo. Compress audio sequences in mono or stereo.

• Encoding creates a filtered and subsampled representation of the input audio stream. Encoding creates a filtered and subsampled representation of the input audio stream.

• A psychoacoustic model creates data to control the quantiser and coding. A psychoacoustic model creates data to control the quantiser and coding.

• The quantiser and coding block creates coding symbols from the mapped input The quantiser and coding block creates coding symbols from the mapped input samples. samples.

• The block 'frame packing' assembles the actual bitstream from the output data of the The block 'frame packing' assembles the actual bitstream from the output data of the other blocks and adds other information (e.g. error correction) if necessary. other blocks and adds other information (e.g. error correction) if necessary.

MPEG-1 MPEG-1 Audio LayerAudio Layer


MPEG Streaming in variable networksMPEG Streaming in variable networks

• Problem: available bandwidthProblem: available bandwidth• Slightly too low, varyingSlightly too low, varying• Shared by other users/applicationsShared by other users/applications

• Target application: InformediaTarget application: Informedia

• MPEG movie database (terabytes)MPEG movie database (terabytes)

• http://www.cineflo.comhttp://www.cineflo.com

• CMU spinoff startup company for CMU spinoff startup company for

adaptive MPEG-1 video transmissionadaptive MPEG-1 video transmission

http://www.cineflo.com/

Filter / Transcoder

System OverviewSystem Overview

Client Data-Base

Video server

• Application-aware networkApplication-aware network

• Network-aware applicationNetwork-aware application

ArchitectureArchitecture

• Maintain two connectionsMaintain two connections• control connection: TCPcontrol connection: TCP• data connection: UDPdata connection: UDP

• Fits with the JAVA security modelFits with the JAVA security model

ServerServerFilterFilterClientClient

Control Control

DataData

Congestion Analysis and FeedbackCongestion Analysis and Feedback

• Client notices changes in loss rate and Client notices changes in loss rate and notifies filter notifies filter ......• Variable-size sliding window and two Variable-size sliding window and two

thresholdsthresholds

• Filter modifies rate by clever manipulation of Filter modifies rate by clever manipulation of data streamdata stream

• Client is less aggressive in recapturing Client is less aggressive in recapturing bandwidthbandwidth


Control Control

DataData

FilterFilter

• Acts as mediator between client and Acts as mediator between client and upstreamupstream

• MPEGMPEG Video format dependent Video format dependent• Performs Performs on-the-fly low-coston-the-fly low-cost computational computational

modifications to data streammodifications to data stream• Paces data streamPaces data stream


Control Control

DataData

Network layer

MPEG-1 Systems StreamMPEG-1 Systems Stream

Padding Audio[0]

Audio[0]

Audio[1]

Audio[1]

Video[0] Video[0] Video[0]

Video[0] Video[0]Video[0] Video[0]

Video[0]

Audio[0] Audio[1]

Pack layer Packet layer

MPEG Sensitivity to Network LossesMPEG Sensitivity to Network Losses

0%

20%

40%

60%

80%

100%

120%

0.05% 0.10% 0.20% 0.50% 1.00% 2.00% 5.00% 10.00%

% Packets dropped (1.5KB packets)

% U

nd

isp

lay

ab

le f

ram

es

Average Lost Frames Average Bad Frames

MPEG Video FilteringMPEG Video Filtering

I B B P B B P B B P B B P B B I

I B P B P B P B P B I

I P P P P I

I P P P I

I P P I

I I

MPEG System Sensitive Video FilteringMPEG System Sensitive Video Filtering

• Reduce network traffic by filtering framesReduce network traffic by filtering frames on-the-fly & low-cost !on-the-fly & low-cost !

• Maintain smoothnessMaintain smoothness

• Maintain synchronization dataMaintain synchronization data• Adjust Packet LayerAdjust Packet Layer

Padding Audio[0] Audio[1]

Padding Audio[0] Audio[1]

-----------B frame--------------

EvaluationEvaluation

0

500

1000

1500

2000

Withoutfiltering

Withfiltering

Stre

amin

g r

ate

(Kb

its/

sec)

MPEG Sending rate

MPEG Receiving rate

0%

20%

40%

60%

80%

100%

Withoutfiltering

Withfiltering

Good frames

Damaged frames

Lost or removed frames

• Constant Constant heavyheavy competing load competing load


Streaming based on estimated needStreaming based on estimated need

• Smarter Streaming for interactivitySmarter Streaming for interactivity

• Break apart I, P, B framesBreak apart I, P, B frames

• Client decides which are more likely to be needed and Client decides which are more likely to be needed and requests those from server for the client cacherequests those from server for the client cache

• Differential weights on frames based on needDifferential weights on frames based on need• Also weighting based on type of frame (I,P,B)Also weighting based on type of frame (I,P,B)

since you can’t decode a B frame without the I and P.since you can’t decode a B frame without the I and P.

• Can only achieve savings of ~ 30% over raw MPEG-1Can only achieve savings of ~ 30% over raw MPEG-1


MPEG-2MPEG-2

• Digital Television (4 - 9 Mb/s)Digital Television (4 - 9 Mb/s)

• Satellite dishes, digital cable videoSatellite dishes, digital cable video

• Larger data sizeLarger data size

• includes CCincludes CC

• More complex encoding (“long time”)More complex encoding (“long time”)

• almost HDTValmost HDTV


HDTVHDTV

2x horizontal and vertical resolution 2x horizontal and vertical resolution

• SDTV: 480 line, 720 pixels per line, 29.97 frames per secondSDTV: 480 line, 720 pixels per line, 29.97 frames per second

x 16 bits/pixex 16 bits/pixel = 168 Mbits/sec uncompressedl = 168 Mbits/sec uncompressed

MPEG-1 brings this to 1.5Mbits/sec at VHS qualityMPEG-1 brings this to 1.5Mbits/sec at VHS quality

• HDTV: expanded to 1080 lines, 1920 pixels per line, 60 fpsHDTV: expanded to 1080 lines, 1920 pixels per line, 60 fps

x 16 bits/pixel = 1990 Mbits/sec uncompressedx 16 bits/pixel = 1990 Mbits/sec uncompressed

MPEG-II like encoding, different audio encoding MPEG-II like encoding, different audio encoding

HDTV Audio Compression is based on the Dolby AC-3 system with HDTV Audio Compression is based on the Dolby AC-3 system with sampling rate 48kHz and perceptually codedsampling rate 48kHz and perceptually coded


Why HDTV?Why HDTV?

• Higher-resolution pictureHigher-resolution picture

• Wider pictureWider picture

• Digital surround sound.Digital surround sound.

• Additional dataAdditional data

• Easy to interface with computersEasy to interface with computers


Current TV StandardsCurrent TV Standards

TV Standards NTSC PAL SECAM

Regions U.S. Asia, Europe, South America

France

Channel Bandwidth

6MHz 8MHz 8MHz

Aspect ratio 4:3 4:3 4:3

NTSC: National Television Systems CommitteeNTSC: National Television Systems Committee

PAL: Phase Alternation LinePAL: Phase Alternation Line

SECAM: Séquential Couleur Avec MèmoireSECAM: Séquential Couleur Avec Mèmoire


HDTV and NTSC SpecificationsHDTV and NTSC Specifications

HDTV USA NTSC

Aspect ratio 16:9 4:3

Largest frame rate 60 frames/sec 30 frames/sec

Vertical refresh rate 60 Hz 60 Hz

Highest resolution 1080 lines 525 lines


Analog bandwidth of HDTV signals?Analog bandwidth of HDTV signals?

• HDTV image size of 1050 by 600 at 30 frames HDTV image size of 1050 by 600 at 30 frames per sec, the bandwidth required to carry that per sec, the bandwidth required to carry that image quality using the analog transmission image quality using the analog transmission system is 18MHz. However, it will require more system is 18MHz. However, it will require more bandwidth to transmit it in digital format.bandwidth to transmit it in digital format.

• With the MPEG-2 compression, the bit rate is With the MPEG-2 compression, the bit rate is compressed from more than 1 Gbps to about 20 compressed from more than 1 Gbps to about 20 Mbps, which transmit digitally only require Mbps, which transmit digitally only require bandwidth 6MHzbandwidth 6MHz

Architecture of HDTV ReceiversArchitecture of HDTV Receivers

DisplayProcessor

AudioDecoder

ImageDecoder

Demodulator Demultiplexer

Decoded video signals

Decoded audio signals

Display format

video signals

audio signals

digital signals

analog carrier

+digitalsignals


Aspect ratio of movies vs. HDTV?Aspect ratio of movies vs. HDTV?

• Aspect ratio of HDTV is 16:9Aspect ratio of HDTV is 16:9

• However, movies have many different aspect ratios:However, movies have many different aspect ratios:

““Movies are always shot so they can be displayed in several Movies are always shot so they can be displayed in several

aspect ratios at different types of movie theaters, from the aspect ratios at different types of movie theaters, from the shoebox-sized foreign movie houses to the ultra big screen Star shoebox-sized foreign movie houses to the ultra big screen Star Wars jobs.” ----- Franco VitalianoWars jobs.” ----- Franco Vitaliano

http://www.vxm.com/21R.107.htmlhttp://www.vxm.com/21R.107.html


Original Timeline of HDTVOriginal Timeline of HDTV

• First began in 60’s at NHK, the Japan Broadcasting Corporation.First began in 60’s at NHK, the Japan Broadcasting Corporation.

• In 1993, FCC suggested an alliance that could create the best possible systemIn 1993, FCC suggested an alliance that could create the best possible system

• November 1998: HDTV transmissions begin at 27 stations in the top 10 markets November 1998: HDTV transmissions begin at 27 stations in the top 10 markets

• May 1999: network affiliates in the top 10 markets must show at least 50% digital May 1999: network affiliates in the top 10 markets must show at least 50% digital programming programming

• November 1999: digital broadcasts in the next 20 largest markets November 1999: digital broadcasts in the next 20 largest markets

• May 2002: remaining commercial stations must convert May 2002: remaining commercial stations must convert

• 2003: public stations must convert to digital broadcasts 2003: public stations must convert to digital broadcasts

• 2004: stations must simulcast at least 75% of their analog programming on HDTV 2004: stations must simulcast at least 75% of their analog programming on HDTV

• 2005: stations must simulcast 100% of their analog programming 2005: stations must simulcast 100% of their analog programming

• 2006: stations relinquish their current analog spectrum 2006: stations relinquish their current analog spectrum

NTSC TV sets will no longer be able to pick up broadcast signalsNTSC TV sets will no longer be able to pick up broadcast signals


Spring 2001 StatusSpring 2001 Status

• 18 digital TV formats are approved by FCC18 digital TV formats are approved by FCC

• More than 27 digital channels being broadcast by ABC, More than 27 digital channels being broadcast by ABC, CBS, FOX, NBCCBS, FOX, NBC

• DirecTV has one HDTV channelDirecTV has one HDTV channel

• Cox is broadcasting two HDTV channelsCox is broadcasting two HDTV channels


Hardware RequirementsHardware Requirements

• Digital DecoderDigital Decoder• converts digital signals to analog converts digital signals to analog • allow current TV set to workallow current TV set to work

• Digital-Ready TV setDigital-Ready TV set• Wide-screen formatWide-screen format• progressive scanningprogressive scanning

• HDTV setHDTV set• Wide-screen formatWide-screen format• can receive 18 digital input formatcan receive 18 digital input format

ComparisonComparison

Current TV

HDTV

ComparisonComparison (current TV) (current TV)

Comparison (HDTV)Comparison (HDTV)


Video vs. computer (ROM) formatsSingle (R) and multiple (RAM) recordings possibleUp to 17 GB of data

• 12 cm optical disc format data storage medium• Replaces optical media such as

• the laserdisc• audio CD, • and CD-ROM.

• Will also replace VHS tape as a distribution format for movies

• MPEG-2 encoding

Digital Video Disc (DVD)Digital Video Disc (DVD)


DVD FeaturesDVD Features

• Language choice (for automatic selection of video scenes, audio tracks, subtitle Language choice (for automatic selection of video scenes, audio tracks, subtitle tracks, and menus). Optional tracks, and menus). Optional

• Special effects playback: freeze, step, slow, fast, and scan (no reverse play or Special effects playback: freeze, step, slow, fast, and scan (no reverse play or reverse step). reverse step).

• Parental lock (for denying playback of discs or scenes with objectionable Parental lock (for denying playback of discs or scenes with objectionable material). Optional material). Optional

• Programmability (playback of selected sections in a desired sequence). Programmability (playback of selected sections in a desired sequence).

• Random play and repeat play. Random play and repeat play.

• Digital audio output (PCM stereo and Dolby Digital). Digital audio output (PCM stereo and Dolby Digital).

• Compatibility with audio CDsCompatibility with audio CDs

• Digital ZoomDigital Zoom

• Six channel audioSix channel audio


MPEG-4MPEG-4

• MPEG 2 plusMPEG 2 plus

• Interactive Graphics ApplicationsInteractive Graphics Applications

• Interactive multimedia (WWW), networked distributionInteractive multimedia (WWW), networked distribution


MPEG-4MPEG-4

• Bitrates from 5kb/s to 10Mb/sBitrates from 5kb/s to 10Mb/s

• Several extension “profiles”Several extension “profiles”

• Very high quality videoVery high quality video

• Better compression than MPEG-1Better compression than MPEG-1

• Low delay audio and error resilienceLow delay audio and error resilience

• Support for “objects”Support for “objects”

• Face AnimationFace Animation

• Support for efficient streamingSupport for efficient streaming

• Limited industry activity at this pointLimited industry activity at this point

MPEG-4MPEG-4from: http://mpeg.telecomitalialab.com/standards/mpeg-4/mpeg-4.htmfrom: http://mpeg.telecomitalialab.com/standards/mpeg-4/mpeg-4.htm

MPEG-4MPEG-4


MPEG-7MPEG-7

• Data + Multimedia Content Description SchemeData + Multimedia Content Description Scheme

• Description Definition Language (XML-based)Description Definition Language (XML-based)

• Still not ‘final’, but closeStill not ‘final’, but close

• Does not deal with data, but meta-data transmissionDoes not deal with data, but meta-data transmission

• Description Scheme + Content Description, e.g:Description Scheme + Content Description, e.g:

• Table of contentTable of content

• Still ImagesStill Images

• SummariesSummaries

• linkslinks

• etc.etc.

• How does the Description data get generated? How is it used?How does the Description data get generated? How is it used?


Mpeg-7 ExamplesMpeg-7 Examples

<VideoText id="VideoText1" textType="Superimposed"><VideoText id="VideoText1" textType="Superimposed">

<MediaTime><MediaTime>

<MediaTimePoint> T0:0:0:0 </MediaTimePoint><MediaTimePoint> T0:0:0:0 </MediaTimePoint>

<MediaDuration> PT6S </MediaDuration><MediaDuration> PT6S </MediaDuration>

</MediaTime></MediaTime>

<Text xml:lang="en-us">CNN World News</Text><Text xml:lang="en-us">CNN World News</Text>

</VideoText></VideoText>

<TextProperty><TextProperty>

<FreeText xml:lang="en"> World Today </FreeText><FreeText xml:lang="en"> World Today </FreeText>

<SyncTime><SyncTime>

<MediaRelTimePoint>PT01N30F </MediaRelTimePoint><MediaRelTimePoint>PT01N30F </MediaRelTimePoint>

<MediaDuration> PT2S </MediaDuration><MediaDuration> PT2S </MediaDuration>

</SyncTime></SyncTime>

</TextProperty></TextProperty>

<Place><Place>


Mpeg-7 Examples Cont’dMpeg-7 Examples Cont’d

<Name xml:lang="en">Kabul</Name><Name xml:lang="en">Kabul</Name>

<GPSCoordinates type="latlon">69.137E 34.531N<GPSCoordinates type="latlon">69.137E 34.531N </GPSCoordinates> </GPSCoordinates>

<Country>Afghanistan</Country><Country>Afghanistan</Country>

<Region>Velayat</Region> <Region>Velayat</Region>

<AdministrativeUnit type="city"> Kabul </AdministrativeUnit><AdministrativeUnit type="city"> Kabul </AdministrativeUnit>

</Place></Place>

MPEG-7MPEG-7


MPEG-21 (Draft)MPEG-21 (Draft)http://mpeg.telecomitalialab.com/standards/mpeg-21/mpeg-21.htmhttp://mpeg.telecomitalialab.com/standards/mpeg-21/mpeg-21.htm


Video Compression StylesVideo Compression Styles

• Symmetric codecs require inverse operations to Symmetric codecs require inverse operations to decompress the format. decompress the format.

• Asymmetric codecs use different compression|Asymmetric codecs use different compression|decompression methods. More processing time is spent decompression methods. More processing time is spent in compressing to achieve low storage to allow for in compressing to achieve low storage to allow for shorter decompression time. shorter decompression time.


Other Compression SchemesOther Compression Schemes

• Quicktime (Apple), Video for WindowsQuicktime (Apple), Video for Windows• Open architecture allowing different codecsOpen architecture allowing different codecs

• Motion JPEG – no interframe compressionMotion JPEG – no interframe compression

• Cinepak is an asymmetric codec designed for 24-bit Cinepak is an asymmetric codec designed for 24-bit video in a 320 X 240 window for single-speed CD-ROM video in a 320 X 240 window for single-speed CD-ROM drives. Compression typically takes 300 times longer drives. Compression typically takes 300 times longer than decompression. than decompression.

• Indeo asymmetric codec (Intel). Playback can take Indeo asymmetric codec (Intel). Playback can take place on a Intel 486 processor without any hardware place on a Intel 486 processor without any hardware assistance. Less efficient than Cinepakassistance. Less efficient than Cinepak

• DVI Digital Video Interactive requires off-line DVI Digital Video Interactive requires off-line supercomputer processing power for the compression.supercomputer processing power for the compression.


QuickTimeQuickTime

• An ISO standard for digital mediaAn ISO standard for digital media

• Created by Apple Computer Inc., 1993Created by Apple Computer Inc., 1993

• Audio, animation, video, and interactive capabilities for PCAudio, animation, video, and interactive capabilities for PC

• Allows integration of MPEG technology into QuickTime. Allows integration of MPEG technology into QuickTime.

• QuickTime is available for MS Windows/NT as wellQuickTime is available for MS Windows/NT as well

• QuickTime movies have file extension .qt and .mov.QuickTime movies have file extension .qt and .mov.

• Description: Description: http://www.apple.com/quicktime/specifications.htmlhttp://www.apple.com/quicktime/specifications.html

• ftp://ftp.intel.com/pub/IAL/multimedia/indeo/utilities/smartv.exeftp://ftp.intel.com/pub/IAL/multimedia/indeo/utilities/smartv.exe

converts quicktime to avi and backconverts quicktime to avi and back

http://www.apple.com/quicktime/specifications.html


Video Players for your PCVideo Players for your PC

• To play a movie on your computer, you need a multimedia player To play a movie on your computer, you need a multimedia player • e.g. an MPEG player, WindowsMediaPlayer, RealPlayer or QuickTime player. e.g. an MPEG player, WindowsMediaPlayer, RealPlayer or QuickTime player.

• These players are also called decoders because they decode the MPEG or These players are also called decoders because they decode the MPEG or QuickTime, RealNetworks, etc. compressed codes. QuickTime, RealNetworks, etc. compressed codes.

• Some software allows you to both encode and decode multimedia files, e.g. to Some software allows you to both encode and decode multimedia files, e.g. to make and play the files. make and play the files. • You’ll use both for your digital video homework assignment.You’ll use both for your digital video homework assignment.

• Some software only allows you to play back multimedia files.Some software only allows you to play back multimedia files.

• When digitizing from a When digitizing from a VCRVCR, then the quality of the videotape recording and , then the quality of the videotape recording and playback process limits the quality the digital video capturing system can achieve. playback process limits the quality the digital video capturing system can achieve. Consumer grade recorders used should at least be SVHS, or Hi-8, to give adequate Consumer grade recorders used should at least be SVHS, or Hi-8, to give adequate quality of the computer representation.quality of the computer representation.

ReferencesReferences

• http://www.cato.org/pubs/regulation/reg16n4b.htmlhttp://www.cato.org/pubs/regulation/reg16n4b.html

• http://web-star.com/hdtv/faq.htmlhttp://web-star.com/hdtv/faq.html

• http://web-star.com/hdtv/perspective.htmlhttp://web-star.com/hdtv/perspective.html

• http://bock.bushwick.com/hdtv_ppt/http://bock.bushwick.com/hdtv_ppt/

• http://web-star.com/hdtv/history.htmlhttp://web-star.com/hdtv/history.html

• http://www.cnn.com/TECH/computing/9910/26/pc.hdtv.idg/http://www.cnn.com/TECH/computing/9910/26/pc.hdtv.idg/

• http://money.cnn.com/services/tickerheadlines/bw/222470357.htmhttp://money.cnn.com/services/tickerheadlines/bw/222470357.htm

http://www.cato.org/pubs/regulation/reg16n4b.html

http://web-star.com/hdtv/faq.html

http://web-star.com/hdtv/perspective.html

http://bock.bushwick.com/hdtv_ppt/

http://web-star.com/hdtv/history.html

http://www.cnn.com/TECH/computing/9910/26/pc.hdtv.idg/

http://money.cnn.com/services/tickerheadlines/bw/222470357.htm


ReferencesReferences

• MPEG-1 System LayerMPEG-1 System Layer

• MPEG-1 Video LayerMPEG-1 Video Layer

• MPEG-1 Audio MPEG-1 Audio LayerLayer

• Definition of Video TermsDefinition of Video Terms


Digital Video

That’s all for today

digital video carnegie mellon © © copyright 2001 michael g. christel and alexander g. hauptmann...

Documents