Dr.B.Yogameena

D6GB Multimedia Systems

Faculty: Dr.B.Yogameena (ymece@tce.edu)

Outline for todayHighlevel introduction to multimedia systems

To discuss…

Course Introduction Multimedia – Definitions Multimedia - Applications Multimedia Data Hypermedia Digital Technology

Text Books: 1. Ze-Nian Li and Mark S. Drew, “Fundamentals of

Multimedia”, Pearson Prentice Hall, October 2003.2. K. Rammohanarao, Z. S. Bolzkovic and D. A.

Milanovic, “Multimedia Communication Systems”, Prentice Hall, May 2002.

3. Yao Wang, Joern Ostermann, and Ya-Qin Zhang, “Video Processing and Communications”, Prentice Hall, 2002.

4. Fred Halsall, “Multimedia Communications: Applications, Networks, Protocols and Standards”, Addison-Wesley, 2001.

Multimedia- ApplicationsMultimedia plays major role in following

areasInstructionBusiness

Advertisements Training materials Presentations Customer support services

Entertainment Interactive Games

Multimedia- Applications

Enabling Technology Accessibility to web based materials Teaching-learning disabled children & adults

Fine Arts & Humanities Museum tours Art exhibitions Presentations of literature

Multimedia- Applications

In Medicine

Source: Cardiac Imaging,YALE centre for advanced cardiac imaging

Multimedia- Applications

In training

Multimedia- Applications

Public awareness campaign

SourceInteractive Multimedia Project Department of food science& nutrition, Colorado State Univ

Example Multimedia ApplicationsVideo teleconferencing, distributed lectures,

telemedicine, tele symphonyWhite board, collaborative document editingAugmented realityDVDs, digital movies, VOIP telephony (Vonage,

Skype) …Networked gamesVideo on demand (from cable TV, satellite etc.),

IPTV (AT&T U-verse)Can you think of more applications?

YouTube.com, founded in Feb 2005Every minute, 10 hours of video is uploaded

Definition: MultimediaMultimedia means the computer

information/data being transferred over the network which is composed of one or more modality where every type of information/data can be represented, stored, transmitted and processed digitally to enrich its content and enhance communication.

Systems operating on multiple modalities: text, audio, images, drawings, animation, video etc.

Sychronizing multiple modalities is important and hard

Data Types in MM system

Text Data

ASCII stands for American Standard Code for Information

Interchange. Computers can only understand numbers, so an

ASCII code is the numerical representation of a character.

This included both unformatted text, comprising strings of characters from a limited character set, and formatted text strings as used for the structuring, access and presentation of electronic documents.

Text Data

Figure 2.1 ASCII table and description (copyright: www.asciitable.com)

Text Data

ASCII uses 7 bits to represent a character. As a result only 127

characters are defined as standard ASCII characters. Characters 128-

255 are called extended ASCII characters.

Figure 2.2 Extended ASCII codes (copyright: www.asciitable.com)

Text Data

EBCDIC

EBCDIC (Extended Binary Coded Decimal Interchange

Code) is a character set used on early IBM computers.

EBCDIC was first introduced in 1965, it was the new

character-coding scheme came with IBM System 360 series.

EBCDIC uses 8 bits to represent a character.

Text Data

UNICODE

The Unicode character uses 16 bits to represent a character, thus more than 65000 characters can be represented. While 65000 characters are sufficient for encoding most of the many thousands of characters used in major languages of the world.

Sound Data

Sound Data

A typical compact disc can hold up to 74 minutes of 16 bit,

44.1 kHz audio that is uncompressed – about 650

megabytes.

Sound Data

Table shows how the size of a file is affected by the sampling rate and

bit length. The file is a one-minute sound clip, recorded and saved in

various forms in the Microsoft Windows WAV file format.

Quality Sampling Rate Resolution File Size

CD 44 kHz 16 bit Stereo 10.3 MB

44 kHz 8 bit Stereo 5.18 MB

FM Radio 22 kHz 16 bit Stereo 5.18 MB

22 kHz 8 bit Stereo 2.59 MB

AM Radio 11 kHz 16 bit Stereo 2.59 MB

11 kHz 8 bit Stereo 1.29 MB

Table 2.1 Variation of file size and sampling rate (60 seconds audio clip in MS WAV format)

Image Data

Image Data

Images, or pictures, are two-dimensional arrays of data

called bitmaps, with each element is called pixel.

Image Data

UnitsDpi - Dots Per InchBit Depth - The number of bits used to hold a pixel. Also

called color depth and pixel depth, the bit depth determines

the number of colors that can be displayed at one time.

Color Depth Number of Colors

4 bits 16

8 bits 256

16 bits 65,536

24 bits 16,777,216

Table Color Depths

Video Data

Video Data

Video, or moving images, is a sequence of images.

To create a sense of continuity, video must be played at a

rate of at least 25 frames per second (fps).

MultimediaMultimedia Hardware Hardware PeripheralsPeripheralsInput devices

Output devices

Storage devices

Communication devices _Modems

_Network Interfaces

Input DevicesInput DevicesKeyboards And MiceScanners And Digital camerasMIDI Keyboards Touch screens Trackballs Tablets

Continue…Continue…Voice recognition systems

Infrared remotes

Continue…Continue…Magnetic Card Encoders And Readers

Video cameras

Output DevicesOutput DevicesMonitorsSpeakersVR helmet and VR immersive displayVideo Devices

Storage DeviceStorage DeviceSyquest drives

Continue…Continue…CD-ROM Drives

Magneto-optical drives

Laserdisc Player

To discuss…

Course Introduction Multimedia – Definitions Multimedia - Applications Multimedia Data Hypermedia Digital Technology

Hypermedia and MultimediaA hypertext system: meant to be read

nonlinearly, by following links that point to other parts of the document, or to other documents

• HyperMedia: not constrained to be text-based, can include other media, e.g., graphics, images, and especially the continuous media – sound and video. The World Wide Web (WWW) — the best

example of a popular hypermedia application.

f(x,y) = reflectance(x,y) * illumination(x,y)Reflectance in [0,1], illumination in [0,inf]

Original 8-bit image,256 gray levels

Quantized to 6 bits ,64 gray levels

Quantized to 3 bits ,8 gray levels

Quantized to 1 bits ,2 gray levels

Quantization-False Contouring

Problem: Limited bandwidthNeed for compressionAudio

CD quality: 44100 samples per seconds with 16 bits per sample, stereo sound

44100*16*2 = 1.411 MbpsFor a 3-minute song: 1.441 * 180 = 254 Mb =

31.75 MBVideo

For 320*240 images with 24-bit colors320*240*24 = 230KB/image15 frames/sec: 15*230KB = 3.456MB3 minutes of video: 3.456*180 = 622MB

Discrete Cosine Transform

DCT converts the information from spatial domain to frequency domain

Consider an unsorted list of 12 numbers between 0 and 3 -> (2, 3, 1, 2, 2, 0, 1, 1, 0, 1, 0, 0). Consider a transformation of the list involving two steps

sort the list Count the frequency of occurrence of each of

the numbers (4,4,3,1 ) spatial info lost, captured freq. info

Discrete Cosine Transform (DCT)Discrete Cosine Transform (DCT)

This is DCT

DCT is an orthogonal transformm so its inverse DCT is an orthogonal transformm so its inverse kernel is the same as forward kernelkernel is the same as forward kernel

This is inverse DCT

100

1

00

1

001

bygiven ,matrix al tridiagonsymmetric the

of orseigen vect theare DCT theof vectorsbasis 3.The

2-MEPG 1,-MPEG JPEG,

coding transformcompactionenergy excellent .2

.orthogonal and real is DCT 1.The

DCT theof Properties

1

c

c

Q

Q

tCCCC

Properties of DCT: real, Properties of DCT: real, orthogonal, energy-orthogonal, energy-

compacting, compacting, eigenvector-eigenvector-basedbased

• Each 8x8 block can be looked at as a weighted sum of these basis functions. • The process of 2D DCT is also the process of finding those weights.

DCT: Why does it do this?DCT takes advantage of redundancies in the

data by grouping pixels with similar frequencies togetherHigher frequencies = lower numberLower frequencies = higher number

If lossy compression is acceptable, then each data unit can then be divided by quantization coefficient (QC)

Zig Zag Scan

……..

8X8

1X64

• To group low frequency coefficients in top of the vector

• Maps 8 x 8 to a 1 x 64 vector.

DCT (cont)

DPCM on DC Components

The DC component value in each 8x8 block is large and varies across blocks, but is often close to that in the previous block.

Differential Pulse Code Modulation (DPCM): Encode the difference between the current and previous 8x8 block.

RLE on AC ComponentsThe 1x64 vectors have a lot of zeros in

them, more so towards the end of the vector

Encode a series of 0s as a (skip,value) pair, where skip is the number of zeros and value is the next non-zero componentSend (0,0) as end-of-block value

using −415 (the DC coefficient) and rounding to the nearest integer

DCT

Subtracting -128

quantization

8X8 block of pixel values taken from original image

(16 is the value of the first pixel from quantization matrix)

−26 −3 0 −3 −2 −6 2 −4 1 −4 1 1 5 1 2 −1 1 −1 2 0 0 0 0 0 −1 −1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Zig-zag scan

−26 −3 0 −3 −2 −6 2 −4 1 −4 1 1 5 1 2 −1 1 −1 2 0 0 0 0 0 −1 −1 EOB

Huffman coding