ee442—multimedia networking jane dong california state university, los angeles

EE442—Multimedia NetworkingEE442—Multimedia Networking

Jane Dong

California State University, Los Angeles

EE442 Multimedia Networking 2

Part IPart I

Introduction to Multimedia NetworkingIntroduction to Multimedia Networking


OverviewOverview

What is MultimediaWhat is Multimedia?

Characteristics of multimedia

Various media types

What is Multimedia networkingWhat is Multimedia networking?

User requirements of multimedia applications on the network

Technologies associated with multimedia networking

Overall structure of multimedia networking


What is multimedia?What is multimedia?

Definition of multimediaHard to find a clear-cut definition

In general, multimedia is an integration of text, graphics, still and moving images, animation, sounds, and any other medium where every type of information can be represented, stored, transmitted and processed digitally

Characteristics of multimediaDigital – key concept

Integration of multiple media type, usually including video or/and audio

May be interactive or non-interactive


Various Media TypesVarious Media Types

Text, Graphics, image, video, animation, sound, etc.

Classifications of various media typesCaptured vs. synthesized media

Captured media (natural) : information captured from the real world

Example: still image, video, audio

Synthesized media (artificial) : information synthesize by the computer

Example: text, graphics, animation

Discrete vs. continuous mediaDiscrete media: spaced-based, media involve the space dimension only

Continuous media: time-based, media involves both the space and the time dimension

(Text, Image, Graphics)

(Video, Sound, Animation)


Classification of Media TypeClassification of Media Type

SoundSound VideoVideo

ImageImage

AnimationAnimation

TextText GraphicsGraphics

Captured From real world

Synthesized By computer

Discrete Discrete

Continuous Continuous


TextText

Plain textUnformatted

Characters coded in binary form

ASCII code

All characters have the same style and font

Rich textFormatted

Contains format information besides codes for characters

No predominant standards

Characters of various size, shape and style, e.g. bold, colorful


Plain Text vs. Rich TextPlain Text vs. Rich Text

An example of Plain textAn example of Plain text Example of Rich textExample of Rich text


GraphicsGraphics

Revisable document that retains structural information

Consists of objects such as lines, curves, circles, etc

Usually generated by graphic editor of computer programs

-4-2

02

4

-4

-2

0

2

4-10

-5

0

5

10

Example of graphics (FIG file)


ImagesImages

2D matrix consisting of pixelsPixel—smallest element of resolution of the image

One pixel is represented by a number of bits

Pixel depth– the number of bits available to code the pixel

Have no structural information

Two categories: scanned vs. synthesized still image

Computer software

Computer software

Capture and A/D conversionCapture and

A/D conversion

Digital still imageDigital still image

Synthesizedimage

Scannedimage

Camera


Images (cont.)Images (cont.)

Examples of imagesBinary image – pixel depth 1

Gray-scale – pixel depth 8

Color image – pixel depth 24

Binary image

Gray-scale imagecolor image


Graphics vs. ImageGraphics vs. Image

GraphicsRevisable documents

Document format retains structural information

Semantic content is preserved in presentation

Described as objects

ImagesNot revisable

Document format is unaware of any structural information

Semantic content is NOT preserved

Described as bitmaps formed of individual pixels


Video vs. AnimationVideo vs. Animation

Both images and graphics can be displayed as a succession of view which create an impression of movement

Video – moving images or moving picturesCaptured or Synthesized

Consists of a series of bitmap images

Each image is called a frame

Frame rate: the speed to playback the video (frame per second)

Animation – moving graphicsGenerated by computer program (animation authoring tools)

Consists of a set of objects

The movements of the objects are calculated and the view is updated at playback


SoundSound1-D time-based signal

Speech vs. non-speech soundSpeech – supports spoken language and has a semantic content

Non-speech – does not convey semantics in general

Natural vs. structured soundNatural sound – Recorded/generated sound wave represented as digital signal

Example: Audio in CD, WAV files

Structured sound – Synthesize sound in a symbolic wayExample: MIDI file

0 100 200 300 400 500 600 700 800 900 1000-0.2

-0.15

-0.1

-0.05

0

0.05

0.1

0.15

0.2


Networked MultimediaNetworked MultimediaLocal vs. networked multimedia

Local: storage and presentation of multimedia information in standalone computers

Sample applications: DVD

Networked: involve transmission and distribution of multimedia information on the network

Sample applications: videoconferencing, web video broadcasting, multimedia Email, etc.

InternetInternetVideo server

Image serverA scenario of multimedia networking


Consideration of Networked MultimediaConsideration of Networked Multimedia

Characteristics of multimedia informationLarge data volume

Exercise: What is the size of a video clip of 60 minutes if the frame size is 640*480, the pixel depth is 24, and the frame rate is 24 fps?Real-time property

Continuous displayDelay requirement of multimedia applications

Properties of current InternetLimitation of bandwidthBest effort network, cannot guarantee quality of multimedia applicationsHeterogeneity

Different user requirementsDifferent user network conditions


Consideration of Networked MultimediaConsideration of Networked Multimedia

Requirements of multimedia applications on the networkDelay requirement

Quality requirementSatisfactory quality of media presentation

Synchronization requirement

Continuous requirement (no jerky video/audio)

Can tolerant some degree of information loss

Challenges of multimedia networkingConflict between media size and bandwidth limit of the network

Conflict between the user requirement of multimedia application and the best-effort network

How to meet different requirements of different users?


Technologies of Multimedia NetworkingTechnologies of Multimedia Networking

Media compression – reduce the data volumeAddress the1st challenge

Image compression

Video compression

Audio compression

Multimedia transmission technologyAddress the 2nd and 3rd challenges

Protocols for real-time transmission

Rate / congestion control

Error control


Multimedia Networking SystemsMultimedia Networking Systems

Live media transmission systemCapture, compress, and transmit the media on the fly (example?)

Send stored media across the networkMedia is pre-compressed and stored at the server. This system delivers the stored media to one or multiple receivers. (example?)

Differences between the two systemsFor live media delivery:

Real-time media capture, need hardware support

Real-time compression– speed is important

Compression procedure can be adjusted based on network conditions

For stored media deliveryOffline compression – better compression result is important

Compression can not be adjusted during transmission


Generic Media Streaming SystemGeneric Media Streaming System

VideoEncoder

Input videoCompressed

Video StreamingServer

InternetInternet

ReceiverVideoDecoder

Video Display

Error control, ratecontrol can be donehere to improve QoS

Error control, providefeedback to the sender

Compressed VideoVideo Packets

ee442—multimedia networking jane dong california state university, los angeles

Documents

image camera slide

animation slide

synthesized image

integration of text

colorful slide

media natural

ee442multimedia networking2

ee442multimedia networking4