1

Introduction to Introduction to Multimedia NetworkingMultimedia Networking

2

What is Multimedia ?What is Multimedia ? Information Perception from External World

Scene: 60% Sound: 20% Touch(feel): 15% Taste: 3% Smell: 2%

Media: means of transport information(digital data) text, audio, image, graphic, video

Multimedia combination of two or more media

Persistent vs Non-persistent(live) information

3

What is Multimedia CommunicationsWhat is Multimedia Communications Multimedia System

a computer system that can accept multimedia information from the external world

Multimedia Applications Local Multimedia Application Distributed Multimedia Application

Multimedia Communications concerns the technology required to manipulate,

transmit, and control multimedia across a networked communication channel

4

Digital Media DevicesDigital Media Devices

CaptureASCII keyboardMIDI keyboardImage scanner3D digitigerVideo frame grabberVideo digitizerAudio digitizer

PresentationFrame buffer, display driverPrinterMIDI synthesizerAudio digital-to-analog converter

ProcessingVideo encoder/decoderAudio encoder/decoderDigital video effects deviceDigital audio effects device3D graphics hardware

5

Analog Media devicesAnalog Media devices

SourcesMicrophoneVideo cameraVideo tape playerAudio tape playerVideodisc playerPhotographic camera

SinkSpeakerVideo displayVideo tape recoderAudio tape recoder

FiltersAnalog video effects deviceAnalog audio effects deviceAudio mixerVideo scan converterVideo overlay device

6

Continuous and Discrete MediaContinuous and Discrete Media Discrete media

time independent media e.g. : text/data, image, graphics

Continuous(temporal) media time-dependent media (real time media) requires continuous playout as time passes e.g : audio, video

7

Continuous MediaContinuous Media

Audio Video

8

Media SynchronizationMedia Synchronization

intra-media synchronization: e.g. audio inter-media synchronization: e.g. movie

9

Why Digital?Why Digital? Analog reality, digital approximations. Conversion from A-to-D, and D-to-A Sampling, Quantization + Encoding Process Advantages of digital:

Cost Reliability High-speed (electronic) storage Programmability

10

How often sample ?How often sample ?

Nyquist theorem f0: max. frequency component in analog signal If sampling frequency 2 f0, original analog signal

can be regenerated.

11

Quantization and EncodingQuantization and Encoding

12

Bit Rate - Digital Audio FormatBit Rate - Digital Audio Format

Format SamplingRate(kHz)

Bandwidth(kHz)

FrequencyRange (Hz)

Bit rate(kbps)

Telephony 8.0 3.0 200 – 3,200 64Teleconferencing 16.0 7.0 50 – 7,000 256

Compact disk 44.1 20.0 20 – 20,000 1,410Dig Audio Tape 48.0 20. 20 – 20,000 1,536

13

Still ImageStill Image Pixel(picture element) examples

black and white: 1bit monochrome: 8bits color: 24bits

PC monitor(1024 pixels x 768 lines) bit map 1024 x 768 x 24 = 18.874 Mbits Transmission time over 14.4Kbps modem: 21.84min

Hints Send the image over a faster channel Reduce the number of bits per pixel Reduce the resolution of the display Remove the redundancy in the display

Image vs. Graphics ?

14

Moving Graphics and ImagesMoving Graphics and Images

Animation: moving graphics Video: motion pictures

temporal composition of spatial images(frames) frame rate suitable for human perception: 25-30 fps

e.g. CIF(Common Intermediate Format) video 30 fps 1 frame: 360 pixels/line, 288 lines/picture 24 bit/pixel # of bits/sec: 360 x 288 x 24 x 30 = 74.65Mbps

15

Human being can sense analog signal

Compression technique employs human perception behavior

Encoding/Decoding for Digital Encoding/Decoding for Digital TransmissionTransmission

16

Signal Compression CapabilitiesSignal Compression Capabilities

17

Compression Standards for MM (1)Compression Standards for MM (1)

GIF (Graphics Interchange Format) For raster graphical data and annimation

JPEG For continuous tone images

MPEG MPEG-1: coding of moving pictures and associated

audio MPEG-2: generic coding of moving pictures and

associated audio MPEG-4: coding of audiovisual objects MPEG-7: MM contents description MPEG-21: MM framework

18

Compression Standards for MM (2)Compression Standards for MM (2)

ITU-T Audiovisual systems: H.310, H.320, H.321, H.322,

H.323, H.324 Video coding standards: H.261, H.262, H.26L Speech coding standards: G.7xx Multimedia Multiplex and Synchronization: H.221,

H.223, H.225, H.245

19

Multimedia Computer and SoftwareMultimedia Computer and Software

Multimedia PC Multimedia system software

Microsoft Video for Windows ActiveMovie DirectShow

Apple’s QuickTime Java Media Framework (JMF)

20

Multimedia Generation & DeliveryMultimedia Generation & Delivery

21

Evolution of Network ServiceEvolution of Network Service

Work in officeBoard GamesLibrary researchStore Shopping

Work on home PCElectronic GamesCD-ROM researchCD-ROM shopping

Telecommute/Desktop collaborateMultiplayer interactive gamesOnline service researchInternet Shopping

Physical Electronic Networked

Contents Digitize network

22

Networks for Multimedia: ExampleNetworks for Multimedia: Example

23

Multimedia CommunicationsMultimedia Communications

Distributed multimedia applications require multimedia networks

Charateristics of Multimedia Networks Broad Bandwidth Multicasting Real-time Constraints Reliability QoS

Multimedia over IP Mbone: Multicast Backbone Protocols: RSVP, RTP, ST2 Applications

Cu-SeeMe Internet Talk Radio

Other audio-visual tools: vat, nv, ivs, ...

24

Factors That Affects Network Factors That Affects Network Performance (1)Performance (1) Throughput

effective bit rate = physical link speed - overhead Physical link speed throughput vs. bandwidth Throughput of ATM network ?

Error Rate BER : bit error rate

10-9~10-12 : optical link 10-7 : satellite link

PER : packet error rate FER : frame error rate

25

Factors That Affects Network Factors That Affects Network Performance(2)Performance(2) Delay(latency)

End-to-End Delay Transit Delay (propagation delay) Transmission Delay : to transmit a block of data

(including routing and buffering) Network Delay = Transit delay + Transmission Delay Interface Delay

Round Trip Delay Delay Variation or Jitter

Important factor in designing a MM network

26

MM Traffic Source CharacteristicsMM Traffic Source Characteristics Throughput Variation with time

Constant Bit Rate (CBR) Fixed Bandwidth Intended for real-time applications

Variable Bit Rate (VBR) Transmit rate varies with time (bursty) Intended for Voice/Video

Unspecified Bit Rate (UBR) Intended for non-critical applications

Time Dependency Bidirectional Symmetry

Symmetric or Asymmetric

27

MM Traffic Requirement for Networks (1)MM Traffic Requirement for Networks (1) Throughput Requirement

High bandwidth requirement High storage bandwidth requirement Streaming requirement

Reliability(error control) Requirement Error control End-to-end delay Tolerance of transmission network errors

audio, video: tolerable due to human perception audio is more sensitive that video text: usually not tolerable

Delay is more important than error in continuos media

28

MM Traffic Requirement for Networks (2)MM Traffic Requirement for Networks (2) Delay Requirement

Multimedia connection may consist of multiple streams

Type of connection Asynchronous : no upper limit on the delay Synchronous : two streams transmit at the same rate and

arrive at the same time Isochronous : small bounded delay

Delay variation is important

29

Quality of Service RequirementQuality of Service Requirement

How Much Bandwidth is Enough? How Much Delay is Acceptable?

interactive applications: 100-300 ms (one-way) non-interactive playback: seconds or minutes

How Much Jitter is Acceptable? buffer space and delay limitations

What Error or Loss Rate is Acceptable? bit Errors, packet errors, and frame errors effect of compression

QoS requirement can be expressed in QoS parameters

30

QoS ParametersQoS Parameters

Traffic throughput(bandwidth) Delay Jitter Transmission reliability Synchronization

31

QoS ProvisionQoS Provision

Traffic shaping Resource reservation and scheduling Resource negotiation

Admission control Policing

32

QoS GuaranteesQoS Guarantees

Traffic "Burstiness" or Variability peak rate, average rate, maximum burst size

Deterministic (100%) Guarantees based on peak traffic rate simple, predictable, conservative

Statistical (< 100%) Guarantees based on peak and mean traffic rates complex, less predictable, higher utilization

No Guarantees "best effort" service

33

Multimedia in NetworksMultimedia in Networks Streaming stored MM

Clients request audio/video files from servers and pipeline reception over the network and display

Interactive: user can control operations (similar to VCR) Delay: from client request until display start can be 1 to 10

seconds Unidirectional Real-time

Similar to existing TV and radio stations, but delivery over the Internet

Non-interactive, just listen/vies Interactive Real-time

Phone or video conference More stringent delay requirement than Streaming &

Unidirectional Video: < 150 msec acceptable Audio: < 150 msec good, < 400 msec acceptable

34

Applications of MM CommunicationsApplications of MM Communications Entertainment

VoD Interactive games Interactive program guide Home shopping

Education Cyber university Distance learning

Industrial/business CAD/CAE Video conferencing Multimedia banking

Medicine Telemedicine Remote diagnosis