T.Sharon-A.Frank1

Multimedia

Need high Bandwidth

Analog vs. Digital

Multi-modality

Real-Time MM

T.Sharon-A.Frank

2

Major Issues

1. Size of Data Frame

2. Need High Bandwidth

3. Digital vs. Analog Representation

4. Multi-Modality Integration/Blending

5. Soft Real-Time Constraints

6. Need Group Communication

7. Need Synchronization

8. Need for Standards

T.Sharon-A.Frank

3

2 .Need High Bandwidth

VOICE RIVER

IMAGE RIVER

BANDWIDTH RIVER

BROADBAND

RAY

VIDEO R

IVER

DATA STREAM

LAN CREEK

MICROBROOK PC R

UN

T.Sharon-A.Frank

4

Channel Rates (Bits/Second)

1Gbps1Mbps1Kbps

Video conferencing

Graphical user interfaces

Video phone

Compressed videoPeripheral sharing

TV

Information, e-mail, voice mail

Remote file systemInquiry

File transfer

HDTVCDAlarm, pager Voice

Telemetry Fax

T.Sharon-A.Frank

5

Applications Requirements

Ses

sion

Dur

atio

n

Channel Rate

101

102

103

105

104

1 Min

1 Hr

1 Day

102 104 106 108 1010

1 KB/s 1 MB/s 1 GB/s

High SpeedData

Information Retrieval

Teleconferencing

CD Audio Entertainment Video

T.Sharon-A.Frank

6

Increasing Throughput RequirementsIn

crea

sing

Bur

stin

ess

Bandwidth64K 1.5M 45M 150M 600M

FRAMERELAY

DEDICATED PRIVATE LINE

ISDNSWITCHEDT-SERIES

SMDS

B-ISDN

X.25PACKET

SWITCHING

DIAL-UP

T.Sharon-A.Frank

7

LAN/WAN fit for MM?

MMLAN/WAN

Slow Fast

Need Fiber Optics, Broadband,High-Speed Networks (HSN)

T.Sharon-A.Frank

8

Solutions

Use High Speed Networks (HSNs)

Use Lossy Compression

Show/See Flash!

T.Sharon-A.Frank

9

3 .Digital vs. Analog Representation

Analog Sound Wave Digital Sample “Quantized” Sound Wave

Original SoundOriginal Sound Digital SamplingDigital Sampling Digital PlaybackDigital Playback

10

• Only certain signal values are valid:

– Relatively immune to corruption by noise.

– Do not degrade when copied or transmitted over network.

• Some information lost.

• Undersampling

– Samples 'too far apart' so cannot accurately reconstruct original signal.

Digital Signals

T.Sharon-A.Frank

11

Solutions

Use Standards

Use Special Hardware (AD/DA)

T.Sharon-A.Frank

12

4 .Multi-Modality Integration/Blending

T.Sharon-A.Frank

13

Multimedia Input

Eye Tracker

Speech Recognizer

Data Glove

Put that there!

T.Sharon-A.Frank

14

Multimedia Output

Maps

0

10

20

30

40

50

60

70

80

90

MIG-27 MIG-29 MIG-31

Speed

MIG-31MIG-29MIG-27720650490SP

FashionedFulcrumFlugger-DMODEL

Data

Graphs

Natural Language

Aircraft Speed Name

MIG-27 490 Flogger-D

MIG-29 650 Fulcrum

MIG-31 720 Fashioned

Tables

Video

There are three types ofSoviet fighters: the MIG-27Flogger-D that travels 490miles per hour, the MIG-29Fulcrum that...

T.Sharon-A.Frank

15

Modality Blending

Syntax

Semantics

Pragmatics

Application

Mode 1 Mode 2 Mode 3 Mode n...

T.Sharon-A.Frank

16

Multimedia-Multimodal Interface

Syntax

Semantics

Pragmatics

Application

Mode 1 Mode 2 Mode 3 Mode n...

T.Sharon-A.Frank

17

5 .Soft Real-Time Constraints

T.Sharon-A.Frank

18

Hardware/Software Pipeline

ScheduleCompress

Compression

TransmitInitiate

TransmitNetworkDriver

Transmission

DigitizeScheduleDigitize

Software

Hardware

16.5ms 33ms 16.5ms 20ms

20ms 1ms 16.5ms ??Depends on network load

Transmitting Video Frame

T.Sharon-A.Frank

19

Solutions

Use Standards

Use Quality of Service (QoS)