iee 5037 multimedia communications lecture 1:...

IEE 5037 Multimedia CommunicationsLecture 1: Introduction

Dr. Tian-Sheuan Changtschang@twins.ee.nctu.edu.twDept. Electronics EngineeringNational Chiao-Tung University

Dept. Electronics Engineering,N

ational Chiao T

ung University Special Thanks to Prof. Hang to provide most of the slides

in this lecture, except highlighted by other sources.

2

Chapter 1 IntroductionWhy need compression

How compression possible

Principles of Compression Techniques

Multimedia Standards

Subjects to be discussed

3

Why Compression?

-- Massive dataSpeech: 8 bits (per sample) x 8K (samples/sec) = 64Kbits/sCD audio:

16 bits x 44.1K (samples/sec) x 2 (channels) = 1.411Mbits/sec(44.1K = 60 (fields) x 245 (lines) x 3 (samples) (J. Watkinson,The Art of Digital Audio, p.28, Focal Press, 1989) )

Digital TV: (4:2:2, NTSC in CCIR 601)Still picture: 720 (pels) x 483 (lines) x 2.0 bytes = 5.564 MbitsMotion picture: 5.564 Mbits x 29.97 (frames/sec) = 167Mbits/sec

Digital HDTV: (ATSC)1920 (pels) x 1080 (lines) x 1.5 bytes x 30 (frames/sec)=746Mbits

4

Picture: “Hotel“, 720x576, 414,720 Byte

Source: T. Wiegand “Digital image communication”

5

JPEG-2000 Compressed to 12,960 Byte

Source: T. Wiegand “Digital image communication”

6

JPEG-2000 Compressed to 3,240 Byte

Source: T. Wiegand “Digital image communication”

7

The Power of Compression

Source: T. Wiegand “Digital image communication”

8

Digital Image Formats

Source: T. Wiegand “Digital image communication”

9

Digital Image Formats

Source: T. Wiegand “Digital image communication”

10

Digital Image Formats and Applications

Source: T. Wiegand “Digital image communication”

11

Examples of Storage Media

Source: T. Wiegand “Digital image communication”

12

Examples of Transmission Media

Source: T. Wiegand “Digital image communication”

13

An Example of Comm. Systems< A Digital TV System >

Our focus

Source: T. Wiegand “Digital image communication”

14

Examples of Data Sources-- Classified based upon compression techniques

General digital data: various types of computer filesTextSpeech: human conversationAudio: musicBi-level images: fax, dithered (two-tone) imagesStill gray-level pictures: graphics, natural imagesMotion pictures: video conferencing, television.

-- Data characteristics can be used to reduce bit rate.

15

How Compression Possible?Characteristics of data:

Stationary statistical model -- Shannon information theoryNon-stationary properties such as local correlation

Characteristics of human perception:Finite resolution of hearing and visionAuditory masking effectColor representationVisual masking effect

16

Signals (Waveforms)

Time/ Space

Amp.

Analog Signals x(t)

Conti. Conti.

Discrete-time (discrete-space) (sampled-data) signal x(m)

Discrete Conti.

Digital signals x(m)

Discrete Discrete

17

Audio SamplesPiano (fur Elise) samples and spectrogram (McClellan et al., DSP First, Prentice-Hall, 1998)

18

Data Compression TechniquesInformation lossless: Redundancy reduction ─ The

original data can be completely recovered.Direct: Huffman codes, arithmetic coding, Ziv-Lempel coding, … (narrow-sense data compression)Predictive: Run length coding, …

Information lossy: Information (entropy) reduction --The reproduced data are approximations of the original data. This may not be meaningful for a computer file.Block coding: vector quantization, transform coding, …Sequential: DPCM, tree coding, …Multi-resolution (non-block): sub-band, wavelet, ...

19

Compression TechniquesWaveform coding: Reproduce waveform,e.g., DPCM, transform, sub-band, …-- Universal but lower efficiency.Content-based coding: Reproduce contents, e.g., (speech) vocoder, (image) contour-texture coding, (video) model-based coding.International standards: JPEG, MPEG, H.261/3, …

20

Compression Algorithm Development

Phase 1: Modeling. Describe redundancy in the form of a model. E.g., prob. distribution, Markov model, …--Techniques are then invented to remove redundancy; e.g.,: predictor, transform, …Phase 2: Coding. Representations of “model”and “residual” (unpredictable) information. --Techniques: e.g., quantization and entropy coding of the prediction errors, …

21

Elements in Waveform CodingDecorrelation: Reduce spatial and temporal redundancy. Techniques: prediction, transform, …Selecting representatives: Reduce the number of possible signals. Techniques: quantization, …Entropy coding: Equalize the probability distribution of the output symbols. Techniques: Huffman codes, Ziv-Lempel coding, ...

22

Multimedia Coding Standards

Complete, practical coding algorithms-- A balance between (compression) performance and (implementation) complexity (and a compromise among various interest parties)Critical for telecommunication products and consumer audio/video media products

23

Standards Organizations

CCITT – Comité Consultaitif International Télégraphique et Téléphonique (International Telegraph and Telephone Consultative Committee) ITU – International Telecommunication UnionISO – International Standardization

OrganizationIEC – International Electrotechnical

Commission

24

Image/Video StandardsISO/IEC JTC1 SC29 – ISO and IEC Joint Technical Committee (on Information Technology) Subcommittee 29 (Coding of audio, picture, multimedia and hypermedia)– Working Group (WG) 1:

JBIG (Joint Bi-level Image Group) – 1-bit to 4/5-bit still pictures

JPEG (Joint Photographic Experts Group) – 8-bit or more still picturesISO/IEC JTC1 SC29– WG 11: MPEG (Moving Picture Experts Group) – Motion pictures– WG 12: MHEG (Multimedia-Hypermedia Experts Group) –Multi/Hyper-media exchange format

25

Video Coding StandardsStandards Typical rates ApplicationsITU-T (CCITT) H.261 128 384k bits/s Videophone over ISDN ISO MPEG-1 (11172-2) 1.2 Mbits/s Video CDISO MPEG-2 (13818-2) 4–10 Mbits Digital TV/HDTV (ITU-T H.262) 20 Mbits/s Over air/networksITU-T H.263 < 64k bits/s Videophone over PSTNISO MPEG-4 (14496-2) Low/high-rates Object-orientedISO MPEG-7 (15938) Database Content descriptionITU-T H.263 v2 < 64k bits/s PSTN/wireless VideophoneITU-T H.264 (JVT,AVC) < 40k bits/s Net/wireless Videophone

ISDN: Integrated Services Digital NetworkPSTN: Public Switched Telephone Network

26

MPEG Audio StandardsStandards (Year) Coding rates Sampling rates/

(channel)

MPEG-1 Layer 1 (92) 32k – 448 kbits/s 32, 44.1, 48k (1-2 chs)

MPEG-1 Layer 2 (92) 32k – 384 kbits/s 32, 44.1, 48k (1-2 chs)

MPEG-1 Layer 3 (93) 32k – 320 kbits/s 32, 44.1, 48k (1-2 chs)

MPEG-2 Layers 1,2,3 (94) 32, 44.1, 48, 16, 22.05, 24k (1-5.1)

MPEG-2 AAC (97) 8-64 kbits/s/ch 8, …, 96k (1-96 chs)

MPEG-4 AAC (99) 8-64 kbits/s/ch 8, …, 96k (1-96 chs)

27

Speech Coding StandardsStandards Typical rates (Year) Quality: MOS (1-5)PCM 64 kbits/s (1972) 4.4 (PSTN)G.721 ADPCM 32 kbits/s (1984) 4.1 (PSTN)GSM 13 kbits/s (1991) 3.6 (Cellular) G.728 (low delay) 16 kbits/s (1992) 4.0IS-96A (CDMA) 0.8-8.55 kbits/s (1993) ~3.4 (Cellular) G.729 8 kbits/s (1995) ~4.2G.723.1 5.3, 6.3 kbits/s (1995) ~4.0Half-rate GSM 5.6 kbits/s (1995) ~3.4 (Cellular) AMR 5.15-12.2 kbits/s (1999) ~3.9 (3GPP)

MOS: Mean Opinion Score -- 5=excellent, 4=good, 3=fair, 2=poor, 1=bad

28

Subjects To Be DiscussedBasic theory: Shannon theory (outline)Lossless coding algorithms: Huffman, arithmetic, run-length.Lossy coding algorithms: (vector) quantization, differential coding, Transform coding, subband coding, wavelet-coding, motion estimation/compensation. Standards: JPEG, JPEG2000, H.261/3 basic, MPEG-1/2/4 video.

29

Organization - from ISO/IEC to WG

Where does MPEG fit into the scheme of things

ISOInternation Standards Organization

IECInternational Electrotechnical Commission

ISO/IECJTC 1

Joint Technical Committee 1

Source: MPEG 101, N6706

30

Organization - from ISO to WG

Where does MPEG fit in the scheme of things

ITTFInternational Technical Task Force

Geneva, CH

SC xxSubcommittee

WG 1JPEG

WG11MPEG

Moving Picture Experts GroupMilan, IT

WG12MHEG

(last meeting 2001/03

SC 29Subcommittee 29

Secretariat - ITSJC/IPSJTokyo, Japan

JTC 1Joint Technical Committee - Information Technology

Secretariat - ANSINew YorK City

ISO/IECGeneva, CH

Source: MPEG 101, N6706

31

Organization - from WG to adhoc

WG11 MPEG Committee

Leonardo Chiariglione

WG11 MPEG Committee

Leonardo Chiariglione

SystemsOlivier Avaro

SystemsOlivier Avaro

VideoJen-Rainer Ohm

VideoJen-Rainer Ohm

AudioSchuyler Quackenbush

AudioSchuyler Quackenbush

MDSIan Burnett

MDSIan Burnett

TestTobias Oelbaum

TestTobias Oelbaum Requirments

Rob KoenenRequirmentsRob Koenen

LiaisonJan Borman

LiaisonJan Borman ISG

Marco MattevelliISG

Marco Mattevelli

IntegrationJean-Claude Durford

IntegrationJean-Claude Durford

SNHCMickael Borgas-Sevenier

SNHCMickael Borgas-Sevenier

Source: MPEG 101, N6706

32

Organization - from WG to adhoc

WG11 MPEG Committee

Leonardo Chiariglione

WG11 MPEG Committee

Leonardo Chiariglione

SystemsOlivier Avaro

SystemsOlivier Avaro

VideoJen-Rainer Ohm

VideoJen-Rainer Ohm

AudioSchuyler Quackenbush

AudioSchuyler Quackenbush

MDSIan Burnett

MDSIan Burnett

TestTobias Oelbaum

TestTobias Oelbaum Requirments

Rob KoenenRequirmentsRob Koenen

LiaisonJan Borman

LiaisonJan Borman ISG

Marco MattevelliISG

Marco Mattevelli

IntegrationJean-Claude Durford

IntegrationJean-Claude Durford

SNHCMickael Borgas-Sevenier

SNHCMickael Borgas-Sevenier

Source: MPEG 101, N6706

33

Membership

MPEG experts

National Bodies

WG11

SC29 NB members of SC29

JTC1 NB members of JTC1

ISO/IECNational Standard

Organisations

Source: MPEG 101, N6706

34

The MPEG Modus Operandi

•ExplorationThe search for new technologySeek Industry expertsOpen seminars

•RequirementsEstablish the scope of workCall for Proposals

•Competitive phaseDo HomeworkResponse to CfPInitial technology selection

•Collaborative phaseCore ExperimentsWorking Drafts

Source: MPEG 101, N6706

35

The MPEG Modus Operandi (cont’d)

•StandardizationBallotsNational Body Comments

•AmendmentAdding new technology

•CorrigendaCorrective actions

•New subdivisionsAdd new non-compatible technology

Source: MPEG 101, N6706

36

Where to from here?

• Ongoing search for technology enhancements

• Call for evidence of new/improved technology

• Significant benefits need to be demonstrated

• Actions include:•amendments or •new subdivisions to existing standards or •entirely new standards

Source: MPEG 101, N6706

37

Where to from here

Amendments • compatible enhancements to an existing specification

Corrigenda• Corrective actions to any element of the specification

Collaboration• Establish liaison with other duly constituted organizations• Exchange requirements and specification.

New subdivision of specification• non-compatible enhancements and additions to technology

Source: MPEG 101, N6706

38

Specification development time-lines

No fix duration specified or required -

•Exploration6-12 months depending on extent of search

•Requirements development6-12 months partly in parallel with exploration

•Competitive phase3-6 months partly in parallel with requirements

•Collaborative phase1 year following completion of competitive phase

total approximately 2 yearsSource: MPEG 101, N6706

39

Completion phase of development

Fixed minimum timeline - balloting

3 months 2 months publish

Specification CD FCD FDIS IS

Amendment PDAM FPDAM FDAM AMD

Corrigenda X DCOR X COR

TechnicalReport PDTR X DTR TR

Documenttype

4 months

All documents start out in life as working drafts (WD)

Source: MPEG 101, N6706

40

Alphabet soup

WD - Working Draft

CD - Committee DraftFCD - Final Committee DraftFDIS© - Final Draft International StandardIS © - International Standard© indicates ISO copyright

PDAM - Proposed Draft AmendmentFPDAM - Final Proposed Draft AmendmentFDAM © - Final Draft AmendmentAMD © - Amendment

Source: MPEG 101, N6706

41

More Alphabet soup

DCOR - Draft CorrigendaCOR © - Corrigenda

PDTR - Proposed Draft Technical ReportDTR © - Draft Technical ReportTR © - Technical Report

DoC - Disposition of Comments

Source: MPEG 101, N6706

42

Step by step

WDs become CDs (PDAM, PDTR, DCOR)(changes from here on out are by ballot / comments only)

CDs are balloted by National Bodies

Ballots are submitted by NB’s to SC 29 with or without comments(at FDIS ballots are JTC 1 level)

Comments are processed and approved at MPEG meetings (DoC)

Specification moves on to next stage

Source: MPEG 101, N6706