multimedia-systems: compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/a...kom -...
TRANSCRIPT
![Page 1: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/1.jpg)
y, Dept. of Computer Science151 166151, M.tu-darmstadt.de Fax. +49 6151 166152
logyTel.+49 6151 869869mstadt.gmd.de Fax. +49 6151 869870
06-compression.fm 1 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz
Multimedia-Systems:Compression
Prof. Dr.-Ing. Ralf SteinmetzDr. L.Wolf, Dr. S.Fischer
TU Darmstadt - Darmstadt University of Technology, Dept. of Electrical Engineering and Information Technolog
KOM - Industrial Process and System Communications, Tel.+49 6Merckstr. 25, D-64283 Darmstadt, Germany, Ralf.Steinmetz@KO
GMD - German National Research Center for Information TechnoIPSI - Integrated Publication and Information Systems Institute, Dolivostr. 15, D-64293 Darmstadt, Germany, Ralf.Steinmetz@dar
![Page 2: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/2.jpg)
User Interfaces
.Synchro-nization
Group Communi-
cations
Programming
ms Communications
ice Networks
ession
Video Audio
06-compression.fm 2 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz Scope
Usa
ge
Applications
Learning & Teaching Design
Ser
vice
s Content Process-
ing
Docu-ments
Security ..S
yste
ms
Databases
Media-Server Operating Syste
Opt. Memories Quality of Serv
Bas
ics
Computer Archi-
tectures
Compr
Image &Graphics
Animation
![Page 3: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/3.jpg)
06-compression.fm 3 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz Contents
1. Motivation
2. Requirements - General
3. Fundamentals - Categories
4. Source Coding
5. Entropy Coding:
6. Hybrid Coding: Basic Encoding Steps
7. JPEG
8. H.261 and related ITU Standards
9. MPEG-1
10. MPEG-2
11. MPEG-4
12. Wavelets
13. Fractal Image Compression
14. Basic Audio and Speech Coding Schemes
15. Conclusion
![Page 4: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/4.jpg)
and 2 Byte/Char
mple
4 Bit/Pixel, 30 frames/s
06-compression.fm 4 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz 1. Motivation
Digital video in computing means for • Text:
• 1 page with 80 char/line and 64 lines/page
• 80 x 64 x 2 x 8 = 80 kBit/page• Image:
• 24 Bit/Pixel, 512 x 512 Pixel/image
• 512 x 512 x 24 = 6 MBit/Image
• Audio: • CD-quality, samplerate44,1 kHz, 16 Bit/sa• Mono: 44,1 x 16 = 706 kBit/s
Stereo: 1.412 MBit/s• Video:
• full frames with 1024 x 1024 Pixel/frame, 21024 x 1024 x 24 x 30 = 720 MBit/s
• more realistic360 x 240 Pixel/frame = 60 MBit/s
Hence compression is necessary
![Page 5: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/5.jpg)
high qualityon
., ease of decoding)
.g., memory req.)
scalability
06-compression.fm 5 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz 2. Requirements - General
compressi
low delay
low complexity (e.gefficient implementation (e
intrinsic
![Page 6: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/6.jpg)
atedia
tion requirements
06-compression.fm 6 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz Requirements
Dialogue and retrieval mode requirements:• Independence of frame size and video frame r• Synchronization of audio, video, and other me
Dialogue mode requirements:• Compression and decompression in real-time
(e.g. 25 frames/s)• End-to-end delay < 150ms
Retrieval mode requirements:• Fast forward and backward data retrieval• Random access within 1/2 s
Software and/or hardware-assisted implementa
![Page 7: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/7.jpg)
nel
hybrid
coding
- entropy
and
source
coding
06-compression.fm 7 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz 3. Fundamentals - Categories
entropy encodingsource coding
- based on semantic of the data
- often lossy
channel coding
- adaptation to communication chan
- introduction of redundancy
entropy coding
- ignoring semantics of the data
- lossless
![Page 8: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/8.jpg)
DPCM
DM
FFT
DCT
Bit PositionSubsampling
Sub-Band Coding
06-compression.fm 8 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz Categories and Techniques
EntropyCoding
Run-Length Coding
Huffman Coding
Arithmetic Coding
SourceCoding
Prediction
Transformation
Layered Coding
Vector Quantization
Hybrid Coding
JPEG
MPEG
H.261, H.263
proprietary: Quicktime, ...
![Page 9: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/9.jpg)
aluesion
redicted
tion is determined
al
06-compression.fm 9 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz 4. Source Coding
DPCMDPCM = Differential Pulse-Code Modulation
Assumptions:• Consecutive samples or frames have similar v• Prediction is possible due to existing correlat
Fundamental Steps:• Incoming sample or frame (pixel or block) is p
by means of previously processed data• Difference between incoming data and predic• Difference is quantized
Challenge: optimal predictor
Further predictive coding technique:• Delta modulation (DM): 1 bit as difference sign
![Page 10: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/10.jpg)
mpress
n
frequencydomain
n
06-compression.fm 10 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz Source Coding: Transformation
Assumptions:• Data in the transformed domain is easier to co• Related processing is feasible
Example:
FFT: Fast Fourier Transformation
DCT: Discrete Cosine Transformation
Inverse
Fourier Transformatio
time domain
Fourier Transformatio
![Page 11: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/11.jpg)
an others
he signal
gfrequency
06-compression.fm 11 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz Source Coding: Sub-Band
Assumption:• Some frequency ranges are more important th
Example:
Application:• vocoder for speech communication• MPEG audio
frequency spectrum of t
transformation / codin
![Page 12: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/12.jpg)
C B...
ion
B...
er ofrrences
06-compression.fm 12 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz 5. Entropy Coding:
Run-LengthAssumption:• Long sequences of identical symbols
Example:
... A B C E E E E E E D A
compress
... A B C E M 6 D A C
symbol
special flag
numboccu
![Page 13: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/13.jpg)
uage
shorter bit strings
0.1, p(D)=0.15, p(E)=0.15symbol code
A
B
C
D
E
11
10
011
010
00
06-compression.fm 13 22.October.99htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz Entropy Coding: Huffman
Assumption:• Some symbols occur more often than others• E.g., character frequencies of the English lang
Fundamental principle:• Frequently occurring symbols are coded with
Example:• Characters to be encoded: A, B, C, D, E• probability to occur: p(A)=0.3, p(B)=0.3, p(C)=
probability
1
0
1
0
1
0
1
0
coding tree
30%
30%
10%
15%
15%
100%
40%
60%
25%
![Page 14: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/14.jpg)
11 10 00 10 11 00
BA B E A E
06-compression.fm 14 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz Entropy Coding: Huffman
Table and example of application to data stream
Other types of Entropy encoding• Arithmetic Encoding (1)
10 11 011 010
B A DC
symbol code
ABCDE
111001101000
![Page 15: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/15.jpg)
teps
lossless
e.g.- runlength- Huffman
entropy
encoding
compresse
data
06-compression.fm 15 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz 6. Hybrid Coding: Basic Encoding S
audio:
video:lossy
lossless (sometimes lossless)lossy
e.g.- resolution- frame rate
e.g.- DCT- sub-band
coding
e.g.- linear- DC, AC
values
data
pre-
paration
data
pro-
cessing
quanti-
zation
source
data
![Page 16: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/16.jpg)
ous-tone still images:
ults
06-compression.fm 16 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz 7. JPEG
“JPEG”: Joint Photographic Expert Group
International Standard:• For digital compression and coding of continu
• Gray-scale• Color
• Since 1992
Joint effort of:• ISO/IEC JTC1/SC2/WG10• Commission Q.16 of CCITT SGVIII
Compression rate of 1:10 yields reasonable res
![Page 17: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/17.jpg)
cs
ata
06-compression.fm 17 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz JPEG
Very general compression scheme
Independence of:• Image resolution• Image and pixel aspect ratio• Color representation• Image complexity and statistical characteristi
Well-defined interchange format of encoded d
Implementation in:• Software only• Software and hardware
“MOTION JPEG” for video compression• Sequence of JPEG-encoded images
![Page 18: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/18.jpg)
entropyencoding
com-
pressed
image
runlength
Huffman
Arithm.
tion
06-compression.fm 18 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz JPEG - Compression Steps
imagepre-
paration
imagepro-
cessingquanti-
zation
source
image
blockMCU
pixelpredictor
FDCT
MCU: Minimum Coded UnitFDCT: Forward Discrete Cosine Transforma
![Page 19: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/19.jpg)
r color) possible
Yi
Xi
right
bottom
* *
* *
*
*
*
line
top
ith 8x8 pixels in other modes
06-compression.fm 19 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz JPEG - Image Preparation
Planes:• 1 ≤ N ≤ 255 components Ci (e.g., one plane pe• Different resolution of individual components
Pixel resolution:• 8 or 12 bit per pixel in lossy modes• 2 to 16 bit per pixel in lossless mode
C1
C2
CNleft
*
*
*
*
data units
data units: samples in lossless mode, blocks w
![Page 20: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/20.jpg)
g
06-compression.fm 20 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz JPEG - Image Preparation
Example 4:2:2 YUV, 4:1:1 YUV, and YUV9 Codin• Luminance (Y):
• brightness• sampling frequency 13.5 MHz
• Chrominance (U, V):• color differences• sampling frequency 6.75 MHz
![Page 21: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/21.jpg)
rent components
ght
* *** **
* *** **
C3
06-compression.fm 21 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz JPEG - Image Preparation
Non-interleaved encoding:
Interleaved encoding:
Minimum Coded Unit (MCU):• Combination of interleaved data units of diffe
top
rileft
bottom
* * * * * * ** * * * * * *
* * * * * * *
* * ** * ** * ** * ** * ** * ** * ** * *
* * ** * ** * ** * *
* * ** * *
C1 C2
![Page 22: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/22.jpg)
mentations:
set-up
entropyencoding
com-
pressed
image
tables
06-compression.fm 22 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz JPEG - Baseline Mode
Baseline mode is mandatory for all JPEG imple• Often restricted to certain resolution• Often only three planes with predefined color
Image preparation:• Pixel resolution of p=8 bit• 8 x 8 pixel blocks (data units)
tables
imagepre-
paration
imagepro-
cessing
quanti-zation
source
image
FDCT tables8x8
blocks
![Page 23: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/23.jpg)
ssing
DCT):
:ents:
ro values
uπ------- 2y 1+( )vπ16
-----------------------------cos
06-compression.fm 23 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz JPEG - Baseline Mode: Image Proce
Forward Discrete Cosine Transformation (F
with:
cu, cv = , for u, v= 0; else cu, cv = 1
Formula applied to each block for all 0 ≤ u, v ≤ 7• Blocks with 8x8 pixel result in 64 DCT coeffici
• 1 DC-coefficient S00: basic color of the block• 63 AC-coefficients: (likely) zero or near-by ze
Different significance of the coefficients:• DC: most important• AC: less important
Svu14---CuCv syx
2x 1+( )16
----------------------cos
y 0=
7
∑x 0=
7
∑=
1
2-------
![Page 24: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/24.jpg)
ssing
* * * * * ** * * * * ** * * * * ** * * * * ** * * * * ** * * * * ** * * * * ** * * * * *
06-compression.fm 24 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz JPEG - Baseline Mode: Image Proce
FDCT transforms:• blocks into blocks• not pixels into pixels
Example:• Calculation of S00
# # # # # # # ## # # # # # # ## # # # # # # ## # # # # # # ## # # # # # # ## # # # # # # ## # # # # # # ## # # # # # # #
* ** ** ** ** ** ** ** *
…
…
![Page 25: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/25.jpg)
nts:mberefficient
06-compression.fm 25 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz JPEG - Baseline Mode: Quantization
Use of quantization tables for the DCT-coefficie• Map interval of real numbers to one integer nu• Allows to use different granularity for each co
![Page 26: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/26.jpg)
(b)
06-compression.fm 26 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz JPEG - Quantization Effect
(a)
![Page 27: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/27.jpg)
oding
1
...
06-compression.fm 27 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz JPEG - Baseline Mode: Entropy Enc
DC-coefficients:• Compute the differences:
• Use differences instead of the DCi values
block block
DIFF = DCi - DCi-
...
DCiDCi-1
![Page 28: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/28.jpg)
ing
kely to be zero
f respective value
AC77
AC07
06-compression.fm 28 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz JPEG - Baseline Mode: Entropy Cod
63 AC coefficients:• Ordering in ‘zig-zag’ form
• reason: coefficients in lower right corner are li• Huffman coding of all coefficients:
• Transformation into a codewhere amount of bits depends on frequency o
• Subsequent runlength coding of zeros
* *** ***** *** ****
* *** ***** *** ****
* *** ***** *** ***** *** ***** *** ****AC70
DC
AC01
![Page 29: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/29.jpg)
mode
mode
06-compression.fm 29 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz JPEG - 4 Modes of Compression
lossy sequential DCT-based
(baseline mode)
expanded lossy DCT-based
lossless mode
hierarchical mode
![Page 30: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/30.jpg)
Mode
06-compression.fm 30 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz JPEG - Extended Lossy DCT-Based
Pixel resolution 8 to 12 bit
Sequential image display:• Top to bottom• Good for small images and fast processing
Progressive image display:• Coarse to fine• Good for large and complicated images
![Page 31: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/31.jpg)
Mode
value
ently
06-compression.fm 31 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz JPEG - Extended Lossy DCT-Based
Principle:• Coefficients stored in buffer after quantization• Order of pixel/block processing changed
By spectral selection:• Selection according to importance of DC, AC • All DC values of whole image first• All AC values in order of importance subsequ
By successive approximation:• Selection according to position of bits• First the most significant bit of all blocks• Then the second significant bit of all blocks• Until the least significant bit of all blocks
![Page 32: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/32.jpg)
the actual value
predictionde01234567
no predictionx=Ax=Bx=Cx=A+B+Cx=A+((B-C)/2)x=B+((A-C)/2)x=(A+B)/2
06-compression.fm 32 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz JPEG - Lossless Mode
Image preparation:• On pixel basis (2-16 bit/pixel)
Image processing:• Selection of a predictor for each pixel
Entropy coding:• Same as lossy mode• Code of chosen predictor and its difference to
c ba x
co
![Page 33: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/33.jpg)
solution - image A’ - B = A - A’
06-compression.fm 33 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz JPEG - Hierarchical Mode
Coding of each image with several resolutions:• Image scaling• Differential encoding• First, coded with lowest resolution - image A• Coded with increasing horizontal & vertical re• Difference between both images is computed• Iteration for higher resolutions
Features:• Requires more storage and higher data rate• Fast decoding process• Used for scalable video• Similar to Photo-CD
![Page 34: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/34.jpg)
bit/s:
oftware
06-compression.fm 34 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz 8. H.261 and related ITU Standards
Video codec for audiovisual services at p x 64k• CCITT standard from 1990• For ISDN• With p=1,..., 30
Technical issues:• Real-time encoding/decoding• Max. signal delay of 150ms• Constant data rate• Implementation in hardware (main goal) and s
![Page 35: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/35.jpg)
solution: Mandatory
essed)
ptional
essed)
cks
06-compression.fm 35 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz H.261 - Image Preparation
Fixed source image format• Image components:
• Luminance signal (Y)• Two color difference signals (Cb,Cr)
• Subsampling according to CCIR 601 (4:1:1)
Quarter Common Intermediate Format (QCIF) re• Y: 176 x 144 pixel• At 29.97 frames/s appr. 9.115 Mbit/s (uncompr
Common Intermediate format (CIF) resolution: O• Y: 352 x 288 pixel• At 29.97 frames/s appr. 36.46 Mbit/s (uncompr
Layered structure:• Block of 8 x 8 pixels• Macroblock of: 4 Y blocks, 1 Cr block, 1 Cb blo• Group of blocks (GOBs) of 3 x 11 macroblock• Picture:
• QCIF picture:3 GOBs• CIF picture: 12 GOBs
![Page 36: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/36.jpg)
gector
06-compression.fm 36 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz H.261 - Image Compression
Intraframe coding:• DCT as in JPEG baseline mode
Interframe coding, motion estimation:
• Search of similar macroblock in previous ima• Position of this macroblock defines motion ve• Search range is up to the implementation:
• i.e., motion vector may always be 0
Frame 1 Frame 2
![Page 37: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/37.jpg)
reshold
f variable length
06-compression.fm 37 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz H.261 - Image Compression
Interframe coding, further steps:• Results:
• Difference between similar macroblocks• Motion vector
• Difference of macroblocks:• DCT if value higher than a specific threshold• No further processing if value less than this th
• Motion vector:• Components are coded yielding code words o
Quantization:• Linear• Adaptation of step size š constant data rate
![Page 38: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/38.jpg)
H.3xx)
H.263
r Encoder Decoder
required
required
optional
optional
06-compression.fm 38 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz Further ITU Video Schemes (H.263,
H.263• extension to H.261• max. bitrate: H.263 approx. 2.5 x H.261
Source Image Formats
Format PixelsH.261
Encoder Decode
SQCIF 128 x 96 optional
QCIF 176 x 144 required
CIF 352 x 144 optional
4CIF 704 x 576not defined
16CIF 1408 x 1152
![Page 39: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/39.jpg)
& 1 P-Frame)ion
pel)
06-compression.fm 39 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz H.263
Differences of H.263 compared to H.261• optimal PB-frames (2 combined pictures: 1 B-• optional overlapped block motion compensat• optional motion vector pointing outside image• half pel motion compensation (instead of full • JPEG is the still picture mode• no included error detection and correction• ..
![Page 40: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/40.jpg)
DN
(ATM)
(instead of N-ISDN)
O-Ethernet)
V.34 Modems)
06-compression.fm 40 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz H.320, H.32x Family
H.320 specifies (as overview) videophone for IS
H.310• adapt MPEG 2 for communication over B-ISDN
H.321• define videoconferencing terminal for B-ISDN
H.322• adapts H.320 for guaranteed QoS LANs (like IS
H.323• videoconferencing over non-guaranteed LANs
H.324• Terminal for low bit rate communication (over
![Page 41: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/41.jpg)
06-compression.fm 41 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz 9. MPEG-1
Motion Picture Expert Group (MPEG)• ISO/IEC working group(s)• ISO/IEC JTC1/SC29/WG11• ISO IS 11172 since 3/93
Starting point: MPEG-1• Audio/video at about 1.5 Mbit/s• Based on experiences with JPEG and H.261
Follow-up standards• MPEG-2• MPEG-4• MPEG-7
![Page 42: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/42.jpg)
system
combined stream
common buffermanagement
06-compression.fm 42 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz MPEG - Features
Consideration of other standards:• JPEG• H.261
Symmetric and asymmetric compression
Constant data rate, should be < 1856 kbit/s
Target rate about 1.5 Mbit/s
audio video
MPEG
coding data streamcoding data stream
![Page 43: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/43.jpg)
06-compression.fm 43 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz MPEG - Video: Preparation Step
Fixed image format
Color subsampling:• Y, Cr, Cb• 4:2:0
Resolution:• Should be at most 768 x 576 pixel• 8 bit/pixel in each layer (i.e., for Y, Cr, Cb)• 14 pixel aspect ratios• 8 frame rates
No user defined MCU like JPEG
No progressive mode like JPEG
![Page 44: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/44.jpg)
motion vectord
s):) frames
06-compression.fm 44 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz MPEG - Video: Processing Step
4 types of frames:
I-frames (intra-coded frames):• Like JPEG• Real-time decoding demands
P-frames (predictive coded frames):• Reference to previous I- or P-frames• Motion vector
• MPEG does not define how to determine the • difference of similar macroblocks is DCT code
• DC and AC coefficients are runlength coded
B-frames (bi-directional predictive coded frame• Reference to previous and subsequent (I or P• Interpolation between macro blocks
D-frames (DC-coded frames):• Only DC-coefficients are DCT coded• For fast forward and rewind
![Page 45: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/45.jpg)
Intracoded)
(Predictive Coded)
(Bidirectionally Coded)
(DC Coded))
06-compression.fm 45 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz MPEG - Video Coding
Sequence of I-, P-, and B-frames:
Sequence:• Defined by application• E.g., I B B P B B P B B I B B P B B P B B…• Order of transmission is different: I P B B ...
I
P
BB
P
BB
IReferences
t
• I-Frames (
• P-Frames
• B-Frames
• (D-Frames
![Page 46: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/46.jpg)
> ...)
inficult
ntropy decoding
06-compression.fm 46 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz MPEG - Video: Implications
Random access• at I-frames• at P-frames: i.e. decode previous I-frame first• at B-frame: i.e. decode I and P-frames first
Editing• decoded data
• loss of quality (encode -> decode -> encode -• application of all video editing functions
• encoded data (previous to entropy encoding)• preservation of quality• transition effects as function in the DCT doma• morphing, non-block conform overlay very dif
• encoded data• preservation of quality• today: too complex, if possible, i.e. need for e
![Page 47: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/47.jpg)
ls
2 5 10 20equency (kHz)
4 kHz
av
06-compression.fm 47 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz MPEG - Audio Coding: Fundamenta
Masking threshold in the frequence domain• narrowband random noise• depends on frequency
0.02 0.05 0.1 0.2 0.5 1fr
So
un
d P
ress
ure
Lev
el (
dB
)
0
20
40
60
80
fm = 0.25 1
absolute thresholdof hearing
maskingpatterns
![Page 48: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/48.jpg)
ls
50 100 150 200
v
st-masking-
06-compression.fm 48 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz MPEG - Audio Coding: Fundamenta
Masking in Time Domain• after and before the event• depends on (to some extent) amplitude
-50 50 100 150 ms 0
SL T
0
20
40
60
Dt t
masker
simultaneous-pre- po
![Page 49: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/49.jpg)
o as MUSICAM in DAB)
entropycoder &frame
packing
06-compression.fm 49 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz MPEG - Audio Coding
Audio channel:• Between 32 and 448 kbit/s• In steps of 16 kbit/s
Definition of 3 layers of quality• Layer 1: max. 448 Kbit/s• Layer 2: max. 384 Kbit/s (most often used, als• Layer 3: max. 320 Kbit/s
sub-bandcoding
quanti-zation
psychoacousticalmodel
32
controls
![Page 50: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/50.jpg)
al) irrelevancy)
asting “MUSICAM” =ding And Multiplexing)
06-compression.fm 50 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz MPEG - Audio Coding
Compatible to encoding of CD-DA and DAT:• Sampling rates:
• 32 kHz, 44,1 kHz, 48 kHz• Sampling precision:
• 16 bit/sample
Audio channels:• Mono (single, 1 channel)• Stereo (2 channels)
• dual channel mode (independent, e.g., bilingu• optional: joint stereo (exploits redundancy and
Application Example: DAB Digital Audio Broadc• uses MPEG layer 2 (compression also known as
(Masking pattern adapted Universal Subband Integrated Co
Delays• max. of 30 ms encoding• max. of 10 ms decoding• based on VLSI
![Page 51: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/51.jpg)
ms
06-compression.fm 51 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz MPEG - Audio and Video Data Strea
Audio Data Stream Layers:
1. Frames
2. Audio access units
3. Slots
Video Data Stream Layers:
1. Video sequence layer
2. Group of pictures layer
3. Single picture layer
4. Slice layer
5. Macroblock layer
6. Block layer
![Page 52: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/52.jpg)
onn image contents
06-compression.fm 52 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz 10. MPEG-2
Follow-Up MPEG StandardsMPEG-2:• Higher data rates for high-quality audio/video• Multiple layers and profiles
MPEG-3• Initially HDTV• MPEG-2 scaled up to subsume MPEG-3
MPEG-4:• Lower data rates for e.g. mobile communicati• Coding and additional functionalities based o
MPEG-7:• Content description• Basis for search and retrieval• See section on databases
![Page 53: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/53.jpg)
deo broadcasting)
06-compression.fm 53 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz MPEG-2
From MPEG-1 to MPEG-2• Improvement in quality
• from VCR to TV to HDTV• No CD-ROM based constraints
• higher data rates• MPEG-1: about 1.5 Mbit/s• MPEG-2: 2-100 Mbit/s
Evolution• 1994: International Standard• Also later known as H.262• Prominent role for digital TV in DVB (digital vi
• commercial MPEG-2 realizations available
![Page 54: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/54.jpg)
ear quantization function
06-compression.fm 54 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz MPEG-2 Video
Inclusion of interlaced video format
Increase resolution, more than CCIR 601
Defined as:• 5 profiles (simple, main,..)• 4 levels (with increasing resolution,...)
Other additional features• DCT coefficients may be coded with a non-lin
![Page 55: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/55.jpg)
rs occurur, i.e scaling
G algorithms applied,r)difference MPEG algorithms
layer)
s
encoded with better FECbits encoded with worse FEC
06-compression.fm 55 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz MPEG-2 Video: Scaling
Motivation• analog: continuous decrease in quality if erro• digital: need for tolerance whenever error occ
Option: Spatial scaling• reduction of resolution• approach
• image sampled with half resolution, then MPEoutput processed with better FEC (base laye
• Image decoded, substracted from original, to applied,output processed with worseFEC (enhanced
Option: Signal to Noise (SNR) scaling• noise introduced by
• quantization errors and visible block structure• approach
• Base layer: DCT output, more significant bits• Enhanced layer:DCT output, less significant
![Page 56: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/56.jpg)
≤ 100 Mbit/s
≤ 60 Mbit/s ≤ 80 Mbit/s
≤ 20 Mbit/s
SpatialScalableProfile
HighProfile
B-frames B-frames
4:2:0 4:2:0 or 4:2:2
SNR Scalable or Spatial Scalable
SNR Scalable or Spatial Scalable
06-compression.fm 56 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz MPEG-2 Video Profiles und Levels
High Level1920 pixels/line
1152 lines
≤ 80 Mbit/s
High-1440 Level1440 pixels/line
1152 lines
≤ 60 Mbit/s
Main Level720 pixels/line
576 lines
≤ 15 Mbit/s ≤ 15 Mbit/s ≤ 15 Mbit/s
Low Level352 pixels/line
288 lines
≤ 4 Mbit/s ≤ 4 Mbit/s
LEVELSand
PROFILES
SimpleProfile
MainProfile
SNR ScalableProfile
No B-frames B-frames B-frames
4:2:0 4:2:0 4:2:0
Not Scalable Not Scalable SNRScalable
![Page 57: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/57.jpg)
MPEG-2ackward compatibility
yLeft_b+
r yRigtht_b+
06-compression.fm 57 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz MPEG-2 Audio
Up to • 5 full bandwidth channels (surround system)
• left and right front• center (in front)• left and right back with (x and y = 0.71)
• and 7 multilingual/commentary channels
Improved quality at or below 64 kbit/s
Compatible to MPEG-1• all MPEG-1 audio format can be processed by• only 3 MPEG-2 audio codec will not provide b
(in the range between 256 - 448 Kbit/s)
Left for Stereo Left_f xCenter+=
Right for Stereo Right_f xCente+=
![Page 58: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/58.jpg)
ntary Stream (PES)“
sport Stream”
re time bases
reams possible
06-compression.fm 58 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz MPEG-2 System
Steps
1. Audio and video combined to “Packetized Eleme
2. PES(es) combined to “Program Stream” or “Tran
Program stream:• Error-free environment• Packets of variable length• One single stream with one timing reference
Transport stream:• Designed for “noisy“ (lossy) media channels• Multiplex of various programs with one or mo• Packets of 188 byte length
Conversion between Program and Transport St
![Page 59: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/59.jpg)
es
e contents
06-compression.fm 59 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz 11. MPEG-4
GoalsMPEG-4 (ISO 14496) originally:• Targeted at systems with very scarce resourc• To support applications like
• Mobile communication• Videophone and E-mail
• Max. data rates and dimensions (roughly):• Between 4800 and 64000 bits/s• 176 columns x 144 lines x 10 frames/s
Further demand:• To provide enhanced functionality
to allow for analysis and manipulation of imag
MPEG-4: Schedule for Standardization• 1993 Work started• 1997: Committee Draft• 1998: Final Committee Draft• 1998: Draft International Standard• 1999-2000: International Standard
![Page 60: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/60.jpg)
l content
rroundings and background
diovisual scenes with AVOs
iding a QoS (Quality-of-
he decoder’s site
Audioobject 2
06-compression.fm 60 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz MPEG-4: Goals (cont.)
Hence: find standardized ways to• Represent units of aural, visual or audiovisua
• "audio/visual objects" or AVOs
• object coding independent of other objects, su• natural and synthetic objects
• Compose these objects together• i.e. creation of compound objects that form au
• Multiplex and synchronize the data associated• for transportation over network channels prov
Service)• Interact with the audiovisual scene generated at t
„Rhubarb
Audioobject 1 video objects
1 2 3Rhubarb“
![Page 61: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/61.jpg)
esentation
s,
pound audiovisual objects
06-compression.fm 61 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz MPEG-4: Scope
Definition of• „System Decoder Model“
• specification for decoder implementations• Description language
• binary syntax of an AV object’s bitstream repr• scene description information
• Corresponding concepts, tools and algorithmespecially for
• content-based compression of simple and com• manipulation of objects• transmission of objects• random access to objects• animation• scaling• error robustness
![Page 62: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/62.jpg)
to 15 frames/s
06-compression.fm 62 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz MPEG-4: Scope (cont.)
Targeted bit rates for video and audio:• VLBV core
• „Very Low Bit-rate Video“• 5 - 64 Kbit/s• image sequences with CIF resolution and up
• Higher-quality video• 64 Kbit/s - 4 Mbit/s• quality like digital TV
• Natural audio coding• 2 - 64 Kbit/s
![Page 63: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/63.jpg)
ation
sm
eded for decoding
06-compression.fm 63 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz MPEG-4: Video and Image Encoding
Encoding / decoding of• Rectangular images and
video• coding similar to
MPEG-1/2• motion prediction• texture coding
• Images and video of arbitrary shape
• as done in conventional approach
• 8x8 DCT or shape-adaptive DCT
• plus coding of shape and transparency inform
Encoder• Must generate timing information
• speed of the encoder clock = time base• desired decoding times and/or expiration time
• by using time stamps attached to the strea
• Can specify the minimum buffer resources ne
![Page 64: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/64.jpg)
en objects
tem into a global coordinate
arameters
top streams, select
primitive AVO
compound object
compound object
06-compression.fm 64 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz MPEG-4: Composition of Scenes
Scene description includes:• Tree to define hierarchical relationships betwe
• Objects’ positions in space and time• by converting the objects’ local coordinate sys
system• Attribute value selection
• e.g. pitch of sound, color, texture, animation p
Description based on some VRML concepts• VRML = „Virtual Reality Modelling Language“
Interaction with scenes• e.g. change viewing point, drag object, start/s
language
„RhubarbRhubarb“
![Page 65: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/65.jpg)
06-compression.fm 65 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz MPEG-4: Example of a Composition
![Page 66: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/66.jpg)
t a reduced spatial resolutiontream
l resolutiontream
layers of different bit-rateser
ignal
06-compression.fm 66 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz MPEG-4: Scaling
Three approaches:• Spatial scalability
• decoder displays textures and visual objects a• by decoding only a subset of the total bit s
• 32 levels max. for textures and still images• 3 levels max. for video sequences
• Temporal scalability• decoder displays video at a reduced tempora
• by decoding only a subset of the total bit s
• 3 levels max.• Quality scalability
• bitstream is parsed into a number of bit stream• either during transmission or in the decod
• subset of the layers still yields a meaningful s
Spatial and temporal scaling both for• Conventional rectangular display and• Objects with arbitrary shape
![Page 67: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/67.jpg)
ic parameters
06-compression.fm 67 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz MPEG-4: Synthetic Objects
Visual objects:• Virtual parts of scenes
• e.g. virtual background• Animation
• e.g. animated faces
Audio objects:• „Text-to-speech“
• speech generation from given text and prosod• face animation control
• „Score driven synthesis“• music generation from a score• more general than MIDI
• Special effects
![Page 68: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/68.jpg)
tecture
udio framesription commands
info, sync. info, QoS req.,...
lementary streamsS requirements
ltiplexing
ultiplexingface specifiedf can be any network,DP/IP, AAL5/ATM
coding
06-compression.fm 68 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz MPEG-4: Layered Networking Archi
Access Units
Adaptation Layer
FlexMux Layer
Elementary Streams
Multiplexed Streams
Network or Local Storage
e.g. video or aor scene desc
A/V object data+ stream type
e.g. multiple ewith similar Qo
Flexible Mu
Transport M- only interTransMux Layer
- layer itsele.g. RTP/U
CoDecCoDecCoDecCoDec
Display / Recording
Media
Coding / De
![Page 69: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/69.jpg)
tecture (cont.)
and bandwidths
06-compression.fm 69 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz MPEG-4: Layered Networking Archi
DMIF „Delivery Multimedia Integration Framework“• Allows to establish multiple party sessions
• interaction with• remote interactive peers
• broadcast systems
• storage systems• establishment of channels with specific QoSs
• Controls• FlexMux layer• TransMux layer
![Page 70: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/70.jpg)
06-compression.fm 70 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz MPEG-4: Error Handling
Mobile communication:• Low bit-rate (< 64 Kbps)• Error-prone
MPEG-4 concepts for error handling:• Resynchronization
• enables receiver to „tune in“ again• based on markers within bitstream
• Data recovery• enables receiver to reconstruct lost data• encode data in an error-resilient manner
• Error concealment• enables receiver to bridge gaps in data• e.g. by repeating parts of old frames
![Page 71: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/71.jpg)
o small blocks compression ratios
source data
roduces inherent scaling
to human perception
06-compression.fm 71 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz 12. Wavelets
MotivationJPEG / DCT problems:• DCT not applicable to whole image, but only t
⇒ block structure becomes visible at high• Scaling as add-on ⇒ additional effort• DCT function is fixed ⇒ can not be adapted to
Improvements by using Wavelets:• Transformation of the whole image
⇒ overcomes visible block structures and int
• Better identification of which data is relevant ⇒ higher compression ratio
![Page 72: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/72.jpg)
ssion
gorithms
step
antizer Encoder
r Decoder
r
or
06-compression.fm 72 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz Wavelets: Compression / Decompre
The same overall structure as for DCT-based al
But: important differences in the transformation
Qu
Inverse WaveletTransformation DeQuantize
Forward WaveletTransformation
Compresso
Decompress
![Page 73: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/73.jpg)
n (as in JPEG)
ine functions
rval
f the image
eing a Wavelet
e.g.:
06-compression.fm 73 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz Wavelets: Fundamental Idea
Image is transformed into the frequency domai
But: based on Wavelet functions instead of cos
Advantage: Wavelets are 0 outside a limited inte
⇒ Wavelet automatically relates only to a part o
⇒ Image needs not be splitted into blocks
Use Wavelet family: {2-j/2*Ψ(2-j*x-k)}, j,k ε Z, Ψ b
cosine:
......
Wavelet
![Page 74: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/74.jpg)
)
ass filters
. . . lowerfrequencies
transformedimage withreduced size
higherfrequencies
06-compression.fm 74 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz Wavelets: Transformation Steps
"Discrete Wavelet Transformation" (Mallat, 1989
Split image recursively by using high and low p
c1
d11
d12
d13
L
H
L
H
L
H
read by column
L Low PassH High Pass
line
read by
![Page 75: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/75.jpg)
nt.)
ge
ageils / at a lower resolution
06-compression.fm 75 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz Wavelets: Transformation Steps (co
In each step i:• Three images dx
i (x=1,2,3):• containing the high frequency parts of the ima• representing "details" of the image• submitted to Wavelet transformation
• or thrown away in case of scaling
• One image ci:• containing the lower frequency parts of the im• representing the original image with less deta• submitted to step i+1
Afterwards:• Quantization• Entropy encoding
as with DCT
![Page 76: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/76.jpg)
rmed)eely chosen
06-compression.fm 76 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz Wavelets: DWT compared with DCT
Advantages of DWT over DCT:• No block artefacts• Inherent scaling
• based on the dxi for i=1,2,3,...
• Lower time complexity for the transformation• DCT: O(n*logn),• DWT: O(n) (n=number of values to be transfo
• Higher flexibility: Wavelet function − can be fr
![Page 77: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/77.jpg)
..In-1 - In-2
In - In-1
...
Waveletcompressor
06-compression.fm 77 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz Wavelets: Further Issues
Edge detection reduces high frequencies:• First extract detected edges• Then apply wavelets to such a filtered image
Application to video:
In-2In-1
Image n
Imt
Computedifferences
.
t
![Page 78: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/78.jpg)
06-compression.fm 78 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz 13. Fractal Image Compression
Image Generation
Mandelbrot• recursive construction of images• infinite “granularity”• self-similarities in images• Zi = RealConst. * Zi-1 + ComplexConst
![Page 79: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/79.jpg)
ration
06-compression.fm 79 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz To apply self-similarity: Image Gene
Examples• (from TUD + Univ. Bochum)
for recursive contruction of images
Sirpinky triangle• to produce self-
similar structures• infinite steps applied
to different source images lead to same result
• known as Sirpinski-triangle
• "Grenzwert" also known as attractor
![Page 80: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/80.jpg)
06-compression.fm 80 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz To Find Self-Similarities
affine function allows for• translation• rotation• scaling• brightness adaptation
IFS: Iterative Function System• ideally completely self-similar• example see right
PIFS: Partitioned Iterative Funcion System• real images are
not completly self-similar• Wimg?
![Page 81: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/81.jpg)
| ≤ s | x-y | for all x,y ε F
any x ε F
trical space pixels of two picturesd point of Wimg
06-compression.fm 81 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz Theoretical Basis
Banach’s Fixed Point Theorem:• Let F be a metrical space• Let W: F→F be a contractive mapping
• i.e. there exists an s, 0<s<1, with | W(x)-W(y)• Then W has exactly one fixed point xf
• i.e. W(xf) = xf • xf can be computed as xf = limn→∞ Wn(x) with
Application to image compression:• Let img be the image to be compressed• Regard the set of all possible images as a me
• metric e.g.: maximum difference between the• Goal: construct Wimg such that img is the fixe
![Page 82: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/82.jpg)
compression
t
mage⇒ easy
06-compression.fm 82 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz Fractal Image Compression and De
Compression: Find appropriate Wimg ⇒ difficul
Decompression: Apply Wimg iteratively to any i
• Stop when error falls below some bound• Error can be calculated by "Collage Theorem"
![Page 83: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/83.jpg)
e
roringRiptation
06-compression.fm 83 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz How to Find Wimg?
Systematic search based on"Partitioned Iterative Function System (PIFS)"• Partition image into "range blocks" Ri
• 8*8 pixel blocks• non-overlapping
• Consider all "domain blocks" Dj of double siz• 16*16 pixel blocks• overlapping
• Find for each Ri the most similar Dj• consider rotations (0o/90o/180o/270o) and mir• adapt brightness and contrast of Dj to that of • translation, rotation, mirroring, brightness ada
define a (partial) affine function• Combine partial functions to Wimg
![Page 84: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/84.jpg)
s
r squares
06-compression.fm 84 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz Further Improvements
Quadtree partitioning:• Problem:
• fixed 8*8 blocks do not reflect image propertie• Solution:
• flexible partition of image into larger or smalle• driven by image structure
Partitioning into rectangles and triangles
![Page 85: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/85.jpg)
about 1:10 to 1:30)
f "pixelization"
g image
ain block pairs
turepossible
rates
06-compression.fm 85 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz Advantages & Drawbacks
+ High quality at high compression rates• At least for images with self-similarities• Here: better than JPEG ("cross-over point" at
+ Zooming into image supported• detailed view possible, interpolation instead o
+ Scalabilitty• decompression steps yield iteratively improvin
- Long compression times• asymmetric mechanisms• improving search techniques for range & dom
- blockwise artifacts with Information losses• Wimg is only approximative
- Not well applicable to images of non-fractal na• E.g. texts, sharp lines & no quality guarantee
- Lower quality than JPEG at low compression
- Error (Fehlerfortpflanzung)
![Page 86: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/86.jpg)
Schemes
-MLQ): 6,3 kbpsion (ACELP) 5,3 kbps
06-compression.fm 86 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz 14. Basic Audio and Speech Coding
Background• ITU driven activities
G.711: PCM • with 64 kbps
G.722 differential PCM (DPCM)• 48, 56, 64 kbps
G.723• Multipulse-maximum Likelihood Quatizer (MP• Algebraic Codebook Excitation Linear Predict• application: speech
![Page 87: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/87.jpg)
n (LD-CELP)
ue to CODEC algorithm)
rd error correction FEC) in Japan)
06-compression.fm 87 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz Schemes for Speech Coding
G.728: Low Delay Code Excited Linear Predictio• used in audio/video conferencing• 16 kbps• one-way end to end delay less than 2 msec (d• complex algorithm
• 16-18 MIPS in floating point required• appr. 40 MIPS whole encoding and decoding
AV.253• still “under consideration” at ITU• 32 kbps
IS-54• VSELP
• good for voice• bad for music
• 13 kbps (appr. 8 kbps voice + 5.05 kbps forwa• driving force: Motorola (similar developments
![Page 88: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/88.jpg)
Networks
r
06-compression.fm 88 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz Speech Coding in Mobile Telephone
RPE-LTP (GSM)• Regular Pulse Excitation - Long-Term Predicto• used in European GSM: speech• 13 kbps
GSM Half-Rate Coders• 5.6 - 6.25 kbps• quality and characteristics similar to RPE-LPT
![Page 89: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/89.jpg)
sl coding)
ch frequency band)
or 200 -300 Hz
300 -450 Hz
2.800 -3.400 Hz
tornoisegenerator
or
or
switch
…
encoded Speech
replicated foreach frequencyband
…
06-compression.fm 89 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz Vocoder: e.g. Inmarsat IMBE Coder
Improved Multiband Excitation Coder IMBE• application: maritime satellite communication• 4,15 kbps for voice (plus 2,25 kbps for channe
Principle: Vocoder • (IMBE voiced and unvoiced individually for ea
200 -300 Hz
300 -450 Hz
2.800 -3.400 Hz
pitch analysis
.DC+ lowpass
DC+ lowpass
DC+ lowpass
modulat
pulsgenera
modulat
modulat
… ……
Speech input
![Page 90: Multimedia-Systems: Compressionextras.springer.com/2000/978-3-540-66757-5/vorlesung/A...KOM - Industrial Process and System Communications, Tel.+49 6151 166151, Merckstr. 25, D-64283](https://reader034.vdocuments.us/reader034/viewer/2022042200/5e9fbde19ad4df1f041e3c8d/html5/thumbnails/90.jpg)
tio
ta rates
s
06-compression.fm 90 22.October.99
htt
p:/
/ww
w.k
om
.e-t
ech
nik
.tu
-da
rmst
adt
.de
htt
p:/
/ww
w.ip
si.g
md
.de
© R
alf
Ste
inm
etz 15. Conclusion
JPEG:• Very general format with high compression ra• SW and HW for baseline mode available
H.261 / H.263:• Established standard by telecom world• Preferable hardware realization
MPEG family of standards:• Video and audio compression for different da• Asymmetric (focus) and symmetric
Proprietary systems: e.g. Quicktime Product• Migration to the use of standards
Next steps: wavelets, fractals, models of object