9. image compression computer engineering, sejong university...

Image Processing

9. Image Compression

Computer Engineering, Sejong University

Dongil Han

2/64

The storage and communication of video requirements are immense• storage requirement for uncompressed video

512pixel x 512 pixel x 3bytes/pixel x 30frame/sec = 23.6MBytes/sec

image compression : reducing the amount of data required to represent a digital imageex) 23.6MBytes =>187KBytes

Fundamentals of image compression• reduce or eliminate the redundant data in the image or

video

Image Compression

.

3/64

application• Digital TV broadcasting• VOD(Video on Demand)• Televideo-conferencing• Medical imaging• Facsimile transmission• or diverse Multi-media environment

=> mainly used for storages and transmissions

Evaluation of compression efficiency• compression ratio

compression ratio = original data/compressed data

• the greater the compression ratio, the smaller the final image will be

Image Compression

.

4/64

Selection criteria for image compression techniques• achievable compression ratio• compression/decompression time• complexity of algorithm• cost of computational resources• availability of computational resources• adoption of standard or not • lossy or loseless compression • the quality of decoded image when lossy compression is

used

Image Compression

.

5/64

Fundamentals

Two basic types of image compression• Lossless compression : encodes and decodes the data

perfectly. resulting image matches the original image exactly• Lossy compression : allows redundant and nonessential

information to be lost

Data compression• process of reducing the amount of data required to represent

a given quantity of information• key idea : reducing the data redundancy

Three basic data redundancies in digital images• coding redundancy• interpixel redundancy• psychovisual redundancy

.

6/64

Elimination of coding redundancy• assigning fewer bits to the more probable data• assigning longer bits to the less probable data• this process is referred to as variable length coding• reduces the total number of encoded bits

• Image data : assigning fewer bits to the gray levels with higher historam

Fundamentals

.

7/64

coding redundancy• discrete random variable rk in the interval [0,1] represents

the gray levels of an image• each rk occurs with probability pr(rk)

where L is the number of gray levels• If the number of bits used to represent each value of rk is

l(rk), then the average number of bits required to represent each pixel is

• Elimination of coding redundancy => minimize the Lavg

n

nrp kkr )(

1

0

)()(L

kkrkavg rprlL

Fundamentals

8/64

Elimination of coding redundancy

bits

rprlLk

krkavg

7.2

)02.0(6)03.0(6)06.0(5)08.0(4

)16.0(3)21.0(2)25.0(2)19.0(2

)()(7

02

code 1 code 2

bits

rprlLk

krkavg

0.3

)02.0(3)03.0(3)06.0(3)08.0(3

)16.0(3)21.0(3)25.0(3)19.0(3

)()(7

01

Fundamentals

9/64

Elimination of coding redundancy

Fundamentals

10/64

Elimination of interpixel redundancy• use similarities between adjacent pixels• use similarities between adjacent fields• Run-length encoding, DPCM, ADPCM, etc.

• Terminologies for representing interpixel redundancy- spatial redundancy- geometric redundancy- interframe redundancy

Fundamentals

.

11/64

Elimination of interpixel redundancy

Fundamentals

12/64

Elimination of interpixel redundancy

Fundamentals

13/64

Elimination of psychovisual redundancy• brightness of region depends on factors other than

simply the light reflected by the regions• certain information has less importance than other

information in normal visual processing• this information is said to be psychovisual redundant• this redundancy can be eliminated without significantly

impairing the quality of image perception=> results in a loss of quantitative information

• related to the sampling, quantization

Fundamentals

.

14/64

Elimination of psychovisual redundancy

Fundamentals

15/64

Fidelity Criteria• reproducible means of quantifying the nature and extent

of information loss– objective fidelity criteria(객관적 충실도 기준)– subjective fidelity criteria(주관적 충실도 기준)

• objective fidelity criteria offer a simple and convenient mechanism for evaluating information loss

• sometimes, subjective evaluations by a human observer is more appropriate

Fundamentals

.

16/64

Example of objective fidelity criteria• root-mean-square(rms) error between an input and output

image• Let be input and estimate of input image,

then the total error between the two images is

where the images are of size MxN• The root-mean-square erms between the two images is

),(ˆ),,( yxfyxf

1

0

1

0

)],(),(ˆ[M

x

N

y

yxfyxf

2/11

0

1

0

2)],(),(ˆ[1

M

x

N

yrms yxfyxf

MNe

Fundamentals

17/64

Example of objective fidelity criteria• mean-square signal-to-noise ratio of the estimated

image, denoted SNRms is

1

0

1

0

2

1

0

1

0

2

)],(),(ˆ[

),(ˆ

M

x

N

y

M

x

N

yms

yxfyxf

yxf

SNR

Fundamentals

18/64

Image Compression Model

Image Compression System• two distinct structural blocks : Encoder and Decoder• source encoder : removes input redundancies• channel encoder : increases the noise immunity• noise free environment, the channel encoder/decoder are

omitted

Source encoder

Channel encoder Channel Channel

decoderSource decoder

Encoder Decoder

),( yxf ),(ˆ yxf

.

19/64


Source encoder• reduces or eliminates any coding, interpixel, psychovisual

redundancies in the input image

Source encoding stages• Mapper : reversible• Quantizer : irreversible• Symbol encoder : reversible

Mapper Quantizer Symbol encoder Channel

Source Encoder

),( yxf

.

20/64


Mapper• transforms the input data into a format designed to reduce

interpixel redundancies in the input image• may or may not reduce the amount of data• example : run-length coding• transformed results : array of coefficients

Quantizer• reduces the accuracy of the mapper’s output• reduces the psychovisual redundancy• must be omitted when error-free compression is desired

Symbol Encoder• reduces the coding redundancy• creates a fixed- or variable-length code

.

21/64


Source Decoder• performs the inverse operations of the source encoder• inverse quantizer block is not included in the general

source decoder

Source Decoding stages• Symbol decoder : inverse operations of the symbol

encoder• Inverse mapper : inverse operations of the mapper

Symbol Decoder

Inverse Mapper

Channel

Source Decoder

),(ˆ yxf

.

22/64

Error-free compression

Error-free compression• means for data reduction• reduces interpixel, coding redundancy

Application• medical or business documents• satellite imagery : cost of collecting the data• digital radiography(방사선 사진) : for diagnostic accuracy• normally provide compression ratios of 2 to 10

Error-free compression techniques• Variable-length coding• Arithmetic coding• Bit-plane coding : Run-length coding• Lossless predictive coding, etc.

.

23/64

Variable-length coding • the simplest approach to error-free compression technique• reduces coding redundancy only • assigns the shortest possible code words to the most

probable gray levels• source symbol could be gray levels or pixel differences of

an image, output of run-length encoding, etc.

Types of variable-length coding • Huffman coding• Truncated Huffman coding• Shift coding• Huffman shift coding, etc.


.

24/64

Huffman coding• In 1952, a paper by David Huffman was published• variable length codes can achieve a higher data density than

fixed length codes• assigns short code for the most frequently occurring data• yields the smallest possible number of code symbols per

source symbol• resulting code is optimal with constraint that the source

symbols be coded one at a time

Huffman coding process1. Ordering the probabilities of symbols2. Combining the lowest probability symbols into a single

symbol. This process is repeated until a reduced source with two symbols is reached

3. To code each reduced source starting with the smallest source4. Working back to the original source


25/64

symbol reductions

Code assignment procedure

Huffman coding

26/64

전치 특성(Prefix property)• 모든 코드는 다른 코드의 prefix가 될 수 없는 성질

– 예 : 영문자 e의 코드가 “01”일 경우 “010”, “011” 혹은“0100” 등의 코드는 존재하지 않는다

– 즉 e의 코드 “01”이 다른 코드의 prefix가 될 수 없다.– 만약 “010”을 b의 코드로 정하면 “010”을 복호화 할 때다음의 모순 발생

=> “01” 을 e로 복호화 혹은=> “010”을 b로 복호화

Another property• bit operation is required

Huffman coding

27/64

Advantages• creates the optimal code for a set of symbols and

probabilities • coding/decoding is accomplished in a simple lookup table• block code : each source symbol is mapped into a fixed

sequence of code symbols• can be decoded without referencing succeeding symbols

Disadvantages• encoding requires two nontrivial passes over the data

(probability calculation, code table creation)• one corrupted bit will wipe out the rest of the data

(disadvantage of variable length coding)• when a large number of symbols is decoded, the

construction of the optimal code is an nontrivial case

Huffman coding

.

28/64

Other near optimal VLCs

29/64

Arithmetic coding• nonblock code : a sequence of source symbols is

assigned a single arithmetic code word• the code word defines an interval of real numbers

between 0 and 1• as the number of source symbols increases, the interval

used to represent it becomes smaller• any number within the subinterval can be used to

represent the source symbol• encoding requires two nontrivial passes over the data

– probability calculation– code table creation


.

30/64

Arithmetic coding example


31/64

LZW(Lempel-Ziv-Welch) coding• assigns fixed-length code words to variable length

sequences of source symbols• for 8-bit images

– first 256 words of the dictionary are assigned to the gray values 0, 1, …, 255

– image sequences are added to the next code word (39-39 => 256, 39-126=>257, etc.)

Properties of LZW coding• requires no a priori knowledge of the probability• codebook is created while the data are being encoded• repeated source symbols such as “the”, “. “ are

compressed effectively• has been integrated into gif, tiff, pdf file format, etc.


.

32/64

LZW coding example


33/64

Bit-plane coding• effective technique for reducing an image’s interpixel

redundancy• process the image’s bit planes individually• compress each binary image via well-known binary

compression methods


.

34/64

Bit-Plane Coding

Bit-plane decomposition• small changes in gray levels could generate many bit

change(e.g. 127(01111111) => 128(10000000))• m-bit gray code => reduces the bit change

– successive code words differ in only one bit position– small changes in gray levels are less likely to affect all

m bit planeDecimal number BCD code gray code

0 0000 0000 1 0001 0001 2 0010 0011 3 0011 0010 4 0100 0110 5 0101 0111 6 0110 0101 7 0111 0100 8 1000 1100 9 1001 1101

.

35/64

original image

original gray coded original gray coded

Bit-Plane Coding

36/64

Run-length coding• reduces interpixel redundancy• binary image compression methods• applied each bit-plane of gray level image• describe successive runs of black and white pixels• standard compression approach in FAX coding• compression technique in bmp file• applied for 1, 2, 4-bit gray level image• additional compression can be realized by variable-length

coding the run lengths themselves

– input : AAAABBBBBCCCCCCCCDEEEE– output : 4A5B8C1D4E– compression ratio : 22/10 = 2.2


.

37/64

Run-length coding

Notes for RLC• If compression ratio is less than 1, send original data

– input : MyDogHasFleas– output : 1M1y1D1o1g1H1a1s1F1l1e1a1s– compression ratio : 13/26 = 0.5

• Use special prefix character (for example: +)– input : ABCDDDDDDDDEEEEEEEEE– output : ABC+8D+9E– compression ratio : 19/9 = 2.11

• It make sense to encode only runs of 3 or longer• If the special prefix character is found in the source : use

3 byte notation(run of length 1: + => +1+)

38/64

Lossless predictive coding• reduces interpixel redundancy• does not require decomposition of an image into bit planes• extracts and codes only the new information in each pixel• new information : the difference between the actual and

predicted value of that pixel


.

39/64

Lossless predictive coding

Original image

Prediction error image

Histogram

Histogram of

prediction error


40/64

Lossy compression

Lossy compression• compromise the accuracy of the reconstructed image in

exchange for increased compression• reduces interpixel, coding redundancy, psychovisual

redundancy

Application• Digital TV : MPEG-2, Video conferencing : H.261• still image coding: JPEG• normally provide compression ratios of 10 to 100

Lossy compression techniques• Lossy predictive coding• Transform coding• Hierarchical coding• Hybrid coding, wavelet coding, etc.

.

41/64

Lossy Predictive Coding

Lossy predictive coding• predictive coding with quantizer• predictions generated by the encoder and decoder are

equivalent• well-known form of lossy predictive coding

- DM(Delta Modulation)- most simple

• other lossy predictive coding techniques- ADM(Adaptive Delta Modulation)- DPCM(Differential Pulse Code Modulation)- ADPCM(Adaptive Differential Pulse Code Modulation)

• optimal predictor : DPCM- minimizes the encoder’s mean-square prediction error

.

42/64

Lossy predictive coding

Lossy predictive coding model


43/64

DM(Delta Modulation)• utilize a 1-bit fixed-length code• delta modulation process

- evaluate the input and present encoded level- allocate 1 when input level exceeds present encoded

level- else allocate 0

• implementation- positive/negative constant delta value is assigned to

each code- code 1 : + delta- code 0 : – delta

• relatively smooth region : granular noise appears• rapidly changing region : slope overload appears


.

44/64

DM(Delta Modulation)


45/64

Problem of Delta Modulation :• big delta : granular noise is increased• small delta: slope overload is increased

ADM(Adaptive Delta Modulation)• adaptively adjust the delta value• relatively smooth region

- reduce the delta value- encodes the small variation of input signal- granular noise is reduced

• rapidly changing region- increase the delta value- follows rapidly the input signal- slope overload is reduced


.

46/64

DPCM(Differential Pulse Code Modulation)• minimizes the encoder’s mean-square prediction error

• optimal predictor• prediction is constrained to a linear combination of m

previous pixels

}]ˆ{[}{ 22nnn ffEeE


47/64

)1,(97.0),(ˆ yxyxf ),1(5.0

)1,(5.0),(ˆ

yxf

yxfyxf

)1,1(5.0),1(75.0

)1,(75.0),(ˆ

yxfyxf

yxfyxf

otherwiseyxf

vhifyxfyxf

),1(97.0

)1,(97.0),(ˆ

|)1,1()1,(|

|)1,1(),1(|

yxfyxfv

yxfyxfh

Linear Prediction Example

48/64

Quantization example


49/64

DPCM example

1 bits/pixel 1.25 bits/pixel

2 bits/pixel

2.125 bits/pixel

3 bits/pixel3.125

bits/pixel

50/64

1.25 bits/pixel

2.125 bits/pixel

3.125 bits/pixel

1 bits/pixel

2 bits/pixel

3 bits/pixel

DPCM error example

51/64

Compression with transform coding• Most information is located in the low frequency region• modifying the transform of an image• map the image into a set of transform coefficients, which

are then quantized and coded• significant number of the coefficients have small magnitude • and can be coarsely quantized with little image distortion

Transform coding techniques• image is divided into subimages of size n x n (8x8, 16x16)• transform each block

- pack as much information as possible into the smallest number of transform coefficients

• more coarsely quantizes the coefficients that carry the least information

• final stage : usually VLC the quantized coefficient is used

Transform Coding

.

52/64

Typical transform coding system

Transform Coding

53/64

Transform coding• transform kernel(basis function) constructs each

transform coding techniques

• Types of transform coding- KLT((Karhunen-Loeve Transform)- DFT(Discrete Fourier Transform)- DCT(Discrete Cosine Transform)- WHT(Walsh-Hadamard Transform)

),,,(),(1

),(

),,,(),(),(

1

0

1

0

1

0

1

0

vuyxhvuFN

yxf

vuyxgyxfvuF

N

u

N

v

N

x

N

y

Transform Coding

.

54/64

Types of transform coding• Discrete Fourier Transform(DFT)

• Walsh-Hadamard Transform(WHT)

• Discrete Cosine Transform(DCT)

NvyuxjNvyuxj evuyxheN

vuyxg /)(2/)(22

),,,(,1

),,,(

1

0

)()()()(

)1(1

),,,(),,,(

m

iiiii vpybupxb

Nvuyxhvuyxg

]2

)12(cos[]

2

)12(cos[)()(),,,(),,,(

N

vy

N

uxvuvuyxhvuyxg

1,...,2,1,2

0,1

)(),(

NvuforN

vuforN

vu

Transform Coding

55/64

Transform kernel example

Walsh-Hadamard basis function for N=4

Discrete-cosine basis function for N=4

Transform Coding

56/64

Approximations using

(a) Fourier

(b) Hadamard

(c) Cosine

(a)

(b)

(c)

difference

8x8 block

Comp. ratio : 2

rms error:

(a):1.28

(b):0.86

(c):0.68

57/64

RMS error comparision• Reconstruction error versus subimage size comparison

Transform Coding

58/64

Transform coding quantization• quantization of transform coefficients

),(

),(),(ˆ

vuZ

vuFroundvuF

Typical normalization matrix : Z(u,v)

Transform Coding

59/64

128136132140150157161154

132140140161154157168161

140154157150154161161164

132154161143154157161161

132147157157161154171164

154143161164164147168171

164150171157150161154171

154164168154150161161168

24232011

43311322

214743895

1591521512

66101544108

105339846

6153513112534

6111020349214

DC component

AC component

Original image

DCT

Transform coding quantization• DCT example

Transform Coding

60/64

24232011

43311322

214743895

1591521512

66101544108

105339846

6153513112534

6111020349214

00000000

00000000

00000000

00000001

00000011

00000100

00001123

000010413

Quantization

9910310011298959272

10112012110387786449

921131048164553524

771031096856372218

6280875129221714

5669574024161314

5560582619141212

6151402416101116

W

Quantization Matrix

Quantization Example

61/64

Video Compression Technique

Typical compression technique with DPCM, DCT and VLC

62/64

Zig-Zag scan Alternate scan

00000000

00000000

00000000

00000001

00000011

00000100

00001123

000010413

13 4 3 0 -2 0 1 1 0 1 -1 -1 1 1 0 0 0 … 0

Zig-Zag scan

Scan• AC values are strung together in a sequence• this irregular ordering keeps low frequency coefficient

together


63/64

[ 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 3 1 0 0 0 0 2 ]

(5, 1) (11, 3) (0, 1) (4, 2)

A

B

C

0

10

1

ABC

0.50.30.2

01011

Symbol Probability Code

Run-Length Coding• zig-zag 시퀀스를 (앞에 있는 ‘0’의 개수, ‘0’이 아닌 상수)의형태로 표현

Huffman Coding• 출현 빈도가 높은 값에 적은 비트 할당

• 무손실 부호화


64/64

160162161159

163160155150

156153151144

153149144139

1027

22911

361723

5121154

Original image ordifference between original & predicted images

0000

0011

0012

01010

00000001001112010

DCTQuantization

/16

Zig-Zag scan

00

12

10

10

10

21

100

Run-Length CodeHuffman Code

10011011100011...

Video Compression Process

9. image compression computer engineering, sejong university...

Documents