digital image processing chapter 11. image compression 11. image compression 11.1. introduction...

Digital image processing Chapter 11. Image compression

11. IMAGE COMPRESSION

11.1. Introduction 11.2. Pixel level coding PCM coding Entropic coding RL coding Arithmetic coding 11.3. Predictive coding techniques Principles of the predictive coding Delta modulation 1-D DPCM 2-D DPCM Comparison of the performances of various predictive coding methods 11.4. Transform-based image compression techniques Bits allocation Transform-based image compression Zonal coding; threshold-based coding Transform-based coding with visibility criteria Adaptive transform-based coding 2-D DCT based image coding Image compression algorithm based on DCT 11.5. Hybrid coding; vectorial DPCM based coding 11.6. Interframe coding Principles of the inter-frame coding Predictive coding with motion compensation Motion estimation algorithms Hybrid inter-frame coding

11. 1. INTRODUCTION

• Def. Compression = minimization of the number of bits needed to represent the visual information in the scene, without noticeable loss of information• Applications:

• image transmission• image storage in multimedia/image databases


The need for image compression:• huge sizes of the data files, in uncompressed form• the storage devices have relatively slow access =>impossible to render/index image info in real time• limited bandwidth of the communication channels => impossible to do image transmission in real time

TEXT IMAGINI AUDIO ANIMATIE VIDEO

TIPUL - ASCII - EBCDIC

-grafica bitmap -fax -imagini statice

-secvente necodate audio sau voce

- sunet si imagini cu 15-19 cadre/s

- TV analogic[ sau imagini cu 24-30 cadre/s

DIMENSIUNE FISIER SI

LARGIME DE BANDA

2kB/pagina - simple (320x200x8biti) 77kB/imagine -complexe (true color) (1100x900x24biti) 3Mb/imagine

-voce/telefon 8kHz/8biti (mono) 6-44kB/s - audio CD 44,1 kHz 16 biti/ stereo 176 kB/s (44,1x2chx16biti)

-secvente 16biti/pixel, 16cadre/s 6,5 Mb/s (32x640x16bitix16cadre/s)

-secvente color pe 24 biti cu 30cadre/s 27,6 Mb/s (640x480x24bitix30cadre/s)

Memory needed for storage before andafter compression

text 2:1, color images 15:1, maps 10:1, stereo sound 6:1, animation 50:1, video 50:1.


•Shannon => the average amount of information per symbol:

L = number of symbols

=> It is possible to lossless encode the source having the entropy H by (H+e) bits/symbol, e>0, e->0. The compression factor (ratio) C:

•Maximal (ideal) compression can be achieved by vector quantization (theoretically). Practically – near optimal only:

1-L

ii2i lbiti/simboplogpH

1

H

B

eH

BC

e)(Hdata compressed theof lbits/symbo ofnumber Average

(B)data eduncompress theof lbits/symbo ofnumber Average


11. 2. PIXEL LEVEL CODING

11.2.1. PCM (pulse code modulation)

• The minimal number of bits per pixel possible to be achieved by PCM:

11.2.2. Entropic coding (Huffman)

= coding one block of M pixels, M – small; without compression: B bits/pixel needed => MB bits/block=> the source–has L=2MB symbols => any block can be encoded on less than MB bits, according to:

=> entropic coding principle: use a codebook with variable length code words, the length of the code word is different for each block as follows:

- the more frequent blocks (pi large) => have -log2pi smaller => shorter code word

- the less frequent blocks (pi small) => have -log2pi larger => longer code word

Sample and hold

y Zero-memory quantizer; encoder

Digital modulation

u(t) u(nT) gi(n) Transmission

R1

2log ;PCM 2

u2

q2 q

2u

2

σ2u= the variance of the signal at the input

of the quantizer; σ2q= quantization error

H p log pi ei 2i 0

L 1

= average bit rate


• Huffman coding algorithm:

1) Order decreasingly the probabilities pi , i=1,2,…,L, and write them in a list of symbols.2) Chose from the list the last 2 elements (with the smallest probabilities, pk , pl) . From the 2

=> create a node with the probability pk+pl => and insert it in the ordered list in the right position; erase pk, pl from the list

3) Asign the codes: 0 to k ; 1 to l4) Repeat steps (1)…(3) Symbol pi -log2(pi) Code

Subtotal

A 15/39 1.38 0 2*15 B 7/39 2.48 111 3*7 C 6/39 2.70 110 3*6 D 6/39 2.70 101 3*6 E 5/39 2.96 100 3*5 Total: 87 bitsBinary tree H=87/39=2.23 bits per pixel in average


1) Decoding:

Simbol Code

A 0 B 111 C 110 D 101 E 100

The received string: 1000100010101010110111 (22 bits)

Decode!


• Versions of the Huffman coding:

* the truncated Huffman code- first L1 symbols – Huffman encoded;- the remaining L-L1 are encoded using a prefix code followed by a fixed length code

* the modified Huffman code- the integer number “i” is represented as i=qL1+j- encode first L1 symbols with a Huffman code- if above L1 => encode q by a prefix code + use a terminator code = Huffman code of j

1Lj0,L

1)(LIntq0,jqLi 1

11

11.2.3. RLC (run length coding)

=> For binary images, 0 – white, 1- black; p(white)>>p(black); encode the length of the white run=> applications: fax; sketches

p(white)=p(0)=p; p(black)=p(1)=q; p→1. Notations:

M = maximum length of a run of zeros; M=2m-1 => m bits to encode the run


11.2.3. RLC - continued

The probability of appearance of a run of l symbols ‘0’, 0 ≤ l ≤ M, followed by a ‘1’:

The average number of symbols/run:

The compression ratio:

The average number of bits per pixel:

The coding efficiency: can be increased by the Huffman coding or by the arithmetic coding of the run lengths (instead of the uniform m bits encoding)

Ml;p

1Ml0;p)(1pg(l)

M

l

(l 1)p (1 p) Mp1 p

1 pl M

M

l 0

M 1

Cm

1 - p

m(1 p)

M

mm

Ba

cursa o-intr continuti "0" de biti de mediu nr.

incheiere de "1" bitul"0" de cursa coda apentru biti nr.

aB

Hh


11.2.3. The arithmetic coding=> A single codeword is used for an entire sequence of symbols=> The code = a number in a sub-range of [0;1).

Source’s symbol Probability Initial sub-range

a1 0,2 [0,00,2)a2 0,2 [0,2 0,4)

a3 0,4 [0,4 0,8)

a4 0,2 [0,8 1,0)

* Decoding: 1) 0.068 [0.0; 0.2) => a1; (0.068-0.0)/(0.2-0.0)=0.34=> 2) 0.34 [0.2; 0.4) => a2; (0.34-0.2)/(0.4-0.2)=0.7

=> codeword:Any number in[0.0624;0.0688)ex. 0.068


Arithmetic decoding - continued

=> 3) 0.7 [0.4; 0.8) => a3; (0.7-0.4)/(0.8-0.4)=0.75=> 4) 0.75 [0.4; 0.8) => a3; (0.75-0.4)/(0.8-0.4)=0.875=> 4) 0.875 [0.8; 1) => a4

=> The decoded sequence: a1 a2 a3 a3 a4

* Drawbacks of the arithmetic coding:- limited by the representation precision for long sequences - an end-of-message flag is needed

* Alternative solutions: re-normalization and rounding

11.2.4. Bits planes encoding=> e.g. 8 planes of 1 bit, independently encoded (MSB …. LSB)

=> disadvantage: sensitive to errors


11. 3. PREDICTIVE CODING

11.3.1. DPCM coding principles= Maximize image compression efficiency by exploiting the spatial redundancy present in an image!


Line-by-linedifference of the luminance

Many close to zero data => spatial redundancy, the brightnessis almost repeating from a point to the next one! => no need toencode all brightness info, only the new one!

see images: one can “estimate” ( “predict”) the brightness of the subsequent spatial point based on the brightness of the previous (one or more) spatial points = PREDICTIVE CODING

11. 3. PREDICTIVE CODING

11.3.1. DPCM coding principles – continued – BASIC FORMULATION OF DPCM:Let {u(m)} – the image pixels, represented on a line by line basis, as a vector. Consider we already encoded u(0), u(1),…, u(n-1); then at the decoder => only available their decoded versions ( original + coding error), denoted: u•(0), u•(1),…, u•(n-1)

=> currently to do: predictive encode u(n), n-current sample:(1) estimate (predict) the grey level of the nth sample based on the knowledge of the previously encoded neighbor pixels u•(0), u•(1),…, u•(n-1):

(2) compute the prediction error:

(3) quantize the prediction error e(n) and keep the quantized e•(n); encode e•(n) PCM and transmit it.

At the decoder: 1) decode e => get e•(n) + 2) build prediction + 3) construct u•(n):

* The encoding & decoding error:

2)...)(n1)u(n(u(n)u

(n)uu(n)e(n)

(n)e(n)u(n)u

q(n)(n)ee(n)(n)uu(n)u(n)


(n)u

11. 3. PREDICTIVE CODING - continued

Basic DPCM codec

DPCM codec: (a) with distorsions; (b) without distorsions


11.3.2. Delta modulation

=> the simplest DPCM encoder; prediction from 1 sample, 1 bit quantizer.

=> alternatively, one can predict the value of the sample as:

with: (n)= zero-mean error; - factor which minimizes the mean square error (leak factor); given by the error

covariance equation:

=> SNR can be estimated as:

1)(nuu(n)e(n)1);(nu(n)u

n)(m)(1(m)(n)E 2u

2

dB)f(1)2(1

1)f(1)}(2{110log(SNR) 10M


(n);1)(nu(n)u

Delta modulation result for images: q=6,5; =1

OriginalCompressed

(compression rate 8:1)


11.3.2. 1-D DPCM (line-by-line DPCM)

=> is the usual version of applying DPCM on digital images. The sequence to encode = one image line; consider the sequence {u(n)} stationary => one can use a p order linear predictor:

where a(k) are chosen to minimize:

**The equations of 1-D DPCM codec:

(1) Predictor:

(2) Quantizer’s input:

(3) Quantizer’s output: e(n)(4) Decoder:

=> Estimation of the SNR:

p

1k

22 ](n)eE[e(n);k)(na(k)uu(n)

22 ](n)eE[

p

1kk)(na(k)u(n)u

(n)uu(n)e(n)

(n)ek)(nua(k)(n)e(n)u(n)up

1k

[dB])f(B)(1

f(B)(110log(SNR)

2

2

10DPCM

=> Average bit rate reduction vs. PCM:

pixelbiti/1

1log

2

1RR

22DPCMPCM


11.3.2. 1-D DPCM - Comparison to PCM; the effect of error propagation:

PCM 8 bits PCM 3 bits

The error in 3 DPCM on 3 bits DPCM 3 bits


11.3.2. 1-D DPCM - Comparison to PCM; the effect of error propagation:


11.3.2. 2-D DPCM

=> prediction as a function of one neighbor on the same line and 3 neighbors on the previous line:

with a(k,l) chosen to minimize

r(1,0) a1r(0,0) a2r(1,-1) a3r(0,1) a4r(0,1)

r(0,1) a1r(1,-1) a2r(0,0) a3r(1,0)a4r(1,-2)

r(1,1) a1r(0,1) a2r(1,0) a3r(0,0) a4r(0,2)

r(1,-1) a1r(0,1) a2r(1,-2) a3r(0,2) a4r(0,0)

2 E2(m,n) r(0,0) - a1r(1,0) - a2r(0,1) - a3r(1,1) - a4r(1,-1)

r(k,l) = covariance of u(m,n). If r(k,l) is separable =>:

22 ](n)eE[

1)n1,(mua1)n1,(mua1)n(m,uan)1,(muan)(m,u 4321

n

m B C D

A X

0a,a,a,a 42132211 ))(1(122 22

21


11.3.2. 2-D DPCM - continued

**Equations of 2-D DPCM:

(1) Predictor:

(2) Quantizer’s input:

(3) Quantizer’s output: e(n)(4) Decoder:

** Comparison of the performances of the predictive coding methods:

1)n1,(mua1)n1,(mua1)n(m,uan)1,(muan)(m,u 4321

n)(m,un)u(m,n)e(m,

n)(m,e+n)(m,un)(m,u

SNR (dB)

Rata (biti/esantion)

35

30

25

20

15

10

5 0

1 2 3 4

2D DPCM (maxim)

2-D DPCM cu predictie cu trei puncte

DPCM linie cu linie

PCM


11.4. TRANSFORM-BASED IMAGE COMPRESSION( “block quantization”)

• The essence of transform-based image compresssion: (1) an image block is transformed using a unitary transform, so that a large percent from

the energy of the block is packed in a small number of transform coefficients (2) the transform coefficients are independently quantized

• The optimal transform-based encoder = the encoder which minimizes the mean square error of the decoded data, for a given number of bits the Karhunen-Loeve based encoder.

• Transform-based image coding algorithm:u = a random vector, u[N×1], of zero mean; v=A∙u – a liniar transform of u, so that v(k) – not correlated => they can be quantized independently => v’(k); u’=B∙v’ – inverse

transform


11.4.1. Main issues to be solved in transform-based image coding: (1) Finding the optimal transform matrices: A[N×N], B[N×N], able to minimize the

mean square value of the distortion for a given number of bits, D:

(2) Designing the optimal quantizers for v, to minimize D.

• Approaches to solve these issues:(1) For a given quantizer and a forward transform matrix A, the optimal reconstruction

matrix B is:

• The optimal matrix A from the max. de-correlation p.o.v.= the KLT of u; the lines of A =the eigenvectors of the covariance matrix R of u. Then: B=A-1=A*T. Unfortunately KLT does not have a fast computation algorithm => is typically replaced in compression by: DCT; DST; DFT; Hadamard; Slant good compromise between decorrelation performance & computational speed

)u-(u)u-(u T ''

1

2' ))()(( EN

1nunuE

N

1D

N

n

1AB

k

kk '

N

...00.

.

.

.

.

.

.

.

.

.

.

.0...0

0...0

2

1

2'' )(kvEk )(')( * kvkvEk


11.4.2. Bits allocation:• The variance of v(k) is typically non-uniform => different number of bits needed on

each coefficient for the same quantization error; v(k) → nk

• The optimal design of the transform-based encoder: how to efficiently allocate the total available number of bits between the transform coefficients, so that D – minimized; for A – some unitary matrix, B=A-1=A*T:

where: σk2 = the variance of v(k); nk = the number of bits allocated for the quantization of

v(k); f(nk) = the distortion rate of the quantizer, monotonic, f(0)=1; f(∞)=0.

• The transformed-based encoder rate is: ; B= the average number of bits/sample

• Bits allocation = finding the optimal nk, nk ≥ 0, which minimizes D for the average number of bits/sample B given.

D E v k v k f nk

N

k

k

N

k

1

N

1

N( ) '( ) ( )

2

0

12

0

1

R n BA kk

N

1

N 0

1


» Bits allocation algorithm:(1) Define

(2) Find θ = the root of the non-linear equation:

(11.30)

One encodes only the coefficients having σk2 > θ.

(3) The number of bits nk is:

(11.31)

=> The minimum mean distortion:

(11.32)

• If instead of imposing B, one imposes D=d: (1) solve (11.32) => θ; (2) solve (11.31) => nk; (3) estimate B.

xxfxxx )())( (' f;' f)h( 1

R hf

BA

k ki k

1

N

2

02

'( )

nh f

k

k

k k

0,

( '( ) / ),

2

2 20

kk kk

kkk nfN

1D

2: 2:

22 )(


» Practical (simplified) algorithm to allocate the bits for nk - integer:= iterative algorithm, j – the current iteration:

(1) At the beginning of the iterations, set:

(2) Compute:

(3) If => stop the algorithm. Otherwise repeat (2).

1;10,00 jNknk

;max),(1

0

1k

N

kI

jk

jk thatsoIknn

1112 jk

jkkk nfnf

BNnk

jk


11.4.3. Transform-based image compression steps:

(1) The image = U[M×N], U={u(m,n)} vector u[MN×1] => A[MN×MN], B[MN×MN], v[MN×1]• In practice: use DCT, DST, DFT, Slant, Hadamard – fast algorithms• Practical implementation of the transform-based image compression:

(1) divide the image in small sized blocks: U[M×N] => I = MN/pq blocks Ui[p×q], i=0,1,…,I-1,p<<M, q<<N(2) transform + independently encode the blocks Ui[p×q].=> Advantages: (a) memory efficient implementation of the transform, MN/pq times

(b) decrease the number of operations for computing the transform on the factor log2MN/log2pq times

Disadvantage: lower compression rate r.

r – compression rate (bits/pixel)

r

2,5 2,0

1,5

1,0

0,5 2x2 4x4 8x8 16x16 32x32 64x64 128x128

Block size

Fig. 11.11 Compression rate when using a KLT-based compression scheme


• Practical algorithm for transform-based image compression:

* The steps of the algorithm:(1) Divide the input image into blocks Ui, transform them 2-D to get Vi, i=0,..., I-1,(2) Bit allocation:

q

p

p q

uj

u'j

ApUiAT

q Quantizer Encoding Transmit/ storage

Decoding A*TVjA*q p

Fig. 11.12 2-D transform image coding

pq

NMI

Bit allocation for DCT for 16x16 pixels block


(3) Quantizer design:

(4) Encode the quantizer’s output (5) Decode & inverse quantization of the coefficients – at the decoder:

=> One can compute: the mean distortion, the mean bit rate:

)(),(,),(

),(~

,0,0lk

lkvlkv

lk

ii

restin

Ilklkvlkv tlki

i,0

),(,),(),( ,

1

0

1

0,

1

0

1

0,

2, ,)(

p

k

q

llk

p

k

q

llklk n

pq

1Rnf

pq

1D

10-1

10-2

10-3

0,25 0,5 1,0 2,0 4,0 R (bits/pixel)

Cosinus, KL Hadamard Sinus


11.4.4. Zonal coding and threshold-based coding:

• Zonal coding: according to the bit allocation matrix => transmit just N t coeficients = coeficients of the transform with large variances => the zonal mask = mask with values 1 for the sent coefficients, 0 otherwise:

• Threshold based coding: transmit only the coefficients having the amplitude above a threshold (regardless their variance); the set of the transmitted coefficients = I’ t

= threshold-based mask

restin0

Ilk1,lkm t

,

,),(

),(:,' lkvlkI t

restin0

Ilk1lkm t

,

,,),(

'


11.4.4. Transformed-based coding with visibility criteria:

L(m,n) Brigthness

Brightness/ contrast transform f()

FUF Visual quantization

F*VF* Inverse transform

contrast/ brightness f-1()

Frequency filtering

H(k,l)

MMSE quantizer

Inverse frequency filtering

1/H(k,l)

v,(k,l) w(k,l) w'(k,l) v'(k,l)

u v v' u' L'(m,n)

a. The encoding system

b. Visual quantization based on visibility criteria

Fig.11.16 Transform-based codec with visibility criteria (F – the unitary Fourier transform matrix)

11.4.5. Adaptive transform-based coding:

• adapt the transform• adapt the bit allocation• adapt the quantization levels


11.4.6. 2-D DCT –based coding:

Fig.11.17 The influence of the different DCT coefficients

2N

1)l(2ncos

2N

1)k(2mcosn)u(m,(l)(k)l)v(k,

1N

0m

1N

0n

u(m,n) (k) (l) v(k,l) cos(2m 1)k

2Ncos

(2n 1)l

2Nl=0

N 1

k=0

N 1

Nk1orfN

2(k)nd

N

1(0) a


11.4.6. 2-D DCT –based coding- continued:


• DCT-based encoding algorithm:


• DCT-based encoding algorithm – continued:


digital image processing chapter 11. image compression 11. image compression 11.1. introduction...

Documents

image compression slide

entropic coding huffman

huffman coding algorithm

entropic coding principle

interframe coding principles

compression text

hybrid coding vectorial

renderindex image info