Introduction - 1

Wavelets, Fi l ter Banks and Multiresolution Signal Processing

“ I t is wi th logic that one proves;i t is wi th intui t ion that one invents.”

Henr i Poincaré

Introduction - 2

A bit of history: from Fourier to Haar to wavelets

Old topic: representations of functions

1807: Joseph Fourier upsets the French Academy

1898: Gibbs’ paper 1899: Gibbs’ correction

++=

Introduction - 3

1910: Alfred Haar discovers the Haar waveletdual to the Fourier construction

Why do this? What makes it work?• basic atoms form an orthonormal set

Note• s ines/cosines and Haar funct ions are ON bases for• both are structured orthonormal bases• they have di fferent t ime and frequency behavior

+++=

L2 ℜ( )

Introduction - 4

1930: Heisenberg discovers that you cannot have your cake and eat i t too!

Uncertainty principle• lower bound on TF product

t w

w

w

t

t

perfectly localin time

perfectly localin frequency

trade-off

f t( )

f t( )

f t( )

F ω( )

F ω( )

F ω( )

w

t

Introduction - 5

1945: Gabor localizes the Fourier transform ⇒ STFT

1980: Morlet proposes the continuous wavelet transform

short-t ime Fourier transform wavelet transform

time

frequency

time

frequency

Introduction - 6

Analogy with the musical scoreBach knew about wavelets!

Introduction - 7

Time-frequency t i l ing for a sine + Delta

t

f

t

f

t

f

t

f

t

t0 T

t0 T t0 T

t0 T t0 T

f

FT

STFT WT

so.. . .

what is a good basis?

Introduction - 8

1983: Lena discovers pyramids (actually, Burt and Adelson)

+-

x x

MR encoder MR decoder

D II

+

coarse

residual

Introduction - 9

1984: Lena gets crit ical(subband coding)

f1

f2π

−π

−π

π LL LH HL HH

H1 2

H0 2

H1 2

H0 2

HL 2

HΗ 2

H1

H0

HH

HL

LH

LL

x

horizontalvertical

Introduction - 10

1986: Lena gets formal. . .(multiresolution theory by Mallat, Meyer. . . )

= +

Introduction - 11

Wavelets, f i l ter banks and multiresolution analysis

Filter banks(DSP)

Wavelets(applied mathematics)

Multiresolution signal analysis(computer vision)

Construction of bases for

signal expansions

x ψi x,⟨ ⟩ψi∑=

+=

Introduction - 12

Wavelets. . .

‘ ‘Al l th is t ime, the guard was looking at her,f i rst through a te lescope,

then through a microscope,and then through an opera glass. ’ ’

Lewis Carrol l , Through the Looking Glass

Introduction - 13

. . . what are they and how to build them?

Orthonormal bases of wavelets• Haar ’s construct ion of a basis for (1910)• Meyer, Batt le-Lemarié, Stromberg (1980’s)• Mal lat and Meyer ’s mult i resolut ion analysis (1986)

Wavelets from iterated f i l ter banks• Daubechies’ construction of compactly supported wavelets• smooth wavelet bases for and computational algorithms

Relation to other constructions• successive ref inements in graphics and interpolat ion• mult i resolut ion in computer v is ion• mult igr id methods in numerical analysis• subband coding in speech and image processing

Goal: f ind ψ(t) such that i ts scales and shifts form an orthonormal basis for .

L2 ℜ( )

L2 ℜ( )

L2 ℜ( )

Introduction - 14

Why expand signals?

Suppose

Advantages• easier to analyze signal in pieces: “div ide and conquer”• extracts important features• pieces can be treated in an independent manner

original coarse detail

signal block 1 block 2

signal

++

=== projection x elementary signalsΣ

+=

Introduction - 15

Example: Example:

• or thogonal basis• biorthogonal basis• t ight f rame

Note• or thonormal basis has successive approximat ion

property, b ior thogonal basis and frames do not• quant izat ion in orthogonal case is easy,

unl ike in the other cases

ℜ2

ϕ1

ϕ0

e0

e1e1 ϕ1= ~

ϕ1

e0 ϕ0=

ϕ1 e1

e0 ϕ0=

ϕ2ϕ0~

Introduction - 16

Why not use Fourier?

Block Fourier transform: bad frequency localization

Gabor transform: i l l -behaved for crit ical sampling

Balian-Low theorem: there is no local Fourier basis with good t ime and frequency localization

• however: good local cosine bases!

• shi ft and modulat ion

t

t

ω

ω

1/ω

1/ωα, α > 1

Introduction - 17

How do f i l ter banks expand signals?

Analogy

2

2

2

2

x

analysis synthesisH1

H0

G1

G0

y1

y0 x0

x1

+ x̂

beam of

white light

Introduction - 18

. . .and multiresolution analysis?

IDEA: successive approximation/refinement of the signal

original

coarse 1

coarse 2

detail 1detail 1

detail 2 ++

+ =

=

=

Introduction - 19

. . . how about wavelets?

“mother” wavelet ψ

Who?• fami l ies of funct ions obtained from “mother” wavelet

by di lat ion and translat ion

Why?• wel l local ized in t ime and frequency• i t has the abi l i ty to “zoom”

Introduction - 20

Haar system

Basis functions

Basis functions across scales

ψ t( )1 0 t 0.5<≤1– 0.5 t 1<≤0 else⎩

⎪⎨⎪⎧

=

ψm n, t( ) 2 m/2–ψ 2 m– t n–( )=

t

t

t

m = 0

m = 1

m = -1

Introduction - 21

Haar system.. .. . . as a basis for L2 ℜ( )

1 2 3 4 56 7 8

0-8 -7 -6 -5 -4 -3

-2-1

f (0)

1 2 3 4 57 8

0-8 -7 -6 -5 -4 -3

-1

f (1)

24

80

-8-6

-5 -4d (1)

80-8 -5-7 -6 -4

-3 -2 -1 1 2 3 4 5 6 7

f (2)

8-8 -7-5

-3-1 1 3 4 5

7d (2)

t

t t

t t

+

+

=

=

=

geometric proof

Introduction - 22

Haar system.. .. . . scaling function and wavelet

The Haar scaling function(indicator of unit interval)

helps in the construction of the wavelet, since

and satisfies a two-scale equation

Note: • Haar wavelet a bi t too

t r iv ia l to be useful . . .1

1ϕ(2t) + ϕ(2t-1)

1

1ϕ(t)

1

1

1

ϕ(t)1

1

1

ϕ(2t) − ϕ(2t−1)1

1ϕ(t)

=

=

ϕ t( )1 0 t 1<≤0 else⎩

⎨⎧

=

ψ t( ) ϕ 2t( ) ϕ– 2t 1–( )=

ϕ t( ) ϕ 2t( ) ϕ 2t 1–( )+=

Introduction - 23

Discrete version of the wavelet transform

Compute WT on a discrete grid

m = -1

m = 0

m = 1

scale shift

Introduction - 24

Perfect reconstruction f i l ter banks

Perfect reconstruction:

Orthogonal system:

2

2

2

2

x

analysis synthesisH1

H0

G1

G0

y1

y0 x0

x1

+ x̂

ππ/2

H0 H1

magnitude responses of analysis filters

G0H0 G1H1+ I=

H0( )*H0 H1( )*H1+ I= G0 H1( )*=

Introduction - 25

Daubechies’ construction.. .. . . i terated f i l ter banks

Iteration wil l generate an orthonormal basis for the space of square-summable sequences

Consider equivalent basis sequences and (generates octave-band frequency analysis)

Interesting question: what happens in the l imit?

l2 ℑ( )G12

G02

G12

G02

G12

G02

++

+

G0i( ) z( ) G1

i( ) z( )

ππ/2π/4π/8

Introduction - 26

Daubechies’ construction.. .. . . i teration algorithm

At ith step associate piecewise constant approximation of length with

Fundamental l ink between discrete and continuous t ime!

1/2i

g0i( ) n[ ]

0 1 2

Time

0

0.2

0.4

0.6

0.8

Amplitude

0 1 2

Time

-0.2

0

0.2

0.4

0.6

0.8

Amplitude

0 1 2

Time

-0.2

0

0.2

0.4

0.6

0.8

Amplitude

0 1 2

Time

-0.2

0

0.2

0.4

0.6

0.8

Amplitude

i = 1

i = 4 i = 3

i = 2

Introduction - 27

Daubechies’ construction.. .. . .scaling function and wavelet

• Haar and sync systems: either good time OR frequency localization• Daubechies system: good time AND frequency localization

Finite length, continuous and , based on L=4 iterated filterMany other constructions: biorthogonal, IIR, multidimensional...

0.5 1.0 1.5 2.0 2.5 3.0

Time

-0.25

0

0.25

0.5

0.75

1

1.25Amplitude

8.0 16.0 24.0 32.0 40.0 48.0

Frequency [radians]

0

0.2

0.4

0.6

0.8

1

Magnitude response

0.5 1.0 1.5 2.0 2.5 3.0

Time

-1

-0.5

0

0.5

1

1.5

Amplitude

8.0 16.0 24.0 32.0 40.0 48.0

Frequency [radians]

0

0.2

0.4

0.6

0.8

1

Magnitude response

scalingfunction

wavelet

ϕ t( ) ψ t( )

Introduction - 28

Daubechies’ construction.. .. . . two-scale equation

Hat function

Daubechies’ scaling function

ϕ t( ) cnϕ 2t n–( )n∑=

=

0.5 1.0 1.5 2.0 2.5 3.0

Time

0

0.5

1

1.5

Amplitude

0.5 1.0 1.5 2.0 2.5 3.0

Time

-0.25

0

0.25

0.5

0.75

1

1.25

Amplitude

=

Introduction - 29

Not every discrete scheme leads to wavelets

How do we know which ones wil l?. . . wait and see.. .

-1 -1

0 0biorthogonalfilter banks

quadratic spline

0

1 1

33

0

33

Introduction - 30

Applications

“That which shr inks must f i rst expand.”

Lao-Tzu, Tao Te Ching

CompressionCommunications

DenoisingGraphics

Introduction - 31

What is mult iresolution?

High resolution Low resolutionsubtract info

add info

= +

Introduction - 32

. . . and why use multiresolution?

A number of applications require signals to be processed and transmitted at multiple resolutions and multiple rates

• d igi ta l audio and video coding• conversions between TV standards• digi ta l HDTV and audio broadcast• remote image databases with searching• storage media with random access• MR coding for mult icast over the Internet• MR graphics

Compression: sti l l a key technique in communications

Introduction - 33

Multiresolution compression.. .. . . the DCT versus wavelet game

Questiongiven Lena (you have never seen before), what is the ‘ ‘best ’ ’ t ransform to code i t?

Fourier versus wavelet bases• l inear versus octave-band frequency scale• DCT versus subband coding• JPEG versus mult i resolut ion

Multiresolution source coding• successive approximat ion• browsing• progressive transmission

Introduction - 34

Compression systems based on l inear transforms

Goal: remove built- in redundancy, send only necessary info

• LT: l inear t ransform (KLT, WT, SBC, DCT, STFT)• Q: quant izat ion• EC: entropy coding

LT Q EC 0100101001

0.5 bits/pixel8 bits/pixel

Introduction - 35

Gibbs phenomenon

“Blocking” effect in image compression

Wavelets• smooth t ransi t ions• mult iscale propert ies• mult i resolut ion

Introduction - 36

A rate-distort ion primer. . .

Compression: rate-distortion is fundamental trade-off• more bi t rate ⇒ less distort ion• less bi t rate ⇒ more distort ion

Standard image coder• operates at one part icular point on D(R) curve

Multiresolution coder ( layered, scalable)• t ravels rate-distort ion curve (successive approximat ion)• computat ion scalabi l i ty

distortion

rate

x

x

x

xx

x x

DSA(R)D(R)

Introduction - 37

Best image coder?.. . wavelet based!

Shapiro’s embedded zero-tree algorithm (EZW)

• standard wavelet decomposi t ion (biorthogonal)• b i t p lane coding and zero-tree structure• beats JPEG whi le achieving successive approximat ion

DWT

bit planecoding

zerotrees

adaptiveentorpycoding

distortion

rate

x

x

x

xx

x x

JPEGEZW

Introduction - 38

Next image coding standard.. . JPEG 2000

All the best coders based on wavelets• 24 ful l proposals and a few part ia l ones• 18 used wavelets, 4 used DCT and 5 used others• top 75% are wavelet-based• top 5 use advanced wavelet or iented quant izat ion• systems requirements ask for mult i resolut ion

Final JPEG 2000 standard is wavelet based

Introduction - 39

Digital video coding

• s ignal decomposi t ion for compression• compat ib le subchannels• t ight control over coding error• easy jo int source/channel coding• robustness to channel errors• easy random access for digi ta l storage

MRprocessing

block

variety ofscales andresolutions

Introduction - 40

Conversion between TV formats

• HDTV/NTSC• inter laced/progressive

50Hz

USA

60Hz Europe

Introduction - 41

Interaction of source and channel coding

full reconstruction coarse reconstruction

MRcoder

high priorityhigh protection

low prioritylittle protection

Introduction - 42

MR transmission for digital broadcast

Embedding of coarse information within detai l• c loud: carr ies coarse info• satellite: carries detail• b lend MR transmission with MR coding

Trade-off in broad-cast ranges [miles]

cloud withsatellites

ai

bi

SR IR MR: λ = 0.5, 0.2

16

63

24

401445

28

high/low resolution

Introduction - 43

MR coding for mult icast over the Internet

‘ ‘ I want to say a special welcome to everyonethat ’s c l imbed into the Internet tonight, and has got into the

MBone --- and I hope i t doesn’ t a l l col lapse! ’ ’

Mick Jagger, Rol l ings Stones on Internet,11/18/94

Motivation: Internet is a heterogeneous mess!

Video multicast over Mbone • v ideo by VIC• software encoder/decoder• learning exper ience

(seminars. . . )

Heterogeneous user population

On-going experienceATM

64

1M

LAN

6464

1M1M

LAN

Introduction - 44

MR coding for mult icast over the Internet

Fact: different users receive different bit rates• t ransmission heterogenei ty

Different users absorb different bit rates• computat ion heterogenei ty

Solution: layered multicast trees• d i fferent layers are t ransmit ted over independent t rees• automat ic subscr ibe/unsubscr ibe• dynamic qual i ty management

S

Q1

Q1

Q2

Q2

Q2 Q2

Q3

Q2

Q3

Q3

D

R

Introduction - 45

Remote image databases with browsing

Introduction - 46

Multiresolution graphics

Example: optimize quality (distortion) for a target rate

rate

dist

ortio

n

Introduction - 47

Skull page

not reproduced due to copyright restrictions