a tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfconnections with fourier...
TRANSCRIPT
![Page 1: A Tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfConnections with Fourier Analysis Fourier analysis gives an understanding of frequency, but \non-stationary" signals](https://reader034.vdocuments.us/reader034/viewer/2022043004/5f869eb0989b5908de490ac9/html5/thumbnails/1.jpg)
A Tutorial on
Wavelets and their Applications
Martin J. Mohlenkamp
University of Colorado at Boulder
Department of Applied Mathematics
This tutorial is designed for people with little or noexperience with wavelets. We will cover the basicconcepts and language of wavelets and computationalharmonic analysis, with emphasis on the applicationsto numerical analysis. The goal is to enable those whoare unfamiliar with the area to interact moreproductively with the specialists.
Thanks to Gregory Beylkin, Willy Hereman, and
Lucas Monzon for help with this tutorial.
![Page 2: A Tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfConnections with Fourier Analysis Fourier analysis gives an understanding of frequency, but \non-stationary" signals](https://reader034.vdocuments.us/reader034/viewer/2022043004/5f869eb0989b5908de490ac9/html5/thumbnails/2.jpg)
Goals
• Enable the wavelet novice to interact
more productively with the specialists, by
– Introducing the basic concepts and
language
– Doing some physics related examples
– Explaining why people like them
Not Goals
• Give the history and assign credit
• Convince you that wavelets are better
than any particular technique for any
particular problem
![Page 3: A Tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfConnections with Fourier Analysis Fourier analysis gives an understanding of frequency, but \non-stationary" signals](https://reader034.vdocuments.us/reader034/viewer/2022043004/5f869eb0989b5908de490ac9/html5/thumbnails/3.jpg)
Outline
• Multiresolution Analysis
• wavelets (traditional)
• properties
• fast algorithms
• Connections with Fourier Analysis
• Local Cosine and the phase plane
• 1D Schrodinger example
• wavelet packets
• Operators in Wavelet Coordinates
• density matrix example
• Review and Questions
![Page 4: A Tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfConnections with Fourier Analysis Fourier analysis gives an understanding of frequency, but \non-stationary" signals](https://reader034.vdocuments.us/reader034/viewer/2022043004/5f869eb0989b5908de490ac9/html5/thumbnails/4.jpg)
Multiresolution Analysis
A multiresolution analysis is a decomposition
of L2(R), into a chain of closed subspaces
· · · ⊂ V2 ⊂ V1 ⊂ V0 ⊂ V−1 ⊂ V−2 ⊂ · · · ⊂ L2(R)
such that
1.⋂
j∈Z Vj = {0} and⋃
j∈Z Vj is dense in L2(R)
2. f(x) ∈ Vj if and only if f(2x) ∈ Vj−1
3. f(x) ∈ V0 if and only if f(x− k) ∈ V0
for any k ∈ Z.
4. There exists a scaling function ϕ ∈ V0
such that {ϕ(x− k)}k∈Z is an orthonormal
basis of V0.
![Page 5: A Tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfConnections with Fourier Analysis Fourier analysis gives an understanding of frequency, but \non-stationary" signals](https://reader034.vdocuments.us/reader034/viewer/2022043004/5f869eb0989b5908de490ac9/html5/thumbnails/5.jpg)
Multiresolution Analysis
Let Wj be the orthogonal complement
of Vj in Vj−1:
Vj−1 = Vj ⊕ Wj,
so that
L2(R) =
⊕
j∈Z
Wj.
Selecting a coarsest scale Vn and finest scale
V0, we truncate the chain to
Vn ⊂ · · · ⊂ V2 ⊂ V1 ⊂ V0
and obtain
V0 = Vn
n⊕
j=1
Wj.
From the scaling function ϕ we can define
the wavelet ψ, such that {ψ(x− k)}k∈Z is an
orthonormal basis of W0.
![Page 6: A Tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfConnections with Fourier Analysis Fourier analysis gives an understanding of frequency, but \non-stationary" signals](https://reader034.vdocuments.us/reader034/viewer/2022043004/5f869eb0989b5908de490ac9/html5/thumbnails/6.jpg)
Multiresolution Analysis
Example: Haar wavelets
ϕ(x) =
1 for 0 < x < 1
0 elsewhere.
V0 = span({ϕ(x− k)}k∈Z) are piecewise
constant functions with jumps only at
integers.
ψ(x) =
1 for 0 < x < 1/2
−1 for 1/2 ≤ x < 1
0 elsewhere.
,
W0 = span({ψ(x− k)}k∈Z) are piecewise
constant functions with jumps only at
half-integers, and average 0 between integers.
![Page 7: A Tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfConnections with Fourier Analysis Fourier analysis gives an understanding of frequency, but \non-stationary" signals](https://reader034.vdocuments.us/reader034/viewer/2022043004/5f869eb0989b5908de490ac9/html5/thumbnails/7.jpg)
Multiresolution Analysis
?
?
?
BBBBBBBBBN
BBBBBBBBBN
BBBBBBBBBN
V3 W3
V2 W2
V1 W1
V0
V0 = V3 ⊕ W3 ⊕ W2 ⊕ W1
![Page 8: A Tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfConnections with Fourier Analysis Fourier analysis gives an understanding of frequency, but \non-stationary" signals](https://reader034.vdocuments.us/reader034/viewer/2022043004/5f869eb0989b5908de490ac9/html5/thumbnails/8.jpg)
Where’s the Wavelet?
Since Wj is a dilation of W0, we can define
ψj,k = 2−j/2ψ(2−jx− k)
and have
Wj = span({ψj,k(x)}k∈Z).
In this example,
W1 = span
W2 = span
W3 = span
![Page 9: A Tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfConnections with Fourier Analysis Fourier analysis gives an understanding of frequency, but \non-stationary" signals](https://reader034.vdocuments.us/reader034/viewer/2022043004/5f869eb0989b5908de490ac9/html5/thumbnails/9.jpg)
The Wavelet Zoo
φ ψ
Haar
0.2 0.4 0.6 0.8 1
0.2
0.4
0.6
0.8
1
0.2 0.4 0.6 0.8 1
-1
-0.5
0.5
1
daub40.5 1 1.5 2 2.5 3
-0.25
0.25
0.5
0.75
1
1.25
-1 -0.5 0.5 1 1.5 2
-1
-0.5
0.5
1
1.5
daub12 2 4 6 8 10
-0.4
-0.2
0.2
0.4
0.6
0.8
1
-4 -2 2 4 6
-1
-0.5
0.5
1
coif4-2 -1 1 2 3
0.5
1
1.5
-2 -1 1 2 3
-2
-1.5
-1
-0.5
0.5
1
coif12-5 -2.5 2.5 5 7.5 10
-0.2
0.2
0.4
0.6
0.8
1
-7.5 -5 -2.5 2.5 5 7.5
-1
-0.5
0.5
![Page 10: A Tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfConnections with Fourier Analysis Fourier analysis gives an understanding of frequency, but \non-stationary" signals](https://reader034.vdocuments.us/reader034/viewer/2022043004/5f869eb0989b5908de490ac9/html5/thumbnails/10.jpg)
Vanishing Moments
Wavelets are usually designed with vanishing
moments:∫ +∞
−∞ψ(x)xmdx = 0, m = 0, . . . ,M − 1,
which makes them orthogonal to the low
degree polynomials, and so tend to compress
non-oscillatory functions.
For example, we can expand in a Taylor series
f(x) = f(0) + f ′(0)x+ · · · + f(M−1)(0)xM−1
(M − 1)!
+ f(M)(ξ(x))xM
M !
and conclude
|〈f, ψ〉| ≤ maxx
∣
∣
∣
∣
∣
f(M)(ξ(x))xM
M !
∣
∣
∣
∣
∣
.
Haar has M = 1.
![Page 11: A Tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfConnections with Fourier Analysis Fourier analysis gives an understanding of frequency, but \non-stationary" signals](https://reader034.vdocuments.us/reader034/viewer/2022043004/5f869eb0989b5908de490ac9/html5/thumbnails/11.jpg)
Quadrature Mirror Filters
Wavelets are designed through properties of
their “quadrature mirror filter” {H,G}.
Haar ↔H =1√2[1,1]
G =1√2[1,−1]
daub4 ↔H =1
4√
2[1 +
√3,3 +
√3,3 −
√3,1 −
√3]
G = [H(3),−H(2), H(1),−H(0)]}
(The values are usually not in closed form.)
For instance, vanishing moments
∫ +∞
−∞ψ(x)xmdx = 0, m = 0, . . . ,M − 1,
are a consequence of
∑
i
G(i)im = 0, m = 0, . . . ,M − 1.
![Page 12: A Tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfConnections with Fourier Analysis Fourier analysis gives an understanding of frequency, but \non-stationary" signals](https://reader034.vdocuments.us/reader034/viewer/2022043004/5f869eb0989b5908de490ac9/html5/thumbnails/12.jpg)
Trade-offs
You can get
• higher M
• more derivatives
• closer to symmetric
• closer to interpolating (coiflets)
if you pay by increasing the filter length,
which causes
• longer (overlapping) supports, and so
worse localization.
• slower transforms
The cost is linear (in M etc.).
![Page 13: A Tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfConnections with Fourier Analysis Fourier analysis gives an understanding of frequency, but \non-stationary" signals](https://reader034.vdocuments.us/reader034/viewer/2022043004/5f869eb0989b5908de490ac9/html5/thumbnails/13.jpg)
Fast Wavelet Transform
Sample onto the finest resolution and then
apply the “quadrature mirror filter” {H,G}.
V3 • ◦ W3
V2 • • ◦ ◦ W2
V1
• • • • ◦ ◦ ◦ ◦ W1
V0 • • • • • • • •
H�
����
AAAAUG
H�
����
AAAAUG
HAAAAU
��
���G
The total cost of this cascade is 2 ·N · L,
where L is the length of the filter.
![Page 14: A Tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfConnections with Fourier Analysis Fourier analysis gives an understanding of frequency, but \non-stationary" signals](https://reader034.vdocuments.us/reader034/viewer/2022043004/5f869eb0989b5908de490ac9/html5/thumbnails/14.jpg)
There are many, many ’lets
By loosening the definitions, you can get
• symmetric
• interpolating
• 2D properties (brushlets)
• . . .
Rules of thumb:
• Use a special purpose wavelet if and only
if you have a special need.
• Use one vanishing moment per digit
desired (truncation level).
• Do not use more derivatives than your
function typically has.
![Page 15: A Tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfConnections with Fourier Analysis Fourier analysis gives an understanding of frequency, but \non-stationary" signals](https://reader034.vdocuments.us/reader034/viewer/2022043004/5f869eb0989b5908de490ac9/html5/thumbnails/15.jpg)
Multiwavelets
(Polynomial version on [0,1])
Fix k ∈ N, and let Vn be the space of
functions that are polynomials of degree less
than k on the intervals (2nj,2n(j + 1)) for
j = 1, . . . ,2−n − 1, and 0 elsewhere.
V0 is spanned by k scaling functions.
W0 is spanned by k multiwavelets.
By construction, the wavelets have k
vanishing moments, and so the same sparsity
properties as ordinary wavelets.
The wavelets are not even continuous, but
this allows weak formulations of the
derivative, which allows better treatment of
boundaries.
![Page 16: A Tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfConnections with Fourier Analysis Fourier analysis gives an understanding of frequency, but \non-stationary" signals](https://reader034.vdocuments.us/reader034/viewer/2022043004/5f869eb0989b5908de490ac9/html5/thumbnails/16.jpg)
Connections with Fourier Analysis
Fourier analysis gives an understanding of
frequency, but “non-stationary” signals beg
for space (time) localization.
This need motivates Computational Harmonic
Analysis and its tools, such as wavelets and
local cosine.
The theory and intuition are still based on
Fourier analysis.
![Page 17: A Tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfConnections with Fourier Analysis Fourier analysis gives an understanding of frequency, but \non-stationary" signals](https://reader034.vdocuments.us/reader034/viewer/2022043004/5f869eb0989b5908de490ac9/html5/thumbnails/17.jpg)
Local Cosine Basis
Partition the line (interval, circle) with
· · · ai < ai+1 · · · , Ii = [ai, ai+1] .
Construct a set of bells: {bi(x)}with
∑
i b2i (x) = 1, bi(x)bi−1(x) even about ai,
and bi(x)bj(x) = 0 if j 6= i± 1.
Construct the cosines which are even on the
left and odd on the right
cji(x) =
√
√
√
√
2
ai+1 − aicos
(
(j + 1/2)π(x− ai)
ai+1 − ai
)
.
{bi(x)cji(x)} forms an orthonormal basis with
fast transform based on the FFT.
![Page 18: A Tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfConnections with Fourier Analysis Fourier analysis gives an understanding of frequency, but \non-stationary" signals](https://reader034.vdocuments.us/reader034/viewer/2022043004/5f869eb0989b5908de490ac9/html5/thumbnails/18.jpg)
Phase Plane Intuition
If a function has ‘instantaneous frequency’
ν(x), it should be represented by those Local
Cosine basis elements whose rectangles
intersect ν(x). There are max{∆l,1} of these.
6
-
ξ
x|I| = l
1/l
∆
ν(x)
![Page 19: A Tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfConnections with Fourier Analysis Fourier analysis gives an understanding of frequency, but \non-stationary" signals](https://reader034.vdocuments.us/reader034/viewer/2022043004/5f869eb0989b5908de490ac9/html5/thumbnails/19.jpg)
Local Cosine Phase Plane
Consider the eigenfunctions of −∆ − C/x,
which have instantaneous frequency
νn =√
C/x+ λn.
6
-x
ξ
��
��
��
νn
P
We can get an efficient representation (with
proof) by adapting, but this is not very
flexible, especially in higher dimensions.
![Page 20: A Tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfConnections with Fourier Analysis Fourier analysis gives an understanding of frequency, but \non-stationary" signals](https://reader034.vdocuments.us/reader034/viewer/2022043004/5f869eb0989b5908de490ac9/html5/thumbnails/20.jpg)
Wavelet Phase Plane
The multiresolution analysis divides the phase
plane differently:
6
-
x
ξ
W1
W2
W3
W4
V4
For wavelets this is intuitive but not rigorous.
![Page 21: A Tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfConnections with Fourier Analysis Fourier analysis gives an understanding of frequency, but \non-stationary" signals](https://reader034.vdocuments.us/reader034/viewer/2022043004/5f869eb0989b5908de490ac9/html5/thumbnails/21.jpg)
Wavelet Phase Plane
Consider again the instantaneous frequency
νn =√
C/x+ λn.
6
-
x
ξ
P
We get an efficient representation without
adapting, so the location of the discontinuity
is not important.
![Page 22: A Tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfConnections with Fourier Analysis Fourier analysis gives an understanding of frequency, but \non-stationary" signals](https://reader034.vdocuments.us/reader034/viewer/2022043004/5f869eb0989b5908de490ac9/html5/thumbnails/22.jpg)
Tones
Sustained high frequencies, such as those in
spherical harmonics
are a problem for Wavelets, but not for local
cosine.
To enable wavelets to handle such functions,
“wavelet packets” were developed.
![Page 23: A Tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfConnections with Fourier Analysis Fourier analysis gives an understanding of frequency, but \non-stationary" signals](https://reader034.vdocuments.us/reader034/viewer/2022043004/5f869eb0989b5908de490ac9/html5/thumbnails/23.jpg)
Wavelet Packets and Best Basis Searches
Idea: by filtering the wavelet spaces, we can
partition phase space in different ways:
V3 • ◦
V2 • • ◦ ◦
V1
• • • • ◦ ◦ ◦ ◦ W1
V0 • • • • • • • •
H�
����
AAAAUG H
��
���
AAAAUG H
��
���
AAAAUG H
��
���
AAAAUG
H�
����
AAAAUG H
��
���
AAAAUG
HAAAAU
��
���G
Any choice of decompositions gives a wavelet
packet representation.
A fast tree search can find the “best basis”.
![Page 24: A Tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfConnections with Fourier Analysis Fourier analysis gives an understanding of frequency, but \non-stationary" signals](https://reader034.vdocuments.us/reader034/viewer/2022043004/5f869eb0989b5908de490ac9/html5/thumbnails/24.jpg)
Wavelet Packets Phase Plane
If we choose:
����
AAAU
����
AAAU
����
AAAU
����
AAAU
����
AAAU
����
AAAU
AAAU
����
then our phase
plane looks like:
-
6
x
ξ
![Page 25: A Tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfConnections with Fourier Analysis Fourier analysis gives an understanding of frequency, but \non-stationary" signals](https://reader034.vdocuments.us/reader034/viewer/2022043004/5f869eb0989b5908de490ac9/html5/thumbnails/25.jpg)
Operators
There are many competing, adaptive ways to
represent functions.
It is more interesting to consider operators
and develop operator calculus.
=
d1
d2
d 3
s 3
d1
d^2
d^ 3
s 3
1Α
2Α
3Α
3Τ
21Γ3Γ 1
1Γ 42Γ 3
2Γ 43Γ 4
4Β 1Β 12 3Β 1
3Β 2 2Β 4
3Β 4
Many operators are sparse in Wavelet bases.
![Page 26: A Tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfConnections with Fourier Analysis Fourier analysis gives an understanding of frequency, but \non-stationary" signals](https://reader034.vdocuments.us/reader034/viewer/2022043004/5f869eb0989b5908de490ac9/html5/thumbnails/26.jpg)
Operators in the Nonstandard Form
The nonstandard form gives a more isotropic,
and often more sparse, representation.
=
d1
s 1
d^2
s 2
d^ 3
s 3
1Α
1Γ
Β 1
2Β2Α
2Γ
3Β3Α
3Γ 3Τ
d1
s1
d2
s2
d 3
s 3
![Page 27: A Tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfConnections with Fourier Analysis Fourier analysis gives an understanding of frequency, but \non-stationary" signals](https://reader034.vdocuments.us/reader034/viewer/2022043004/5f869eb0989b5908de490ac9/html5/thumbnails/27.jpg)
Operators in Wavelets
Hamiltonian
−∆ − 300
|x|
Density Matrix
with 15
eigenfunctions
![Page 28: A Tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfConnections with Fourier Analysis Fourier analysis gives an understanding of frequency, but \non-stationary" signals](https://reader034.vdocuments.us/reader034/viewer/2022043004/5f869eb0989b5908de490ac9/html5/thumbnails/28.jpg)
Operators in Wavelets
Some operators can be computed rapidly.
0 5 10 15 20 25 30 35 400
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Number of iterations
Sig
nific
ant c
oeffi
cien
t rat
io
straightforward disretizationfull wavelet representationadapted wavelet projection at 1e−8adapted wavelet projection at 1e−5
Here the density matrix is computed via the
sign iteration
T0 = T/||T ||2Tk+1 = (3Tk − T3
k )/2, k = 0,1, . . .
![Page 29: A Tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfConnections with Fourier Analysis Fourier analysis gives an understanding of frequency, but \non-stationary" signals](https://reader034.vdocuments.us/reader034/viewer/2022043004/5f869eb0989b5908de490ac9/html5/thumbnails/29.jpg)
Philosophical Review
Multiscale assumption: Efficient when high
frequencies (sharp features) happen for a
short amount of time/space.
Cleanly adaptive: Refine or coarsen the
“grid” by adding or deleting basis
functions.
Automatically adaptive: Simply truncate
small coefficients.
Tunable: Decide on the needed precision and
properties first, then choose which
wavelet.