Wavelet Transforms in Time SeriesAnalysis

Andrew Tangborn

Global Modeling and Assimilation Office, Goddard Space Flight Center

Andrew.V.Tangborn@nasa.gov

301-614-6178

Outline

1. Fourier Transforms.

2. What is a Wavelet?

3. Continuous and Discrete Wavelet Transforms

4. Construction of Wavelets through dilation equations.

5. Example - Haar wavelets

6. Daubechies Compactly Supported wavelets.

7. Data compression, efficient representation.

8. Soft Thresholding.

9. Continuous Transform - Morlet Wavelet

10. Applications to approximating error correlations

Fourier Transforms

• A good way to understand how wavelets work and why they are useful is by

comparing them with Fourier Transforms.

• The Fourier Transform converts a time series into the frequency domain:

Continuous Transform of a function f(x):

f̂(ω) =∞∫

−∞f(x)e−iωxdx

where f̂(ω) represents the strength of the function at frequency ω, where ω iscontinuous.

Discrete Transform of a function f(x):

f̂(k) =∞∫

−∞f(x)e−ikxdx

where k is a discrete number.

for discrete data f(xj), j = 1, ..., N

f̂k =N∑

j=1

fje(−i2π(k−1)(j−1)/N)

• The Fast Fourier Transform (FFT) is o(NlogN) operations.

Example: Single Frequency Signal

f(t) = sin(2πt)

0 1 2 3 4 5 6 7 8 9 10−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

Signal

0 1 2 3 4 5 6 7 8 9 10−0.5

−0.4

−0.3

−0.2

−0.1

0

0.1

0.2

0.3

0.4

0.5Real Coefficient Imaginary Coefficient

Fourier Transform

• Discrete Fourier Transform (DFT) locates the single frequency and re-

flection.

• Complex expansion is not exactly sine series - causes some spread.

Two Frequency Signal

f(t) = sin(2πt) + 2 sin(4πt)

0 1 2 3 4 5 6 7 8 9 10−3

−2

−1

0

1

2

3

Signal

0 1 2 3 4 5 6 7 8 9 10−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1Real Coefficients Imaginary Coefficients

Fourier Transform

• Both signal frequencies are represented by the Fourier Coefficients.

Intermittent Signal

f(t) = sin(2πt) + 2 sin(4πt) when .37 < t < .55

f(t) = sin(2πt) otherwise

0 1 2 3 4 5 6 7 8 9 10−3

−2

−1

0

1

2

3

Signal

0 1 2 3 4 5 6 7 8 9 10−0.5

−0.4

−0.3

−0.2

−0.1

0

0.1

0.2

0.3

0.4

0.5Real coefficients Imaginary Coefficients

Fourier Transform

• First frequency is found, but higher intermittent frequency appears as

many frequencies, and is not clearly identified.

What is a Wavelet?

• A function that is localized in time and frequency, generally with a zero

mean.

• It is also a tool for decomposing a signal by location and frequency.

Consider the Fourier transform:

A signal is only decomposed into its frequency components.

No information is extracted about location and time.

• What happens when applying a Fourier transform to a signal that has

a time varying frequency?

0 2 4 6 8 10 12 14 16 18 20−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

The Fourier transform will only give some information on

which frequencies are present, but will give no

information on when they occur.

Schematic Representation of Decomposition

Original Signal

Time −>

Fre

quen

cy −

>

• The signal is represented by an amplitude that is changing in time.

• There is no explicit information on frequency.

Fourier Transform

Time −>

Fre

quen

cy −

>

• The Fourier transform results in a representation that depends only on frequency.

• It gives no information on time.

Wavelet Transform

Time −>

Freq

uenc

y −>

• The wavelet transform contains information on both the time location and fre-quency of a signal.

Some typical (but not required) properties of wavelets

• Orthogonality - Both wavelet transform matrix and wavelet functions can be

orthogonal.

Useful for creating basis functions for computation.

• Zero Mean (admissibility condition) - Forces wavelet functions to wiggle (oscillate

between positive and negative).

• Compact Support - Efficient at representing localized data and functions.

How are Wavelets Defined?

• Families of basis functions that are based on

(1) dilations:

ψ(x) → ψ(2x)

(2) translations:

ψ(x) → ψ(x + 1)

of a given general ”mother wavelet” ψ(x).

• How do we use ψ(x)?

The general form is:

ψjk(x) = 2j/2ψ(2jx− k)

where

j: dilation index

k: translation index

2k/2 needed for normalization

• How do we get ψ(x)?

Dilation Equations

Construction of Wavelets

• We consider here only orthogonally/compactly supported

wavelets

- Orthogonality means:

∞∫

−∞ψjk(x)ψj

′k′(x)dx = δkk′δjj′

• Wavelets are constructed from scaling functions, φ(x) :

φ(x) come from the dilation equation:

φ(x) =∑

kckφ(2x− k)

ck: Finite set of filter coefficients

• General features:

- Fewer non-zero ck’s mean more compact and less smooth functions

- More non-zero ck’s mean less compact and more smooth functions

Restrictions on the Filter Coefficients

• Normality:∞∫

−∞φ(x)dx = 1

∞∫

−∞

∑

kckφ(2x− k)dx = 1

∑

kck

∞∫

−∞φ(2x− k)dx = 1

∑

kck

∞∫

−∞

1

2φ(2x− k)d(2x− k) = 1

∑

kck

∞∫

−∞

1

2φ(ξ)d(ξ) = 1

∑

kck(

1

2)(1) = 1

or∑

kck = 2

• Simple Examples

- Smallest number of ck is 1: just get φ(x) = δ, → zero support.

- Haar Scaling Function: c0 = 1, c1 = 1.

Haar Scaling Function

The scaling function equation is:

φ(x) = φ(2x) + φ(2x− 1)

The only function that satisfies this is:

φ(x) = 1 if 0 ≤ x ≤ 1

φ(x) = 0 otherwise

−1 −0.5 0 0.5 1 1.5 2−0.5

0

0.5

1

1.5

x

y

• Translation and Dilation of φ(x):

φ(2x) = 1 if 0 ≤ x ≤ 1/2

φ(2x) = 0 elsewhere

φ(2x− 1) = 1 if 1/2 ≤ x ≤ 1

φ(2x− 1) = 0 elsewhere

so the sum of the two functions is then:

−1 −0.5 0 0.5 1 1.5 2−0.5

0

0.5

1

1.5

x

y

Haar Wavelet• Wavelets are constructed by taking differences of scaling

functions

ψ(x) =∑

k(−1)kc1−kφ(2x− k)

differencing is caused by the (−1)k:

so the basic Haar wavelet is:

−1 −0.5 0 0.5 1 1.5 2−1.5

−1

−0.5

0

0.5

1

1.5

and the family comes from dilating and translating:

ψjk(x) = 2j/2ψ(2jx− k)

so that the j = 1 wavelets are:

−1 −0.5 0 0.5 1 1.5 2−1.5

−1

−0.5

0

0.5

1

1.5

Orthogonality of Haar wavelets

• Translation ⇒ no overlap

−1 −0.5 0 0.5 1 1.5 2−1.5

−1

−0.5

0

0.5

1

1.5

• Dilation ⇒ cancellation

−1 −0.5 0 0.5 1 1.5 2−1.5

−1

−0.5

0

0.5

1

1.5

Daubechies’ Compactly Supported Wavelets

The following plots are wavelets created using larger numbers of filter coefficients, but having allthe properties of orthogonal wavelets. For example, D4 has coefficients:

ck =1

4(1 +

√3),

1

4(3 +

√3),

1

4(3 −

√3),

1

4(1 −

√3)

(Reference: Daubechies, 1988)

0 20 40 60 80 100 120−0.4

−0.3

−0.2

−0.1

0

0.1

0.2

0.3

0.4

0.5

D4

0 20 40 60 80 100 120−0.4

−0.3

−0.2

−0.1

0

0.1

0.2

0.3

D12

0 20 40 60 80 100 120−0.4

−0.3

−0.2

−0.1

0

0.1

0.2

0.3

D20

Continuous and Discrete Wavelet TransformsContinuous

(Twavf)(a, b) = |a|−1/2∫

dtf(t)ψ(t− 1

b)

where

a: translation parameter

b: dilation parameter

Discrete

Twavm,n (f) = a−m/2o

∫

dtf(t)ψ(a−mo t− nbo)

m: dilation parameter

n: translation parameter

ao,bo depend on the wavelet used

Fast Wavelet Transform(Reference: S. Mallat, 1989)

• Uses the discrete data:[

f0 f1 f2 f3 f4 f5 f6 f7

]

• Pyramid Algorithm ⇒ o(N) !! - Start at finest scale and calculate differences

j=0

j=1

j=2

f1 f

2f3

f4 f

5 f6

f7 f

8

a0,0 b

0,0

a1,0 a

1,1b1,0 b

1,1

a2,0 a

2,1a

2,2 a2,3

b2,0 b

2,1 b2,2

b2,3

and averages

- Use Averages at next coarser scale to get new set of differences (bj,k) and averages(aj,k)

And the coefficients are simply the differences (bj,k) and the average for the coarsestscale (a0,0):

[

a0,0 b0,0 b1,0 b1,1 b2,0 b2,1 b2,2 b2,3

]

For Haar Wavelet:aj−1,k = c0aj,k + c1aj,k+1

bj−1,k = c0aj,k − c1aj,k+1

- The differences are the coefficients at each scale. Averages used for next scale.

• How do we store all this? (eg, Numerical Recipes algorithm).

Intermittent Signal - Wavelet Transform

0 1 2 3 4 5 6 7 8 9 10−3

−2

−1

0

1

2

3

Signal

0 20 40 60 80 100 120−5

−4

−3

−2

−1

0

1

2

3

4

j=2

j=3j=4

j=5

• Higher Frequency intermittent signal shows up in the j = 5, 6 scales, and only in themiddle portion of the domain. Localized frequency data is found.

Data Compression - Efficient Representation

0 1 2 3 4 5 6 7 8 9 10−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

Original Localized Data, 128 points

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

16 term Wavelet Reconstruction

0 1 2 3 4 5 6 7 8 9 10−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

16 term Fourier Reconstruction

• Localized function is represented with more accuracy using 16 wavelet coefficientsbecause only significant coefficients need to be retained. Fourier coefficients are global.

Denoising by Soft Thresholding

Basic Idea:

• Wavelet Representation highly compresses coherent data into justa few coefficients.

Magnitude of each coefficient is relatively large.

• White noise contains energy at all time scales and time locations.Representation is spread to many (if not all) wavelet coefficients.

Noise coefficients are relatively small in amplitude.

• How do we remove the noise contribution in the coefficients?Ans: Shrink all of them just a little.

• How much do we shrink each coefficient?t =

√

2log(n)σ/√n

where n= number of data points and σ=noise standard deviation.

Example

0 50 100 150 200 250 300−1.5

−1

−0.5

0

0.5

1

1.5

2

2.5

Noisy Signal with white noise of known variance.

0 50 100 150 200 250 300−1.5

−1

−0.5

0

0.5

1

1.5

2

Wavelet coefficients Shrunk toward zero.

• Soft Thresholding takes advantage of the fact that white noise is rep-resented equally by all coefficients.

• Data compression means that coherent signal is represented by just afew coefficients with relatively large values. White noise (or non-white

as well) is spread over many coefficients, adding just a small amount tothe magnitude of each.

The Continuous Morlet Wavelet Transform

The Morlet mother wavelet is a complex exponential (Fourier) with a

Gaussian envelop which ensures localization:

ψ(t) = exp(iω0t)exp(−t2/2σ2)

where ω0 is the frequency and σ is a measure of the spread or support.

Note that while the footprint is infinite, the exponential decay createsan effective footprint which is relatively compact.

Translations and dilations of the Morlet wavelet:

ψ(b, a)(t) =1

aexp

[

iω0

(

t− b

a

)]

exp

−(

t− b

a

)2

/2σ2

The Morlet wavelet using matlab

In matlab, the Morlet mother wavelet can be constructed using thecommand:

[psi,x] = Morlet(-8,8,128);

on 128 grid points, and domain of [-8,8].

−8 −6 −4 −2 0 2 4 6 8−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

x

Ψ

The Morlet transform with Matlab

Given a time series:

y(t) =

{

sin(2πt) + sin(32πt) when .3 ≤ t ≤ .6sin(2πt) otherwise

on 128 grid points, the Morlet transform can be calculated and plotted

using the command:

coef=cwt(y,[1 2 4 8 16 32 64 128],’morl’,’plot’)

• the top level of the coefficient plot shows bright (or large) values forscale 128 (largest scale).

• The next two layers are not zero, indicating leakage between different

time scales.

0 0.2 0.4 0.6 0.8 1−1.5

−1

−0.5

0

0.5

1

1.5

2

x

f(x)

Absolute Values of Ca,b Coefficients for a = 1 2 4 8 16 ...

time (or space) b

sca

les a

20 40 60 80 100 120

1

2

4

8

16

32

64

128

Example: Southern Oscillation Index Time Series 1951-2005

The SOI is the monthly pressure fluctuations in air pressure between Tahiti andDarwin. It is generally a noisy time series and benefits from some smoothing. Softthresholding is a way to remove the noise in the signal without removing important in-formation. The multivariate ENSO signal comes from sea level pressure, surface wind,sea surface tempemperature, surface air temperature and total amount of cloudiness.

0 100 200 300 400 500 600 700−8

−6

−4

−2

0

2

4

6

Time in months

SOI s

ignal

Original SOI signal 1951−2005

0 100 200 300 400 500 600 700−8

−6

−4

−2

0

2

4

6

Time in Months

SOI s

ignal

Denoised SOI Signal

Multivariate ENSO Signal from CDC

Morlet Wavelet Decomposition of SOI time series

0 100 200 300 400 500 600 700−8

−6

−4

−2

0

2

4

6

Month

SOI

Monthly Southern Oscillation Index, 1951−2006

Absolute Values of Ca,b Coefficients for a = 1 2 3 4 5 ...

Time in months

Tim

e S

cale

in M

onth

s

100 200 300 400 500 600 1

100

166

232

298

364

430

529

628

50

100

150

200

Morlet Coefficients for SOI

time in months

Tim

e s

cale

in m

on

ths

100 200 300 400 500 600 1

19

37

55

73

91

109

127

145

163

50

100

150

200

Application to Approximation of Error Correlations

• Error correlations are essential for carrying out atmospheric data assimilation.

• They tell us how errors in model outputs are spatially related, and how to spread informationfrom observational data into the model.

• Error correlations are generally the most computationally and memory intensive parts of a dataassimilation system.

• Below is the error correlation of a chemical constituent assimilation system (a), and waveletapproximations created by retaining 10 % (b), 10% (c) and 2% (d) of the wavelet coeffients.

Extracting time scales from climate signals:

Reference: Seasonal-to-Interannual variability of Ethiopia/Horn

of Africa Monsoon. Part 1: Associations of Wavelet Filtered

large-Scale Atmospheric Circulation and Global Sea Surface

Temperature, Segelet et al. J. of Climate, 2009.

How can we use information separated out by time-scale

to make predictions?

Reference: Webster and Hoyos, BAMS, Vol 85, 2004.

• Prediction of Asian monsoons on 15-35 day timescale.

• Morphology of the Monsoon Intraseasonal Oscillation (MISO)

• Madden-Julian Oscillation (MJO):

- Eastward propagating convection

- Largest variance in 20-40 day spectral band

Facts about South Asian Summers

(1) Convection stronger in eastern Indian Ocean.

(2) E. Indian Ocean convection lags western convection.

(3) Northern Indian Ocean convection primarily in the East.

(4) Convection or east equitorial Indian Ocean out of phase with con-vecton over India.

(5) North Indian Ocean convection coincides with development of mon-

soons over south asia.