stationary time series ams 586 1. the moving average time series of order q, ma(q) where {z t |t t}...
TRANSCRIPT
![Page 1: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/1.jpg)
Stationary Time Series
AMS 586
1
![Page 2: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/2.jpg)
The Moving Average Time series of order q, MA(q)
where {Zt|t T} denote a white noise time series with variance 2.
Let {Xt|t T} be defined by the equation.
Then {Xt|t T} is called a Moving Average time series of order q. (denoted by MA(q))
2
2211 qtqtttt ZZZZX
![Page 3: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/3.jpg)
qi
qihXXCOV
hq
ihii
htt
0
if),(
0
2
The autocorrelation function for an MA(q) time series
The autocovariance function for an MA(q) time series
qi
qihh
q
ii
hq
ihii
0
if0 0
2
0
The mean value for an MA(q) time series
tXE
3
![Page 4: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/4.jpg)
The autocorrelation function for an MA(q) time series
qi
qihh
q
ii
hq
ihii
0
if0 0
2
0
Comment
“cuts off” to zero after lag q.
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
q
4
![Page 5: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/5.jpg)
The Autoregressive Time series of order p, AR(p)
where {Zt|t T} is a white noise time series with variance 2.
Let {Xt|t T} be defined by the equation.
2211 tptpttt ZXXXX
Then {Zt|t T} is called a Autoregressive time series of order p. (denoted by AR(p))
5
![Page 6: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/6.jpg)
The mean of a stationary AR(p)
Assuming {Xt|t T} is stationary, and take expectations of the equation, we obtain the mean μ:
2211 tptpttt EZEEXEXEXEX
6
p
tXE
211
Now we can center (remove the mean of) the time series as follows:
)(
)()( 2211
tptp
ttt
ZX
XXX
![Page 7: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/7.jpg)
Computing the autocovariance of a stationary AR(p)
Now assuming {Xt|t T} is stationary with mean zero:
7
Multiplying by Xt-h, h ≥ 0, and take expectations of the equation, we obtain the Yule-Walker equations for the autocovariance. Note, for a zero mean sequence:
)()(
)()()( 2211
htthtptp
htthtthtt
XZEXXE
XXEXXEXXE
2211 tptpttt ZXXXX
)()()()(),() htthtthtthtt XXEXEXEXXEXXCOVh
![Page 8: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/8.jpg)
Note: For h > 0, we have:
The Autocovariance function (h) of a stationary AR(p) series
Satisfies the equations:
21 10 pp
101 1 pp
212 1 pp
and
011 ppp
phhh p 11 for h > p
Yule Walker Equations
8
0)( htt XZE
For h = 0, we have: 2)()( ttt ZVarXZE
![Page 9: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/9.jpg)
2
1
01 1 p p
with
phhh p 11for h > p
111 1 pp
212 1 pp
111 ppp
The Autocorrelation function (h) of a stationary AR(p) series
Satisfies the equations:
and
9
![Page 10: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/10.jpg)
or:
h
pp
hh
rc
rc
rch
111
22
11
and c1, c2, … , cp are determined by using the starting values of the sequence (h).
pp xxx 11
pr
x
r
x
r
x111
21
where r1, r2, … , rp are the roots of the polynomial
10
![Page 11: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/11.jpg)
Conditions for stationarity
Autoregressive Time series of order p, AR(p)
11
![Page 12: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/12.jpg)
The value of Xt increases in magnitude and Zt eventually becomes negligible.
i.e. 11 ttt ZXX
If 1 = 1 and = 0.
The time series {Xt|t T} satisfies the equation:
The time series {Xt|t T} exhibits deterministic behavior.
11 tt XX
12
![Page 13: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/13.jpg)
For a AR(p) time series, consider the polynomial
pp xxx 11
pr
x
r
x
r
x111
21
with roots r1, r2 , … , rp
then {Xt|t T} is stationary if |ri| > 1 for all i.
If |ri| < 1 for at least one i then {Xt|t T} exhibits deterministic behavior.
If |ri| ≥ 1 and |ri| = 1 for at least one i then {Xt|t T} exhibits non-stationary random behavior.
13
![Page 14: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/14.jpg)
since:
h
pp
hh
rc
rc
rch
111
22
11
i.e. the autocorrelation function, (h), of a stationary AR(p) series “tails off” to zero.
lim 0h
h
and |r1 |>1, |r2 |>1, … , | rp | > 1 for a stationary AR(p) series then
0
0.2
0.4
0.6
0.8
1
1.2
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
14
![Page 15: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/15.jpg)
Special Cases: The AR(1) time
Let {Xt|t T} be defined by the equation.
11 ttt ZXX
15
![Page 16: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/16.jpg)
Consider the polynomial
xx 11
1
1r
x
with root r1= 1/1
1. {xt|t T} is stationary if |r1| > 1 or |1| < 1 .
2. If |ri| < 1 or |1| > 1 then {Xt|t T} exhibits deterministic behavior.
3. If |ri| = 1 or |1| = 1 then {Xt|t T} exhibits non-stationary random behavior.
16
![Page 17: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/17.jpg)
Special Cases: The AR(2) time
Let {Xt|t T} be defined by the equation.
2211 tttt ZXXX
17
![Page 18: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/18.jpg)
Consider the polynomial
2211 xxx
21
11r
x
r
x
where r1 and r2 are the roots of (x)
1. {Xt|t T} is stationary if |r1| > 1 and |r2| > 1 .
2. If |ri| < 1 or |1| > 1 then {Xt|t T} exhibits deterministic behavior.
3. If |ri| ≥ 1 for i = 1,2 and |ri| = 1 for at least on i then {Xt|t T} exhibits non-stationary random behavior.
This is true if 1+2 < 1 , 2 –1 < 1 and 2 > -1.These inequalities define a triangular region for 1 and 2.
18
![Page 19: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/19.jpg)
Patterns of the ACF and PACF of AR(2) Time SeriesIn the shaded region the roots of the AR operator are complex
h kk
h kk
h kk
h kk
1
21
-1
2-2
III
IIIIV
2
19
![Page 20: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/20.jpg)
The Mixed Autoregressive Moving Average Time Series of order p,q The ARMA(p,q) series
20
![Page 21: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/21.jpg)
The Mixed Autoregressive Moving Average Time Series of order p and q, ARMA(p,q)
Let 1, 2, … p , 1, 2, … p , denote p + q +1 numbers (parameters).
Let {Zt|t T} denote a white noise time series with variance 2.
– uncorrelated– mean 0, variance 2.
Let {Xt|t T} be defined by the equation. 2211 ptpttt XXXX
Then {Xt|t T} is called a Mixed Autoregressive- Moving Average time series - ARMA(p,q) series.
2211 qtqttt ZZZZ
21
![Page 22: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/22.jpg)
Mean value, variance, autocovariance function,
autocorrelation function of anARMA(p,q) series
22
![Page 23: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/23.jpg)
Similar to an AR(p) time series, for certain values of the parameters 1, …, p an ARMA(p,q) time series may not be stationary.
An ARMA(p,q) time series is stationary if the roots (r1, r2, … , rp ) of the polynomial
(x) = 1 – 1x – 2x2 - … - p xp
satisfy | ri| > 1 for all i.
23
![Page 24: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/24.jpg)
Assume that the ARMA(p,q) time series {Xt|t T} is stationary:
Let = E(Xt). Then
2211 ptpttt XEXEXEXE
21 p
1 21 p
2211 qtqttt ZEZEZEZE
0000 21 q
or p
tXE
211
24
![Page 25: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/25.jpg)
The Autocovariance function, (h), of a stationary
mixed autoregressive-moving average time series {Xt|t T} be determined by the equation:
ptpttt XXXX 2211
Thus
p 211 now
11 ptptt XXX
qtqttt ZZZZ 2211
qtqttt ZZZZ 2211
25
![Page 26: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/26.jpg)
Hence
tht XXEh
phtpht XXE 11
tqhtqhththt XZZZZ 2211
tphtptht XXEXXE 11
tqhtqthttht XZEXZEXZE 11
phh p 11
qhhh zxqzxzx 11
thtzx XZEh where26
![Page 27: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/27.jpg)
ptptht XXuE 11
qtqttt ZZZZ 2211
pthtptht XZEXZE 11
qthtqthttht ZZEZZEZZE 11
phh zxpzx 11
qhhh zzqzzzz 11
thtzx XZEh note
.0 if 0 where hXZEh thtzx
.0 if 0
.0 if and
2
h
hZZEh thtzz
27
![Page 28: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/28.jpg)
We need to calculate:
qzxzxzx ,,1,0
20 zx
hzx note phh zxpzx 11
qhhh zzqzzzz 11
.0 if 0 and hhzx
.0 if 0
.0 if 2
h
hhzz
222201 zxzx
222 012 zxzxzx
22
22
2
222 etc 28
![Page 29: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/29.jpg)
The autocovariance function (h) satisfies:
phhh p 11
qhhh zxqzxzx 11
For h = 0, 1. … , q:
pp 10 1 qzxqzxzx 10 1
101 1 pp 101 qzxqzx
pqqq p 11 0zxq
for h > q:
phhh p 1129
![Page 30: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/30.jpg)
We then use the first (p + 1) equations to determine: (0), (1), (2), … , (p)
We use the subsequent equations to determine:(h) for h > p.
30
![Page 31: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/31.jpg)
Example:The autocovariance function, (h), for an ARMA(1,1) time series:
11 hh 11 hh zxzx
For h = 0, 1:
10 1 10 1 zxzx
01 1 01 zx
for h > 1: 11 hh
or 10 1 2
1112
01 1 21
31
![Page 32: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/32.jpg)
Substituting (0) into the second equation we get:
or
21
2111
211 11
22
1
1111
1
11
Substituting (1) into the first equation we get:
2111
222
1
11111 1
10
22
1
1112
12
111111
1
111
22
1
1121
1
21
32
![Page 33: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/33.jpg)
for h > 1: 11 hh
22
1
111111 1
112
22
1
1111211 1
123
22
1
1111111 1
11
hhh
33
![Page 34: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/34.jpg)
The Backshift Operator B
34
![Page 35: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/35.jpg)
Consider the time series {Xt : t T} and Let M denote the linear space spanned by the set of random variables {Xt : t T}
(i.e. all linear combinations of elements of {Xt : t T} and their limits in mean square).
M is a vector space
Let B be an operator on M defined by:
BXt = Xt-1.
B is called the backshift operator.35
![Page 36: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/36.jpg)
Note: 1.
2. We can also define the operator Bk withBkXt = B(B(...BXt)) = Xt-k.
3. The polynomial operator p(B) = c0I + c1B + c2B2 + ... + ckBk
can also be defined by the equation.p(B)Xt = (c0I + c1B + c2B2 + ... + ckBk)Xt . = c0IXt + c1BXt + c2B2Xt + ... + ckBkXt
= c0Xt + c1Xt-1 + c2Xt-2 + ... + ckXt-k
ktktt XcXcXcB
21 21
ktktt BXcBXcBXc 21 21
11211 21 ktktt XcXcXc
36
![Page 37: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/37.jpg)
4. The power series operator p(B) = c0I + c1B + c2B2 + ...
can also be defined by the equation.p(B)Xt = (c0I + c1B + c2B2 + ... )Xt
= c0IXt + c1BXt + c2B2Xt + ...
= c0Xt + c1Xt-1 + c2Xt-2 + ...
5. If p(B) = c0I + c1B + c2B2 + ... and q(B) = b0I + b1B + b2B2 + ... are such that
p(B)q(B) = I i.e. p(B)q(B)Xt = IXt = Xt than q(B) is denoted by [p(B)]-1.
37
![Page 38: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/38.jpg)
Other operators closely related to B:
1. F = B-1 ,the forward shift operator, defined by FXt = B-1Xt = Xt+1 and
2. ∇= I - B ,the first difference operator, defined by ∇Xt = (I - B)Xt = Xt - Xt-1 .
38
![Page 39: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/39.jpg)
The Equation for a MA(q) time series
Xt= 0Zt + 1Zt-1 +2Zt-2 +... +qZt-q + can be written
Xt= (B) Zt + where
(B) = 0I + 1B +2B2 +... +qBq
39
![Page 40: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/40.jpg)
The Equation for a AR(p) time series
Xt= 1Xt-1 +2Xt-2 +... +pXt-p + +Zt
can be written
(B) Xt= + Zt
where
(B) = I - 1B - 2B2 -... - pBp
40
![Page 41: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/41.jpg)
The Equation for a ARMA(p,q) time series
Xt= 1Xt-1 +2Xt-2 +... +pXt-p + + Zt + 1Zt-1 +2Zt-2 +... +qZt-q
can be written
(B) Xt= (B) Zt + where
(B) = 0I + 1B +2B2 +... +qBq
and
(B) = I - 1B - 2B2 -... - pBp
41
![Page 42: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/42.jpg)
Some comments about the Backshift operator B
1. It is a useful notational device, allowing us to write the equations for MA(q), AR(p) and ARMA(p, q) in a very compact form;
2. It is also useful for making certain computations related to the time series described above;
42
![Page 43: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/43.jpg)
The partial autocorrelation function
A useful tool in time series analysis
43
![Page 44: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/44.jpg)
The partial autocorrelation function
Recall that the autocorrelation function of an AR(p) process satisfies the equation:
x(h) = 1x(h-1) + 2x(h-2) + ... +px(h-p)
For 1 ≤ h ≤ p these equations (Yule-Walker) become:x(1) = 1 + 2x(1) + ... +px(p-1)
x(2) = 1x(1) + 2 + ... +px(p-2)
...
x(p) = 1x(p-1)+ 2x(p-2) + ... +p. 44
![Page 45: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/45.jpg)
In matrix notation:
pxx
xx
xx
x
x
x
pp
p
p
p
2
1
121
211
111
2
1
These equations can be used to find 1, 2, … , p, if the time series is known to be AR(p) and the autocorrelation x(h) function is known.
45
![Page 46: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/46.jpg)
In this case p
ppp ,,, 21
If the time series is not autoregressive the equations can still be used to solve for 1, 2, … , p, for any value of p >1.
are the values that minimizes the mean square error:
2
1
)()(...p
ixit
pixt XXEESM
46
![Page 47: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/47.jpg)
121
211
111
21
211
111
)(
kk
k
k
kkk
xx
xx
xx
xxx
xx
xx
kkkk
Definition: The partial auto correlation function at lag k is defined to be:
47
![Page 48: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/48.jpg)
Comment:
The partial auto correlation function, kk is determined from the auto correlation function, (h)
48
![Page 49: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/49.jpg)
Some more comments:
1. The partial autocorrelation function at lag k, kk, can be interpreted as a corrected autocorrelation between Xt and Xt-k conditioning on the intervening variables Xt-1, Xt-2, ... , Xt-k+1 .
2. If the time series is an AR(p) time series than
kk = 0 for k > p
3. If the time series is an MA(q) time series than
x(h) = 0 for h > q 49
![Page 50: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/50.jpg)
A General Recursive Formula for Autoregressive Parameters and the
Partial Autocorrelation function (PACF)
50
![Page 51: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/51.jpg)
Letkk
kk
kkk ,,,, 321
denote the autoregressive parameters of order k satisfying the Yule Walker equations:
kkk
kkk13221
223121 kkk
kkk
kkk
kk
kk
kk 332211
51
![Page 52: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/52.jpg)
Then it can be shown that:
k
jj
kj
k
jjk
kjk
kkkk
1
11
1,111
1
and
kjkjkkk
kj
kj ,,2 ,1 11,1
1
52
![Page 53: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/53.jpg)
Proof:
The Yule Walker equations:
kkk
kkk13221
223121 kkk
kkk
kkk
kk
kk
kk 332211
53
![Page 54: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/54.jpg)
In matrix form:
kkk
k
k
kk
k
k
22
1
21
2
1
1
1
1
kkk ρβΡ or
k
k
kk
k
k
k
kk
k
k
k
22
1
21
2
1
and ,
1
1
1
ρβΡ
kkk ρΡβ1
54
![Page 55: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/55.jpg)
The equations for
1
2
11
12
11
1
1
1
1
1
kkk
k
k
kk
k
k
1,111
13
12
11 ,,,,
kkkk
kkk
55
![Page 56: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/56.jpg)
11,1
11
1or
k
k
kk
k
k
kk
ρβ
Aρ
AρΡ
001
000
100
where
A and
113
12
11
11 ,,,, k
kkkkk β
The matrix A reverses order56
![Page 57: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/57.jpg)
kkkk
kk ρAρβΡ
1,11
1
The equations may be written
11,11
1
kkkkk βAρ
Multiplying the first equations by
kkkkkkk
k βρΡAρΡβ
11
1,11
1
1
kΡ
or kkkk
kk AρΡββ1
1,11
1
kkkk
k ρΡAβ1
1,1
k
kkk Aββ 1,1 57
![Page 58: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/58.jpg)
Substituting this into the second equation
or
11,11,1
kkkk
kkkk AββAρ
kkk
kkkk Aβρβρ
11,1 1
and kk
kkk
kk
ρβ
Aβρ
1 1
1,1
58
![Page 59: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/59.jpg)
Hence
k
jj
kj
k
jjk
kjk
kkkk
1
11
1,111
1
and
kjkjkkk
kj
kj ,,2 ,1 11,1
1
kkk
kk Aβββ 1,11
or
59
![Page 60: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/60.jpg)
Some Examples
60
![Page 61: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/61.jpg)
Example 1: MA(1) time seriesSuppose that {Xt|t T} satisfies the following
equation:
Xt = 12.0 + Zt + 0.5 Zt – 1
where {Zt|t T} is white noise with = 1.1.Find:1. The mean of the series,2. The variance of the series,3. The autocorrelation function.4. The partial autocorrelation function.
61
![Page 62: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/62.jpg)
SolutionNow {Xt|t T} satisfies the following equation:
Xt = 12.0 + Zt + 0.5 Zt – 1
Thus:
1. The mean of the series,
= 12.0
The autocovariance function for an MA(1) is
222 21
22
1 0.5 1.1 01 0 1.5125 0
1 0.5 1.1 1 0.605 1
0 1 0 1 0 1
hh h
h h h h
h h h
62
![Page 63: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/63.jpg)
Thus:
2. The variance of the series,
(0) = 1.5125
and
3. The autocorrelation function is:
0.6051.5125
1 0 1 0
1 0.4 10
0 1 0 1
h hh
h h h
h h
63
![Page 64: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/64.jpg)
( )
1 1 1
1 1 2
1 2
1 1 1
1 1 2
1 2 1
kkk k
k k k
k
k
k k
4. The partial auto correlation function at lag k is defined to be:
Thus (1)11 1
11 0.4
1
2 2(2)
22 2 2 2
1 1
1 2 2 1 0.4 0.16.19048
1 1 1 0.4 0.841 1
1 1
64
![Page 65: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/65.jpg)
(3)33 3
1 1 1 1 0.4 0.4
1 1 2 0.4 1 0
2 1 3 0 0.4 0 0.0640.0941
1 .4 0 0.681 1 2
.4 1 .41 1 1
0 .4 12 1 1
(4)44 4
1 1 2 1 1 .4 0 .4
1 1 1 2 .4 1 .4 0
2 1 1 3 0 .4 1 0
3 2 1 4 0 0 .4 0 0.02560.0469
1 .4 0 0 0.54561 1 2 3
.4 1 .4 01 1 1 2
0 .4 1 .42 1 1 1
0 0 .4 13 2 1 1
(5)55 5
0.010240.0234
0.4368 65
![Page 66: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/66.jpg)
66 77 88 990.0117, 0.0059, 0.0029, 0.0015
10,10 11,11 12,120.0007, 0.0004, 0.00029
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
0 1 2 3 4 5 6 7 8 9 10 11
Graph: Partial Autocorrelation function kk
66
![Page 67: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/67.jpg)
Exercise: Use the recursive method to calculate kk
111
1 1, 1
1
1
kk
k j k jjk
k k k kkj j
j
and
11, 1 1 1, 2, , k k k
j j k k k j j k
11 1,1 1we start with
67
![Page 68: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/68.jpg)
Exercise: Use the recursive method to calculate kk
212 2 1 12 2,2 21
1 1
0.4.19048
1 1 0.4
and2 1 1
1 1 2.2 1 1j
1 .19048 0.4
1.19048 0.4 .0.476192
2 23 3 1 2 1 13 3,3 2 2
1 1 2 2
0.0941, etc1
68
![Page 69: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/69.jpg)
Example 2: AR(2) time series
Suppose that {Xt|t T} satisfies the following equation:
Xt = 0.4 Xt – 1 + 0.1 Xt – 2 + 1.2 + Zt
where {Zt|t T} is white noise with = 2.1.Is the time series stationary?Find:1. The mean of the series,2. The variance of the series,3. The autocorrelation function.4. The partial autocorrelation function.
69
![Page 70: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/70.jpg)
1. The mean of the series
1 2
1.22.4
1 1 0.4 0.1
3. The autocorrelation function.Satisfies the Yule Walker equations
1 1 2 1 1
2 1 1 2 1
1 0.4 0.1
1 0.4 0.1
1 1 2 1 1then 0.4 0.1
where h h h h h
h h
70
![Page 71: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/71.jpg)
hence
1
2
0.40.4444
0.90.4
0.4 0.1 2.7780.9
1 1 2 1 1then 0.4 0.1
where h h h h h
h h
h 0 1 2 3 4 5 6
h 1.0000 0.4444 0.2778 0.1556 0.0900 0.0516 0.0296
h 7 8 9 10 11 12 13
h 0.0170 0.0098 0.0056 0.0032 0.0018 0.0011 0.0006
71
![Page 72: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/72.jpg)
2. the variance of the series
2 2
1 1 2 1
2.10 5.7522
1 1 0.4 0.4444 0.1 0.2778
4. The partial autocorrelation function.
1,1 1 0.4444
1
1 22,2
1
1
1 1 0.4444
0.4444 .27780.1000
1 0.44441
0.4444 11
72
![Page 73: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/73.jpg)
1 1
1 2
2 1 33,3
1 2
1 1
2 1
1 1 0.4444 0.4444
1 0.4444 1 0.2778
0.2778 0.4444 0.15560
1 1 0.4444 0.2778
1 0.4444 1 0.4444
1 0.2778 0.4444 1
,in fact 0 for 3k k k
The partial autocorrelation function of an AR(p) time series “cuts off” after p.
73
![Page 74: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/74.jpg)
Example 3: ARMA(1, 2) time series
Suppose that {Xt|t T} satisfies the following equation:
Xt = 0.4 Xt – 1 + 3.2 + Zt + 0.3 Zt – 1 + 0.2 Zt – 1
where {Zt|t T} is white noise with = 1.6.Is the time series stationary?Find:1. The mean of the series,2. The variance of the series,3. The autocorrelation function.4. The partial autocorrelation function.
74
![Page 75: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/75.jpg)
Theoretical Patterns of ACF and PACF
75
Type of Model
Typical Pattern of ACF
Typical Pattern of
PACF
AR (p) Decays exponentially or
with damped sine wave pattern or
both
Cut-off after lags p
MA (q) Cut-off after lags q
Declines exponentially
ARMA (p,q)
Exponential decay Exponential decay
![Page 76: Stationary Time Series AMS 586 1. The Moving Average Time series of order q, MA(q) where {Z t |t T} denote a white noise time series with variance 2](https://reader036.vdocuments.us/reader036/viewer/2022062511/55176f0b5503460e6e8b4ea0/html5/thumbnails/76.jpg)
Reference
• GEP Box, GM Jenkins, GC Reinsel (1994) Time series analysis: Forecasting and control, Prentice-Hall.
• Brockwell, Peter J. and Davis, Richard A. (1991). Time Series: Theory and Methods. Springer-Verlag.
• We also thank colleagues who posted their notes as on-line open resources for time series analysis.
76