stat 497 lecture notes 3 stationary time series processes (arma processes or box-jenkins processes)...

STAT 497LECTURE NOTES 3

STATIONARY TIME SERIES PROCESSES(ARMA PROCESSES OR BOX-JENKINS

PROCESSES)

1

AUTOREGRESSIVE PROCESSES

• AR(p) PROCESS:

or

where

tptptt aYYY 11

ttp aYB

.1 221

ppp BBBB

2

AR(p) PROCESS

• Because the process is

always invertible.• To be stationary, the roots of p(B)=0 must lie

outside the unit circle.• The AR process is useful in describing

situations in which the present value of a time series depends on its preceding values plus a random shock.

;11

p

jj

jj

3

AR(1) PROCESS

where atWN(0, )

• Always invertible.

• To be stationary, the roots of (B)=1B=0

must lie outside the unit circle.

t

c

ttt

B

ttt

aYBaYB

aYY

1111

2a

4

AR(1) PROCESS

• OR using the characteristic equation, the roots

of m=0 must lie inside the unit circle.

B=1 |B|<|1| ||<1 STATIONARITY CONDITION

5

AR(1) PROCESS

• This process sometimes called as the Markov

process because the distribution of Yt given

Yt-1,Yt-2,… is exactly the same as the

distribution of Yt given Yt-1.

6

AR(1) PROCESS

• PROCESS MEAN:

t

c

t

ttt

aYB

aYY

offunction abut mean process the

not is which

1

11

7

AR(1) PROCESS

• AUTOCOVARIANCE FUNCTION: K

kttkk

kttkttk

ktttk

kttkttk

YaE

YaEYYE

YaYE

YYEYYCov

1

1

1

,

8

AR(1) PROCESS tt YaE10

tt aY 1

12 tt aY

21 a

01 02

02

.1 where1 2

2

02

00

aa

9

AR(1) PROCESS

0

1 1,

k

k

kk k

.1,: 1 kACF kkk

When ||<1, the process is stationary and the ACF decays exponentially.

10

AR(1) PROCESS

• 0 < < 1 All autocorrelations are positive.• 1 < < 0 The sign of the autocorrelation

shows an alternating pattern beginning a negative value.

1,0

1,: 1

k

kPACF kk

11

AR(1) PROCESS

• RSF: Using the geometric series

t

it

iitt

aBB

aBaB

Y

22

0

1

1

11

0 1 2

0, iii

12

AR(1) PROCESS

• RSF: By operator method _ We know that

1 and 1 BBBB

0,

0

0

111

1

2212

11

221

j

BBB

BB

jj

13

AR(1) PROCESS

• RSF: By recursion

tttt

tttt

tttttt

ttttt

aaaY

aaaY

aaYaaY

YYaYY

122

33

1232

122

12

1 where

14

THE SECOND ORDER AUTOREGRESSIVE PROCESS

• AR(2) PROCESS: Consider the series satisfying

15

tttt aYYY 2211

t

c

t

tttt

aYBB

aYYY

212

21

221121

11

1

where atWN(0, ).2a

AR(2) PROCESS

• Always invertible.• Already in Inverted Form.• To be stationary, the roots of

must lie outside the unit circle. OR the roots of the characteristic equation

must lie inside the unit circle.16

01 221 BBB

0212 mm

AR(2) PROCESS

17

stationary be toprocess AR(2)for condition required the

22

112,1 1

2

4

m

2

1

121121

221221

mmmm

mmmm

AR(2) PROCESS

• Considering both real and complex roots, we have the following stationary conditions for AR(2) process (see page 84 for the proof)

18

1

1

1

2

12

21

AR(2) PROCESS

• THE AUTOCOVARIANCE FUNCTION: Assuming stationarity and that at is independent of Yt-k, we have

19

kttkk

ktttt

kttk

YaE

YaYYE

YYE

2211

2211

AR(2) PROCESS

20

2

2211

2211

2211

0

a

tt

tttt

tt

YaE

YaYYE

YYE

AR(2) PROCESS

21

1201

11201

12211

11

tt

tttt

tt

YaE

YaYYE

YYE

2

011 1

AR(2) PROCESS

22

0211

20211

22211

22

tt

tttt

tt

YaE

YaYYE

YYE

02

222

21

2 1

AR(2) PROCESS

23

2

1222

22

0

20

2

222

21

202

11

222110

11

1

11

a

a

a

AR(2) PROCESS

ACF: It is known as Yule-Walker Equations

24

0,2211 kkkk

0,2211 kkkk

ACF shows an exponential decay or sinusoidal behavior.

AR(2) PROCESS

• PACF:

25

2,0

1

1

221

212

2

11

k

kk

PACF cuts off after lag 2.

AR(2) PROCESS• RANDOM SHOCK FORM: Using the Operator

Method

26

tt

B

aYBB

2

211

1 BB

2,

2

2211

213

13

22

12

11

jjjj

The p-th ORDER AUTOREGRESSIVE PROCESS: AR(p) PROCESS

• Consider the process satisfying

27

tptptt aYYY 11

where atWN(0, ).2a

tt

B

pp aYBB

p

11

provided that roots of all lie outside the unit circle

01 1 ppBB

AR(p) PROCESS

• ACF: Yule-Walker Equations

• ACF: tails of as a mixture of exponential decay or damped sine wave (if some roots are complex).

• PACF: cuts off after lag p.

28

0,11 kpkpkk

0,11 kpkpkk

MOVING AVERAGE PROCESSES

• Suppose you win 1 TL if a fair coin shows a head and lose 1 TL if it shows tail. Denote the outcome on toss t by at.

• The average winning on the last 4 tosses=average pay-off on the last tosses:

29

up shows tail,1

up shows head ,1ta

321 41

41

41

41

tttt aaaa MOVING AVERAGE PROCESS



30

.,0~ where

1

2

1

11

at

t

B

qq

qtqttt

WNa

aBB

aaaY

q


• Because , MA processes

are always stationary.

• Invertible if the roots of q(B)=0 all lie outside

the unit circle.

• It is a useful process to describe events

producing an immediate effects that lasts for

short period of time.31

q

ii

jj

1

2

0

2 1

THE FIRST ORDER MOVING AVERAGE PROCESS_MA(1) PROCESS


32

.,0~ where

1

2

1

1

at

t

B

ttt

WNa

aBaaY

tYE:Mean Process The

2220:Variance Process The aatYVar

MA(1) PROCESS

• From autocovariance generating function

33

122

111

2

12

1

BB

BB

BBB

a

a

a

1,0

1,

0,12

22

k

k

k

a

a

k

MA(1) PROCESS

• ACF

34

1,0

1,1 2

k

kk

ACF cuts off after lag 1.

General property of MA(1) processes: 2|k|<1

MA(1) PROCESS

• PACF:

35

4

2

2111 11

1

6

22

4221

21

22 11

11

1

1

112

2

k,

k

k

kk

MA(1) PROCESS

• Basic characteristic of MA(1) Process:– ACF cuts off after lag 1.– PACF tails of exponentially depending on the sign

of .– Always stationary.– Invertible if the root of 1B=0 lie outside the unit

circle or the root of the characteristic equation m=0 lie inside the unit circle.

INVERTIBILITY CONDITION: ||<1.

36

MA(1) PROCESS

• It is already in RSF.• IF:

37

tt

tti

ii

tt

tt

aYBB

aYB

aYB

aBY

22

0

1

11

1

1= 2=2

MA(1) PROCESS

• IF: By operator method

38

1,

111

12

21

j

BBB

BB

jj

THE SECOND ORDER MOVING AVERAGE PROCESS_MA(2) PROCESS• Consider the moving average process of order 2:

39

.,0~ where

1

2

2212211

2

at

t

B

tttt

WNa

aBBaaaY


22

222

12

0:Variance Process The

aaa

tYVar

MA(2) PROCESS• From autocovariance generating function

40

22

22

1211211

22

21

2

122

2

12

1

BBBB

BB

BBB

a

a

a

2,0

2,

1,1

0,1

22

22

1

22

21

2

k

k

k

k

a

a

a

k

MA(2) PROCESS• ACF

• ACF cuts off after lag 2.• PACF tails of exponentially or a damped sine

waves depending on a sign and magnitude of parameters. 41

2,0

2,1

1,1

1

22

21

2

22

21

21

k

k

k

k

MA(2) PROCESS

• Always stationary.• Invertible if the roots of

all lie outside the unit circle.OR if the roots of

all lie inside the unit circle.

42

01 221 BB

0212 mm

MA(2) PROCESS

• Invertibility condition for MA(2) process

43

11

1

1

2

12

21

MA(2) PROCESS

• It is already in RSF form.• IF: Using the operator method:

44

2,

111

1

2211

221

221

j

BBBB

BB

jjj

The q-th ORDER MOVING PROCESS_ MA(q) PROCESS

45

.,0~ where

1

2

1

11

at

t

BB

qq

qtqttt

WNa

aBB

aaaY

q


2222

12

0:Variance Process The

aqaa

tYVar

Consider the MA(q) process:

MA(q) PROCESS

• The autocovariance function:

• ACF:

46

qk

qkqkqkkak

,0

,,2,1,12

qk

qkq

qkqkk

k

,0

,,2,1,1 22

1

1

THE AUTOREGRESSIVE MOVING AVERAGE PROCESSES_ARMA(p, q)

PROCESSES• If we assume that the series is partly

autoregressive and partly moving average, we obtain a mixed ARMA process.

47

qtqttptptt aaaYYY 1111

.,0~ and where 2attt WNaYY

tqtp aBYB

qpARMA

:,

ARMA(p, q) PROCESSES• For the process to be invertible, the roots of lie outside the unit circle.• For the process to be stationary, the roots of lie outside the unit circle.• Assuming that and share

no common roots,Pure AR Representation:Pure MA Representation:

48

0Bq

0Bp 0Bp 0Bq

B

BBaYB

q

ptt

B

BBaBY

p

qtt

ARMA(p, q) PROCESSES

• Autocovariance function

• ACF

• Like AR(p) process, it tails of after lag q.• PACF: Like MA(q), it tails of after lag p.

49

1,11 qkpkpkk

1,11 qkpkpkk

ARMA(1, 1) PROCESSES

• The ARMA(1, 1) process can be written as

50

tt

tttt

aBYB

aaYY

1111

.,0~ where 2at WNa

•Stationary if ||<1.

•Invertible if ||<1.

ARMA(1, 1) PROCESSES

• Autocovariance function:

51

kttkttk

kttkttktt

ktttt

kttk

YaEYaE

YaEYaEYYE

YaaYE

YYE

11

11

11

ARMA(1,1) PROCESS

• The process variance

52

2220

110

1

212

12

aaa

aaY

tttt

tt

tatatY

ta

YaEYaE

ARMA(1,1) PROCESS

53

20

11

0

101

212

a

aaY

tttt

ttt

YaEYaE

2

2

0

22200

121

1

aa

ARMA(1,1) PROCESS

• Both ACF and PACF tails of after lag 1.

54

1,

1,

11

11

k

kk

kk

kkk

ARMA(1,1) PROCESS

• IF:

55

0,

111

1

1

221

j

BBBB

BB

jj

ARMA(1,1) PROCESS

• RSF:

56

0,

111

1

1

221

j

BBBB

BB

jj

AR(1) PROCESS

57

AR(2) PROCESS

58

MA(1) PROCESS

59

MA(2) PROCESS

60

ARMA(1,1) PROCESS

61

ARMA(1,1) PROCESS (contd.)

62

stat 497 lecture notes 3 stationary time series processes (arma processes or box-jenkins processes)...

Documents