![Page 1: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/1.jpg)
1
ECON 427:ECONOMICFORECASTING
Ch8. ARIMA models
OTexts.org/fpp2/
![Page 2: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/2.jpg)
Outline
1 Stationarity and differencing
2 Non-seasonal ARIMA models
3 Estimation and order selection
4 ARIMA modelling in R
5 Forecasting
6 Seasonal ARIMA models
7 ARIMA vs ETS
2
![Page 3: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/3.jpg)
Stationarity
DefinitionIf {yt} is a stationary time series, then for all s, thedistribution of (yt, . . . , yt+s) does not depend on t.
A stationary series is:
roughly horizontalconstant varianceno patterns predictable in the long-term
3
![Page 4: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/4.jpg)
Stationarity
DefinitionIf {yt} is a stationary time series, then for all s, thedistribution of (yt, . . . , yt+s) does not depend on t.
A stationary series is:
roughly horizontalconstant varianceno patterns predictable in the long-term
3
![Page 5: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/5.jpg)
Stationary?
3600
3700
3800
3900
4000
0 50 100 150 200 250 300
Day
Dow
Jon
es In
dex
4
![Page 6: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/6.jpg)
Stationary?
−100
−50
0
50
0 50 100 150 200 250 300
Day
Cha
nge
in D
ow J
ones
Inde
x
5
![Page 7: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/7.jpg)
Stationary?
4000
5000
6000
1950 1955 1960 1965 1970 1975 1980
Year
Num
ber
of s
trik
es
6
![Page 8: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/8.jpg)
Stationary?
40
60
80
1975 1980 1985 1990 1995
Year
Tota
l sal
es
Sales of new one−family houses, USA
7
![Page 9: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/9.jpg)
Stationary?
100
200
300
1900 1920 1940 1960 1980
Year
$
Price of a dozen eggs in 1993 dollars
8
![Page 10: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/10.jpg)
Stationary?
80
90
100
110
1990 1991 1992 1993 1994 1995
Year
thou
sand
s
Number of pigs slaughtered in Victoria
9
![Page 11: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/11.jpg)
Stationary?
0
2000
4000
6000
1820 1840 1860 1880 1900 1920
Year
Num
ber
trap
ped
Annual Canadian Lynx Trappings
10
![Page 12: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/12.jpg)
Stationary?
400
450
500
1995 2000 2005 2010
Year
meg
alitr
es
Australian quarterly beer production
11
![Page 13: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/13.jpg)
Stationarity
DefinitionIf {yt} is a stationary time series, then for all s, thedistribution of (yt, . . . , yt+s) does not depend on t.
Transformations help to stabilize the variance.For ARIMA modelling, we also need to stabilize themean.
12
![Page 14: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/14.jpg)
Stationarity
DefinitionIf {yt} is a stationary time series, then for all s, thedistribution of (yt, . . . , yt+s) does not depend on t.
Transformations help to stabilize the variance.For ARIMA modelling, we also need to stabilize themean.
12
![Page 15: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/15.jpg)
Non-stationarity in the mean
Identifying non-stationary series
time plot.The ACF of stationary data drops to zerorelatively quicklyThe ACF of non-stationary data decreases slowly.For non-stationary data, the value of r1 is oftenlarge and positive.
13
![Page 16: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/16.jpg)
Example: Dow-Jones index
3600
3700
3800
3900
4000
0 50 100 150 200 250 300
Day
Dow
Jon
es In
dex
14
![Page 17: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/17.jpg)
Example: Dow-Jones index
0.00
0.25
0.50
0.75
1.00
0 5 10 15 20 25
Lag
AC
F
Series: dj
15
![Page 18: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/18.jpg)
Example: Dow-Jones index
−100
−50
0
50
0 50 100 150 200 250 300
Day
Cha
nge
in D
ow J
ones
Inde
x
16
![Page 19: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/19.jpg)
Example: Dow-Jones index
−0.10
−0.05
0.00
0.05
0.10
0 5 10 15 20 25
Lag
AC
F
Series: diff(dj)
17
![Page 20: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/20.jpg)
Differencing
Differencing helps to stabilize the mean.The differenced series is the change betweeneach observation in the original series:y′t = yt − yt−1.The differenced series will have only T − 1 valuessince it is not possible to calculate a difference y′1for the first observation.
18
![Page 21: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/21.jpg)
Second-order differencing
Occasionally the differenced data will not appearstationary and it may be necessary to difference thedata a second time:
y′′t = y′t − y′t−1= (yt − yt−1)− (yt−1 − yt−2)= yt − 2yt−1 + yt−2.
y′′t will have T − 2 values.In practice, it is almost never necessary to gobeyond second-order differences.
19
![Page 22: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/22.jpg)
Second-order differencing
Occasionally the differenced data will not appearstationary and it may be necessary to difference thedata a second time:
y′′t = y′t − y′t−1= (yt − yt−1)− (yt−1 − yt−2)= yt − 2yt−1 + yt−2.
y′′t will have T − 2 values.In practice, it is almost never necessary to gobeyond second-order differences.
19
![Page 23: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/23.jpg)
Second-order differencing
Occasionally the differenced data will not appearstationary and it may be necessary to difference thedata a second time:
y′′t = y′t − y′t−1= (yt − yt−1)− (yt−1 − yt−2)= yt − 2yt−1 + yt−2.
y′′t will have T − 2 values.In practice, it is almost never necessary to gobeyond second-order differences.
19
![Page 24: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/24.jpg)
Seasonal differencing
A seasonal difference is the difference between anobservation and the corresponding observation fromthe previous year.
y′t = yt − yt−mwherem = number of seasons.
For monthly datam = 12.For quarterly datam = 4.
20
![Page 25: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/25.jpg)
Seasonal differencing
A seasonal difference is the difference between anobservation and the corresponding observation fromthe previous year.
y′t = yt − yt−mwherem = number of seasons.
For monthly datam = 12.For quarterly datam = 4.
20
![Page 26: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/26.jpg)
Seasonal differencing
A seasonal difference is the difference between anobservation and the corresponding observation fromthe previous year.
y′t = yt − yt−mwherem = number of seasons.
For monthly datam = 12.For quarterly datam = 4.
20
![Page 27: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/27.jpg)
Electricity production
usmelec %>% autoplot()
200
300
400
1980 1990 2000 2010
Time
.
21
![Page 28: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/28.jpg)
Electricity production
usmelec %>% log() %>% autoplot()
5.1
5.4
5.7
6.0
1980 1990 2000 2010
Time
.
22
![Page 29: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/29.jpg)
Electricity production
usmelec %>% log() %>% diff(lag=12) %>%autoplot()
0.0
0.1
1980 1990 2000 2010
Time
.
23
![Page 30: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/30.jpg)
Electricity production
usmelec %>% log() %>% diff(lag=12) %>%diff(lag=1) %>% autoplot()
−0.15
−0.10
−0.05
0.00
0.05
0.10
1980 1990 2000 2010
Time
.
24
![Page 31: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/31.jpg)
Electricity production
Seasonally differenced series is closer to beingstationary.Remaining non-stationarity can be removed withfurther first difference.
If y′t = yt− yt−12 denotes seasonally differenced series,then twice-differenced series i
y∗t = y′t − y′t−1= (yt − yt−12)− (yt−1 − yt−13)= yt − yt−1 − yt−12 + yt−13 .
25
![Page 32: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/32.jpg)
Seasonal differencing
When both seasonal and first differences areapplied. . .
it makes no difference which is done first—theresult will be the same.If seasonality is strong, we recommend thatseasonal differencing be done first becausesometimes the resulting series will be stationaryand there will be no need for further firstdifference.
It is important that if differencing is used, thedifferences are interpretable.
26
![Page 33: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/33.jpg)
Seasonal differencing
When both seasonal and first differences areapplied. . .
it makes no difference which is done first—theresult will be the same.If seasonality is strong, we recommend thatseasonal differencing be done first becausesometimes the resulting series will be stationaryand there will be no need for further firstdifference.
It is important that if differencing is used, thedifferences are interpretable.
26
![Page 34: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/34.jpg)
Seasonal differencing
When both seasonal and first differences areapplied. . .
it makes no difference which is done first—theresult will be the same.If seasonality is strong, we recommend thatseasonal differencing be done first becausesometimes the resulting series will be stationaryand there will be no need for further firstdifference.
It is important that if differencing is used, thedifferences are interpretable. 26
![Page 35: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/35.jpg)
Interpretation of differencing
first differences are the change between oneobservation and the next;seasonal differences are the change betweenone year to the next.
But taking lag 3 differences for yearly data, forexample, results in a model which cannot be sensiblyinterpreted.
27
![Page 36: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/36.jpg)
Interpretation of differencing
first differences are the change between oneobservation and the next;seasonal differences are the change betweenone year to the next.
But taking lag 3 differences for yearly data, forexample, results in a model which cannot be sensiblyinterpreted.
27
![Page 37: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/37.jpg)
Unit root tests
Statistical tests to determine the required order ofdifferencing.
1 Augmented Dickey Fuller test: null hypothesis isthat the data are non-stationary andnon-seasonal.
2 Kwiatkowski-Phillips-Schmidt-Shin (KPSS) test:null hypothesis is that the data are stationaryand non-seasonal.
3 Other tests available for seasonal data.
28
![Page 38: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/38.jpg)
KPSS test
library(urca)summary(ur.kpss(goog))
#### ######################### # KPSS Unit Root Test ### ########################### Test is of type: mu with 7 lags.#### Value of test-statistic is: 10.7223#### Critical value for a significance level of:## 10pct 5pct 2.5pct 1pct## critical values 0.347 0.463 0.574 0.739
ndiffs(goog)
## [1] 1
29
![Page 39: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/39.jpg)
KPSS test
library(urca)summary(ur.kpss(goog))
#### ######################### # KPSS Unit Root Test ### ########################### Test is of type: mu with 7 lags.#### Value of test-statistic is: 10.7223#### Critical value for a significance level of:## 10pct 5pct 2.5pct 1pct## critical values 0.347 0.463 0.574 0.739
ndiffs(goog)
## [1] 1
29
![Page 40: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/40.jpg)
Automatically selecting differences
STL decomposition: yt = Tt + St + RtSeasonal strength Fs = max
(0, 1− Var(Rt)
Var(St+Rt)
)If Fs > 0.64, do one seasonal difference.
usmelec %>% log() %>% nsdiffs()
## [1] 1
usmelec %>% log() %>% diff(lag=12) %>% ndiffs()
## [1] 1
30
![Page 41: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/41.jpg)
Automatically selecting differences
STL decomposition: yt = Tt + St + RtSeasonal strength Fs = max
(0, 1− Var(Rt)
Var(St+Rt)
)If Fs > 0.64, do one seasonal difference.
usmelec %>% log() %>% nsdiffs()
## [1] 1
usmelec %>% log() %>% diff(lag=12) %>% ndiffs()
## [1] 130
![Page 42: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/42.jpg)
Your turn
For the visitors series, find an appropriatedifferencing (after transformation if necessary) toobtain stationary data.
31
![Page 43: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/43.jpg)
Backshift notation
A very useful notational device is the backward shiftoperator, B, which is used as follows:
Byt = yt−1 .
In other words, B, operating on yt, has the effect ofshifting the data back one period. Two applicationsof B to yt shifts the data back two periods:
B(Byt) = B2yt = yt−2 .
For monthly data, if we wish to shift attention to “thesame month last year,” then B12 is used, and thenotation is B12yt = yt−12.
32
![Page 44: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/44.jpg)
Backshift notation
A very useful notational device is the backward shiftoperator, B, which is used as follows:
Byt = yt−1 .
In other words, B, operating on yt, has the effect ofshifting the data back one period.
Two applicationsof B to yt shifts the data back two periods:
B(Byt) = B2yt = yt−2 .
For monthly data, if we wish to shift attention to “thesame month last year,” then B12 is used, and thenotation is B12yt = yt−12.
32
![Page 45: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/45.jpg)
Backshift notation
A very useful notational device is the backward shiftoperator, B, which is used as follows:
Byt = yt−1 .
In other words, B, operating on yt, has the effect ofshifting the data back one period. Two applicationsof B to yt shifts the data back two periods:
B(Byt) = B2yt = yt−2 .
For monthly data, if we wish to shift attention to “thesame month last year,” then B12 is used, and thenotation is B12yt = yt−12.
32
![Page 46: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/46.jpg)
Backshift notation
A very useful notational device is the backward shiftoperator, B, which is used as follows:
Byt = yt−1 .
In other words, B, operating on yt, has the effect ofshifting the data back one period. Two applicationsof B to yt shifts the data back two periods:
B(Byt) = B2yt = yt−2 .
For monthly data, if we wish to shift attention to “thesame month last year,” then B12 is used, and thenotation is B12yt = yt−12.
32
![Page 47: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/47.jpg)
Backshift notation
The backward shift operator is convenient fordescribing the process of differencing.
A firstdifference can be written as
y′t = yt − yt−1 = yt − Byt = (1− B)yt .
Note that a first difference is represented by (1− B).Similarly, if second-order differences (i.e., firstdifferences of first differences) have to be computed,then:
y′′t = yt − 2yt−1 + yt−2 = (1− B)2yt .
33
![Page 48: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/48.jpg)
Backshift notation
The backward shift operator is convenient fordescribing the process of differencing. A firstdifference can be written as
y′t = yt − yt−1 = yt − Byt = (1− B)yt .
Note that a first difference is represented by (1− B).Similarly, if second-order differences (i.e., firstdifferences of first differences) have to be computed,then:
y′′t = yt − 2yt−1 + yt−2 = (1− B)2yt .
33
![Page 49: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/49.jpg)
Backshift notation
The backward shift operator is convenient fordescribing the process of differencing. A firstdifference can be written as
y′t = yt − yt−1 = yt − Byt = (1− B)yt .
Note that a first difference is represented by (1− B).
Similarly, if second-order differences (i.e., firstdifferences of first differences) have to be computed,then:
y′′t = yt − 2yt−1 + yt−2 = (1− B)2yt .
33
![Page 50: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/50.jpg)
Backshift notation
The backward shift operator is convenient fordescribing the process of differencing. A firstdifference can be written as
y′t = yt − yt−1 = yt − Byt = (1− B)yt .
Note that a first difference is represented by (1− B).Similarly, if second-order differences (i.e., firstdifferences of first differences) have to be computed,then:
y′′t = yt − 2yt−1 + yt−2 = (1− B)2yt .
33
![Page 51: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/51.jpg)
Backshift notation
Second-order difference is denoted (1− B)2.Second-order difference is not the same as asecond difference, which would be denoted1− B2;In general, a dth-order difference can be writtenas
(1− B)dyt.A seasonal difference followed by a firstdifference can be written as
(1− B)(1− Bm)yt .34
![Page 52: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/52.jpg)
Backshift notation
The “backshift” notation is convenient because theterms can be multiplied together to see the combinedeffect.
(1− B)(1− Bm)yt = (1− B− Bm + Bm+1)yt= yt − yt−1 − yt−m + yt−m−1.
For monthly data,m = 12 and we obtain the sameresult as earlier.
35
![Page 53: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/53.jpg)
Backshift notation
The “backshift” notation is convenient because theterms can be multiplied together to see the combinedeffect.
(1− B)(1− Bm)yt = (1− B− Bm + Bm+1)yt= yt − yt−1 − yt−m + yt−m−1.
For monthly data,m = 12 and we obtain the sameresult as earlier.
35
![Page 54: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/54.jpg)
Outline
1 Stationarity and differencing
2 Non-seasonal ARIMA models
3 Estimation and order selection
4 ARIMA modelling in R
5 Forecasting
6 Seasonal ARIMA models
7 ARIMA vs ETS
36
![Page 55: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/55.jpg)
Autoregressive models
Autoregressive (AR) models:
yt = c + φ1yt−1 + φ2yt−2 + · · · + φpyt−p + εt,where εt is white noise. This is a multiple regressionwith lagged values of yt as predictors.
8
10
12
0 20 40 60 80 100
Time
AR(1)
15.0
17.5
20.0
22.5
25.0
0 20 40 60 80 100
Time
AR(2)
37
![Page 56: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/56.jpg)
AR(1) model
yt = 2− 0.8yt−1 + εt
εt ∼ N(0, 1), T = 100.
8
10
12
0 20 40 60 80 100
Time
AR(1)
38
![Page 57: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/57.jpg)
AR(1) model
yt = c + φ1yt−1 + εt
When φ1 = 0, yt is equivalent to WNWhen φ1 = 1 and c = 0, yt is equivalent to a RWWhen φ1 = 1 and c 6= 0, yt is equivalent to a RWwith driftWhen φ1 < 0, yt tends to oscillate betweenpositive and negative values.
39
![Page 58: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/58.jpg)
AR(2) model
yt = 8 + 1.3yt−1 − 0.7yt−2 + εt
εt ∼ N(0, 1), T = 100.
15.0
17.5
20.0
22.5
25.0
0 20 40 60 80 100
Time
AR(2)
40
![Page 59: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/59.jpg)
Stationarity conditions
We normally restrict autoregressive models tostationary data, and then some constraints on thevalues of the parameters are required.General condition for stationarityComplex roots of 1− φ1z− φ2z2 − · · · − φpzp lieoutside the unit circle on the complex plane.
For p = 1: −1 < φ1 < 1.For p = 2:−1 < φ2 < 1 φ2 + φ1 < 1 φ2 − φ1 < 1.More complicated conditions hold for p ≥ 3.Estimation software takes care of this.
41
![Page 60: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/60.jpg)
Stationarity conditions
We normally restrict autoregressive models tostationary data, and then some constraints on thevalues of the parameters are required.General condition for stationarityComplex roots of 1− φ1z− φ2z2 − · · · − φpzp lieoutside the unit circle on the complex plane.
For p = 1: −1 < φ1 < 1.For p = 2:−1 < φ2 < 1 φ2 + φ1 < 1 φ2 − φ1 < 1.More complicated conditions hold for p ≥ 3.Estimation software takes care of this. 41
![Page 61: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/61.jpg)
Moving Average (MA) models
Moving Average (MA) models:
yt = c + εt + θ1εt−1 + θ2εt−2 + · · · + θqεt−q,where εt is white noise. This is a multiple regressionwith past errors as predictors. Don’t confuse this withmoving average smoothing!
18
20
22
0 20 40 60 80 100
Time
MA(1)
−5.0
−2.5
0.0
2.5
0 20 40 60 80 100
Time
MA(2)
42
![Page 62: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/62.jpg)
MA(1) model
yt = 20 + εt + 0.8εt−1
εt ∼ N(0, 1), T = 100.
18
20
22
0 20 40 60 80 100
Time
MA(1)
43
![Page 63: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/63.jpg)
MA(2) model
yt = εt − εt−1 + 0.8εt−2
εt ∼ N(0, 1), T = 100.
−5.0
−2.5
0.0
2.5
0 20 40 60 80 100
Time
MA(2)
44
![Page 64: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/64.jpg)
MA(∞) models
It is possible to write any stationary AR(p) process asan MA(∞) process.Example: AR(1)
yt = φ1yt−1 + εt= φ1(φ1yt−2 + εt−1) + εt= φ21yt−2 + φ1εt−1 + εt= φ31yt−3 + φ21εt−2 + φ1εt−1 + εt. . .
Provided−1 < φ1 < 1:yt = εt + φ1εt−1 + φ21εt−2 + φ31εt−3 + · · ·
45
![Page 65: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/65.jpg)
MA(∞) models
It is possible to write any stationary AR(p) process asan MA(∞) process.Example: AR(1)
yt = φ1yt−1 + εt= φ1(φ1yt−2 + εt−1) + εt= φ21yt−2 + φ1εt−1 + εt= φ31yt−3 + φ21εt−2 + φ1εt−1 + εt. . .
Provided−1 < φ1 < 1:yt = εt + φ1εt−1 + φ21εt−2 + φ31εt−3 + · · ·
45
![Page 66: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/66.jpg)
Invertibility
Any MA(q) process can be written as an AR(∞)process if we impose some constraints on theMA parameters.Then the MA model is called “invertible”.Invertible models have some mathematicalproperties that make them easier to use inpractice.Invertibility of an ARIMA model is equivalent toforecastability of an ETS model.
46
![Page 67: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/67.jpg)
Invertibility
General condition for invertibilityComplex roots of 1 + θ1z + θ2z2 + · · · + θqzq lie outsidethe unit circle on the complex plane.
For q = 1: −1 < θ1 < 1.For q = 2:−1 < θ2 < 1 θ2 + θ1 > −1 θ1 − θ2 < 1.More complicated conditions hold for q ≥ 3.Estimation software takes care of this.
47
![Page 68: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/68.jpg)
Invertibility
General condition for invertibilityComplex roots of 1 + θ1z + θ2z2 + · · · + θqzq lie outsidethe unit circle on the complex plane.
For q = 1: −1 < θ1 < 1.For q = 2:−1 < θ2 < 1 θ2 + θ1 > −1 θ1 − θ2 < 1.More complicated conditions hold for q ≥ 3.Estimation software takes care of this.
47
![Page 69: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/69.jpg)
ARIMA models
Autoregressive Moving Average models:
yt = c + φ1yt−1 + · · · + φpyt−p+ θ1εt−1 + · · · + θqεt−q + εt.
Predictors include both lagged values of yt andlagged errors.Conditions on coefficients ensure stationarity.Conditions on coefficients ensure invertibility.
Autoregressive Integrated Moving Average modelsCombine ARMA model with differencing.(1− B)dyt follows an ARMA model.
48
![Page 70: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/70.jpg)
ARIMA models
Autoregressive Moving Average models:
yt = c + φ1yt−1 + · · · + φpyt−p+ θ1εt−1 + · · · + θqεt−q + εt.
Predictors include both lagged values of yt andlagged errors.Conditions on coefficients ensure stationarity.Conditions on coefficients ensure invertibility.
Autoregressive Integrated Moving Average modelsCombine ARMA model with differencing.(1− B)dyt follows an ARMA model.
48
![Page 71: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/71.jpg)
ARIMA models
Autoregressive Moving Average models:
yt = c + φ1yt−1 + · · · + φpyt−p+ θ1εt−1 + · · · + θqεt−q + εt.
Predictors include both lagged values of yt andlagged errors.Conditions on coefficients ensure stationarity.Conditions on coefficients ensure invertibility.
Autoregressive Integrated Moving Average modelsCombine ARMA model with differencing.(1− B)dyt follows an ARMA model. 48
![Page 72: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/72.jpg)
ARIMA models
Autoregressive Integrated Moving Average modelsARIMA(p, d, q) modelAR: p = order of the autoregressive partI: d = degree of first differencing involved
MA: q = order of the moving average part.
White noise model: ARIMA(0,0,0)Random walk: ARIMA(0,1,0) with no constantRandom walk with drift: ARIMA(0,1,0) with const.AR(p): ARIMA(p,0,0)MA(q): ARIMA(0,0,q) 49
![Page 73: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/73.jpg)
Backshift notation for ARIMA
ARMA model:yt = c + φ1Byt + · · · + φpBpyt + εt + θ1Bεt + · · · + θqBqεtor (1− φ1B− · · · − φpBp)yt = c + (1 + θ1B + · · · + θqBq)εt
ARIMA(1,1,1) model:
(1− φ1B) (1− B)yt = c + (1 + θ1B)εt↑ ↑ ↑
AR(1) First MA(1)difference
Written out:yt = c + yt−1 + φ1yt−1 − φ1yt−2 + θ1εt−1 + εt
50
![Page 74: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/74.jpg)
Backshift notation for ARIMA
ARMA model:yt = c + φ1Byt + · · · + φpBpyt + εt + θ1Bεt + · · · + θqBqεtor (1− φ1B− · · · − φpBp)yt = c + (1 + θ1B + · · · + θqBq)εt
ARIMA(1,1,1) model:
(1− φ1B) (1− B)yt = c + (1 + θ1B)εt↑ ↑ ↑
AR(1) First MA(1)difference
Written out:yt = c + yt−1 + φ1yt−1 − φ1yt−2 + θ1εt−1 + εt
50
![Page 75: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/75.jpg)
R model
Intercept form(1− φ1B− · · · − φpBp)y′
t = c + (1 + θ1B + · · · + θqBq)εt
Mean form(1− φ1B− · · · − φpBp)(y′
t − µ) = (1 + θ1B + · · · + θqBq)εt
y′t = (1− B)dytµ is the mean of y′
t.c = µ(1− φ1 − · · · − φp).
51
![Page 76: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/76.jpg)
US personal consumption
−2
−1
0
1
2
1970 1980 1990 2000 2010
Year
Qua
rter
ly p
erce
ntag
e ch
ange
US consumption
52
![Page 77: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/77.jpg)
US personal consumption
(fit <- auto.arima(uschange[,"Consumption"]))
## Series: uschange[, "Consumption"]## ARIMA(2,0,2) with non-zero mean#### Coefficients:## ar1 ar2 ma1 ma2 mean## 1.3908 -0.5813 -1.1800 0.5584 0.7463## s.e. 0.2553 0.2078 0.2381 0.1403 0.0845#### sigma^2 estimated as 0.3511: log likelihood=-165.14## AIC=342.28 AICc=342.75 BIC=361.67
“‘
ARIMA(2,0,2) model:yt = c + 1.391yt−1 − 0.581yt−2 − 1.180εt−1 + 0.558εt−2 + εt,
where c = 0.746× (1− 1.391 + 0.581) = 0.142 and εt is white noise with astandard deviation of 0.593 =
√0.351.
53
![Page 78: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/78.jpg)
US personal consumption
(fit <- auto.arima(uschange[,"Consumption"]))
## Series: uschange[, "Consumption"]## ARIMA(2,0,2) with non-zero mean#### Coefficients:## ar1 ar2 ma1 ma2 mean## 1.3908 -0.5813 -1.1800 0.5584 0.7463## s.e. 0.2553 0.2078 0.2381 0.1403 0.0845#### sigma^2 estimated as 0.3511: log likelihood=-165.14## AIC=342.28 AICc=342.75 BIC=361.67
“‘
ARIMA(2,0,2) model:yt = c + 1.391yt−1 − 0.581yt−2 − 1.180εt−1 + 0.558εt−2 + εt,
where c = 0.746× (1− 1.391 + 0.581) = 0.142 and εt is white noise with astandard deviation of 0.593 =
√0.351.
53
![Page 79: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/79.jpg)
US personal consumption
fit %>% forecast(h=10) %>% autoplot(include=80)
−1
0
1
2
2000 2005 2010 2015 2020
Time
usch
ange
[, "C
onsu
mpt
ion"
]
level
80
95
Forecasts from ARIMA(2,0,2) with non−zero mean
54
![Page 80: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/80.jpg)
Understanding ARIMA models
If c = 0 and d = 0, the long-term forecasts will goto zero.If c = 0 and d = 1, the long-term forecasts will goto a non-zero constant.If c = 0 and d = 2, the long-term forecasts willfollow a straight line.If c 6= 0 and d = 0, the long-term forecasts will goto the mean of the data.If c 6= 0 and d = 1, the long-term forecasts willfollow a straight line.If c 6= 0 and d = 2, the long-term forecasts willfollow a quadratic trend. 55
![Page 81: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/81.jpg)
Understanding ARIMA models
Forecast variance and dThe higher the value of d, the more rapidly theprediction intervals increase in size.For d = 0, the long-term forecast standarddeviation will go to the standard deviation of thehistorical data.
Cyclic behaviourFor cyclic forecasts, p ≥ 2 and some restrictionson coefficients are required.If p = 2, we need φ21 + 4φ2 < 0. Then averagecycle of length
(2π)/[arc cos(−φ1(1− φ2)/(4φ2))
]. 56
![Page 82: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/82.jpg)
Outline
1 Stationarity and differencing
2 Non-seasonal ARIMA models
3 Estimation and order selection
4 ARIMA modelling in R
5 Forecasting
6 Seasonal ARIMA models
7 ARIMA vs ETS
57
![Page 83: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/83.jpg)
Maximum likelihood estimation
Having identified the model order, we need toestimate the parameters c, φ1, . . . , φp, θ1, . . . , θq.
MLE is very similar to least squares estimationobtained by minimizing
T∑t−1
e2t .
The Arima() command allows CLS or MLEestimation.Non-linear optimization must be used in eithercase.Different software will give different estimates.
58
![Page 84: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/84.jpg)
Maximum likelihood estimation
Having identified the model order, we need toestimate the parameters c, φ1, . . . , φp, θ1, . . . , θq.
MLE is very similar to least squares estimationobtained by minimizing
T∑t−1
e2t .
The Arima() command allows CLS or MLEestimation.Non-linear optimization must be used in eithercase.Different software will give different estimates. 58
![Page 85: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/85.jpg)
Partial autocorrelations
Partial autocorrelationsmeasure relationshipbetween yt and yt−k, when the effects of other timelags — 1, 2, 3, . . . , k− 1 — are removed.
αk = kth partial autocorrelation coefficient= equal to the estimate of bk in regression:
yt = c + φ1yt−1 + φ2yt−2 + · · · + φkyt−k.
Varying number of terms on RHS gives αk fordifferent values of k.There are more efficient ways of calculating αk.α1 = ρ1same critical values of±1.96/
√T as for ACF.
59
![Page 86: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/86.jpg)
Partial autocorrelations
Partial autocorrelationsmeasure relationshipbetween yt and yt−k, when the effects of other timelags — 1, 2, 3, . . . , k− 1 — are removed.
αk = kth partial autocorrelation coefficient= equal to the estimate of bk in regression:
yt = c + φ1yt−1 + φ2yt−2 + · · · + φkyt−k.
Varying number of terms on RHS gives αk fordifferent values of k.There are more efficient ways of calculating αk.α1 = ρ1same critical values of±1.96/
√T as for ACF.
59
![Page 87: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/87.jpg)
Partial autocorrelations
Partial autocorrelationsmeasure relationshipbetween yt and yt−k, when the effects of other timelags — 1, 2, 3, . . . , k− 1 — are removed.
αk = kth partial autocorrelation coefficient= equal to the estimate of bk in regression:
yt = c + φ1yt−1 + φ2yt−2 + · · · + φkyt−k.
Varying number of terms on RHS gives αk fordifferent values of k.There are more efficient ways of calculating αk.α1 = ρ1same critical values of±1.96/
√T as for ACF.
59
![Page 88: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/88.jpg)
Example: US consumption
−2
0
2
1970 1980 1990 2000 2010
Year
Qua
rter
ly p
erce
ntag
e ch
ange
US consumption
60
![Page 89: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/89.jpg)
Example: US consumption
−0.1
0.0
0.1
0.2
0.3
4 8 12 16 20
Lag
AC
F
−0.1
0.0
0.1
0.2
0.3
4 8 12 16 20
Lag
PAC
F
61
![Page 90: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/90.jpg)
ACF and PACF interpretation
AR(1)
ρk = φk1 for k = 1, 2, . . . ;α1 = φ1 αk = 0 for k = 2, 3, . . . .
So we have an AR(1) model when
autocorrelations exponentially decaythere is a single significant partialautocorrelation.
62
![Page 91: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/91.jpg)
ACF and PACF interpretation
AR(p)
ACF dies out in an exponential or dampedsine-wave mannerPACF has all zero spikes beyond the pth spike
So we have an AR(p) model when
the ACF is exponentially decaying or sinusoidalthere is a significant spike at lag p in PACF, butnone beyond p
63
![Page 92: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/92.jpg)
ACF and PACF interpretation
MA(1)
ρ1 = θ1 ρk = 0 for k = 2, 3, . . . ;αk = −(−θ1)k
So we have an MA(1) model when
the PACF is exponentially decaying andthere is a single significant spike in ACF
64
![Page 93: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/93.jpg)
ACF and PACF interpretation
MA(q)
PACF dies out in an exponential or dampedsine-wave mannerACF has all zero spikes beyond the qth spike
So we have an MA(q) model when
the PACF is exponentially decaying or sinusoidalthere is a significant spike at lag q in ACF, butnone beyond q
65
![Page 94: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/94.jpg)
Example: Mink trapping
30000
60000
90000
1860 1880 1900
Year
Min
ks tr
appe
d (t
hous
ands
)
Annual number of minks trapped
66
![Page 95: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/95.jpg)
Example: Mink trapping
−0.4
−0.2
0.0
0.2
0.4
0.6
5 10 15
Lag
AC
F
−0.2
0.0
0.2
0.4
0.6
5 10 15
Lag
PAC
F
67
![Page 96: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/96.jpg)
Information criteria
Akaike’s Information Criterion (AIC):AIC = −2 log(L) + 2(p + q + k + 1),
where L is the likelihood of the data,k = 1 if c 6= 0 and k = 0 if c = 0.
Corrected AIC:AICc = AIC + 2(p+q+k+1)(p+q+k+2)
T−p−q−k−2 .
Bayesian Information Criterion:BIC = AIC + [log(T)− 2](p + q + k− 1).
Good models are obtained by minimizing either theAIC, AICc or BIC. Our preference is to use the AICc.
68
![Page 97: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/97.jpg)
Information criteria
Akaike’s Information Criterion (AIC):AIC = −2 log(L) + 2(p + q + k + 1),
where L is the likelihood of the data,k = 1 if c 6= 0 and k = 0 if c = 0.
Corrected AIC:AICc = AIC + 2(p+q+k+1)(p+q+k+2)
T−p−q−k−2 .
Bayesian Information Criterion:BIC = AIC + [log(T)− 2](p + q + k− 1).
Good models are obtained by minimizing either theAIC, AICc or BIC. Our preference is to use the AICc.
68
![Page 98: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/98.jpg)
Information criteria
Akaike’s Information Criterion (AIC):AIC = −2 log(L) + 2(p + q + k + 1),
where L is the likelihood of the data,k = 1 if c 6= 0 and k = 0 if c = 0.
Corrected AIC:AICc = AIC + 2(p+q+k+1)(p+q+k+2)
T−p−q−k−2 .
Bayesian Information Criterion:BIC = AIC + [log(T)− 2](p + q + k− 1).
Good models are obtained by minimizing either theAIC, AICc or BIC. Our preference is to use the AICc.
68
![Page 99: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/99.jpg)
Information criteria
Akaike’s Information Criterion (AIC):AIC = −2 log(L) + 2(p + q + k + 1),
where L is the likelihood of the data,k = 1 if c 6= 0 and k = 0 if c = 0.
Corrected AIC:AICc = AIC + 2(p+q+k+1)(p+q+k+2)
T−p−q−k−2 .
Bayesian Information Criterion:BIC = AIC + [log(T)− 2](p + q + k− 1).
Good models are obtained by minimizing either theAIC, AICc or BIC. Our preference is to use the AICc. 68
![Page 100: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/100.jpg)
Outline
1 Stationarity and differencing
2 Non-seasonal ARIMA models
3 Estimation and order selection
4 ARIMA modelling in R
5 Forecasting
6 Seasonal ARIMA models
7 ARIMA vs ETS
69
![Page 101: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/101.jpg)
How does auto.arima() work?
A non-seasonal ARIMA process
φ(B)(1− B)dyt = c + θ(B)εtNeed to select appropriate orders: p, q, d
Hyndman and Khandakar (JSS, 2008) algorithm:
Select no. differences d and D via KPSS test andseasonal strength measure.Select p, q by minimising AICc.Use stepwise search to traverse model space.
70
![Page 102: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/102.jpg)
How does auto.arima() work?
AICc = −2 log(L) + 2(p + q + k + 1)[1 + (p+q+k+2)
T−p−q−k−2
].
where L is the maximised likelihood fitted to the differenced data,k = 1 if c 6= 0 and k = 0 otherwise.
Step1: Select current model (with smallest AICc) from:ARIMA(2, d, 2)ARIMA(0, d, 0)ARIMA(1, d, 0)ARIMA(0, d, 1)
Step 2: Consider variations of current model:vary one of p, q, from current model by±1;p, q both vary from current model by±1;Include/exclude c from current model.
Model with lowest AICc becomes current model.Repeat Step 2 until no lower AICc can be found.
71
![Page 103: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/103.jpg)
How does auto.arima() work?
AICc = −2 log(L) + 2(p + q + k + 1)[1 + (p+q+k+2)
T−p−q−k−2
].
where L is the maximised likelihood fitted to the differenced data,k = 1 if c 6= 0 and k = 0 otherwise.
Step1: Select current model (with smallest AICc) from:ARIMA(2, d, 2)ARIMA(0, d, 0)ARIMA(1, d, 0)ARIMA(0, d, 1)
Step 2: Consider variations of current model:vary one of p, q, from current model by±1;p, q both vary from current model by±1;Include/exclude c from current model.
Model with lowest AICc becomes current model.Repeat Step 2 until no lower AICc can be found.
71
![Page 104: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/104.jpg)
How does auto.arima() work?
AICc = −2 log(L) + 2(p + q + k + 1)[1 + (p+q+k+2)
T−p−q−k−2
].
where L is the maximised likelihood fitted to the differenced data,k = 1 if c 6= 0 and k = 0 otherwise.
Step1: Select current model (with smallest AICc) from:ARIMA(2, d, 2)ARIMA(0, d, 0)ARIMA(1, d, 0)ARIMA(0, d, 1)
Step 2: Consider variations of current model:vary one of p, q, from current model by±1;p, q both vary from current model by±1;Include/exclude c from current model.
Model with lowest AICc becomes current model.Repeat Step 2 until no lower AICc can be found.
71
![Page 105: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/105.jpg)
Choosing your own model
ggtsdisplay(internet)
80
120
160
200
0 20 40 60 80 100
internet
−0.5
0.0
0.5
1.0
5 10 15 20
Lag
AC
F
−0.5
0.0
0.5
1.0
5 10 15 20
Lag
PAC
F
72
![Page 106: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/106.jpg)
Choosing your own model
ggtsdisplay(diff(internet))
−15
−10
−5
0
5
10
15
0 20 40 60 80 100
diff(internet)
−0.4
0.0
0.4
0.8
5 10 15 20
Lag
AC
F
−0.4
0.0
0.4
0.8
5 10 15 20
Lag
PAC
F
73
![Page 107: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/107.jpg)
Choosing your own model
(fit <- Arima(internet,order=c(3,1,0)))
## Series: internet## ARIMA(3,1,0)#### Coefficients:## ar1 ar2 ar3## 1.1513 -0.6612 0.3407## s.e. 0.0950 0.1353 0.0941#### sigma^2 estimated as 9.656: log likelihood=-252## AIC=511.99 AICc=512.42 BIC=522.37 74
![Page 108: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/108.jpg)
Choosing your own model
auto.arima(internet)
## Series: internet## ARIMA(1,1,1)#### Coefficients:## ar1 ma1## 0.6504 0.5256## s.e. 0.0842 0.0896#### sigma^2 estimated as 9.995: log likelihood=-254.15## AIC=514.3 AICc=514.55 BIC=522.08 75
![Page 109: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/109.jpg)
Choosing your own model
auto.arima(internet, stepwise=FALSE,approximation=FALSE)
## Series: internet## ARIMA(3,1,0)#### Coefficients:## ar1 ar2 ar3## 1.1513 -0.6612 0.3407## s.e. 0.0950 0.1353 0.0941#### sigma^2 estimated as 9.656: log likelihood=-252## AIC=511.99 AICc=512.42 BIC=522.37
76
![Page 110: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/110.jpg)
Choosing your own model
checkresiduals(fit)
−5
0
5
0 20 40 60 80 100
Residuals from ARIMA(3,1,0)
−0.2
−0.1
0.0
0.1
0.2
5 10 15 20
Lag
AC
F
0
5
10
15
20
−10 −5 0 5 10
residuals
coun
t
77
![Page 111: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/111.jpg)
Choosing your own model
##
## Ljung-Box test
##
## data: Residuals from ARIMA(3,1,0)
## Q* = 4.4913, df = 7, p-value = 0.7218
##
## Model df: 3. Total lags used: 10
78
![Page 112: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/112.jpg)
Choosing your own model
fit %>% forecast %>% autoplot
100
150
200
250
0 30 60 90
Time
inte
rnet
level
80
95
Forecasts from ARIMA(3,1,0)
79
![Page 113: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/113.jpg)
Modelling procedure with Arima
1 Plot the data. Identify any unusual observations.2 If necessary, transform the data (using a Box-Cox
transformation) to stabilize the variance.3 If the data are non-stationary: take first differences of the
data until the data are stationary.4 Examine the ACF/PACF: Is an AR(p) or MA(q) model
appropriate?5 Try your chosen model(s), and use the AICc to search for a
better model.6 Check the residuals from your chosen model by plotting
the ACF of the residuals, and doing a portmanteau test ofthe residuals. If they do not look like white noise, try amodified model.
7 Once the residuals look like white noise, calculateforecasts.
80
![Page 114: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/114.jpg)
Modelling procedure with auto.arima
1 Plot the data. Identify any unusual observations.2 If necessary, transform the data (using a Box-Cox
transformation) to stabilize the variance.
3 Use auto.arima to select a model.
6 Check the residuals from your chosen model by plottingthe ACF of the residuals, and doing a portmanteau test ofthe residuals. If they do not look like white noise, try amodified model.
7 Once the residuals look like white noise, calculateforecasts.
81
![Page 115: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/115.jpg)
Modelling procedure8/ arima models 177
1. Plot the data. Identifyunusual observations.Understand patterns.
2. If necessary, use a Box-Cox transformation tostabilize the variance.
Select modelorder yourself.
Use automatedalgorithm.
3. If necessary, differencethe data until it appearsstationary. Use unit-roottests if you are unsure.
4. Plot the ACF/PACF ofthe differenced data and
try to determine pos-sible candidate models.
5. Try your chosen model(s)and use the AICc to
search for a better model.
6. Check the residualsfrom your chosen model
by plotting the ACF of theresiduals, and doing a port-
manteau test of the residuals.
Use auto.arima() to findthe best ARIMA model
for your time series.
Do theresidualslook like
whitenoise?
7. Calculate forecasts.
yes
no
Figure 8.10: General process for fore-casting using an ARIMA model.
82
![Page 116: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/116.jpg)
Seasonally adjusted electrical equipment
eeadj <- seasadj(stl(elecequip, s.window="periodic"))
autoplot(eeadj) + xlab("Year") +
ylab("Seasonally adjusted new orders index")
80
90
100
110
2000 2005 2010
Year
Sea
sona
lly a
djus
ted
new
ord
ers
inde
x
83
![Page 117: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/117.jpg)
Seasonally adjusted electrical equipment
1 Time plot shows sudden changes, particularly bigdrop in 2008/2009 due to global economicenvironment. Otherwise nothing unusual and noneed for data adjustments.
2 No evidence of changing variance, so no Box-Coxtransformation.
3 Data are clearly non-stationary, so we take firstdifferences.
84
![Page 118: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/118.jpg)
Seasonally adjusted electrical equipment
ggtsdisplay(diff(eeadj))
−10
−5
0
5
10
2000 2005 2010
diff(eeadj)
−0.2
0.0
0.2
12 24 36
Lag
AC
F
−0.2
0.0
0.2
12 24 36
Lag
PAC
F
85
![Page 119: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/119.jpg)
Seasonally adjusted electrical equipment
4 PACF is suggestive of AR(3). So initial candidatemodel is ARIMA(3,1,0). No other obviouscandidates.
5 Fit ARIMA(3,1,0) model along with variations:ARIMA(4,1,0), ARIMA(2,1,0), ARIMA(3,1,1), etc.ARIMA(3,1,1) has smallest AICc value.
86
![Page 120: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/120.jpg)
Seasonally adjusted electrical equipment
(fit <- Arima(eeadj, order=c(3,1,1)))
## Series: eeadj## ARIMA(3,1,1)#### Coefficients:## ar1 ar2 ar3 ma1## 0.0044 0.0916 0.3698 -0.3921## s.e. 0.2201 0.0984 0.0669 0.2426#### sigma^2 estimated as 9.577: log likelihood=-492.69## AIC=995.38 AICc=995.7 BIC=1011.72
87
![Page 121: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/121.jpg)
Seasonally adjusted electrical equipment
6 ACF plot of residuals from ARIMA(3,1,1) modellook like white noise.
checkresiduals(fit)
−10
−5
0
5
10
2000 2005 2010
Residuals from ARIMA(3,1,1)
−0.15
−0.10
−0.05
0.00
0.05
0.10
0.15
12 24 36
Lag
AC
F
0
10
20
30
−10 −5 0 5 10
residuals
coun
t
#### Ljung-Box test#### data: Residuals from ARIMA(3,1,1)## Q* = 24.034, df = 20, p-value = 0.2409#### Model df: 4. Total lags used: 24
88
![Page 122: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/122.jpg)
Seasonally adjusted electrical equipment
##
## Ljung-Box test
##
## data: Residuals from ARIMA(3,1,1)
## Q* = 24.034, df = 20, p-value = 0.2409
##
## Model df: 4. Total lags used: 24
89
![Page 123: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/123.jpg)
Seasonally adjusted electrical equipment
fit %>% forecast %>% autoplot
60
80
100
120
2000 2005 2010 2015
Time
eead
j level
80
95
Forecasts from ARIMA(3,1,1)
90
![Page 124: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/124.jpg)
Outline
1 Stationarity and differencing
2 Non-seasonal ARIMA models
3 Estimation and order selection
4 ARIMA modelling in R
5 Forecasting
6 Seasonal ARIMA models
7 ARIMA vs ETS
91
![Page 125: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/125.jpg)
Point forecasts
1 Rearrange ARIMA equation so yt is on LHS.2 Rewrite equation by replacing t by T + h.3 On RHS, replace future observations by their
forecasts, future errors by zero, and past errorsby corresponding residuals.
Start with h = 1. Repeat for h = 2, 3, . . ..
92
![Page 126: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/126.jpg)
Point forecasts
ARIMA(3,1,1) forecasts: Step 1
(1− φ1B− φ2B2 − φ3B3)(1− B)yt = (1 + θ1B)εt,
[1− (1 + φ1)B + (φ1 − φ2)B2 + (φ2 − φ3)B3 + φ3B4
]yt
= (1 + θ1B)εt,
yt − (1 + φ1)yt−1 + (φ1 − φ2)yt−2 + (φ2 − φ3)yt−3+ φ3yt−4 = εt + θ1εt−1.
yt = (1 + φ1)yt−1 − (φ1 − φ2)yt−2 − (φ2 − φ3)yt−3− φ3yt−4 + εt + θ1εt−1.
93
![Page 127: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/127.jpg)
Point forecasts
ARIMA(3,1,1) forecasts: Step 1
(1− φ1B− φ2B2 − φ3B3)(1− B)yt = (1 + θ1B)εt,[1− (1 + φ1)B + (φ1 − φ2)B2 + (φ2 − φ3)B3 + φ3B4
]yt
= (1 + θ1B)εt,
yt − (1 + φ1)yt−1 + (φ1 − φ2)yt−2 + (φ2 − φ3)yt−3+ φ3yt−4 = εt + θ1εt−1.
yt = (1 + φ1)yt−1 − (φ1 − φ2)yt−2 − (φ2 − φ3)yt−3− φ3yt−4 + εt + θ1εt−1.
93
![Page 128: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/128.jpg)
Point forecasts
ARIMA(3,1,1) forecasts: Step 1
(1− φ1B− φ2B2 − φ3B3)(1− B)yt = (1 + θ1B)εt,[1− (1 + φ1)B + (φ1 − φ2)B2 + (φ2 − φ3)B3 + φ3B4
]yt
= (1 + θ1B)εt,
yt − (1 + φ1)yt−1 + (φ1 − φ2)yt−2 + (φ2 − φ3)yt−3+ φ3yt−4 = εt + θ1εt−1.
yt = (1 + φ1)yt−1 − (φ1 − φ2)yt−2 − (φ2 − φ3)yt−3− φ3yt−4 + εt + θ1εt−1.
93
![Page 129: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/129.jpg)
Point forecasts
ARIMA(3,1,1) forecasts: Step 1
(1− φ1B− φ2B2 − φ3B3)(1− B)yt = (1 + θ1B)εt,[1− (1 + φ1)B + (φ1 − φ2)B2 + (φ2 − φ3)B3 + φ3B4
]yt
= (1 + θ1B)εt,
yt − (1 + φ1)yt−1 + (φ1 − φ2)yt−2 + (φ2 − φ3)yt−3+ φ3yt−4 = εt + θ1εt−1.
yt = (1 + φ1)yt−1 − (φ1 − φ2)yt−2 − (φ2 − φ3)yt−3− φ3yt−4 + εt + θ1εt−1.
93
![Page 130: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/130.jpg)
Point forecasts (h=1)
yt = (1 + φ1)yt−1 − (φ1 − φ2)yt−2 − (φ2 − φ3)yt−3− φ3yt−4 + εt + θ1εt−1.
ARIMA(3,1,1) forecasts: Step 2
yT+1 = (1 + φ1)yT − (φ1 − φ2)yT−1 − (φ2 − φ3)yT−2− φ3yT−3 + εT+1 + θ1εT.
ARIMA(3,1,1) forecasts: Step 3
yT+1|T = (1 + φ1)yT − (φ1 − φ2)yT−1 − (φ2 − φ3)yT−2− φ3yT−3 + θ1eT.
94
![Page 131: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/131.jpg)
Point forecasts (h=1)
yt = (1 + φ1)yt−1 − (φ1 − φ2)yt−2 − (φ2 − φ3)yt−3− φ3yt−4 + εt + θ1εt−1.
ARIMA(3,1,1) forecasts: Step 2
yT+1 = (1 + φ1)yT − (φ1 − φ2)yT−1 − (φ2 − φ3)yT−2− φ3yT−3 + εT+1 + θ1εT.
ARIMA(3,1,1) forecasts: Step 3
yT+1|T = (1 + φ1)yT − (φ1 − φ2)yT−1 − (φ2 − φ3)yT−2− φ3yT−3 + θ1eT.
94
![Page 132: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/132.jpg)
Point forecasts (h=1)
yt = (1 + φ1)yt−1 − (φ1 − φ2)yt−2 − (φ2 − φ3)yt−3− φ3yt−4 + εt + θ1εt−1.
ARIMA(3,1,1) forecasts: Step 2
yT+1 = (1 + φ1)yT − (φ1 − φ2)yT−1 − (φ2 − φ3)yT−2− φ3yT−3 + εT+1 + θ1εT.
ARIMA(3,1,1) forecasts: Step 3
yT+1|T = (1 + φ1)yT − (φ1 − φ2)yT−1 − (φ2 − φ3)yT−2− φ3yT−3 + θ1eT.
94
![Page 133: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/133.jpg)
Point forecasts (h=2)
yt = (1 + φ1)yt−1 − (φ1 − φ2)yt−2 − (φ2 − φ3)yt−3− φ3yt−4 + εt + θ1εt−1.
ARIMA(3,1,1) forecasts: Step 2
yT+2 = (1 + φ1)yT+1 − (φ1 − φ2)yT − (φ2 − φ3)yT−1− φ3yT−2 + εT+2 + θ1εT+1.
ARIMA(3,1,1) forecasts: Step 3
yT+2|T = (1 + φ1)yT+1|T − (φ1 − φ2)yT − (φ2 − φ3)yT−1− φ3yT−2.
95
![Page 134: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/134.jpg)
Point forecasts (h=2)
yt = (1 + φ1)yt−1 − (φ1 − φ2)yt−2 − (φ2 − φ3)yt−3− φ3yt−4 + εt + θ1εt−1.
ARIMA(3,1,1) forecasts: Step 2
yT+2 = (1 + φ1)yT+1 − (φ1 − φ2)yT − (φ2 − φ3)yT−1− φ3yT−2 + εT+2 + θ1εT+1.
ARIMA(3,1,1) forecasts: Step 3
yT+2|T = (1 + φ1)yT+1|T − (φ1 − φ2)yT − (φ2 − φ3)yT−1− φ3yT−2.
95
![Page 135: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/135.jpg)
Point forecasts (h=2)
yt = (1 + φ1)yt−1 − (φ1 − φ2)yt−2 − (φ2 − φ3)yt−3− φ3yt−4 + εt + θ1εt−1.
ARIMA(3,1,1) forecasts: Step 2
yT+2 = (1 + φ1)yT+1 − (φ1 − φ2)yT − (φ2 − φ3)yT−1− φ3yT−2 + εT+2 + θ1εT+1.
ARIMA(3,1,1) forecasts: Step 3
yT+2|T = (1 + φ1)yT+1|T − (φ1 − φ2)yT − (φ2 − φ3)yT−1− φ3yT−2.
95
![Page 136: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/136.jpg)
Prediction intervals
95% prediction interval
yT+h|T ± 1.96√vT+h|T
where vT+h|T is estimated forecast variance.
vT+1|T = σ2 for all ARIMA models regardless ofparameters and orders.Multi-step prediction intervals for ARIMA(0,0,q):
yt = εt +q∑i=1θiεt−i.
vT|T+h = σ21 +
h−1∑i=1θ2i
, for h = 2, 3, . . . .
96
![Page 137: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/137.jpg)
Prediction intervals
95% prediction interval
yT+h|T ± 1.96√vT+h|T
where vT+h|T is estimated forecast variance.
vT+1|T = σ2 for all ARIMA models regardless ofparameters and orders.Multi-step prediction intervals for ARIMA(0,0,q):
yt = εt +q∑i=1θiεt−i.
vT|T+h = σ21 +
h−1∑i=1θ2i
, for h = 2, 3, . . . .96
![Page 138: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/138.jpg)
Prediction intervals
95% Prediction interval
yT+h|T ± 1.96√vT+h|T
where vT+h|T is estimated forecast variance.
Multi-step prediction intervals for ARIMA(0,0,q):
yt = εt +q∑i=1θiεt−i.
vT|T+h = σ21 +
h−1∑i=1θ2i
, for h = 2, 3, . . . .
AR(1): Rewrite as MA(∞) and use above result.Other models beyond scope of this subject.
97
![Page 139: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/139.jpg)
Prediction intervals
95% Prediction interval
yT+h|T ± 1.96√vT+h|T
where vT+h|T is estimated forecast variance.
Multi-step prediction intervals for ARIMA(0,0,q):
yt = εt +q∑i=1θiεt−i.
vT|T+h = σ21 +
h−1∑i=1θ2i
, for h = 2, 3, . . . .
AR(1): Rewrite as MA(∞) and use above result.Other models beyond scope of this subject.
97
![Page 140: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/140.jpg)
Prediction intervals
Prediction intervals increase in size withforecast horizon.Prediction intervals can be difficult to calculateby handCalculations assume residuals are uncorrelatedand normally distributed.Prediction intervals tend to be too narrow.
the uncertainty in the parameter estimates has notbeen accounted for.the ARIMA model assumes historical patterns willnot change during the forecast period.the ARIMA model assumes uncorrelated future errors 98
![Page 141: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/141.jpg)
Your turn
For the usgdp data:
if necessary, find a suitable Box-Coxtransformation for the data;fit a suitable ARIMA model to the transformeddata using auto.arima();check the residual diagnostics;produce forecasts of your fitted model. Do theforecasts look reasonable?
99
![Page 142: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/142.jpg)
Outline
1 Stationarity and differencing
2 Non-seasonal ARIMA models
3 Estimation and order selection
4 ARIMA modelling in R
5 Forecasting
6 Seasonal ARIMA models
7 ARIMA vs ETS
100
![Page 143: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/143.jpg)
Seasonal ARIMA models
ARIMA (p, d, q)︸ ︷︷ ︸ (P,D,Q)m︸ ︷︷ ︸↑ ↑
Non-seasonal part Seasonal part ofof the model of the model
wherem = number of observations per year.
101
![Page 144: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/144.jpg)
Seasonal ARIMA models
E.g., ARIMA(1, 1, 1)(1, 1, 1)4 model (without constant)
(1−φ1B)(1−Φ1B4)(1−B)(1−B4)yt = (1+θ1B)(1+Θ1B4)εt.
6 6 6 6 6 6(Non-seasonal
AR(1)
)(SeasonalAR(1)
)(Non-seasonaldifference
)(
Seasonaldifference
)(Non-seasonal
MA(1)
)(SeasonalMA(1)
)
102
![Page 145: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/145.jpg)
Seasonal ARIMA models
E.g., ARIMA(1, 1, 1)(1, 1, 1)4 model (without constant)(1−φ1B)(1−Φ1B4)(1−B)(1−B4)yt = (1+θ1B)(1+Θ1B4)εt.
6 6 6 6 6 6(Non-seasonal
AR(1)
)(SeasonalAR(1)
)(Non-seasonaldifference
)(
Seasonaldifference
)(Non-seasonal
MA(1)
)(SeasonalMA(1)
)
102
![Page 146: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/146.jpg)
Seasonal ARIMA models
E.g., ARIMA(1, 1, 1)(1, 1, 1)4 model (without constant)(1−φ1B)(1−Φ1B4)(1−B)(1−B4)yt = (1+θ1B)(1+Θ1B4)εt.
6 6 6 6 6 6(Non-seasonal
AR(1)
)(SeasonalAR(1)
)(Non-seasonaldifference
)(
Seasonaldifference
)(Non-seasonal
MA(1)
)(SeasonalMA(1)
)
102
![Page 147: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/147.jpg)
Seasonal ARIMA models
E.g., ARIMA(1, 1, 1)(1, 1, 1)4 model (without constant)(1−φ1B)(1−Φ1B4)(1−B)(1−B4)yt = (1+θ1B)(1+Θ1B4)εt.
All the factors can be multiplied out and the generalmodel written as follows:
yt = (1 + φ1)yt−1 − φ1yt−2 + (1 + Φ1)yt−4− (1 + φ1 + Φ1 + φ1Φ1)yt−5 + (φ1 + φ1Φ1)yt−6−Φ1yt−8 + (Φ1 + φ1Φ1)yt−9 − φ1Φ1yt−10+ εt + θ1εt−1 + Θ1εt−4 + θ1Θ1εt−5.
103
![Page 148: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/148.jpg)
Common ARIMA models
The US Census Bureau uses the following modelsmost often:
ARIMA(0,1,1)(0,1,1)m with log transformationARIMA(0,1,2)(0,1,1)m with log transformationARIMA(2,1,0)(0,1,1)m with log transformationARIMA(0,2,2)(0,1,1)m with log transformationARIMA(2,1,2)(0,1,1)m with no transformation
104
![Page 149: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/149.jpg)
Seasonal ARIMA models
The seasonal part of an AR or MA model will be seenin the seasonal lags of the PACF and ACF.ARIMA(0,0,0)(0,0,1)12 will show:
a spike at lag 12 in the ACF but no othersignificant spikes.The PACF will show exponential decay in theseasonal lags; that is, at lags 12, 24, 36, . . . .
ARIMA(0,0,0)(1,0,0)12 will show:exponential decay in the seasonal lags of the ACFa single significant spike at lag 12 in the PACF.
105
![Page 150: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/150.jpg)
European quarterly retail trade
autoplot(euretail) +xlab("Year") + ylab("Retail index")
92
96
100
2000 2005 2010
Year
Ret
ail i
ndex
106
![Page 151: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/151.jpg)
European quarterly retail trade
euretail %>% diff(lag=4) %>% ggtsdisplay()
−2
0
2
2000 2005 2010
.
0.0
0.5
4 8 12 16
Lag
AC
F
0.0
0.5
4 8 12 16
Lag
PAC
F
107
![Page 152: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/152.jpg)
European quarterly retail trade
euretail %>% diff(lag=4) %>% diff() %>%ggtsdisplay()
−2
−1
0
1
2000 2005 2010
.
−0.4
−0.2
0.0
0.2
4 8 12 16
Lag
AC
F
−0.4
−0.2
0.0
0.2
4 8 12 16
Lag
PAC
F
108
![Page 153: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/153.jpg)
European quarterly retail trade
d = 1 and D = 1 seems necessary.Significant spike at lag 1 in ACF suggestsnon-seasonal MA(1) component.Significant spike at lag 4 in ACF suggests seasonalMA(1) component.Initial candidate model: ARIMA(0,1,1)(0,1,1)4.We could also have started withARIMA(1,1,0)(1,1,0)4.
109
![Page 154: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/154.jpg)
European quarterly retail trade
fit <- Arima(euretail, order=c(0,1,1),seasonal=c(0,1,1))
checkresiduals(fit)
−1.0
−0.5
0.0
0.5
1.0
2000 2005 2010
Residuals from ARIMA(0,1,1)(0,1,1)[4]
−0.2
−0.1
0.0
0.1
0.2
0.3
4 8 12 16
Lag
AC
F
0
5
10
15
−1.0 −0.5 0.0 0.5 1.0
residuals
coun
t
##
## Ljung-Box test
##
## data: Residuals from ARIMA(0,1,1)(0,1,1)[4]
## Q* = 10.654, df = 6, p-value = 0.09968
##
## Model df: 2. Total lags used: 8
110
![Page 155: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/155.jpg)
European quarterly retail trade
#### Ljung-Box test#### data: Residuals from ARIMA(0,1,1)(0,1,1)[4]## Q* = 10.654, df = 6, p-value = 0.09968#### Model df: 2. Total lags used: 8
111
![Page 156: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/156.jpg)
European quarterly retail trade
ACF and PACF of residuals show significant spikesat lag 2, and maybe lag 3.AICc of ARIMA(0,1,2)(0,1,1)4 model is 74.27.AICc of ARIMA(0,1,3)(0,1,1)4 model is 68.39.
fit <- Arima(euretail, order=c(0,1,3),seasonal=c(0,1,1))
checkresiduals(fit)
112
![Page 157: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/157.jpg)
European quarterly retail trade
ACF and PACF of residuals show significant spikesat lag 2, and maybe lag 3.AICc of ARIMA(0,1,2)(0,1,1)4 model is 74.27.AICc of ARIMA(0,1,3)(0,1,1)4 model is 68.39.
fit <- Arima(euretail, order=c(0,1,3),seasonal=c(0,1,1))
checkresiduals(fit)
112
![Page 158: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/158.jpg)
European quarterly retail trade
## Series: euretail
## ARIMA(0,1,3)(0,1,1)[4]
##
## Coefficients:
## ma1 ma2 ma3 sma1
## 0.2630 0.3694 0.4200 -0.6636
## s.e. 0.1237 0.1255 0.1294 0.1545
##
## sigma^2 estimated as 0.156: log likelihood=-28.63
## AIC=67.26 AICc=68.39 BIC=77.65
113
![Page 159: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/159.jpg)
European quarterly retail trade
checkresiduals(fit)
−1.0
−0.5
0.0
0.5
2000 2005 2010
Residuals from ARIMA(0,1,3)(0,1,1)[4]
−0.2
−0.1
0.0
0.1
0.2
4 8 12 16
Lag
AC
F
0
5
10
15
−1.0 −0.5 0.0 0.5 1.0
residuals
coun
t
#### Ljung-Box test#### data: Residuals from ARIMA(0,1,3)(0,1,1)[4]## Q* = 0.51128, df = 4, p-value = 0.9724#### Model df: 4. Total lags used: 8
114
![Page 160: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/160.jpg)
European quarterly retail trade
#### Ljung-Box test#### data: Residuals from ARIMA(0,1,3)(0,1,1)[4]## Q* = 0.51128, df = 4, p-value = 0.9724#### Model df: 4. Total lags used: 8
115
![Page 161: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/161.jpg)
European quarterly retail trade
autoplot(forecast(fit, h=12))
90
95
100
2000 2005 2010 2015
Time
eure
tail level
80
95
Forecasts from ARIMA(0,1,3)(0,1,1)[4]
116
![Page 162: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/162.jpg)
European quarterly retail trade
auto.arima(euretail)
## Series: euretail## ARIMA(1,1,2)(0,1,1)[4]#### Coefficients:## ar1 ma1 ma2 sma1## 0.7362 -0.4663 0.2163 -0.8433## s.e. 0.2243 0.1990 0.2101 0.1876#### sigma^2 estimated as 0.1587: log likelihood=-29.62## AIC=69.24 AICc=70.38 BIC=79.63
117
![Page 163: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/163.jpg)
European quarterly retail trade
auto.arima(euretail,stepwise=FALSE, approximation=FALSE)
## Series: euretail## ARIMA(0,1,3)(0,1,1)[4]#### Coefficients:## ma1 ma2 ma3 sma1## 0.2630 0.3694 0.4200 -0.6636## s.e. 0.1237 0.1255 0.1294 0.1545#### sigma^2 estimated as 0.156: log likelihood=-28.63## AIC=67.26 AICc=68.39 BIC=77.65 118
![Page 164: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/164.jpg)
Cortecosteroid drug salesH
02 sales (million scripts)
Log H02 sales
1995 2000 2005
0.50
0.75
1.00
1.25
−0.8
−0.4
0.0
Year
119
![Page 165: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/165.jpg)
Cortecosteroid drug sales
0.0
0.2
0.4
1995 2000 2005
Year
Seasonally differenced H02 scripts
−0.2
0.0
0.2
0.4
12 24 36
Lag
AC
F
−0.2
0.0
0.2
0.4
12 24 36
Lag
PAC
F
120
![Page 166: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/166.jpg)
Cortecosteroid drug sales
Choose D = 1 and d = 0.Spikes in PACF at lags 12 and 24 suggest seasonalAR(2) term.Spikes in PACF suggests possible non-seasonalAR(3) term.Initial candidate model: ARIMA(3,0,0)(2,1,0)12.
121
![Page 167: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/167.jpg)
Cortecosteroid drug sales
Model AICc
ARIMA(3,0,1)(0,1,2)12 -485.48ARIMA(3,0,1)(1,1,1)12 -484.25ARIMA(3,0,1)(0,1,1)12 -483.67ARIMA(3,0,1)(2,1,0)12 -476.31ARIMA(3,0,0)(2,1,0)12 -475.12ARIMA(3,0,2)(2,1,0)12 -474.88ARIMA(3,0,1)(1,1,0)12 -463.40
122
![Page 168: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/168.jpg)
Cortecosteroid drug sales
(fit <- Arima(h02, order=c(3,0,1), seasonal=c(0,1,2),lambda=0))
## Series: h02## ARIMA(3,0,1)(0,1,2)[12]## Box Cox transformation: lambda= 0#### Coefficients:## ar1 ar2 ar3 ma1 sma1 sma2## -0.1603 0.5481 0.5678 0.3827 -0.5222 -0.1768## s.e. 0.1636 0.0878 0.0942 0.1895 0.0861 0.0872#### sigma^2 estimated as 0.004278: log likelihood=250.04## AIC=-486.08 AICc=-485.48 BIC=-463.28
123
![Page 169: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/169.jpg)
Cortecosteroid drug sales
checkresiduals(fit, lag=36)
−0.2
−0.1
0.0
0.1
0.2
1995 2000 2005
Residuals from ARIMA(3,0,1)(0,1,2)[12]
−0.1
0.0
0.1
12 24 36
Lag
AC
F
0
10
20
30
−0.2 −0.1 0.0 0.1 0.2
residuals
coun
t
##
## Ljung-Box test
##
## data: Residuals from ARIMA(3,0,1)(0,1,2)[12]
## Q* = 50.712, df = 30, p-value = 0.01045
##
## Model df: 6. Total lags used: 36
124
![Page 170: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/170.jpg)
Cortecosteroid drug sales
##
## Ljung-Box test
##
## data: Residuals from ARIMA(3,0,1)(0,1,2)[12]
## Q* = 50.712, df = 30, p-value = 0.01045
##
## Model df: 6. Total lags used: 36
125
![Page 171: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/171.jpg)
Cortecosteroid drug sales
(fit <- auto.arima(h02, lambda=0))
## Series: h02
## ARIMA(2,1,3)(0,1,1)[12]
## Box Cox transformation: lambda= 0
##
## Coefficients:
## ar1 ar2 ma1 ma2 ma3 sma1
## -1.0194 -0.8351 0.1717 0.2578 -0.4206 -0.6528
## s.e. 0.1648 0.1203 0.2079 0.1177 0.1060 0.0657
##
## sigma^2 estimated as 0.004203: log likelihood=250.8
## AIC=-487.6 AICc=-486.99 BIC=-464.83
126
![Page 172: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/172.jpg)
Cortecosteroid drug sales
checkresiduals(fit, lag=36)
−0.2
−0.1
0.0
0.1
0.2
1995 2000 2005
Residuals from ARIMA(2,1,3)(0,1,1)[12]
−0.1
0.0
0.1
12 24 36
Lag
AC
F
0
10
20
30
40
−0.2 −0.1 0.0 0.1 0.2
residuals
coun
t
##
## Ljung-Box test
##
## data: Residuals from ARIMA(2,1,3)(0,1,1)[12]
## Q* = 46.149, df = 30, p-value = 0.03007
##
## Model df: 6. Total lags used: 36
127
![Page 173: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/173.jpg)
Cortecosteroid drug sales
##
## Ljung-Box test
##
## data: Residuals from ARIMA(2,1,3)(0,1,1)[12]
## Q* = 46.149, df = 30, p-value = 0.03007
##
## Model df: 6. Total lags used: 36
128
![Page 174: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/174.jpg)
Cortecosteroid drug sales
(fit <- auto.arima(h02, lambda=0, max.order=9,
stepwise=FALSE, approximation=FALSE))
## Series: h02
## ARIMA(4,1,1)(2,1,2)[12]
## Box Cox transformation: lambda= 0
##
## Coefficients:
## ar1 ar2 ar3 ar4 ma1 sar1 sar2 sma1
## -0.0425 0.2098 0.2017 -0.2273 -0.7424 0.6213 -0.3832 -1.2019
## s.e. 0.2167 0.1813 0.1144 0.0810 0.2074 0.2421 0.1185 0.2491
## sma2
## 0.4959
## s.e. 0.2135
##
## sigma^2 estimated as 0.004049: log likelihood=254.31
## AIC=-488.63 AICc=-487.4 BIC=-456.1
129
![Page 175: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/175.jpg)
Cortecosteroid drug sales
checkresiduals(fit, lag=36)
−0.2
−0.1
0.0
0.1
1995 2000 2005
Residuals from ARIMA(4,1,1)(2,1,2)[12]
−0.1
0.0
0.1
12 24 36
Lag
AC
F
0
10
20
30
−0.2 −0.1 0.0 0.1 0.2
residuals
coun
t
##
## Ljung-Box test
##
## data: Residuals from ARIMA(4,1,1)(2,1,2)[12]
## Q* = 36.456, df = 27, p-value = 0.1057
##
## Model df: 9. Total lags used: 36
130
![Page 176: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/176.jpg)
Cortecosteroid drug sales
##
## Ljung-Box test
##
## data: Residuals from ARIMA(4,1,1)(2,1,2)[12]
## Q* = 36.456, df = 27, p-value = 0.1057
##
## Model df: 9. Total lags used: 36
131
![Page 177: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/177.jpg)
Cortecosteroid drug sales
Training data: July 1991 to June 2006Test data: July 2006–June 2008
getrmse <- function(x,h,...){
train.end <- time(x)[length(x)-h]test.start <- time(x)[length(x)-h+1]train <- window(x,end=train.end)test <- window(x,start=test.start)fit <- Arima(train,...)fc <- forecast(fit,h=h)return(accuracy(fc,test)[2,"RMSE"])
}
getrmse(h02,h=24,order=c(3,0,0),seasonal=c(2,1,0),lambda=0)getrmse(h02,h=24,order=c(3,0,1),seasonal=c(2,1,0),lambda=0)getrmse(h02,h=24,order=c(3,0,2),seasonal=c(2,1,0),lambda=0)getrmse(h02,h=24,order=c(3,0,1),seasonal=c(1,1,0),lambda=0)getrmse(h02,h=24,order=c(3,0,1),seasonal=c(0,1,1),lambda=0)getrmse(h02,h=24,order=c(3,0,1),seasonal=c(0,1,2),lambda=0)getrmse(h02,h=24,order=c(3,0,1),seasonal=c(1,1,1),lambda=0)getrmse(h02,h=24,order=c(3,0,3),seasonal=c(0,1,1),lambda=0)getrmse(h02,h=24,order=c(3,0,2),seasonal=c(0,1,1),lambda=0)getrmse(h02,h=24,order=c(2,1,3),seasonal=c(0,1,1),lambda=0)getrmse(h02,h=24,order=c(2,1,4),seasonal=c(0,1,1),lambda=0)getrmse(h02,h=24,order=c(2,1,5),seasonal=c(0,1,1),lambda=0)getrmse(h02,h=24,order=c(4,1,1),seasonal=c(2,1,2),lambda=0)
132
![Page 178: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/178.jpg)
Cortecosteroid drug sales
Model RMSE
ARIMA(4,1,1)(2,1,2)[12] 0.0615ARIMA(3,0,1)(0,1,2)[12] 0.0622ARIMA(3,0,1)(1,1,1)[12] 0.0630ARIMA(2,1,4)(0,1,1)[12] 0.0632ARIMA(2,1,3)(0,1,1)[12] 0.0634ARIMA(3,0,3)(0,1,1)[12] 0.0638ARIMA(2,1,5)(0,1,1)[12] 0.0640ARIMA(3,0,1)(0,1,1)[12] 0.0644ARIMA(3,0,2)(0,1,1)[12] 0.0644ARIMA(3,0,2)(2,1,0)[12] 0.0645ARIMA(3,0,1)(2,1,0)[12] 0.0646ARIMA(3,0,0)(2,1,0)[12] 0.0661ARIMA(3,0,1)(1,1,0)[12] 0.0679
133
![Page 179: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/179.jpg)
Cortecosteroid drug sales
Models with lowest AICc values tend to giveslightly better results than the other models.AICc comparisons must have the same orders ofdifferencing. But RMSE test set comparisons caninvolve any models.Use the best model available, even if it does notpass all tests.
134
![Page 180: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/180.jpg)
Cortecosteroid drug sales
fit <- Arima(h02, order=c(3,0,1), seasonal=c(0,1,2),
lambda=0)
autoplot(forecast(fit)) +
ylab("H02 sales (million scripts)") + xlab("Year")
0.5
1.0
1.5
1995 2000 2005 2010
Year
H02
sal
es (
mill
ion
scrip
ts)
level
80
95
Forecasts from ARIMA(3,0,1)(0,1,2)[12]
135
![Page 181: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/181.jpg)
Outline
1 Stationarity and differencing
2 Non-seasonal ARIMA models
3 Estimation and order selection
4 ARIMA modelling in R
5 Forecasting
6 Seasonal ARIMA models
7 ARIMA vs ETS
136
![Page 182: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/182.jpg)
ARIMA vs ETS
Myth that ARIMA models are more general thanexponential smoothing.Linear exponential smoothing models all specialcases of ARIMA models.Non-linear exponential smoothing models haveno equivalent ARIMA counterparts.Many ARIMA models have no exponentialsmoothing counterparts.ETS models all non-stationary. Models withseasonality or non-damped trend (or both) havetwo unit roots; all other models have one unit root.137
![Page 183: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/183.jpg)
Equivalences
ETS model ARIMA model Parameters
ETS(A,N,N) ARIMA(0,1,1) θ1 = α− 1ETS(A,A,N) ARIMA(0,2,2) θ1 = α + β − 2
θ2 = 1− αETS(A,A,N) ARIMA(1,1,2) φ1 = φ
θ1 = α + φβ − 1− φθ2 = (1− α)φ
ETS(A,N,A) ARIMA(0,0,m)(0,1,0)mETS(A,A,A) ARIMA(0,1,m + 1)(0,1,0)mETS(A,A,A) ARIMA(1,0,m + 1)(0,1,0)m
138
![Page 184: ECON 427: ECONOMIC FORECASTING - University …fuleky/econ427/8-arima.pdfOutline 1 Stationarity and di˙erencing 2 Non-seasonal ARIMA models 3 Estimation and order selection 4 ARIMA](https://reader030.vdocuments.us/reader030/viewer/2022040902/5e74ba8f1d65ed39112dc895/html5/thumbnails/184.jpg)
Your turn
For the condmilk series:Do the data need transforming? If so, find a suitabletransformation.Are the data stationary? If not, find an appropriatedifferencing which yields stationary data.Identify a couple of ARIMA models that might beuseful in describing the time series.Which of your models is the best according to theirAIC values?Estimate the parameters of your best model and dodiagnostic testing on the residuals. Do the residualsresemble white noise? If not, try to find anotherARIMA model which fits better.Forecast the next 24 months of data using yourpreferred model.Compare the forecasts obtained using ets().
139