econometric forecasting
TRANSCRIPT
-
7/28/2019 Econometric Forecasting
1/86
Introduction Model-free extrapolation Univariate time-series models
Econometric Forecasting
Robert M. [email protected]
University of Viennaand
Institute for Advanced Studies Vienna
November 10, 2012
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
http://find/ -
7/28/2019 Econometric Forecasting
2/86
Introduction Model-free extrapolation Univariate time-series models
Outline
Introduction
Model-free extrapolation
Univariate time-series models
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
http://find/ -
7/28/2019 Econometric Forecasting
3/86
Introduction Model-free extrapolation Univariate time-series models
The basic problem
Given some data (observations) on a (possibly multivariate)variable x , i.e. x 1, . . . , x N , we want to nd a good approximationto the (as yet unknown) observation x N + h . We use Chatfield snotation: x N (h) is a hstep forecast for x N + h given observations (atime series) until and including x N .
The information set available at t = N for the forecast necessarily
includes the observed time series but it may be much larger inpractice.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
http://find/ -
7/28/2019 Econometric Forecasting
4/86
Introduction Model-free extrapolation Univariate time-series models
Forecasting and predictingTo many authors, forecasting and prediction are equivalent. Someauthors distinguish the terms: prediction is the technical word,forecasting relates predictions to the substance-matterenvironment.
Clements and Hendry dene: predictability is a theoreticalpropertyunconditional and conditional distributions differ,forecastability is the possibility that this property can be exploitedin practice.
The words prognosis and projection are related but their usage ismore restricted.
The participle forecasted is incorrect but ubiquitous in theliterature.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
http://find/http://goback/ -
7/28/2019 Econometric Forecasting
5/86
Introduction Model-free extrapolation Univariate time-series models
Aims of forecasting
1. Curiosity: we want to know about the future;2. Decision making: may specify a loss function for prediction,
but fully developed examples are rare;3. Policy evaluation: modied policy may change expectations
and thus model behavior (Lucas critique);4. Model evaluation: quality of models as descriptive tools and
of model-based predictions may differ.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
http://find/ -
7/28/2019 Econometric Forecasting
6/86
Introduction Model-free extrapolation Univariate time-series models
Forecasts: classication according to objectivity
1. magical : no recognizable cause-effect relationship (oracles);2. subjective : judgmental forecasts, Delphi method (averaging
over questionnaires);3. objective : univariate (one time series only), multivariate
(several variables summarized in a vector).
In practice, most institutional forecasts use a mix of subjective andobjective elements.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
http://find/http://goback/ -
7/28/2019 Econometric Forecasting
7/86
Introduction Model-free extrapolation Univariate time-series models
Forecasts: classication according to being automatic
1. automatic methods: forecasts based on a well-dened rule
without any further user intervention;2. non-automatic methods: forecasts require some action(decision) by the user (e.g. Fair s add factors).
Most procedures studied in the literature are automatic. In
practice, forecasts almost always utilize some user intervention.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
http://find/http://goback/ -
7/28/2019 Econometric Forecasting
8/86
Introduction Model-free extrapolation Univariate time-series models
Model-free and model-based prediction
1. model-free procedures : extrapolation by free hand,exponential smoothing, trend tting;
2. model-based procedures : data-driven (time series) ortheory-driven (e.g. econometric models).
Some extrapolation methods can be justied by time-series models.It is easier to evaluate model-based procedures, as the models can
be simulated. With actual data, extrapolation can be a surprisinglygood benchmark.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
http://find/ -
7/28/2019 Econometric Forecasting
9/86
Introduction Model-free extrapolation Univariate time-series models
Model-based forecasting: classication by complexityA Nature of the DGPi Stationary DGP
ii Co-integrated DGPiii Evolutionary, non-stationary DGP
B Knowledge leveli Known DGP; known ii Known DGP; unknown
iii Unknown DGP; unknown C Dimensionality of the systemi Scalar processii Closed vector processiii Open vector process
D Form of analysisi Asymptotic analysis
ii Finite-sample analysisE Forecast horizoni 1stepii Multi-step
F Linearity of the systemi Linearii Non-linear
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
http://find/ -
7/28/2019 Econometric Forecasting
10/86
Introduction Model-free extrapolation Univariate time-series models
Self-fullling and self-defeating forecasts
If decisions are based on forecasts, forecasts may affect theforecasted variables. Effects can be positive or negative:
1. self-fullling forecasts: a bad growth forecast may cause
pessimism and decrease demand; a high ination forecast mayraise incentives for wage bargaining;
2. self-defeating forecasts: a high unemployment forecast maycause active labor market policies; a high ination forecastmay cause central banks to implement anti-inationarypolicies.
A good excuse for inaccurate economic forecasts?
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
http://find/http://goback/ -
7/28/2019 Econometric Forecasting
11/86
Introduction Model-free extrapolation Univariate time-series models
Out-of-sample and in-sample forecasts
1. a true out-of-sample forecast x N (h) only uses information overthe time range t N . If it is model-based, all parameters areestimated for t N , and this includes data-based elements of
model specication;2. an in-sample forecast uses information over t N + h. Such
information may be exogenous variables, or a model is ttedto a time range ending even after N + h. Forecast errors willbe residuals , not true prediction errors.
In forecasting, good performance in out-of-sample prediction isviewed as the acid test for a good forecast model.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
I d i M d l f l i U i i i i d l
http://find/ -
7/28/2019 Econometric Forecasting
12/86
Introduction Model-free extrapolation Univariate time-series models
Forecast failure
Demographic projections are relatively accurate, even forlonger terms;the accuracy of meteorological forecasts has improved overthe last decades;demand forecasts for new products have often failedcompletely (computers, TV sets), and they will continue to doso;speculative markets are very difficult to forecast (Sir Clive
Granger );short-run macroeoconomic forecasts did not improve overthe last decades (excuses: changing environment, humanaction, . . . ).
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
I t d ti M d l f t l ti U i i t ti i d l
http://find/ -
7/28/2019 Econometric Forecasting
13/86
Introduction Model-free extrapolation Univariate time-series models
Literature on forecastingChatfield, C. (2001) Time Series Forecasting , Chapman&Hall: accessible surveyMakridakis, S., Wheelwright, S.C., and R.J.Hyndman (1998) Forecasting: Methods and Applications ,Wiley: introduction for business economistsClements, M., and D.F. Hendry (1998) Forecasting economic time series , Cambridge University Press: academicClements, M., and D.F. Hendry (1999) Forecasting Non-Stationary Economic Time Series , MIT Press: a secondpart
Clements, M. (2005) Evaluating Econometric Forecasts of Economic and Financial Variables , Palgrave-Macmillan: asummaryJournals Journal of Forecasting and International Journal of Forecasting : the state of the art
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model free extrapolation Univariate time series models
http://find/http://goback/ -
7/28/2019 Econometric Forecasting
14/86
Introduction Model-free extrapolation Univariate time-series models
The free-hand method
Instinctively, we attempt to t smoothed curves through time
series and to extrapolate them at the end to generate a forecast.
The forecast will be very subjective and depends on the amount of smoothing.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
15/86
Introduction Model-free extrapolation Univariate time-series models
Example: the Austrian unemployment rate
1950 1960 1970 1980 1990 2000 2010
2
3
4
5
6
7
8
9
%
Extrapolation of the last few years yields a low prediction,smoothing over the last decade yields an upward direction.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
16/86
Introduction Model free extrapolation Univariate time series models
Exponential smoothing: the idea
The variants of exponential smoothing are sophisticated objectiveversions of the free-hand method.
Technical terminology:
a causal lter determines the (ltered) value of a data pointx N from observations x t , t < N ;a smoother determines the (smoothed) value of a data pointx N from observations x t , t < N and t N .
Extrapolation calculates a smoother in order to apply a causal lterafterwards.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
17/86
Introduction Model free extrapolation Univariate time series models
Single exponential smoothing (SES)
SES determines the ltered value x t from a weighted average overa past observation and a past ltered value:
x t = x t + (1 )x t 1
The constant (0, 1) is a damping or smoothing factor . Thisequation is called the recurrence form of SES. Note that a
smoothed past needs to be known here.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
18/86
p
Properties of SES: sum representation
Repeated substitution yields
x t = t 1
j =0(1 ) j x t j + (1 )t x 0,
often given without the last term, assuming x 0 = 0. Large implies strong discounting of the past and weak smoothing. In
any case, the past enters with geometrically declining weights.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
19/86
p
Properties of SES: error-correction form
Re-arranging the recurrence form yields the error-correction formof SES:
x t = x t 1 + (x t x t 1)
The new smoothed value is the old smoothed value plus somecorrection for the prediction error, if we interpret x t 1 as a
forecast for x t .
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
20/86
How to choose
Two main suggestions:
1. Choose from the interval [0.1, 0.3]. 0.1 implies strongsmoothing, 0.3 implies weak smoothing.
2. Determine by optimizing prediction over the sample, i.e.
min
( x t +1 x t )2
Problem with second option: cannot be an estimate, as nogenerating model is assumed.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
21/86
How to choose a starting x
Three options:
1. Start from the actual data x 1 = x 1. Problems for small and
small samples;2. Start from a sample average x 1 = x . Variants calculate meansfrom parts of the sample;
3. backcasting : run the SES lter backward to determine x t 1from x t and x t , using a starting value later in the series, nallydetermine x 1.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
22/86
SES and discounted least squares
If the level of a series changes slowly, it may be advantageous todetermine a location not by least squares
min
t 1
j =0
(x t )2 ,
but by discounted least squares
min
t 1
j =0
j
(x t j )2
for a local mean int . This yields SES for = 1 .
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
23/86
The SES forecast
SES denes a forecast x N (h) by at extrapolation:
x N (1) = x N (2) = . . . = x N
This is not appropriate for trending variables.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
24/86
Example: the Austrian unemployment rate
1950 1960 1970 1980 1990 2000 2010
2
3
4
5
6
7
8
9
%
SES for the unemployment rate (red = 0 .1, blue = 0 .3) andimplied forecasts.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
25/86
Double exponential smoothing (DES)
Double exponential smoothing runs SES twice:
Lt = x t + (1 ) Lt 1 ,T t = Lt + (1 ) T t 1,
with L a local level and T a local trend estimate. Both L and T are smoothed versions of the data.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
26/86
The DES forecast
hstep forecasts extrapolate the last observations on Lt and T t :
x N (h) = 2 + h1 LN 1 + h1
T N
= 2 LN T N +
1 (LN T N ) h
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
27/86
Example: DES on the Austrian unemployment rate
1950 1960 1970 1980 1990 2000 2010
2
3
4
5
6
7
8
9
%
DES for the unemployment rate ( = 0 .3, red L, green T ) andimplied forecasts (blue).
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
http://find/http://goback/ -
7/28/2019 Econometric Forecasting
28/86
Introduction Model-free extrapolation Univariate time-series models
-
7/28/2019 Econometric Forecasting
29/86
Holts linear trend method
Holts method generalizes DES and introduces a second tuningparameter. It has two recursion equations, for local trend (orrather slope) T and local level L:
Lt = x t + (1 ) (Lt 1 + T t 1) ,T t = (Lt Lt 1) + (1 ) T t 1.
L averages data and forecast, T averages old slope and new slope
estimate from L. Meaning of T differs from DES! A very popularmethod.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
30/86
Forecasting using Holts method
The standard denition forh
step forecasts isx N (h) = LN + hT N ,
such that x t 1 (1) = Lt 1 + T t 1 is a smoothed version of x t .Gardner& McKenzie suggest to forecast from Holts methodvia
x N (h) = LN +h
j =1
j T N .
This forecast corresponds to an ARIMA(1,1,2) generating model,while Holts method relies on ARIMA(0,2,2). Chatfield warnsthat all smoothing extrapolations are not genuinely justied byprediction in time-series models. They would imply absurdparameter restrictions.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
31/86
Example: Holt on the Austrian unemployment rate
1950 1960 1970 1980 1990 2000 2010
2
3
4
5
6
7
8
9
%
Holt procedure applied to the unemployment rate ( = 0 .3, = 0 .1, redL) and implied forecasts (blue). Trend changes slowly and still pointsupward at the end. Least-squares tting would suggest even moreextreme parameter values.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
32/86
Holt-Winters seasonal method
Quarterly and monthly economic data often has considerable
seasonal variation. Traditional seasonal models distinguishmultiplicative seasonality (seasonal factors) and additive seasonality(seasonal dummy intercepts). The Holt-Winters method allows forslow changes in these seasonal factors and intercepts.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
33/86
Holt-Winters: the recursionsMultiplicative version:
Lt = x t
S t s + (1 ) (Lt 1 + T t 1) ,
T t = (Lt Lt 1) + (1 ) T t 1,
S t = x t
Lt + (1 ) S t s .
Additive version:
Lt = (x t S t s ) + (1 ) (Lt 1 + T t 1) ,T t = (Lt Lt 1) + (1 ) T t 1,S t = (x t Lt ) + (1 ) S t s .
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
34/86
Remarks on Holt-Winters
Typically, s = 4 or s = 12.The procedure needs three smoothing parameters that areoften determined by least-squares tting. Low prescribes atime-constant, deterministic seasonal cycle.Convenient starting values for T are sample averages over x . For S , one may use s averages over the specic season.Averages may be restricted to a rst portion of the sample.While the Holt method corresponds to an ARIMA(0,2,2)generating model, there is no simple time-series model that justies Holt-Winters. Nonetheless, the method works well inpractice.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
35/86
The Holt-Winters prediction formulae
The last available seasonal pattern is extrapolated into the future.
Multiplicative version:
x
N (k
) = (L
N +T
N k
)S
N + k
s ,
where S N + k s is replaced by the last available correspondingseasonal if k > s .
Additive version:x N (k ) = LN + T N k + S N + k s .
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
36/86
Example: Holt-Winters on the Austrian unemployment rate
1950 1960 1970 1980 1990 2000 2010
2
4
6
8
1 0
1 2
1 4
%
Additive Holt-Winters ( = 0 .3, = 0 .1, = 0 .5) applied to themonthly Austrian unemployment series. Red: shifted L, blue:forecast.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
37/86
The Brockwell & Davis small-trends method
The time-series researchers Brockwell & Davis suggested anappealingly simple alternative to the complex Holt-Wintersalgorithm:
1. Calculate annual averages for the series, interpret them astrend, and subtract the trend from the observed x t to yield aseasonal, but not trending x t ;
2. Calculate averages for each season in x t over all years, whichyields an estimate of the seasonal cycle;
3. Extrapolate the trend (which one?) plus cycle into the futureto obtain a forecast.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/http://goback/ -
7/28/2019 Econometric Forecasting
38/86
Example: Brockwell-Davis small-trends on Austrian
unemployment rate
1950 1960 1970 1980 1990 2000 2010
2
4
6
8
1 0
1 2
1 4
%
Brockwell-Davis small-trends method applied to the monthlyAustrian unemployment series. Red: seasonal cycle, green: trend,blue: forecast.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
39/86
Remarks on the small-trends method
The method assumes a time-constant seasonal cycle, which
often does not appear to be realistic;The method fails if trends really play a role. One may modifythe method tting simple functions of time and extrapolatingthem.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
40/86
Forecasting using time-series modelsThese methods tentatively assume a data-generating process, thedegree of belief in their model varies among researchers: Allmodels are wrong, some are useful [G.E.P. Box ]Notes:
useful may refer to forecasting;
Model usually refers to a parametric model class. Thebest-tting or true parameter value is unknown and has to beestimated;knowing the true model class does not guarantee the bestforecasting performance if parameters have to be estimated.The wrong model class may outperform the wrong parameterin the true class: the true model class can be useless;simple linear time-series models are good forecasters even if the data-generation process is nonlinear .
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
41/86
The stages of time-series prediction
Data
Model selection
Model estimation
Prediction
?
?
?
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
42/86
The general form of time-series models
The current X t depends on its past and on an error:
X t = g (X t 1, X t 2, . . . ; ) + t ,
where g is a nonlinear or linear function, is an unknownparameter, ( t ) is an unobserved error process.( t ) is often assumed i.i.d. but is at least a martingale-difference sequence (MDS) dened by
E ( t | I t 1) = 0 .
I t 1 is an information set containing the process past. White noise(uncorrelated) ( t ) is not sufficient for prediction!
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
43/86
Prediction using a time-series model
Suppose is known. Then
E (X t | I t 1) = g (X t 1, X t 2 , . . . ; ) + E ( t | I t 1)= g (X t 1, X t 2 , . . . ; )
is a convenient forecast X t 1(1). It is easily shown that itminimizes the expected squared prediction error E (e t )2 withe t = X t X t among all feasible X t .
If is unknown, it is estimated from the sample and plugged in, asif it were known. If the model class is correct and the sample islarge, many authors claim that the reduction in accuracy is minor.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/http://goback/ -
7/28/2019 Econometric Forecasting
44/86
General nonlinear time-series models
Many nonlinear models do not obey the linear-errors scheme. Thegeneral form is:
X t = g (X t 1 , X t 2, . . . ; t ; ) .Here, even if were known, we have
E (X t | I t 1) = g (X t 1, X t 2, . . . ; 0; ) .
The only correct solution is stochastic prediction .
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
45/86
Stochastic prediction
1. Choose a generating distribution for the errors t ;2. Draw from a random processor and generate J replications of
X ( j )t = g X t 1 , X t 2, . . . ; ( j ) ; ;
3. Average over the J replications
X J t 1(1) =1J
J
j =1
X ( j )t ,
to approximate the expectation (law of large numbers, LLN);4. Analogous steps can be taken for hstep prediction with
h > 1.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
46/86
From what distribution to draw
Two main suggestions:
1. Fit a parametric model, for example normal distribution, tothe residuals and estimate the parameters: parametricbootstrapping, Monte Carlo;
2. Draw from a discrete uniform law over the sample residuals:(nonparametric) bootstrapping.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
47/86
Linear models with rational lag functions
ARMA (autoregressive moving average ) models of the form
X t = 1X t 1 + . . . + p X t p + t + 1 t 1 + . . . + q t q
are the most popular time-series models for data-based prediction.Their popularity may still be due to the book by Box andJenkins (1970,1976). We repeat the main results for special cases(AR, MA) in brief.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
48/86
The autoregressive model
The AR(p ) model
X t = 1X t 1 + 2X t 2 + . . . + p X t p + t
can be estimated by least squares. Note that t is specied asMDS, not simply as white noise, in forecasting applications.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
49/86
Stability of the AR model
The AR(p ) model is said to be stable if it permits a stationaryprocess that satises the equation and if future depends on thepast in that solution. For example, X t = 2 X t 1 + t has astationary solution that is useless for forecasting. A stable model is
also called asymptotically stationary .
The AR(p ) model is stable if its characteristic polynomial equation
1 1z 2z 2 . . . p z p = 0
has only roots greater than one in modulus. For small p , thisproperty is easily checked by hand.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
50/86
Determining the lag orderp of an AR model
Time-series analysis knows three approaches for lag-order search:1. Plot the empirical partial autocorrelation function (PACF)
P (k ) that should differ from 0 for k p and equal 0 fork > p (recommended by Box& Jenkins );
2. t AR(p ) models for different p and test residuals for whitenoise: choose the smallest p such that the test is passed(unreliable);
3. t AR(p ) models for different p and calculate informationcriteria IC (p ) for each model: choose the p that minimizesthe criterion.
The information-criterion approach is the most suitable one forforecasting.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
51/86
Information criteriaThere are two main classes of information criteria:
1. Consistent criteria: as the sample size N , the true lagorder tends to be found with probability onefor example,Schwarz BIC;
2. efficient criteria: as the sample size N , the forecastbased on the selected model minimizes the expectedmean-squared errorfor example, Akaike s AIC
AIC = log 2(p ) +2p N
,
with 2(p ) an estimated errors variance from an AR(p ) model.
If the aim is forecasting, criteria of the second class, which includesAIC, AIC u , AIC c , FPE, may be a natural choice.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
52/86
Forecasting from an AR(p ) model
Suppose the AR(p ) model has generated the data and theparameters are known. Then:
E (X t +1 | I t ) = 1X t + 2X t 1 + . . . + p X t p +1 + E ( t +1 | I t ),
and hence, as is MDS,
X t (1) = 1X t + 2X t 1 + . . . + p X t p +1 .
If parameters are unknown and also p has been determinedempirically, plugging in j yields a feasible approximation that isacceptable for larger N .
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
53/86
Multi-step forecasting from an AR(p ) model
Two-step forecasts are obtained by iteration, plugging in one-stepforecasts:
E (X t +2 | I t ) = 1E (X t +1 | I t )+ 2X t + . . . + p X t p +2 + E ( t +2 | I t ),
orX t (2) = 1X t (1) + 2X t + . . . + p X t p +2 ,
and this iteration can be continued for larger horizons.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
54/86
Multi-step forecasting: direct modeling
In iterated plugging-in, the forecastX t (h) is formed as
X t (h) = h X t + h+1 X t 1 + . . . + h+ p 1X t p +1 ,
with j depending on 1, . . . , p . Alternatively, one may t models
of the type X t = h X t h + . . . + p X t p + t
to the sample and use
X t (h) = h X t + . . . + p X t p + h .
The relative merits of this direct modeling method are an issue of ongoing research. For small h and correctly specied models,iterated forecasting can be shown to be better.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
55/86
The moving-average model
The MA(q ) model
X t = t + 1 t 1 + 2 t 2 + . . . + q t q
is to be estimated by a non-linear least-squares procedure. Againnote that t is specied as MDS in forecasting applications.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
http://find/ -
7/28/2019 Econometric Forecasting
56/86
Introduction Model-free extrapolation Univariate time-series models
-
7/28/2019 Econometric Forecasting
57/86
What does non-invertibility imply?
Two cases:1. If any of the polynomial roots have modulusexactly equal
one , prediction becomes very difficult. Sometimes, this
non-invertibility is due to pre-processing the data by ltering,seasonal adjustment, differencing;2. If any roots have modulus less than one , there exists an
observationally equivalent MA model with all roots larger thanone. This non-invertibility is due to a non-optimal estimationroutine.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
58/86
Determining the lag orderq of an MA modelAgain, time-series analysis knows three approaches for lag-ordersearch:
1. Plot the empirical autocorrelation function (ACF) orcorrelogram (k ) that should differ from 0 for k q andequal 0 for k > q (recommended by Box& Jenkins );
2. t MA(q ) models for different q and test residuals for whitenoise: choose the smallest q such that the test is passed(unreliable);
3. t MA(q ) models for different q and calculate information
criteria IC (q ) for each model: choose the q that minimizesthe criterion.
Again, the IC approach is the most suitable one for forecasting, andthere may be a preference for using efficient criteria, such as AIC.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
59/86
Forecasting from an MA(q ) model
Suppose the MA(q ) model has generated the data and theparameters are known. Then, one may reconstruct true t and use:
E (X t +1 | I t ) = 1 t + 2 t 1 + . . . + q t q +1 + E ( t +1 | I t ),
and hence, as is MDS,
X t (1) = 1 t + 2 t 1 + . . . + q t q +1 .
If parameters are unknown, q is determined empirically, and t must be estimated, one may still plug in estimates. Alternatively,program routines may use the inverted AR( ) model and cut off the sum at some large value.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
60/86
Multi-step forecasting from an MA(q ) model
Two-step forecasts are simple in principle, according to:
E (X t +2 | I t ) = 1E ( t +1 | I t ) + 2 t + . . . + q t q +2 + E ( t +2 | I t ),
orX t (2) = 2 t + . . . + q t q +2 ,
and this scheme can be continued for larger horizons h . For h > q ,X t (h) = 0. MA processes are nite dependent .
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
61/86
The autoregressive moving-average model
The general ARMA(p , q ) model
X t = 1X t 1+ 2X t 2+ . . . + p X t p + t + 1 t 1+ 2 t 2+ . . . + q t q
is to be estimated by a non-linear least-squares procedure. Againnote that t is specied as MDS in forecasting applications.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
http://find/ -
7/28/2019 Econometric Forecasting
62/86
Stability of the ARMA model
The ARMA(p , q ) model inherits its properties from its AR and MAcomponents.
1. For a unique denition, the characteristic polynomials for theAR and MA parts must not have common zeros, otherwise a
simpler representation ARMA(p 1, q 1) exists and theadditional parameters cannot be estimated;2. under condition # 1, if the AR polynomial has only roots
larger than one, the ARMA model is stable;
3. under condition # 1, if the MA polynomial has only rootslarger than one, the ARMA model is invertible, i.e. there is anAR( ) representation.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
h l d d f d l
http://find/ -
7/28/2019 Econometric Forecasting
63/86
Determining the lag ordersp and q of an ARMA modelIn principle, time-series analysis knows three approaches forlag-order search:
1. Plot advanced tools such as the empirical extended autocorrelation function (EACF) and guess a goodcombination of lag orders by visual inspection (rarely used);
2. t ARMA(p , q ) models for different p and q and test residualsfor white noise: choose the smallest p and q such that thetest is passed (unreliable);
3. t ARMA(p , q ) models for different p and q and calculate
information criteria IC (q ) for each model: choose the pair(p , q ) that minimizes the criterion.
Again, there may be a preference for using efficient criteria, suchas AIC.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
F i f ARMA( ) d l
http://find/ -
7/28/2019 Econometric Forecasting
64/86
Forecasting from an ARMA(p , q ) modelForecasting must proceed carefully, using a variant of the methodused in AR models. Suppose an ARMA(2, 2) model has generatedthe data and the parameters are known. Then, one mayreconstruct true t and use:
E (X t +1 | I t ) = 1X t + 2X t 1 + E ( t +1 | I t ) + 1 t + 2 t 1,
and hence, as is MDS,
X t (1) = 1X t + 2X t 1 + 1 t + 2 t 1.
In practice, parameters are estimated, p and q are determinedempirically, and t must be estimated, and these estimates areplugged in. Alternatively, program routines may use the invertedAR( ) model and cut off the sum at some large value.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
M l i f i f ARMA( ) d l
http://find/ -
7/28/2019 Econometric Forecasting
65/86
Multi-step forecasting from an ARMA(p , q ) model
Two-step forecasts are obtained by plugging in one-step forecastsfor the true values, according to:
E (X t +2 | I t ) = 1E (X t +1 | I t ) + 2X t +E ( t +2 | I t ) + 1E ( t +1 | I t ) + 2 t ,
orX t (2) = 1X t (1) + 2X t + 2 t ,
and this scheme can be continued for larger horizons h . For h > q (here, q = 2), the MA part disappears.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
I t t d d l
http://find/ -
7/28/2019 Econometric Forecasting
66/86
Integrated models
The class of integrated models is the most popular class of non-stationary time-series models. An integrated process (X t ) isdened by the property that it is not stationary but d -th orderdifferences ( d X t ) are stationary. Only d = 1 and d = 2 are of
empirical interest.
The class of integrated processes is a very special class of non-stationary processes. They model near-polynomial andrandom-walk trends well but not structural breaks, outliers,increasing variation, and other observed non-stationary features.Note that data cannot be non-stationary .
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
N t ti
http://find/ -
7/28/2019 Econometric Forecasting
67/86
Notation
Box & Jenkins called a process X t ARIMA(p , d , q ) if d X t is astable and well-dened ARMA(p , q ) process but d 1X t is not.
Engle & Granger called a process X t d-th order integrated , insymbols I(d ) if d X t is stationary but d 1X t is not stationary.This is slightly more general.
Note X t = X t X t 1 and 2X t = X t 2X t 1 + X t 2.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
How to handle data from integrated models
http://find/ -
7/28/2019 Econometric Forecasting
68/86
How to handle data from integrated models
If data stem from an I (d ) process, the idea is:1. take d th order differences;
2. t ARMA models to the differenced data;3. forecast according to the identied ARMA structures;4. possibly integrate back (accumulate) to obtain forecasts for
the original variable.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
How to decide whether data stem from integrated
http://find/ -
7/28/2019 Econometric Forecasting
69/86
How to decide whether data stem from integratedprocesses
Two main ideas:1. Box & Jenkins suggest to consider the correlogram. If it
decays too slowly, take differences. Use the differencing orderthat makes the correlogram as simple as possible:over-differencing would make it more volatile;
2. Most economists today base this decision on the test byDickey & Fuller and comparable tests. The null
hypothesis is the unit root: if the test does not reject, takedifferences.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
http://find/ -
7/28/2019 Econometric Forecasting
70/86
Introduction Model-free extrapolation Univariate time-series models
Nonlinear forecasting models
-
7/28/2019 Econometric Forecasting
71/86
Nonlinear forecasting models
General nonlinear models are rarely used in econometricforecasting. Three specic classes are popular:
1. ARCH models: autoregressive conditional heteroskedasticity;2. threshold models;3. articial neural networks (ANN).
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
ARCH
http://find/ -
7/28/2019 Econometric Forecasting
72/86
ARCH
The original ARCH model byEngle (1982) assumes, in itssimplest form, that X t is white noise. If X t follows
X t = + t ,E 2
t | I t
1= ht =
0+
12
t
1+ . . . + r 2
t
r ,
(X t ) is said to be an ARCH(r ) process. The model is stable withvar X t < if
1. 0 > 0;
2. j 0, 0 < j r ;3. r j =1 j < 1.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
GARCH
http://find/ -
7/28/2019 Econometric Forecasting
73/86
GARCH
The most popular ARCH generalization today is still the GARCHmodel by Bollerslev . The lagged unobserved conditionalvariance serves to reect an ARMAtype geometric decay of volatility shocks. The GARCH(1,1) model reads
X t = + t ,E 2t | I t 1 = ht = 0 + 12t 1 + ht 1 ,
which models well (log differences of) near random walks in thenancial world.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
ARMA-ARCH
http://find/ -
7/28/2019 Econometric Forecasting
74/86
ARMA ARCH
To the time-series forecaster who models serially correlatedvariables, the most interesting extensions are ARMA-ARCH modelswith non-trivial mean equation and an ARCHtype variance equation , for example:
X t = + X t 1 + t ,E 2t | I t 1 = h t = 0 + 12t 1 .
Note that this form already appears in the work of Engle (1982),where monthly U.K. ination was modelled.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
Stable ARMA-ARCH models
http://find/http://goback/ -
7/28/2019 Econometric Forecasting
75/86
Stable ARMA ARCH models
If the mean equation fullls the usual ARMA stability conditionsand the variance equation fullls the ARCH stability conditions, t is white noise and (X t ) is a stationary homoskedastic process.The models view ht (volatility, risk?) as time-dependent butunconditional var X t as time-constant.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
ARCH models for forecasting: worth the additional work?
http://find/ -
7/28/2019 Econometric Forecasting
76/86
ARCH models for forecasting: worth the additional work?
1. Forecasts for the level of X t are as good as the meanequation, the ARCH parameters only enter indirectly, theyserve to estimate e.g. more efficiently and they specify thestandard error of ;
2. for any data with monthly or lower frequency, modelling theARCH part is not worth the work, gains in efficiency are low;
3. it is tempting to forecast X 2t on the basis of the ARCH modelbut such forecasts are often surprisingly poor;
4. current research opines that a systematic forecast of localrisk aiming at commercial advice to risk-conscious tradersbased on ARCH models is not possible or at least verydifficult (Granger ).
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
Threshold models: the idea
http://find/ -
7/28/2019 Econometric Forecasting
77/86
Threshold models: the idea
Economic agents may react differently to positive and negativeshocks, to small and to large shocks.
It may make sense to consider models such as:X t = ( j ) X t 1 + t , r j 1 < X t 1 < r j , j = 1 , . . . , k ,
where r k = and r 0 = . These models are called SETAR
(self-exciting threshold autoregressive) models.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
Stability of SETAR models
http://find/ -
7/28/2019 Econometric Forecasting
78/86
y
First-order SETAR models are stable if the outer regimeshave | j | < 1 (sufficient only);higher-order SETAR models have very complex stabilityconditions;most empirical modelling is done for two or three regimes.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna Introduction Model-free extrapolation Univariate time-series models
The benets of SETAR for prediction
http://find/http://goback/ -
7/28/2019 Econometric Forecasting
79/86
p
Threshold reaction is often found in macroeconomics;thresholds r j and coefficients for rarely observed regimes needextremely large samples for reliable estimation;
forecasting must be based on stochastic prediction, as themodel is nonlinear, distributions of t play a key role;the models may forecast poorly even when they are thedata-generating processes and may be outperformed by linear
models.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna Introduction Model-free extrapolation Univariate time-series models
Neural networks: the idea
http://find/ -
7/28/2019 Econometric Forecasting
80/86
0 . 0
0 . 2
0 . 4
0 . 6
0 . 8
1 . 0
x
(
x )
Single neurons are activated according to 0-1 functions. The sum of many neurons allows a smooth transition from no reaction (0) tounbearable pain (1). Basing models on sigmoid functions rather thanlinear ones may often be more realistic.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna Introduction Model-free extrapolation Univariate time-series models
Layers of neurons
http://find/http://goback/ -
7/28/2019 Econometric Forecasting
81/86
y
Neurons may be linked to further neurons (synapses), whichpermits multiple layers. The simplest version of neural nets usedin practice has just one hidden layer of such synapses. Assumethe stimulus or input are past x , and the reaction or output is
current x . This forecast net function follows Chatfield :
x t = 0 w c 0 +H
h=1
w h0h w ch +h
j =1
w jh x t j ,
where all h are sigmoid functions.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna Introduction Model-free extrapolation Univariate time-series models
Training the net
http://find/http://goback/ -
7/28/2019 Econometric Forecasting
82/86
g
Weights and numbers of layers are typically optimized over anestimation interval ( training set ) and are then used for predictionbased on the identied architecture . Extending the training set to
an intermediate sample to update the weights is called learning .
Neural nets are really just a class of nonlinear time-series models.Their reported forecasting successes may be rooted in the usage of sigmoid reaction functions.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
State-space modelling: the idea
http://find/ -
7/28/2019 Econometric Forecasting
83/86
Assume an unobserved multivariate state t determines theobserved variable of interest X t . There is an observation equation
X t = h
t t + nt ,
with a noise term nt and a vector of linear weights ht . The statebehaves according to a transition equation
t = G t t 1 + w t ,
with square matrix G and another noise term w t .
In this general form, the model cannot be identied.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
Example: autoregressive models in state-space form
http://find/ -
7/28/2019 Econometric Forecasting
84/86
For example, assume p = 4. Dene
G t G =
1 2 3 41 0 0 0
0 1 0 00 0 1 0
,
and t = ( X t , . . . , X t 3) , ht = (1 , 0, 0, 0) , w t = ( t , 0, . . . , 0) ,nt = 0. Thus, any AR( p ) model has a state-space representation,and so has any ARMA model.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
The benets of state-space models
http://find/ -
7/28/2019 Econometric Forecasting
85/86
Most time-series models can be represented in theirstate-space form.
State-space models are more a technique of representingmodels than a separate class of models.The basic form may be convenient, and it allows manygeneralizations beyond standard time-series models.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
Introduction Model-free extrapolation Univariate time-series models
A structural model according toHarvey
http://find/ -
7/28/2019 Econometric Forecasting
86/86
The simplest of the unobserved-components (UC) models due toHarvey has two state variables and :
X t = t + n t , t = t 1 + t 1 + w 1, t ,
t = t 1 + w 2, t .
With white-noise input, X t is a special I(2) or ARIMA(p , 2, q )process. UC adepts claim that the differentparameterizationvariances of errors instead of ARMAcoefficientsis more natural. UC models sometimes performsurprisingly well in forecasting economic data.
Econometric Forecasting University of Vienna and Institute for Advanced Studies Vienna
http://find/