arma estimation
TRANSCRIPT
-
8/10/2019 ARMA Estimation
1/62
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
MODEL IDENTIFICATION, ESTIMATION AND
DIAGNOSTIC CHECKING
Vlayoudom Marimoutou
AMSE Master 2
Autumn 2013
December 11, 2013
1/60
-
8/10/2019 ARMA Estimation
2/62
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
1 INTRODUCTION2 ESTIMATION OFARMA(p, q) PROCESSES
Estimation ofAR(p)processes
estimation of anAR(p)for a fixed (and known)pEstimation ofAR()processes
Estimation ofARMA(p, q)process: Maximum likelihood estimationLikelihood function forAR processes
Exact versus conditional MLE
likelihood function forMA process
Numerical optimization of the log likelihood3 ASYMPTOTIC PROPERTIES OF ML ESTIMATORS AND
INFERENCE4 IDENTIFICATION OF AN ARMA PROCESS: SELECTINGp
ANDq
Information criteriaOther model selection approaches5 DIAGNOSTIC CHECKING6 FORECASTING
The MSE optimal forecast is the conditional mean
Forecast evaluation
Relative evaluation:Diebold-Mariano
- 2/60
-
8/10/2019 ARMA Estimation
3/62
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
1 INTRODUCTION
2 ESTIMATION OF
ARMA(p, q) PROCESSES
3 ASYMPTOTICPROPERTIES OF ML
ESTIMATORS AND
INFERENCE
4 IDENTIFICATION OF ANARMA PROCESS:
SELECTINGp ANDq
5 DIAGNOSTIC CHECKING
6 FORECASTING
7 MODEL IDENTIFICATION
IN THE GENERAL CASE
8 R CODE
9 REFERENCES
3/60
-
8/10/2019 ARMA Estimation
4/62
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
Suppose you want to fit anARMA(p, q)model to aunivariatestationaryprocess {Xt}.The first thing will be to set values for p and q. We will call this step
identification.
Next, we would like toestimatethe correspondingARMA(p, q). Themost popular method for estimatingARMAprocesses is Maximum
Likelihood (ML).
ARprocesses can be also estimated by OLS. It can be shown that this
approach is asymptotically equivalent to ML. This is very convenient
because OLS estimator have closed analytical expressions and are
straight forward to compute.
The final step will be to check whether the residuals of th estimated
model verify the assumptions that have been imposed on the
innovations.
4/60
-
8/10/2019 ARMA Estimation
5/62
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
1 INTRODUCTION
2 ESTIMATION OF
ARMA(p, q) PROCESSES
3 ASYMPTOTICPROPERTIES OF ML
ESTIMATORS AND
INFERENCE
4 IDENTIFICATION OF ANARMA PROCESS:
SELECTINGp ANDq
5 DIAGNOSTIC CHECKING
6 FORECASTING
7 MODEL IDENTIFICATION
IN THE GENERAL CASE
8 R CODE
9 REFERENCES
5/60
-
8/10/2019 ARMA Estimation
6/62
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
ESTIMATION OFAR(p) PROCESSES
.
6/60
O I O C O ES A O AS O C O S I CA O D AG OS C C C G FO CAS G MO
-
8/10/2019 ARMA Estimation
7/62
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
ESTIMATION OFAR(p) PROCESSES
Assume for the time being that(p, q)are known. We will consider firstestimation ofAR processes (its simpler) and then we will move to
estimation of generalARMA(p, q)processes.
Here we distinguish two cases:
Xtfollows anAR(p)for some finitep
Xtfollows anAR()process.
6/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
8/62
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
ESTIMATION OFAR(p) PROCESSES
Let {Xt}be a stationaryAR(p)process withp < , i.e.
Xt=0+1Xt1+...+pXtp+t
wheretis a zero mean innovation (either w.n, m.d.s. or i.i.d.) with variance2.
It is possible to estimate=
01. . . p)
by applying the standard OLS
formula and the resulting estimator is
T- consistent and asymptotically
normal
7/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
9/62
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
ESTIMATION OFAR(p) PROCESSES
We now provide a sketch of the proof for the simplest case. Let {Xt} be anAR(1)process given by
Xt=Xt1+t
wheretis am.d.sand
||
-
8/10/2019 ARMA Estimation
10/62
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
ESTIMATION OFAR(p) PROCESSES
Consistency
olsp
Xt1tX2t1
p 0
Under the previous assumptions,X2t is also stationary and ergodic, thus by
LLN,
X2t1T
p var(Xt1) = 2
(1 2)As for the numerator, notice that iftis am.d.s, so is
Xt1t. Therefore, a
LLN also applies to the numerator and Xt1tT
p
E(X
t1
t) =0. Then
ols=+ op(T)
Op(T)=+
op(1)
Op(1)=+op(1)
9/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
11/62
O
ESTIMATION OFAR(p) PROCESSES
Asymptotic normality
T12
ols
=T1/2Xt1t
T1
X2t1
=
T12Xt1tvar(Xt)
+op(1)
By the CLT for m.d.s
T12 Xt1t d N0, var(Xt)2
Thus
T12 ols
d
N0,
2
var(X
t)= N0, (1
2)Remark: What happens with the distribution ofolsif 1?
A similar proof can be established for a general stationaryAR(p)process.
10/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
12/62
ESTIMATION OFAR(p) PROCESSES
AR(p)(for finitep) are easier to estimate thanMA processes since OLScan be applied.
We know that invertibleARMAprocess can be written as anAR()process.
Berk showed that if anAR(k)process is fitted toXt, wherektends to infinitywith the sample size, it is possible to obtainconsistent and asymptotically
normally distributedestimators of the relevant coefficients.
More explicitly,khas to verify two conditions:
11/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
13/62
ESTIMATION OFAR(p) PROCESSES
an upper bound conditionsk3/T
0 (that says thatkshould not
increase too quickly),
and a lower bound one,T1/2
j=k+1i c =0 (that says thatkmustnot increase too slowly).
How to choosekin practice?
The above mentioned conditions do not help much in choosing the
appropriatekin applications. Ng an Perron (2005) have shown that the
value ofkchosen by the AIC and the BIC does not verify the lower
bound condition. This implies that the estimates are consistent but not
asymptotically normal
Kuersteiner (2005) has proved that the general to specific model
selection verifies both the upper and the lower bound conditions. Then,
the resulting estimates are consistent and asymptotically normal.
12/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
14/62
ESTIMATION OFARMA(p, q) PROCESS: MAXIMUM LIKELIHOOD ESTIMATION
Let {Xt} be anARMA(p, q)process
p(L)Xt=+q(L)t
and let=
1,...,p, 1,...,q, , 2
be the vector containing all the
unknown parameters. Suppose that we have observed a sample of size
T, (x1,x2,...).
The ML approach will amount to calculate the joint probability density
of(XT,...,X1)
fXT,XT1,...,X1 (xT,xT1,...,x1; ) (1)
which might be loosely interpreted as the probability of having
observed this particular sample.
The Maximum likelihood estimator (MLE) ofis the value thatmaximizes(1), i.e, the probability of having observed this sample.
13/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
15/62
ESTIMATION OFARMA(p, q) PROCESS: MAXIMUM LIKELIHOOD ESTIMATION
The first step is to specify a particular distribution for the white noise
innovations of the process,t. Typically, it will be assumed thattis a
Gaussian white noise,t i.i.dN(0, 2).This assumption is strong but when Gaussianity does not hold , the
estimator computed by maximizing the Gaussian likelihood has, under
certain conditions, the same asymptotic properties as if the process
were indeed normal. In this cases, the estimator of the parameters of a
non Gaussian process computed on a Gaussian likelihood is calledQuasi or Pseudo MLE
The second step would be to calculate the likelihood function(1). Theexact closed form of the likelihood function of a generalARMA
processes is complicated. In the following we will illustrate how the
likelihood is computed for simpleAR andMA processes.The final step is to obtain the values ofthat maximize the loglikelihood. Unless the process is a pureAR process (in which cas OLS
can be applied), numerical optimization procedures should be applied
in order to obtain the estimates.
14/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
16/62
LIKELIHOOD FUNCTION FORAR PROCESSES
Let {Xt} be anAR(1)process
Xt=c+Xt1+t
where ||
-
8/10/2019 ARMA Estimation
17/62
LIKELIHOOD FUNCTION FORAR PROCESSES
As for theX3, the distribution ofX3conditional onX2 =x2andX1 =x1 is
fX3|X2,X1 (x3|x2,x1; ) = 1
22e
(x3cx2)2
22
The joint distribution of theX1,X2 andX3 can be written as
fX3,X2,X1 (x3x2,x1; )
= fX3|X2,X1 (x3|x2,x1; )fX2|X1 (x2|x1; )= fX3|X2,X1 (x3|x2,x1; )fX2|X1 (x2|x1; )fX1 (x1; )
Furthermore, notice that the values ofX1,X2,...,XT1 matter forXtonlythrough the value ofXt1, and then
16/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
18/62
LIKELIHOOD FUNCTION FORAR PROCESSES
fXt|Xt1,...,X1 (xt|xt1,...,x1; ) =fXt|Xt1,...,X1 (xt|xt1; ) (2)
fXt|Xt1,...,X1 (xt|xt1; ) = 1
22e
(xtcxt1)2
22 (3)
Thus, the joint density of the firsttobservations is
fXt|Xt1,...,X1 (xT|xT1,...,x1; ) (4)=fXt|Xt1 (xt|xt1; )fXt1|Xt2 (xt1|xt2; )...fX1 (x1; ) (5)
=fX1 (x1; )
Tt=2
fXt|Xt1 (xt|xt1; ) (6)
17/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
19/62
LIKELIHOOD FUNCTION FORAR PROCESSES
Taking logs
L() =log(fX1 (x1; )) +T
t=2logfXt|Xt1 (xt|xt1; ) (7)
and L is called the log-likelihood function.
Clearly, the value ofthat maximizes(6)and(7)is the same but themaximization problem is simpler in the latter case, and the the log likelihood
function is always preferred.
18/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
20/62
LIKELIHOOD FUNCTION FORAR PROCESSES
The next step would be to compute the value offor which the exact loglikelihood in(2)is maximized. This amounts to deriving the log likelihood
and equating the first derivatives to zero. The result is a system of non linearequations onand the sample for which there is no close solution in termsof(x1,...,xT. Then, iterative numerical procedures are needed to obtain.
And alternative procedure is to regard the value ofy1 as deterministic and
then
logfXT,XT1,...,X1 (xT,xT1,...,x1; )
=logfXT|XT1 (xT|xT1; )fXT1|XT2 (xt1|xt2; )...fX1 (x1; )
=
T
t=2
log fXT|XT1 (xT|xT1; )=
T 1
2
log
22
Tt=2
(xt c xt1)2
22
19/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
21/62
LIKELIHOOD FUNCTION FORAR PROCESSES
Notice that the conditional MLE ofc and are obtained by minimizing
Tt=2
(xt c xt1)2
It follows that maximization of the log likelihood is equivalent to theminimization of the sum of squared residuals. More generally, the
conditional MLE for anAR(p)process can be obtained from an OLSregression ofyton a constant andp of its own lagged values. In contrast to
exact MLE, the conditional maximum likelihood are trivial to compute.
Moreover, if the sample sizeTis sufficiently large, the first observationmakes a negligible contribution to the total likelihood provided ||
-
8/10/2019 ARMA Estimation
22/62
LIKELIHOOD FUNCTION FORAR PROCESSES
The conditional MLE of the innovation is found by differentiating (9) with
respect to2 and setting the result equal to zero. It can be checked that
2 =
Tt=2
(xt c xt1)2T 1 ,and in general, the MLE estimator of2 of anAR(p)process is given by thesum of squared residuals over(Tp)
21/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
23/62
LIKELIHOOD FUNCTION FORMA PROCESS
Let {Xt} be a gaussianMA(1)processXt=+t+t1
withtisiid N(0, 2). Let=
,,2
be the population parameters to be
estimated. Then,Xt|t
N+t1, 2andfXt|t1 (xt|t1) =
122
e
(xtt1)
2
22
It is not possible to compute this density sincet1is not directly observed
from the data. Suppose that it is known that0 =0. Then, all the sequenceof innovations could be computed recursively as
22/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
24/62
LIKELIHOOD FUNCTION FORMA PROCESS
1=x
1
2 =x2 x1
...
t=xt xt1Then the conditional density is given by
fXt|Xt1,...,X1 (xt|xt1,...,x1, 0 =0; ) =fXt|t1 (xt|t1)
= 1
22e
2t22
23/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
25/62
LIKELIHOOD FUNCTION FORMA PROCESS
while the conditional log likelihood is
L() = T2
log
22 2t
22
The conditional log likelihood function is a complicated non linear function
and, in contrats to theAR case, they should be found by numerical
optimization.
If is far from 1 in absolute value, then the effect of imposing0 =0 willquickly die out. However, if is over 1 in absolute value, the error ofimposing this restriction accumulates over time. Then, if the output of
estimating is greater than1 should discard the results. The numericaloptimization should be attempted again with the reciprocal of used asstarting value.
24/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
26/62
NUMERICAL OPTIMIZATION OF THE LOG LIKELIHOOD
Once the log likelihood has been computed, the next step is to find the valuethat maximizes L(). With the exception of pureAR processes, for whichclosed analytical expressions of the estimators are available, numerical
optimization procedures should be employed to obtain the estimates. These
procedures make different guesses for, evaluate the likelihood at thesevalues and try to infer from these there the valuefor whichis largest. SeeHamilton, section 5.7 for a description of these methods.
The search procedure may be greatly accelerated if the optimization
algorithm begins with parameter values which are close to the optimum
values. For this reason, simple preliminary estimates ofare often
employed to begin the search. See Brockwell and Davis, Section 8.2-8.4 fora description of these preliminary estimators.
25/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
27/62
1 INTRODUCTION
2 ESTIMATION OF
ARMA(p, q) PROCESSES
3
ASYMPTOTIC
PROPERTIES OF ML
ESTIMATORS AND
INFERENCE
4 IDENTIFICATION OF ANARMA PROCESS:
SELECTINGp ANDq
5 DIAGNOSTIC CHECKING
6 FORECASTING
7 MODEL IDENTIFICATION
IN THE GENERAL CASE
8
R CODE
9 REFERENCES
26/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
28/62
ML estimators have very good asymptotic properties.
Under certain conditions it can be shown that they are consistent,asymptotically normal and efficient, since the variance covariance
matrix equals the inverse of Fischer information matrix.
Let {Xt} be anARMA(p, q)process,0be the vector containing the trueparameter values andis the MLE of. Assuming that neither0 norfallson the boundary of the parameter space then
T
0
d N0,I1where
Iis the Fischer Information matrix
I= E
2L()
|=0
27/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
29/62
An estimator of the variance covariance matrix is
I= T1
2L()
|=
which is often calculated numerically. Another popular estimator of the
Fischer information matrix is the so-called outer product estimator
I= T1T
t=1
ht()
ht()
whereht() =logf(y
t|y
t1,y
t2,...; )/
|=denotes the vector of
derivatives of the log of the conditional density of thet thobservation withrespect to the elements of the parameter vector, evaluated at.
28/60
-
8/10/2019 ARMA Estimation
30/62
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
31/62
1 INTRODUCTION
2 ESTIMATION OF
ARMA(p, q) PROCESSES
3 ASYMPTOTIC
PROPERTIES OF ML
ESTIMATORS AND
INFERENCE
4 IDENTIFICATION OF ANARMA PROCESS:
SELECTINGp ANDq
5 DIAGNOSTIC CHECKING
6 FORECASTING
7 MODEL IDENTIFICATION
IN THE GENERAL CASE
8 R CODE
9 REFERENCES
30/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
32/62
How to selectp andq?
ACFandPACFare of help but their usefulness is limited in the general
ARMA(p, q)case.In this section we will see other ways of selecting these values: model
selection criteria
If you assume that your model contains an infinite number of
parameters or that the type of models you are considering for
estimation do not include the true model, thegoalwill be to select onemodel that best approximates the true model from a set of finite
dimensional candidate models. In large samples, a model selection
criteria that chooses the model with minimum mean squared error
distribution is said to be aasymptotically efficient
Many researchers assume that the true model is of finite dimension andis included in the set of candidate models. The goal in this case is to
correctly choose the true model from the list of candidates. A model
selection criteria that identifies the correct model asymptotically with
probability 1 is said to be consistent
31/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
INFORMATION CRITERIA
-
8/10/2019 ARMA Estimation
33/62
Information criteria are mechanisms designed for choosing an
appropriate number of parameters to be included in the model. Byminimizing some particular distances between the true and the candidates
models (the Kullback-Leidler discrepancy, see Gourieroux-Montfort), it is
possible to arrive to expressions that usually have the general form:
IT(k) =log(2k) + k
C(T)
T penalty term
(8)
Some popular IC are:
1 Akaike:C(T) =2
2 Schwartz or Bayesian (BIC):C(T) =log T
3 Hannan and Quinn (HQ):C(T) =2 log(log T)
32/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
INFORMATION CRITERIA
-
8/10/2019 ARMA Estimation
34/62
The numbers of parameterskto be included in the model is chosen as
k= arg minkm
IT(k)
wherem is some pre-specified maximum numbers of parameters
Remark 1The AIC is not consistent, in the sense that it might not choose
asymptotically the correct model with probability one. It tends to
overestimate the number of parameters to be included in the model.
However, this does not mean that this method is useless. It minimizes the
MSE for one-step ahead forcasting
33/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
INFORMATION CRITERIA
-
8/10/2019 ARMA Estimation
35/62
Remark 2The BIC and the HQ criteria are consistent that is
limT
P(k=k0) =1 (9)
In fact, the consistency result (9) holds for any criterion of the type (8)
withlimT C(T)/T=0 and limT C(T) =
34/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
OTHER MODEL SELECTION APPROACHES
-
8/10/2019 ARMA Estimation
36/62
General to specific criterion (Ng and Perron, 1995). This methodamounts to
1 Set the maximum numbers of parameters to be estimated. For instance, an
AR(k)model withk=7.2 Estimate the "general" model, (anAR(7)in this case).
3 Test for the statistical significance of the coefficients associated with thehigher order lags - with either a tor anFtest (yt7 in this case).
4 Exclude the nonsignificant parameters, reestimate the smaller model and
test again for significance. Repeat until all the remaining estimates are
significantly different from zero.
Bootstrap model selecting techniques.
35/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
37/62
1 INTRODUCTION
2 ESTIMATION OF
ARMA(p, q) PROCESSES
3 ASYMPTOTIC
PROPERTIES OF ML
ESTIMATORS AND
INFERENCE
4
IDENTIFICATION OF ANARMA PROCESS:
SELECTINGp ANDq
5 DIAGNOSTIC CHECKING
6 FORECASTING
7 MODEL IDENTIFICATION
IN THE GENERAL CASE
8 R CODE
9 REFERENCES
36/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
38/62
Typically, the goodness of fit of a statistical model is judged by
comparing the observed data with the values predicted by the estimated
model.
If the fitted model is appropriate, the residuals should behave in a
similar way as the true innovations of the process.
Thus, we should plot the residuals of the estimated model to check whether
they verify the main assumptions: they are zero mean, homoskedastic (with
constant variance) and with no significant correlations. If the graph shows a
cyclic or trended component or a non- constant variance, it can be
interpreted as a sign of misspecification.
The autocorrelation function of these residuals should not display any
significant correlations. Let {et} be the sequence of residuals, given byet=Xt Xt
where Xtare the fitted values.
37/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
39/62
Then, the autocorrelation function is given by
e(h) =
Tht=1(et e)(et+h e)T
t=1(et e)2, h= 1, 2,...
Assume first that the true parameter values were known. Then,et=t. In
this case, we know (see Chapter 1) that Ted
N(0,IH), where = ((1),...,(H))and IHis theHHidentity matrix, then the nullhypothesis that the firstHautocorrelations are non significant can be tested
using the Box-PierceQ statistic:
T
Hi=1
2(i) d
2H (10)
38/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
40/62
However, in practice the values of the parameters are unknown and they
should be estimated. The fact that the parameters employed to computeetare not the true ones but are estimates has an impact on the asymptotic
distribution.
More specifically, the asymptotic variance of the sample autocorrelations isnot the identity matrix anymore. Hence, in this case (10) no longer holds.
However, under certain assumptions it is still possible to use the sample
autocorrelations for diagnosis of the model. The following proposition
presents the distribution of the sample autocorrelations of the sample
residuals, when the parameters of the model are estimated.
39/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
41/62
PROPOSITION
Suppose that Yt=Xt+t, t= 1,..., T, where Ytandtare scalar and Xt
andare k 1vectors. If{Yt,Xt} are jointly stationary and ergodic,E(XtXt)is a full rank matrix,
E(t|t1, t2, ...,Xt,Xt1,...) =0
and
E
2t|t1, t2,...,Xt,Xt1,...
= 2 >0
then
Te
d
N(0,IH
Q)
where qjk(the jk element of the HH matrix Q) is given by
E(Xttj)E(XtX
t)1
E(Xttk) /2
Proof. see Hayashi, p165
40/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
42/62
Further, ifXtis anARMA(p, q)process and the model is correctly specified,it can be shown that the Box Pierce Q -statistic for diagnosting checking
when the parameters are estimated is approximately distributed as
Qe=T
Hi=1
2e (i) d 2(Hpq)
The adequacy of the model is rejected at level if
Qe=Ti=1
H2e (i)> 21(Hpq)
Finally, Ljung and Box (1978) suggestreplacing Qeby
Qe= n(n+2)Hi=12e (i)
n isince they claim that the statistics offers a better approximation to the2
distribution.
41/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
43/62
1 INTRODUCTION
2 ESTIMATION OF
ARMA(p, q) PROCESSES
3 ASYMPTOTIC
PROPERTIES OF ML
ESTIMATORS AND
INFERENCE
4
IDENTIFICATION OF AN
ARMA PROCESS:
SELECTINGp ANDq
5 DIAGNOSTIC CHECKING
6 FORECASTING
7 MODEL IDENTIFICATION
IN THE GENERAL CASE
8 R CODE
9 REFERENCES
42/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
44/62
Ah-step ahead forecast,xt+h|t, is designed to minimize expected loss,conditional on timetinformation. Examples of widely-used lossfunctions:
MSE:xt+h xt+h|t
2
MAD:xt+h xt+h|t
Quad-Quad: 1xt+h xt+h|t
2+2I[xt+hxt+h
-
8/10/2019 ARMA Estimation
45/62
Letxt+h =Et[xt+h]
Letxt+h be any other value
Et
(xt+h xt+h)2
= Et
xt+h xt+h
+xt+h xt+h
2=Etxt+h x
t+h
2+2 xt+h x
t+h xt+h xt+h+ x
t+h
xt+h
2
=Vt[xt+h] +2Et
xt+h xt+h
xt+h xt+h
+Et
xt+h xt+h
2=Vt[xt+h] +2
xt+h xt+h
Et
xt+h xt+h
+Et
xt+h xt+h
2=Vt[xt+h] +2 xt+h xt+h .0+Etx
t+h
xt+h2
=Vt[xt+h] + (xt+h xt+h)2
44/60
OUTLINE
INTRODUCTION
ESTIMATION
ASYMPTOTIC PROPERTIES
IDENTIFICATION
DIAGNOSTIC CHECKING
FORECASTING
MODEL ID
THE M SE OPTIMAL FORECAST IS THE CONDITIONAL MEAN
-
8/10/2019 ARMA Estimation
46/62
MSE optimal forecast for anAR(1):
xt=1xt1+t
Et[xt+1] =Et[1xt+t+1] =1Et[xt] +Et[t+1] =1xt+ 0
Et[xt+2] =Et[1xt+1t+2] =1Et[xt+1] +Et[t+2] =1(1xt) +0= 21xt+ 0
45/60
OUTLINE
INTRODUCTION
ESTIMATION
ASYMPTOTIC PROPERTIES
IDENTIFICATION
DIAGNOSTIC CHECKING
FORECASTING
MODEL ID
FORECAST EVALUATION
-
8/10/2019 ARMA Estimation
47/62
Two primary criteria to evaluate forecasts:
Objective
Relative
Objective: Mincer Zarnowitz regressions
xt+h =+xt+h|t+t
H0 : = 0, =1;H1 : =0 =1Use any test: Wald, LR, LM
Can be generalized to include any variable available when the forecast
was produced
xt+h =+xt+h|t+zt+t
46/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
FORECAST EVALUATION
-
8/10/2019 ARMA Estimation
48/62
H0 : = 0, =1, =0;H1 : =0 =1 j=0
ztmust be in the timetinformation set
Important when working with macro data
47/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
RELATIVE EVALUATION:D IEBOLD-M ARIANO
-
8/10/2019 ARMA Estimation
49/62
Two forecasts,xAt+h|tandxBt+h|t
Two losses
lAt = (yt+h yAt+h|t)2 andlBt = (yt+h yBt+h|t)2
Losses do not need to be MSE
if equally good or bad,E[lAt] =E[lBt] or E[l
AtElBt] =0
Define=lA
t lB
t
48/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
RELATIVE EVALUATION:D IEBOLD-M ARIANO
-
8/10/2019 ARMA Estimation
50/62
Implemented as at-test thatE[t] =0
H0 :
E[t] =
0;HA
1 :E
[t]0
Composite alternative
Sign indicates which model is favored
DM=
V[
]
One complication: {t} cannot be assumed to be uncorrelated, so amore complicated variance estimator is required
Newey-West covariance estimator:
2 = 0+2
Ll=1
1 l
L+1
l
49/60
-
8/10/2019 ARMA Estimation
51/62
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
52/62
1 INTRODUCTION
2 ESTIMATION OF
ARMA(p, q) PROCESSES
3 ASYMPTOTIC
PROPERTIES OF ML
ESTIMATORS AND
INFERENCE
4 IDENTIFICATION OF AN
ARMA PROCESS:
SELECTINGp ANDq
5 DIAGNOSTIC CHECKING
6 FORECASTING
7 MODEL IDENTIFICATION
IN THE GENERAL CASE
8 R CODE
9 REFERENCES
51/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
53/62
Identifying a model forytrefers to the methodology for selecting
The appropriate transformations for obtaining
stationarity (such as variance stabilization
transformations and differencing)
values forp, qandd(the integration order)
deciding whether a deterministic component shouldor should not be included in the model.
The followings steps should be followed to identify the model:
52/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
54/62
Step 1. Plot the data and choose the proper transformations.
The plot usually shows whether the data contains a trend, a seasonal
component, outliers, non constant variances, etc. then, it might suggest
some proper transformations.
The most commonly used transformations are variance stabilizing
transformations, typically, a logarithmic transformations or, more
generally, a Box Cox transformation, and/or differencing.
Always apply the variance stabilizing transformations before taking any
number differences.
Step 2. Compute and examine the sample ACF and sample PACF of the
(variance-stabilized) process.
If these functions decay very slowly, this would be a sign of non stationarity.
Unit root tests should also be used at this stage. Typically, the number of
differences would be 0, 1 or 2.
53/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
55/62
Step 3. Compute the ACF and PACF of the transformed variable to identify
pandq.
Identifying the order of sampleAR or MA polynomials is, in theory, easy
with the table above. However, it is more difficult to identify the orders of an
ARMAprocess. In these cases, other model selection mechanisms, such as
information criteria (see below) can be implemented.
Step 4. Test the deterministic trend term whend>0.
Ifd>0 and a trend in the data not suspected should not be included.However, if there is a reason to believe that a trend should be include, one
can include this term in the model and the, discard it if the coefficient is not
significant.
For some interesting exemples, see Wei, Chapter 6 and Brockwell and
Davies, Chapter 9.
54/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
ESTIMATION
-
8/10/2019 ARMA Estimation
56/62
Once the appropriate transformations have been performed, the resulting
process is stationary (basically, variance stabilizing transformations and/or
differencing)
At this stage, the estimation techniques developed for the stationaryARMA
case are applicable to the resulting stationary process.
55/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
57/62
1 INTRODUCTION
2 ESTIMATION OF
ARMA(p, q) PROCESSES
3 ASYMPTOTIC
PROPERTIES OF MLESTIMATORS AND
INFERENCE
4 IDENTIFICATION OF AN
ARMA PROCESS:
SELECTINGp ANDq
5 DIAGNOSTIC CHECKING
6 FORECASTING
7 MODEL IDENTIFICATION
IN THE GENERAL CASE
8 R CODE
9 REFERENCES
56/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
Download theR Library "Time Series"
-
8/10/2019 ARMA Estimation
58/62
A first step in analysing time series is to examine the autocorrelations (ACF)
and partial autocorrelations (PACF).R provides rhe functionsacf()and
pacf()for computing and plotting ofACFandPACF
sim.ar
-
8/10/2019 ARMA Estimation
59/62
Diagnostic Checking
A first step in diagnostic checking of fitted models is to analyze the residuals
from the fit for any signs of randomness. Rhas the functiontsdiag(), whichproduces a diagnostic plot of fitted time series model
tsdiag(fit)
it produces output containing a plot of the residuals, the autocorrelation of
the residual and thep- values of the Ljung-Box statistic for the first 10 lags.
The functionBox.test()computes the test statistic for a given lag
Box-test(fit$residuals, lag=1)
58/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
-
8/10/2019 ARMA Estimation
60/62
Prediction of ARMA-Models
predict( )can be used for predicting future values of the levels under themodel
AR.pred
-
8/10/2019 ARMA Estimation
61/62
1 INTRODUCTION
2 ESTIMATION OF
ARMA(p, q) PROCESSES
3 ASYMPTOTIC
PROPERTIES OF MLESTIMATORS AND
INFERENCE
4 IDENTIFICATION OF AN
ARMA PROCESS:
SELECTINGp ANDq
5 DIAGNOSTIC CHECKING
6 FORECASTING
7 MODEL IDENTIFICATION
IN THE GENERAL CASE
8 R CODE
9 REFERENCES
60/60
OUTLINE INTRODUCTION ESTIMATION ASYMPTOTIC PROPERTIES IDENTIFICATION DIAGNOSTIC CHECKING FORECASTING MODEL ID
Brockwell and Davies, (1991), Chapters 8, 9
-
8/10/2019 ARMA Estimation
62/62
Hamilton, 1994, Chapter 5
Wei, Chapters 6,7.
60/60