econometrics regression analysis with time series …docentes.fe.unl.pt/~azevedoj/web...

Serial Correlation Heteroskedasticity Example Heteroskedasticity and Serial Correlation

EconometricsRegression Analysis with Time Series Data:

Serial Correlation and Heteroskedasticity

Joao Valle e Azevedo

Faculdade de EconomiaUniversidade Nova de Lisboa

Spring Semester

Joao Valle e Azevedo (FEUNL) Econometrics Lisbon, May 2011 1 / 34


Time Series Analysis

Serially Correlated Errors: Consequences

With assumptions TS.1 through TS.3, OLS estimators are unbiased

With assumptions TS.1’ through TS.3’, OLS estimators areconsistent

Could have serial correlation in the errors (TS.5 or TS.5’ could beviolated)

But for inference, results are NOT valid if TS.5 (or TS.5’) fail

With serial correlation in the errors, usual OLS variances are NOTvalid




Serially Correlated Errors

But notice, can have consistency even in the presence of serial correlationin the errors and lagged dependent variables as regressors

Example:

yt = β0 + β1yt−1 + utI where E [ut |yt−1] = 0 and {ut} are serially correlated

However, if

yt = β0 + β1yt−1 + ut and ut = ρut−1 + et , t = 2, ..., n

et are i.i.d., |ρ| < 1 and E [et |ut−1, ut−2, ...] = E [et |yt−1, yt−2, ...] = 0

Then,

Cov(yt−1, ut) = E [yt−1(ρut−1 + et)] = ρE (yt−1, ut−1)

= ρE [yt−1(yt−1 − β0 − β1yt−2)] 6= 0 unless ρ = 0




Serially Correlated Errors (Cont.)

In this case the OLS estimators are not consistent for β0,β1. This is aspecial (although typical) form of autocorrelation

But in the previous case, it turns out that:

yt = β0 + β1yt−1 + ρ(yt−1 − β0 − β1yt−2) + et

= β0(1− ρ) + (β1 + ρ)yt−1 − ρβ1yt−2 + et

= α0 + α1yt−1 + α2yt−2 + et

I where E [et |yt−1, yt−2, ...] = 0 and

E [yt |yt−1, yt−2, ...] = E [yt |yt−1, yt−2] = α0 + α1yt−1 + α2yt−2

Thus, the ”relevant” model is an AR(2) model for y. With furtherconditions on the parameters (that ensure stability) we can estimatethe α′js consistently




Serially Correlated Errors (Cont.)

Conclusion: Models with lagged dependent variables and serialcorrelation in the errors can often be easily transformed into modelswithout serial correlation in the errors (e.g., by adding more lags ofthe dependent variable as regressors)




Testing for AR(1) Serial Correlation (Cont.)

Want to be able to test whether the errors are serially correlated ornot

yt = β0 + β1xt1 + ...+ βkxtk + ut

Want to test the null that ρ = 0 in ut = ρut−1 + et ,t=2,...,nI where ut is the model error term, |ρ| < 1 and et is an i.i.d. sequence

with E [et |ut−1, ut−2, ...] = 0 and Var [et |ut−1] = σ2e

With strictly exogenous regressors, E (ut |X) = 0, t = 1, 2, ..., n, thetest is straightforward

I Obtain the OLS residuals of the original modelI Then, regress the residuals on (one period) lagged residuals (use OLS)I Then, use a typical t-test for H0 : ρ = 0 (don’t need intercept)I We assume TS.1 through TS.4 hold. If TS.4 fails use a

heteroskedasticity robust statistic

Detects correlation between ut and ut−1 but not between ut and ut−2




Testing for AR(1) Serial Correlation (Cont.)

An alternative is the Durbin-Watson (DW) statistic, which iscalculated by most Econometric packages. Need TS.1 through TS.6holding

DW =n∑

t=2

(ut − ut−1)2/

n∑t=1

u2t ≈ 2(1− ρ)

where ρ is that of the previous test

If the DW statistic is around 2, then we do not reject H0 (absence ofserial correlation of AR(1) type), while if it is significantly < 2 wereject the null (against the alternative H1 : ρ > 0, the most typical)




Testing for AR(q) Serial Correlation

If the regressors are not strictly exogenous, previous t or DW test willNOT be valid. Have alternatives under TS.1’ through TS.4’ (thatare valid also with TS.1 through TS.4). Also, can have more generalforms of serial correlation (e.g., AR(q) serial correlation)

yt = β0 + β1xt1 + ...+ βkxtk + ut

I where ut = ρ1ut−1 + ρ2ut−2 + ...+ ρqut−q + etI et is an i.i.d. sequence with E [et |ut−1, ut−2, ...] = 0 and

Var [et |xt , ut−1, ut−2, ..., ut−q] = σ2e

I The null is H0 : ρ1 = ρ2 = ... = ρq = 0

Regress the OLS residuals (or y) on the q lagged residuals and all ofthe x ’s (this makes the test valid without assuming strict exogeneity.Don’t need to include the x ’s if strict exogeneity is assumed)Then use a usual test of multiple exclusion restrictions (on thecoefficients of the lagged residuals)




Testing for AR(q) Serial Correlation (Cont.)

Can use F test or LM test (or even t test if q=1), where the LMversion is called the Breusch-Godfrey test: LM = (n − q)R2 ∼ χ2

q

(approx.) under the null where R2 is from the residuals regression.

If there is heteroskedasticity in the error term ut , can useheteroskedasticity-robust statistics




Correcting Serial Correlation with strictly exogenousregressors

If there is serial correlation in the errors, inference ”as usual” withOLS is Not valid

Let us assume TS.1 through TS.4 hold in our model, but TS.5 (noserial correlation) fails to hold. Also, assume stationarity and weakdependence as in TS.1’

Assume errors follow an AR(1) so ut = ρut−1 + et , t = 2, ..., n, where|ρ| < 1 and et is an i.i.d. sequence with E [et |X ] = 0 andVar [et |X ] = σ2

e

I Var [ut |X ] = σ2e/(1− ρ2)

We can transform the equation so we have no serial correlation in theerrors




Correcting Serial Correlation with strictly exogenousregressors (Cont.)

Consider the case of one regressor (similar analysis with more)

yt = β0 + β1xt + ut

Then,

ρyt−1 = ρβ0 + ρβ1xt−1 + ρut−1

Subtract this from previous equation to get:

yt − ρyt−1 = (1− ρ)β0 + β1(xt − ρxt−1) + et

I since et = ut − ρut−1 for t ≥ 2

In this quasi-differenced model for t ≥ 2, TS.1 through TS.5 hold!




GLS Estimationyt − ρyt−1 = (1− ρ)β0 + β1(xt − ρxt−1) + et for t ≥ 2

Can transform equation for t=1, y1 = β0 + β1x1 + u1, so that TS.1through TS.5 hold for t ≥ 1 (not yet the case since u1 and et have adifferent variance)

Var(ut |X ) = σ2e/(1− ρ2), so can multiply equation by (1− ρ2)1/2 to

have TS.1 through TS.5 holding for t ≥ 1

(1− ρ2)1/2y1 = (1− ρ2)1/2β0 + (1− ρ2)1/2β1x1 + (1− ρ2)1/2u1

In this quasi-differenced model, TS.1 through TS.5 hold!




GLS Estimation (Cont.)

Transformed model is:

yt = β0xt0 + β1xt1 + et , t ≥ 1

yt = yt − ρyt−1, t ≥ 2 and (1− ρ2)1/2y1 for t = 1

xt1 = xt − ρxt−1, t ≥ 2 and (1− ρ2)1/2x1 for t = 1

xt0 = (1− ρ), t ≥ 2 and (1− ρ2)1/2 for t = 1




GLS Estimation (Cont.)

If ρ is known, can estimate the transformed regression by OLS

This is called the GLS estimator of the original model

GLS is BLUE if TS.1 through TS.5 hold in the transformed model

Can use t and F tests from the transformed equation to conductinference on the parameters of the original equation

These tests are valid (asymptotically) if TS.1 through TS.5 hold inthe transformed model (along with stationarity and weak dependencein the original variables) and distributions (conditional on X) areexact (and with minimum variance) if TS.6 holds for the et




Feasible GLS

Problem: don’t know ρ, need to get an estimate first

I Run OLS on the original model and then regress residuals on laggedresiduals (with OLS)

I Use OLS on the transformed regression (using the estimated ρ). This isa feasible GLS (FGLS) estimator (or Prais-Winsten estimator)

I If we just forget the first equation (t=1) we have theCochrane-Orcutt estimator (also a FGLS estimator)

I These FGLS estimators are not unbiased, but are consistent if TS.1through TS.5 hold in the transformed model (along with stationarityand weak dependence in the original model)

I There is no difference, asymptotically, between the two procedures




Feasible GLS

t and F tests from the transformed equations are valid(asymptotically). If TS.6 holds for the et , relevant distributions alsohold only asymptotically. FGLS is asymptotically more efficient thanOLS

This basic method can be extended to allow for higher order serialcorrelation, AR(q), in the error term. Econometrics packages dealautomatically with estimation of AR serial correlation




Serial Correlation-Robust Standard Errors

FGLS asymptotic distributions rely on strict exogeneity of theregressors

If strict exogeneity does not hold, we can still use OLS (it isconsistent with only contemporaneous exogeneity, or TS.1’, alongwith TS.2’ and TS.3’). Further, we can calculate serialcorrelation-robust standard errors

Idea is to scale the OLS standard errors to take into account serialcorrelation

Actually, with only TS.1’ through TS.3’ we can deriveHeteroskedasticity and Serial Correlation robust standard errorsand corresponding t-statistics




Serial Correlation-Robust Standard Errors (Cont.)

Estimate the model with OLS to get the residuals ut , the standarddeviation of the regression, σ, and the ”usual” standard errors”se(β1)”Run the auxiliary regression of xt1 on xt2, ..., xtk , get the residuals, rt ,and form at = rt utChoose a g - typically the integer part of n1/4

Compute

ν =n∑

t=1

a2t + 2

g∑h=1

[1− h/(g + 1)]

( n∑t=h+1

at at−h

)Then,

Robust se(β1) = [”se(β1)”/σ]2√ν

I and similarly for any βj




Serial Correlation-Robust Standard Errors (Cont.)

These Heteroskedasticity and Serial Correlation robust standarderrors can be poorly behaved in small samples (typical in Macro timeseries) in the presence of ”heavy” serial correlation

Also, standard errors can be very large (OLS is far from efficiencywith ”heavy” serial correlation and small samples)

If you believe the regressors are strictly exogenous and errors followan AR(1) (or more generally an AR(q)), use FGLS

Even if you think the errors do not follow an AR(1) (but regressors arestill strictly exogenous) apply FGLS (as if errors followed an AR(1))and compute robust standard errors in the quasi-differenced model

So, use only Heteroskedasticity and Serial Correlation robuststandard errors if strict exogeneity fails

Will not deal with F statistics robust to serial correlation




Heteroskedasticity in Time Series Regressions

With assumptions TS.1 through TS.3, OLS estimators are unbiased(no role of homoskedasticity, TS.4)

With assumptions TS.1’ through TS.3’, OLS estimators areconsistent (no role of homoskedasticity, TS.4’)

But for inference, ”usual” OLS results are NOT valid if TS.4 (orTS.4’) fail

With assumptions TS.1’ through TS.3’, we can deriveHeteroskedasticity and Serial Correlation robust standard errors

We can do better than this if TS.1’, TS.2’, TS.3’ and TS.5’ hold(need only heteroskedasticity robust standard errors)

Can compute heteroskedasticity robust standard errors and userobust t and LM statistics, exactly as in the cross sectional case




Testing for Heteroskedasticity in Time Series Regressions

However, in small samples we know these robust standard errors canbe large. Therefore, we may want to test for heteroskedasticity

It turns out that we can also use the tests for heteroskedasticity ofthe cross-sectional case (Chapter 8)

But for this, need TS.1’, TS.2’, TS.3’ and TS.5’ holding. So, cannothave serial correlation in the errors

Also, e.g., for the Breusch-Pagan test where we specifyu2 = δ0 + δ1x1 + ...+ δkxk + ν and test H0 : δ1 = δ2 = ... = δk = 0,we need ν to be homoskedastic and serially uncorrelated (which isdifferent from serial correlation in the u’s)

Later, we will see how to deal with dynamic heteroskedasticity (wheresomething like the ν’s above can be correlated across time)




Testing for Heteroskedasticity in Time Series Regressions

So, if we find heteroskedasticity can use heteroskedasticity robuststatistics. If we know the form of heteroskedasticity can also useWeighted Least Squares (WLS), along the same lines of thecross-sectional case. But still, TS.5’ (No serial Correlation) must bein place!




Autoregressive Conditional heteroskedasticity

Now assume TS.1 through TS.5 hold, so we have Conditional (onX!) Homoskedasticity and no Serial Correlation in the Error term

Let yt = β0 + β1xt1 + ...+ βkxtk + ut

We can have E [u2t |X] = Var(ut |X) = Var(ut) = σ2 for t=1,2,...,n but

still:

E [u2t |X, ut−1, ut−2, ...] = E [u2

t |X, ut−1] = α0 + α1u2t−1 or

u2t = α0 + α1u

2t−1 + νt where E [ν|X, ut−1, ut−2, ...] = 0

I with α0 > 0 and α1 ≥ 0




Autoregressive Conditional heteroskedasticity (Cont.)

νt are not independent of past u since νt ≥ −α0 − α1u2t−1 (variance

must be positive)

Also α1 < 1. If this model holds, we have Autoregressive Conditionalheteroskedasticity (ARCH) in the errors

So, even if the errors are not correlated (TS.5 holds), its squares canbe correlated

OLS is still BLUE with ARCH errors. Usual OLS inference is valid ifTS.6 (Normality) holds

Even if Normality does not hold we know that usual OLS inference isvalid asymptotically with TS.1’ through TS.5’ (so can have ARCHerrors in this setting as well)




Autoregressive Conditional heteroskedasticity (Cont.)

So, why do we care about ARCH errors?

Because we can obtain asymptotically more efficient estimators thanOLS (albeit non-linear), with a sort of WLS obtained by firstestimating u2

t = α0 + α1u2t−1 + νt (by OLS, using squared residuals).

No more details in this course

ARCH behavior is interesting in itself, can analyze behavior ofvolatility




Example - BVLG index, Lisbon




Example - BVLG index, Lisbon (Cont.)

BVLG Lisbon daily returns, January 2, 1995 to November 23, 2001

Just take the index

Apply logs

Take first differences





Under the efficient market hypothesis, stock returns should beuncorrelated with past returns. Also, should not have calendareffects...But this is (was?) Portugal

I Dependent variable: Daily ReturnsI Regressors: Day of the week dummies, Tuesday* for Tuesdays from Jan 88 to Apr

89 and three lags of returnsI F test is for the hypothesis that all Day of the week dummies are equalI *, ** and *** denote significance at the 10%, 5% and 1% respectively





Dependent variable: Daily Returns

Regressors: Day of the week dummies, Tuesday* for Tuesdays from Jan 88 to Apr 89 andthree lags of returns

F test is for the hypothesis that all Day of the week dummies are equal

*, ** and *** denote significance at the 10%, 5% and 1% respectively





Besides, there is volatility clustering in the returns series





White Heteroskedasticity Test:

F-statistic 63.32189 Probability 0.000000

Obs*R-squared 112.7329 Probability 0.000000

Test Equation:

Dependent Variable: RESID^2

Method: Least Squares

Sample: 3 1004

Included observations: 1002

Variable Coefficient Std. Error t-Statistic Prob.

C 0.000144 1.82E-05 7.933013 0.0000

Y(-1) -0.000222 0.001125 -0.197479 0.8435

Y(-1)^2 0.299471 0.026700 11.21598 0.0000

R-squared 0.112508 Mean dependent var 0.000213

Adjusted R-squared 0.110731 S.D. dependent var 0.000573

S.E. of regression 0.000541 Akaike info criterion -12.20501

Sum squared resid 0.000292 Schwarz criterion -12.19031

Log likelihood 6117.708 F-statistic 63.32189

Durbin-Watson stat 2.075790 Prob(F-statistic) 0.000000y denotes returns

Assume there are no calendar effect but y , y(−1), are correlated, yt = β0 + β1yt−1 + utFind evidence of heteroskedasticity

But if we accept that TS.4’ holds, the significant coefficient on y(−1)2 can be due toARCH effects





ARCH effects

F-statistic 114.8229 Probability 0.000000

Obs*R-squared 103.1921 Probability 0.000000

Test Equation:

Dependent Variable: RESID^2

Method: Least Squares

Sample(adjusted): 4 1004

Included observations: 1001 after adjusting endpoints

Variable Coefficient Std. Error t-Statistic Prob.

C 0.000145 1.83E-05 7.903650 0.0000

RESID^2(-1) 0.321081 0.029964 10.71555 0.0000

R-squared 0.103089 Mean dependent var 0.000213

Adjusted R-squared 0.102191 S.D. dependent var 0.000573

S.E. of regression 0.000543 Akaike info criterion -12.19544

Sum squared resid 0.000295 Schwarz criterion -12.18564

Log likelihood 6105.819 F-statistic 114.8229

Durbin-Watson stat 2.064939 Prob(F-statistic) 0.000000

Find evidence of ARCH effects

Remember, can have serial correlation in the squared errors even if there is no serialcorrelation in the errors




Heteroskedasticity and Serial Correlation

Can have violation of TS.4 (Homoskedasticity) and TS.5 (No serialcorrelation) simultaneously. Assume still that TS.1 through TS.3 hold(along with stationarity and weak dependence)

yt = β0 + β1xt1 + ...+ βkxtk + ut

ut = νt√

ht

νt = ρνt−1 + et

I where X are independent of et , ht is a function of the regressorsI Also the process {et} has mean zero, constant variance and is serially

uncorrelated

yt√ht

=β0√ht

+β1xt1√

ht+ ...+

βkxtk√ht

+ut√ht




Heteroskedasticity and Serial Correlation (Cont.)

We can estimate the function h exactly as in chapter 8

This transformed equation has still serial correlation but noheteroskedasticity

Use the estimated h in the equation above and applyCochrane-Orcutt or Prais-Winsten

This leads to a feasible GLS estimator that is asymptotically efficient.Test statistics from Cochrane-Orcutt or Prais-Winsten areasymptotically valid


econometrics regression analysis with time series …docentes.fe.unl.pt/~azevedoj/web...

Documents