![Page 1: Regression Analysis. Introduction Derive the α and β Assess the use of the T-statistic Discuss the importance of the Gauss- Markov assumptions Describe](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515dff9550346dd6f8b4c9f/html5/thumbnails/1.jpg)
Regression Analysis
![Page 2: Regression Analysis. Introduction Derive the α and β Assess the use of the T-statistic Discuss the importance of the Gauss- Markov assumptions Describe](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515dff9550346dd6f8b4c9f/html5/thumbnails/2.jpg)
Introduction
• Derive the α and β
• Assess the use of the T-statistic
• Discuss the importance of the Gauss-Markov assumptions
• Describe the problems associated with autocorrelation, how to measure it and possible remedies
• Introduce the problem of heteroskedasicity
![Page 3: Regression Analysis. Introduction Derive the α and β Assess the use of the T-statistic Discuss the importance of the Gauss- Markov assumptions Describe](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515dff9550346dd6f8b4c9f/html5/thumbnails/3.jpg)
Values and Fitted Values
termerrortheise
valuefittedaisywhere
xye
yye
xy
i
i
iii
iii
ii
ˆ
ˆ
ˆ
![Page 4: Regression Analysis. Introduction Derive the α and β Assess the use of the T-statistic Discuss the importance of the Gauss- Markov assumptions Describe](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515dff9550346dd6f8b4c9f/html5/thumbnails/4.jpg)
Deriving the α and β
• The aim of a least squares regression is to minimize the distance between the regression line and error terms (e).
• and ... )ˆˆ( 2
11
2 trwxyeSMin t
n
tt
n
tt
![Page 5: Regression Analysis. Introduction Derive the α and β Assess the use of the T-statistic Discuss the importance of the Gauss- Markov assumptions Describe](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515dff9550346dd6f8b4c9f/html5/thumbnails/5.jpg)
The Constant
xy
xy
ˆˆ
ˆˆ
![Page 6: Regression Analysis. Introduction Derive the α and β Assess the use of the T-statistic Discuss the importance of the Gauss- Markov assumptions Describe](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515dff9550346dd6f8b4c9f/html5/thumbnails/6.jpg)
The Slope Coefficient (β)
n
tt
n
ttt
xx
xxyy
1
2
1
)(
))((
)(
)(ˆxVar
yxCov
![Page 7: Regression Analysis. Introduction Derive the α and β Assess the use of the T-statistic Discuss the importance of the Gauss- Markov assumptions Describe](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515dff9550346dd6f8b4c9f/html5/thumbnails/7.jpg)
T-test
• When conducting a t-test, we can use either a 1 or 2 tailed test, depending on the hypothesis
• We usually use a 2 tailed test, in this case our alternative hypothesis is that our variable does not equal 0. In a one tailed test we would stipulate whether it was greater than or less than 0.
• Thus the critical value for a 2 tailed test at the 5% level of significance is the same as the critical value for a 1 tailed test at the 2.5% level of significance.
![Page 8: Regression Analysis. Introduction Derive the α and β Assess the use of the T-statistic Discuss the importance of the Gauss- Markov assumptions Describe](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515dff9550346dd6f8b4c9f/html5/thumbnails/8.jpg)
T-test
• We can also test whether our coefficient equals 1.
)ˆ(
1ˆ
SE
t
![Page 9: Regression Analysis. Introduction Derive the α and β Assess the use of the T-statistic Discuss the importance of the Gauss- Markov assumptions Describe](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515dff9550346dd6f8b4c9f/html5/thumbnails/9.jpg)
Gauss-Markov Assumptions
• There are 4 assumptions relating to the error term.
• The first is that the expected value of the error term is zero
• The second is that the error terms are not correlated
• The third is that the error term has a constant variance
• The fourth is that the error term and explanatory variable are not correlated.
![Page 10: Regression Analysis. Introduction Derive the α and β Assess the use of the T-statistic Discuss the importance of the Gauss- Markov assumptions Describe](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515dff9550346dd6f8b4c9f/html5/thumbnails/10.jpg)
Gauss-Markov assumptions
• More formally we can write them as:
0),(
)()(
0),(
0)(
22
1
tt
tt
tt
t
xuCov
uEuVar
uuCov
uE
![Page 11: Regression Analysis. Introduction Derive the α and β Assess the use of the T-statistic Discuss the importance of the Gauss- Markov assumptions Describe](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515dff9550346dd6f8b4c9f/html5/thumbnails/11.jpg)
Additional Assumptions
• There are a number of additional assumptions such as normality of the error term and n (number of observations) exceeding k (the number of parameters).
• If these assumptions hold, we say the estimator is BLUE
![Page 12: Regression Analysis. Introduction Derive the α and β Assess the use of the T-statistic Discuss the importance of the Gauss- Markov assumptions Describe](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515dff9550346dd6f8b4c9f/html5/thumbnails/12.jpg)
BLUE
• Best or minimum variance
• Linear or straight line
• Unbiased or the estimator is accurate on average over a large number of samples.
• Estimator
![Page 13: Regression Analysis. Introduction Derive the α and β Assess the use of the T-statistic Discuss the importance of the Gauss- Markov assumptions Describe](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515dff9550346dd6f8b4c9f/html5/thumbnails/13.jpg)
Consequences of BLUE
• If the estimator is not BLUE, there are serious implications for the regression, in particular we can not rely on the t-tests.
• In this case we need to find a remedy for the problem.
![Page 14: Regression Analysis. Introduction Derive the α and β Assess the use of the T-statistic Discuss the importance of the Gauss- Markov assumptions Describe](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515dff9550346dd6f8b4c9f/html5/thumbnails/14.jpg)
Autocorrelation
• Autocorrelation occurs when the second Gauss-Markov assumption fails.
• It is often caused by an omitted variable
• In the presence of autocorrelation the estimator is not longer Best, although it is still unbiased. Therefore the estimator is not BLUE.
![Page 15: Regression Analysis. Introduction Derive the α and β Assess the use of the T-statistic Discuss the importance of the Gauss- Markov assumptions Describe](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515dff9550346dd6f8b4c9f/html5/thumbnails/15.jpg)
Durbin-Watson Test
• This tests for 1st order autocorrelation only• In this case the autocorrelation follows the
first-order autoregressive process
ttt uu 1
![Page 16: Regression Analysis. Introduction Derive the α and β Assess the use of the T-statistic Discuss the importance of the Gauss- Markov assumptions Describe](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515dff9550346dd6f8b4c9f/html5/thumbnails/16.jpg)
Durbin-Watson Test- decision framework
0 dl du 2 44-du b-dl
Zone of indecision
Zone of indecision
![Page 17: Regression Analysis. Introduction Derive the α and β Assess the use of the T-statistic Discuss the importance of the Gauss- Markov assumptions Describe](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515dff9550346dd6f8b4c9f/html5/thumbnails/17.jpg)
DW Statistic
• The DW test statistic lies between 0 and 4, if it lies below the dl point, we have positive autocorrelation. If it lies between du and 4-du, we have no autocorrelation and if above 4-dl we have negative autocorrelation.
• The dl and du value can be found in the DW d-statistic tables (at the back of most text books)
![Page 18: Regression Analysis. Introduction Derive the α and β Assess the use of the T-statistic Discuss the importance of the Gauss- Markov assumptions Describe](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515dff9550346dd6f8b4c9f/html5/thumbnails/18.jpg)
Lagrange Multiplier (LM) Statistic
• Tests for higher order autocorrelation• The test involves estimating the model and
obtaining the error term .• Then run a second regression of the error term on
lags of itself and the explanatory variable: (the number of lags depends on the order of the autocorrelation, i.e. second order)
ttttt xuuu 322110
![Page 19: Regression Analysis. Introduction Derive the α and β Assess the use of the T-statistic Discuss the importance of the Gauss- Markov assumptions Describe](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515dff9550346dd6f8b4c9f/html5/thumbnails/19.jpg)
LM Test
• The test statistic is the number of observations multiplied by the R-squared statistic.
• It follows a chi-squared distribution, the degrees of freedom are equal to the order of autocorrelation tested for (2 in this case)
• The null hypothesis is no autocorrelation, if the test statistic exceeds the critical value, reject the null and therefore we have autocorrelation.
![Page 20: Regression Analysis. Introduction Derive the α and β Assess the use of the T-statistic Discuss the importance of the Gauss- Markov assumptions Describe](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515dff9550346dd6f8b4c9f/html5/thumbnails/20.jpg)
Remedies for Autocorrelation
• There are 2 main remedies:
• The Cochrane-Orcutt iterative process
• An unrestricted version of the above process
![Page 21: Regression Analysis. Introduction Derive the α and β Assess the use of the T-statistic Discuss the importance of the Gauss- Markov assumptions Describe](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515dff9550346dd6f8b4c9f/html5/thumbnails/21.jpg)
Heteroskedasticity
• This occurs when the variance of the error term is not constant
• Again the estimator is not BLUE, although it is still unbisased it is no longer Best
• It often occurs when the values of the variables vary substantially in different observations, i.e. GDP in Cuba and the USA.
![Page 22: Regression Analysis. Introduction Derive the α and β Assess the use of the T-statistic Discuss the importance of the Gauss- Markov assumptions Describe](https://reader035.vdocuments.us/reader035/viewer/2022062511/5515dff9550346dd6f8b4c9f/html5/thumbnails/22.jpg)
Conclusion
• The residual or error term is the difference between the fitted value and actual value of the dependent variable.
• There are 4 Gauss-Markov assumptions, which must be satisfied if the estimator is to be BLUE
• Autocorrelation is a serious problem and needs to be remedied
• The DW statistic can be used to test for the presence of 1st order autocorrelation, the LM statistic for higher order autocorrelation.