multiple regression analysis - kasetsart universityfin.bus.ku.ac.th/01131591 financial...

I. Basic Concepts

KULKUNYA PRAYARACH, PH.D.

Multiple Regression Analysis

II. Multicollinearity IV. HeteroscedasticityIII. Autocorrelation V. Research & Group Work

1

I. Basic Concepts




OUTLINE

Basic Concept: Multiple Regression

MULTICOLLINEARITY

AUTOCORRELATION

HETEROSCEDASTICITY

REASEARCH IN FINANCE

2

I. Basic Concepts




𝑌𝑖 = 𝛽1 + 𝛽2𝑋1𝑖 + 𝛽3𝑋2𝑖 + 𝛽4𝑋3𝑖 + 𝑢𝑖

BASIC CONCEPTS: Multiple Regression

3

I. Basic Concepts




BASIC CONCEPTS: Normality Assumption for

• CLRM assumes that each is distributed normally with

𝑌𝑖 = 𝛽1 + 𝛽2𝑋1𝑖 + 𝛽3𝑋2𝑖 + 𝛽4𝑋3𝑖 + 𝑢𝑖

4

I. Basic Concepts




BASIC CONCEPTS: Why we need Normality Assumptions of

5

I. Basic Concepts




1. Influence of the omitted or neglected variables is small and at best

random Central Limit Theorem (CLT)

2. Even if the number of variables is not very large or if these variables

are not strictly independent, their sum may still be normally distributed

3. Must be normally distributed in order to make assumption of OLS

estimators , are normally distributed

4. Normal distribution is a comparatively simple distribution involving

only two parameters (mean and variance)

5. Let’s say sample < 100 , normality assumption assumes a critical

role. If the sample size is reasonably large, normality is relaxed.

6. Large samples, t and F statistics have appropriately.

TEST ‘BLUE’ Condition

BASIC CONCEPTS: Why we need Normality Assumptions of

6

I. Basic Concepts




0

20

40

60

80

100

120

140

ม.ค.-09

เม.ย.-0

9

ก.ค.-

09

ต.ค.-09

ม.ค.-10

เม.ย.-1

0

ก.ค.-

10

ต.ค.-10

ม.ค.-11

เม.ย.-1

1

ก.ค.-

11

ต.ค.-11

ม.ค.-12

เม.ย.-1

2

OIL OIL_SA

• …is statistical methods of removing the seasonal

component of a time series that is used when analyzing

non-seasonal trends

• Many economic phenomena have seasonal cycles

Seasonally Adjusted :Census X12 Method

0

20

40

60

80

100

120

140

Jan Feb Mar Apr MayJune Jul Aug Sep Oct Nov Dec

Dubai Crude Oil Price

2009 2010 2011 2012

DATA PREPARATION: Seasonally Adjusted

7

I. Basic Concepts




DATA PREPARATION: Seasonally Adjusted

8


1 : Multiple Regression

: William H. Greene, Dr. Kulkunya Prayarach

VIF (βi) = 1 / (1-R2)

If Autocorrelation

D.W. not 2, then AR(1)

(

If Multicollinearity

VIF > 10, then drop variable

(

If Heteroscedasticity (p ≤ 0.05)

Transform Regression

Yi /xi = b0\Xi, +b1

Yi/Xi2 = b0\ Xi2, +b1/Xi

Yi/ 2i = b0, +b1Xi /2

i

(

ECONOMETRIC PROBLEMS

Multicollinearity

Run: Xi = f(X1, X2,..,Xk)

Rule of Thumb: VIF ≤ 10 No Multi

VIF (i) = 1 / 1 –R2)

(

Stationary

(Unit Root Test: ADF)

H0: Non Station (unit root)

Stationary : I(0) (Reject H0), p ≤ 0.05

Non Stationary : I(1) (Fail to Reject H0) p> 0.05

Stationary Data at

I(0) or I(1)

(

First Diff D(data)

Autocorrelation

Test: Durbin Watson (D.W.) 2

No Autocorrelation

( Heteroscedasticity

Test: White Test

H0 : Homoscedasticity, p > 0.05

( Clean Econometrix Problems

GO AHEAD!!! RUN OLS

ALTERNATIVE MODELS

VAR/VECM

Granger

Causality Test

ARCH/GARCH

9

I. Basic Concepts




• …is a stochastic process whose joint probability distribution does

not change when shifted in time or space

>>> Parameters (mean, variance) will not change overtime or position

I(0)

Stationary at level

DATA PREPARATION: Stationary

10

I. Basic Concepts




Random Walk without Drift

DATA PREPARATION: Random Walk (Unit Root Process)

Random Walk with Drift

11

I. Basic Concepts




… a test of stationary (or nonstationary)

Where ut is a white noise error term.

Test Augmented Dickey-Fuller (ADF) Test for Unit Root Test

Test H0 : then UNIT ROOT (nonstationary) ~

Random walk without drift

>>> CANNOT simply regress Yt on its lagged value Yt-1

where

DATA PREPARATION: Unit Root Test

12

I. Basic Concepts




STEP 1: First Differentiate

STEP 2 : Test Unit Root again

Test H0: ~ >>> Unit root (ACCEPT)

STEP 3 : Second Differentiate

Test H0: if reject then NO Unit root

DATA PREPARATION: How to Solve Unit Root Problem

13

I. Basic Concepts




Exchange Rate

272931333537

1/1/20

09

1/4/20

09

1/7/20

09

1/10/2

009

1/1/20

10

1/4/20

10

1/7/20

10

1/10/2

010

1/1/20

11

1/4/20

11

1/7/20

11

1/10/2

011

1/1/20

12

1/4/20

12

0

20

40

60

80

100

120

140

160

1/3/20

063/2

2/200

66/8

/2006

8/24/2

006

11/9/

2006

1/30/2

007

4/18/2

007

7/5/20

079/2

0/200

75

Dec

07

19

Feb

08

5 M

ay 0

8

18

Jul 0

8

2 O

ct 0

8

17

Dec

08

3 M

ar 0

9

18

May

09

31

Jul 0

9

15

Oct

09

30

Dec

09

16

Mar

10

31

May

10

13

Au

g 1

0

28

Oct

10

12

Jan

11

29

Mar

11

13

Jun

11

26

Au

g 1

1

10

No

v 1

1

25

Jan

12

10

Ap

r 1

2

Oil Price (WTI)

14

I. Basic Concepts




DATA PREPARATION: Gaussian, Standard or Classical Linear

Regression Model (CLRM)

15

I. Basic Concepts




# of stock

Ab

no

rmal

pro

fit

%

Assumption 1:

16

I. Basic Concepts




Taylor Series ExpansionGauss-Newton iterativeNewton-Raphson iterative

Method

Nonlinear Regression

Assumption 2:

17

I. Basic Concepts




Assumption 3:

18

I. Basic Concepts




Assumption 4:

19

I. Basic Concepts




Assumption 5:

20

I. Basic Concepts




I. Conceptual Framework

III. My MappingIV. Linkages:

Internal Factor, External Factor, Shock

II. Empirical Evidence

There must be sufficient variability in the values

taken by the regressors. Assumption 6:

21

I. Basic Concepts




• X variables

Should be vary

Assumption 7:

22

I. Basic Concepts




• What is the nature of multicollinearity?

• Is Multicollinearity really a problem?

• What are its practical consequences?

• How does one detect it?

• What remedial measures can be taken to alleviate the

problem of multicollinearity?

Assumption 8:

MULTICOLLINEARITY: Is Multicollinearity seriously Problem?

23

I. Basic Concepts




MULTICOLLINEARITY: Is Multicollinearity seriously Problem?

• The Nature of Multicollinearity is the existence of a “perfect” or exact,

linear relationship among some or all explanatory variables of a

regression model

24

I. Basic Concepts




Best

Linear

Unbiased Estimator

Collinearity does

not destroy the

property of BLUE

MULTICOLLINEARITY: Consequences of Multicollinearity

25

I. Basic Concepts




1. High R2 but few significant t ratios.

Example: R2 = 0.8 but individual t tests wilshow that none or few of the partial slope coefficients are statisticallly different from zero.

2. High pair-wise correlations among regressors.

3. Examination of partial correlations

MULTICOLLINEARITY: Detecting of Multicollinearity

26

I. Basic Concepts




4. Auxiliary regression

5. Eigenvalues and condition index

if 100 < k <1000 moderate multicollinearity

k > 1000 severe multicollinearity

6. Tolerance and variance inflation factors

TOL >>> 0 or VIF > 10

MULTICOLLINEARITY: Detecting of Multicollinearity

27

I. Basic Concepts




1. Do nothing

“Multicollinearity is God’s will, not a problem with OLS or statistical techique in general” (Blanchard)

2. Rule of Thumb Procedures

(1) A priori information

(2) Combining cross-sectional and time series data

(3) Dropping variable(s) and specification bias

(4) Transformation of variables

(5) (Additional or new data) Increase a size of sample

(6) Polynomial Regression

(7) Factor analysis

MULTICOLLINEARITY: Remedial Measures

28

I. Basic Concepts




1. What is the nature of autocorrelation?

2. What are the theoretical and practical consequences of

autocorrelation?

3. How does one remedy the problem of autocorrelation?

Assumption 9:

Autocorrelation: Nature of Autocorrelation

29

I. Basic Concepts




Positive serial correlation Negative serial correlation

Zero correlation

Autocorrelation: Nature of Autocorrelation

30

I. Basic Concepts




1. Specification Bias: Excluded variables Case

2. Nonstationarity

3. Spurious problem

Autocorrelation: Types of Autocorrelation

31

I. Basic Concepts




Best

Linear

Unbiased Estimator

Autocorrelation

destroy

property of BLUE

• Autocorrelation destroys the property of BLUE due to not minimum

variance

• The residual variance is likely to underestimate

• The usual t and F tests of significance are no longer valid, and if

applied, are likely to give seriously misleading conclusions about

the statiscal signifcance of the estimated regression coefficients

Autocorrelation: Consequences of using OLS in the Presence of Autocorrelation

32

I. Basic Concepts




1. Graph Residual Plot

2. Run Test

3. Durbin-Watson Test

4. Breusch-Godfrey (BG) test ~ LM test

nonstochastic regressors, higher-order autoregressive : AR(1) , AR(2))

Autocorrelation: Detecting Autocorrelation

33

I. Basic Concepts




1. Transform the original model >>>

o Generalized least-square (GLS) Method

o Feasible Generalized least-square (FGLS) method

2. First-Difference Method

3. When is not known then estimate from the residuals AR(1)

4. Change Model to ARCH and GARCH Models

5. Change Model to ARMA or ARIMA

Autocorrelation: Remedial Measure

34

I. Basic Concepts




Assumption 10:

Heteroscedasticity: Nature of Heteroscedasticity

35

I. Basic Concepts




What is the nature of heteroscedasticity?

What are its consequences?

How does one detect it?

What are the remedial measures?


36

I. Basic Concepts




Why the variances of ui may be variable?

1. Following the error-learning models, as people learn their

errors of behavior become smaller over time.

2. Growth oriented companies

3. As data collecting techniques improves, is likely to

decrease.

4. The presence of outliers

5. Skewness


37

I. Basic Concepts




Best

Linear

Unbiased Estimator

“If we persist in using the usual testing procedure despite heteroscedasticity, whatever conclusions we draw or inferences we make may be very misleading”

Heteroscedasticity

destroy

property of BLUE

Heteroscedasticity: Consequences of using OLS in the Presence of

Heteroscedasticity

38

I. Basic Concepts




1. Graph Residual Plot against Y and X

2. Park Test

3. Glejser Test

4. Spearman’s Rank Correlation Test

5. Glejser Test

6. Goldfeld-Quandt Test

7. Breusch-Pagon-Godfrey Test (BPG)

8. White’s General Heteroscedasticity Test

Heteroscedasticity: Detecting of Heteroscedasticity

39

I. Basic Concepts




1. Weighted Least Square (WLS) o Weighted by Y, 1/X, Different variables

o Error Term

Heteroscedasticity: Remedial Measures

40

I. Basic Concepts




Omitting Variables

Assumption 11:

41

I. Basic Concepts




42

I. Basic Concepts




43

I. Basic Concepts




Heteroscedasticity

44

I. Basic Concepts




Variable Definitions

45

I. Basic Concepts




WORK SHOP

#246

I. Basic Concepts




WORK ORDERS : Multiple Regression

(1) Run Multiple Regression

Take care of seasonal effect and smooth data (by taking log)

(2) Test Multicollinearity and remedy if happens

(3) Test Autocorrelation and remedy if happens

(4) Test Heteroscedasticity and remedy if happens

(5) Analyze your results

47

multiple regression analysis - kasetsart universityfin.bus.ku.ac.th/01131591 financial...

Documents