multiple regression. psyc 6130, prof. j. elder 2 multiple regression multiple regression extends...

Multiple Regression

PSYC 6130, PROF. J. ELDER 2

Multiple Regression

• Multiple regression extends linear regression to allow for 2 or more independent variables.

• There is still only one dependent (criterion) variable.

• We can think of the independent variables as ‘predictors’ of the dependent variable.

• The main complication in multiple regression arises when the predictors are not statistically independent.


Example 1: Predicting Income

Age

Hours Worked

MultipleRegression Income


Example 2: Predicting Final Exam Grades

Assignments

Midterm

MultipleRegression Final


Coefficient of Multiple Determination

• The proportion of variance explained by all of the independent variables together is called the coefficient of multiple determination (R2).

• R is called the multiple correlation coefficient.

• R measures the correlation between the predictions and the actual values of the dependent variable.

• The correlation riY of predictor i with the criterion (dependent variable) Y is called the validity of predictor i.


Uncorrelated Predictors

21 Yr 2

2Yr

Total variance

Variance explained by assignments Variance explained by midterm

2 2 2 2 21 2=Total proportion of variance explained = Y Y Y YR r r


Uncorrelated Predictors• Recall the regression formula for a single predictor:

• If the predictors were not correlated, we could easily generalize this formula:

Y Xz rz

1 1 2 2Y Y Yz r z r z


Example 1. Predicting Income

Correlations

1 .040* .229**.012 .000

3975 3975 3975.040* 1 .187**

.012 .000

3975 3975 3975

.229** .187** 1

.000 .0003975 3975 3975

Pearson CorrelationSig. (2-tailed)NPearson CorrelationSig. (2-tailed)

N

Pearson CorrelationSig. (2-tailed)N

AGE

HOURS WORKEDFOR PAY OR INSELF-EMPLOYMENT- in Reference Week

TOTAL INCOME

AGE

HOURSWORKEDFOR PAY

OR INSELF-

EMPLOYMENT - inReference Week

TOTALINCOME

Correlation is significant at the 0.05 level (2-tailed).*.

Correlation is significant at the 0.01 level (2-tailed).**.


Correlated Predictors

21 Yr 2

2Yr

Total variance

Variance explained by assignments Variance explained by midterm

2 2 21 2=Total proportion of variance explained < Y YR r r


Correlated Predictors

• Due to the correlation in the predictors, the optimal regression weights must be reduced:

1 1 2 2Yz z z

1 2 12 2 1 121 22 2

12 12

where

and 1 1

Y Y Y Yr r r r r rr r

1 2 beta weights (standardized partial re

andgres

are callesion coeffi

d thc

s)

eient

2 22 1 2 1 2 12

1 1 2 2 212

21

Y Y Y YY Y

r r r r rR r rr


Raw-Score Formulas

0 1 1 2 2Y B B X B X

1 2

1 1 2 2

0 1 1 2 2

where

and

and

Y Y

X X

s sB Bs s

B Y B X B X



Correlations

1 .040* .229**.012 .000

3975 3975 3975.040* 1 .187**

.012 .000

3975 3975 3975

.229** .187** 1

.000 .0003975 3975 3975

Pearson CorrelationSig. (2-tailed)NPearson CorrelationSig. (2-tailed)

N

Pearson CorrelationSig. (2-tailed)N

AGE

HOURS WORKEDFOR PAY OR INSELF-EMPLOYMENT- in Reference Week

TOTAL INCOME

AGE

HOURSWORKEDFOR PAY

OR INSELF-

EMPLOYMENT - inReference Week

TOTALINCOME


Correlation is significant at the 0.01 level (2-tailed).**.

1 1 2 2Yz z z

1 2 12 2 1 121 22 2

12 12

where

and 1 1


2 22 1 2 1 2 12

1 1 2 2 212

21

Y Y Y YY Y

r r r r rR r rr



020

4060

80

0

20

40

60

800

1

2

3

4

5

6

7

x 104

Age (years)Hours worked per week (hours)

Ann

ual I

ncom

e (C

AD

)


Degrees of freedom

1 wheresample sizenumber of predictors

df n knk


Semipartial (Part) Correlations

• The semipartial correlations measure the correlation between each predictor and the criterion when all other predictors are held fixed.

• In this way, the effects of correlations between predictors are eliminated.

• In general, the semipartial correlations are smaller than the validities.


Calculating Semipartial Correlations

• One way to calculate the semipartial correlation for a predictor (say Predictor 1) is to partial out the effects of all other predictors on Predictor 1and then calculate the correlation between the residual of Predictor 1 and the criterion.

• For example, we could partial out the effects of age on hours worked, and then measure the correlation between income and the residual hours worked.


Calculating Semipartial Correlations

• A more straightforward method:

1 2 12(1.2) 2

121Y Y

Yr r rr

r

(1.2)where is the semipartial correlation between Predictor 1 and Yr Y

i.e., the correlation between and Predictor 1 after partialling out the effects of Predictor 2 on Predictor 1.

Y


Example 2: Predicting Final Exam Grades

Assignments

Midterm

MultipleRegression Final


Example 2. Predicting Final Exam Grades (PSYC 6130A, 2005-2006)

Correlations

1 .356 .127.233 .680

13 13 13.356 1 .615*.233 .025

13 13 13.127 .615* 1.680 .025

13 13 13

Pearson CorrelationSig. (2-tailed)NPearson CorrelationSig. (2-tailed)NPearson CorrelationSig. (2-tailed)N

Assignments

Midterm

Final

Assignments Midterm Final


212 120.356 0.127r r 2

1 120.127 0.016Yr r 22 20.615 0.378Y Yr r


Example 2. Predicting Final Exam Grades (PSYC 6130A, 2005-2006)

212 120.356 0.127r r 2

1 120.127 0.016Yr r 22 20.615 0.378Y Yr r

1 1 2 2Yz z z

1 2 12 2 1 121 22 2

12 12

where

and 1 1


2 22 1 2 1 2 12

1 1 2 2 212

21

Y Y Y YY Y

r r r r rR r rr


Example 2. Predicting Final Exam Grades

0 1 1 2 2Y B B X B X

1 2

1 1 2 2

0 1 1 2 2

where

and

and

Y Y

X X

s sB Bs s

B Y B X B X


Example 2. Predicting Final Exam Grades

7080

90100

2040

6080

0

50

100

150

Assignment grade (%)Midterm grade (%)

Fina

l gra

de (%

)


SPSS Output


Example 3. 2006-07 6130 Grades

• Try doing the calculations on this dataset for practice.

multiple regression. psyc 6130, prof. j. elder 2 multiple regression multiple regression extends...

Documents