epi809/spring 2008 1 models with two or more quantitative variables
Post on 20-Dec-2015
215 views
TRANSCRIPT
EPI809/Spring 2008EPI809/Spring 2008 11
Models With Two or Models With Two or More Quantitative More Quantitative
VariablesVariables
EPI809/Spring 2008EPI809/Spring 2008 22
Types of Types of Regression ModelsRegression Models
ExplanatoryVariable
1stOrderModel
3rdOrderModel
2 or MoreQuantitative
Variables
2ndOrderModel
1stOrderModel
2ndOrderModel
Inter-ActionModel
1Qualitative
Variable
DummyVariable
Model
1Quantitative
Variable
ExplanatoryVariable
1stOrderModel
3rdOrderModel
2 or MoreQuantitative
Variables
2ndOrderModel
1stOrderModel
2ndOrderModel
Inter-ActionModel
1Qualitative
Variable
DummyVariable
Model
1Quantitative
Variable
EPI809/Spring 2008EPI809/Spring 2008 33
First-Order Model With First-Order Model With 2 Independent Variables2 Independent Variables
1.1. Relationship Between 1 Dependent & 2 Relationship Between 1 Dependent & 2 Independent Variables Is a Linear FunctionIndependent Variables Is a Linear Function
2.2. Assumes No Interaction Between Assumes No Interaction Between XX11 & & XX22
Effect of Effect of XX11 on on EE((YY) Is the Same Regardless of ) Is the Same Regardless of
XX22 Values Values
EPI809/Spring 2008EPI809/Spring 2008 44
First-Order Model With First-Order Model With 2 Independent Variables2 Independent Variables
1.1. Relationship Between 1 Dependent & 2 Relationship Between 1 Dependent & 2 Independent Variables Is a Linear FunctionIndependent Variables Is a Linear Function
2.2. Assumes No Interaction Between Assumes No Interaction Between XX11 & & XX22
Effect of Effect of XX11 on on EE((YY) Is the Same Regardless of ) Is the Same Regardless of
XX22 Values Values
3.3. ModelModel E Y X Xi i( ) 0 1 1 2 2E Y X Xi i( ) 0 1 1 2 2
EPI809/Spring 2008EPI809/Spring 2008 55
No InteractionNo Interaction
EPI809/Spring 2008EPI809/Spring 2008 66
No InteractionNo Interaction
E(Y)E(Y)
XX11
44
88
1212
0000 110.50.5 1.51.5
EE((YY) = 1 + 2) = 1 + 2XX11 + 3 + 3XX22
EPI809/Spring 2008EPI809/Spring 2008 77
No InteractionNo Interaction
E(Y)E(Y)
XX11
44
88
1212
0000 110.50.5 1.51.5
EE((YY) = 1 + 2) = 1 + 2XX11 + 3 + 3XX22
EE((YY) = 1 + 2) = 1 + 2XX11 + 3( + 3(00) = 1 + 2) = 1 + 2XX11
EPI809/Spring 2008EPI809/Spring 2008 88
No InteractionNo Interaction
E(Y)E(Y)
XX11
44
88
1212
0000 110.50.5 1.51.5
EE((YY) = 1 + 2) = 1 + 2XX11 + 3 + 3XX22
EE((YY) = 1 + 2) = 1 + 2XX11 + 3( + 3(11) = 4 + 2) = 4 + 2XX11
EE((YY) = 1 + 2) = 1 + 2XX11 + 3( + 3(00) = 1 + 2) = 1 + 2XX11
EPI809/Spring 2008EPI809/Spring 2008 99
No InteractionNo Interaction
E(Y)E(Y)
XX11
44
88
1212
0000 110.50.5 1.51.5
EE((YY) = 1 + 2) = 1 + 2XX11 + 3( + 3(22) = 7 + 2) = 7 + 2XX11
EE((YY) = 1 + 2) = 1 + 2XX11 + 3 + 3XX22
EE((YY) = 1 + 2) = 1 + 2XX11 + 3( + 3(11) = 4 + 2) = 4 + 2XX11
EE((YY) = 1 + 2) = 1 + 2XX11 + 3( + 3(00) = 1 + 2) = 1 + 2XX11
EPI809/Spring 2008EPI809/Spring 2008 1010
No InteractionNo Interaction
E(Y)E(Y)
XX11
44
88
1212
0000 110.50.5 1.51.5
EE((YY) = 1 + 2) = 1 + 2XX11 + 3( + 3(22) = 7 + 2) = 7 + 2XX11
EE((YY) = 1 + 2) = 1 + 2XX11 + 3 + 3XX22
EE((YY) = 1 + 2) = 1 + 2XX11 + 3( + 3(11) = 4 + 2) = 4 + 2XX11
EE((YY) = 1 + 2) = 1 + 2XX11 + 3( + 3(00) = 1 + 2) = 1 + 2XX11
EE((YY) = 1 + 2) = 1 + 2XX11 + 3( + 3(33) = 10 + 2) = 10 + 2XX11
EPI809/Spring 2008EPI809/Spring 2008 1111
No InteractionNo Interaction
Effect (slope) of Effect (slope) of XX11 on on EE((YY) does not depend on ) does not depend on XX22 value value
E(Y)E(Y)
XX11
44
88
1212
0000 110.50.5 1.51.5
EE((YY) = 1 + 2) = 1 + 2XX11 + 3(2) = 7 + + 3(2) = 7 + 22XX11
EE((YY) = 1 + 2) = 1 + 2XX11 + 3 + 3XX22
EE((YY) = 1 + 2) = 1 + 2XX11 + 3(1) = 4 + + 3(1) = 4 + 22XX11
EE((YY) = 1 + 2) = 1 + 2XX11 + 3(0) = 1 + + 3(0) = 1 + 22XX11
EE((YY) = 1 + 2) = 1 + 2XX11 + 3(3) = 10 + + 3(3) = 10 + 22XX11
EPI809/Spring 2008EPI809/Spring 2008 1212
Case, i Yi X1i X2i
1 1 1 3
2 4 8 5
3 1 3 2
4 3 5 6
: : : :
Case, i Yi X1i X2i
1 1 1 3
2 4 8 5
3 1 3 2
4 3 5 6
: : : :
First-Order Model First-Order Model WorksheetWorksheet
Run regression with Run regression with YY, , XX11, , XX22
EPI809/Spring 2008EPI809/Spring 2008 1313
Types of Types of Regression ModelsRegression Models
ExplanatoryVariable
1stOrderModel
3rdOrderModel
2 or MoreQuantitative
Variables
2ndOrderModel
1stOrderModel
2ndOrderModel
Inter-ActionModel
1Qualitative
Variable
DummyVariable
Model
1Quantitative
Variable
ExplanatoryVariable
1stOrderModel
3rdOrderModel
2 or MoreQuantitative
Variables
2ndOrderModel
1stOrderModel
2ndOrderModel
Inter-ActionModel
1Qualitative
Variable
DummyVariable
Model
1Quantitative
Variable
EPI809/Spring 2008EPI809/Spring 2008 1414
Interaction Model With Interaction Model With 2 Independent Variables2 Independent Variables
1.1. Hypothesizes Interaction Between Pairs Hypothesizes Interaction Between Pairs of of XX Variables Variables
Response to One Response to One XX Variable Varies at Variable Varies at Different Levels of Another Different Levels of Another XX Variable Variable
EPI809/Spring 2008EPI809/Spring 2008 1515
Interaction Model With Interaction Model With 2 Independent Variables2 Independent Variables
1.1. Hypothesizes Interaction Between Pairs Hypothesizes Interaction Between Pairs of of XX Variables Variables
Response to One Response to One XX Variable Varies at Variable Varies at Different Levels of Another Different Levels of Another XX Variable Variable
2.2. Contains Two-Way Cross Product Terms Contains Two-Way Cross Product Terms
E Y X X X Xi i i i( ) 0 1 1 2 2 3 1 2E Y X X X Xi i i i( ) 0 1 1 2 2 3 1 2
EPI809/Spring 2008EPI809/Spring 2008 1616
1.1.Hypothesizes Interaction Between Pairs of Hypothesizes Interaction Between Pairs of XX VariablesVariables Response to One Response to One XX Variable Varies at Different Variable Varies at Different
Levels of Another Levels of Another XX Variable Variable
2.2.Contains Two-Way Cross Product Terms Contains Two-Way Cross Product Terms
3.3.Can Be Combined With Other Models Can Be Combined With Other Models Example: Dummy-Variable ModelExample: Dummy-Variable Model
E Y X X X Xi i i i( ) 0 1 1 2 2 3 1 2E Y X X X Xi i i i( ) 0 1 1 2 2 3 1 2
Interaction Model With Interaction Model With 2 Independent Variables2 Independent Variables
EPI809/Spring 2008EPI809/Spring 2008 1717
Effect of Interaction Effect of Interaction
EPI809/Spring 2008EPI809/Spring 2008 1818
Effect of Interaction Effect of Interaction
1.1. Given:Given:
E Y X X X Xi i i i( ) 0 1 1 2 2 3 1 2E Y X X X Xi i i i( ) 0 1 1 2 2 3 1 2
EPI809/Spring 2008EPI809/Spring 2008 1919
Effect of Interaction Effect of Interaction
1.1. Given:Given:
2.2. WithoutWithout Interaction Term, Effect of Interaction Term, Effect of XX11
on on YY Is Measured by Is Measured by 11
E Y X X X Xi i i i( ) 0 1 1 2 2 3 1 2E Y X X X Xi i i i( ) 0 1 1 2 2 3 1 2
EPI809/Spring 2008EPI809/Spring 2008 2020
Effect of Interaction Effect of Interaction
1.1. Given:Given:
2.2. WithoutWithout Interaction Term, Effect of Interaction Term, Effect of XX11
on on YY Is Measured by Is Measured by 11
3.3. WithWith Interaction Term, Effect of Interaction Term, Effect of XX11 on on
YY Is Measured by Is Measured by 11 + + 33XX22
• Effect changes As Effect changes As XX22 changes changes
E Y X X X Xi i i i( ) 0 1 1 2 2 3 1 2E Y X X X Xi i i i( ) 0 1 1 2 2 3 1 2
EPI809/Spring 2008EPI809/Spring 2008 2121
Interaction Model RelationshipsInteraction Model Relationships
EPI809/Spring 2008EPI809/Spring 2008 2222
Interaction Model RelationshipsInteraction Model Relationships
E(Y)E(Y)
XX11
44
88
1212
0000 110.50.5 1.51.5
EE((YY) = 1 + 2) = 1 + 2XX11 + 3 + 3XX2 2 + 4+ 4XX11XX22
EPI809/Spring 2008EPI809/Spring 2008 2323
Interaction Model RelationshipsInteraction Model Relationships
E(Y)E(Y)
XX11
44
88
1212
0000 110.50.5 1.51.5
EE((YY) = 1 + 2) = 1 + 2XX11 + 3 + 3XX2 2 + 4+ 4XX11XX22
EE((YY) = 1 + 2) = 1 + 2XX11 + 3( + 3(00) + 4) + 4XX11((00) = 1 + 2) = 1 + 2XX11
EPI809/Spring 2008EPI809/Spring 2008 2424
Interaction Model RelationshipsInteraction Model Relationships
E(Y)E(Y)
XX11
44
88
1212
0000 110.50.5 1.51.5
EE((YY) = 1 + 2) = 1 + 2XX11 + 3 + 3XX2 2 + 4+ 4XX11XX22
EE((YY) = 1 + 2) = 1 + 2XX11 + 3( + 3(11) + 4) + 4XX11((11) = 4 + 6) = 4 + 6XX11
EE((YY) = 1 + 2) = 1 + 2XX11 + 3( + 3(00) + 4) + 4XX11((00) = 1 + 2) = 1 + 2XX11
EPI809/Spring 2008EPI809/Spring 2008 2525
Interaction Model RelationshipsInteraction Model Relationships
Effect (slope) of Effect (slope) of XX11 on on EE((YY) does depend on ) does depend on XX22 value value
E(Y)E(Y)
XX11
44
88
1212
0000 110.50.5 1.51.5
EE((YY) = 1 + 2) = 1 + 2XX11 + 3 + 3XX2 2 + 4+ 4XX11XX22
EE((YY) = 1 + 2) = 1 + 2XX11 + 3( + 3(11) + 4) + 4XX11((11) = 4 + ) = 4 + 66XX11
EE((YY) = 1 + 2) = 1 + 2XX11 + 3( + 3(00) + 4) + 4XX11((00) = 1 + ) = 1 + 22XX11
EPI809/Spring 2008EPI809/Spring 2008 2626
Interaction Model WorksheetInteraction Model Worksheet
Case, i Yi X1i X2i X1i X2i
1 1 1 3 3
2 4 8 5 40
3 1 3 2 6
4 3 5 6 30
: : : : :
Case, i Yi X1i X2i X1i X2i
1 1 1 3 3
2 4 8 5 40
3 1 3 2 6
4 3 5 6 30
: : : : :
MultiplyMultiply X X11 byby X X2 2 to get to get XX11XX22. .
Run regression with Run regression with YY, , XX11, , XX2 2 , , XX11XX22
EPI809/Spring 2008EPI809/Spring 2008 2727
Thinking challengeThinking challenge
• Assume Y: Milk yield, X1: food intake and Assume Y: Milk yield, X1: food intake and X2: weight X2: weight
Assume the following model with Assume the following model with interactioninteraction
Interpret the interactionInterpret the interaction
YY = 1 + 2 = 1 + 2XX11 + 3 + 3XX2 2 + 4+ 4XX11XX22 ^
EPI809/Spring 2008EPI809/Spring 2008 2828
Types of Types of Regression ModelsRegression Models
ExplanatoryVariable
1stOrderModel
3rdOrderModel
2 or MoreQuantitative
Variables
2ndOrderModel
1stOrderModel
2ndOrderModel
Inter-ActionModel
1Qualitative
Variable
DummyVariable
Model
1Quantitative
Variable
ExplanatoryVariable
1stOrderModel
3rdOrderModel
2 or MoreQuantitative
Variables
2ndOrderModel
1stOrderModel
2ndOrderModel
Inter-ActionModel
1Qualitative
Variable
DummyVariable
Model
1Quantitative
Variable
EPI809/Spring 2008EPI809/Spring 2008 2929
Second-Order Model With Second-Order Model With 2 Independent Variables2 Independent Variables
1.1. Relationship Between 1 Relationship Between 1 Dependent & 2 or More Independent Dependent & 2 or More Independent Variables Is a Quadratic FunctionVariables Is a Quadratic Function
2.2. Useful 1Useful 1StSt Model If Non-Linear Model If Non-Linear Relationship SuspectedRelationship Suspected
EPI809/Spring 2008EPI809/Spring 2008 3030
Second-Order Model With Second-Order Model With 2 Independent Variables2 Independent Variables
1.1. Relationship Between 1 Relationship Between 1 Dependent & 2 or More Independent Dependent & 2 or More Independent Variables Is a Quadratic FunctionVariables Is a Quadratic Function
2.2. Useful 1Useful 1StSt Model If Non-Linear Model If Non-Linear Relationship SuspectedRelationship Suspected
3.3. Model Model E Y X X X X
X X
i i i i
i i
( )
0 1 1 2 2 3 1 2
4 12
5 22
E Y X X X X
X X
i i i i
i i
( )
0 1 1 2 2 3 1 2
4 12
5 22
EPI809/Spring 2008EPI809/Spring 2008 3131
Second-Order Model WorksheetSecond-Order Model Worksheet
Case, i Yi X1i X2i X1i X2i X1i2 X2i
2
1 1 1 3 3 1 9
2 4 8 5 40 64 25
3 1 3 2 6 9 4
4 3 5 6 30 25 36
: : : : : : :
Case, i Yi X1i X2i X1i X2i X1i2 X2i
2
1 1 1 3 3 1 9
2 4 8 5 40 64 25
3 1 3 2 6 9 4
4 3 5 6 30 25 36
: : : : : : :
MultiplyMultiply X X11 byby X X2 2 to get to get XX11XX22; then ; then XX1122, , XX22
22. .
Run regression with Run regression with YY, , XX11, , XX2 2 , , XX11XX22, , XX1122, , XX22
22..
EPI809/Spring 2008EPI809/Spring 2008 3232
Models With One Models With One Qualitative Independent Qualitative Independent
VariableVariable
EPI809/Spring 2008EPI809/Spring 2008 3333
Types of Types of Regression ModelsRegression Models
ExplanatoryVariable
1stOrderModel
3rdOrderModel
2 or MoreQuantitative
Variables
2ndOrderModel
1stOrderModel
2ndOrderModel
Inter-ActionModel
1Qualitative
Variable
DummyVariable
Model
1Quantitative
Variable
ExplanatoryVariable
1stOrderModel
3rdOrderModel
2 or MoreQuantitative
Variables
2ndOrderModel
1stOrderModel
2ndOrderModel
Inter-ActionModel
1Qualitative
Variable
DummyVariable
Model
1Quantitative
Variable
EPI809/Spring 2008EPI809/Spring 2008 3434
Dummy-Variable ModelDummy-Variable Model
1.1. Involves Categorical Involves Categorical XX Variable With Variable With 2 Levels2 Levels
e.g., Male-Female; College-No Collegee.g., Male-Female; College-No College
2.2. Variable Levels Coded 0 & 1Variable Levels Coded 0 & 13.3. Number of Dummy Variables Is 1 Less Than Number of Dummy Variables Is 1 Less Than
Number of Levels of VariableNumber of Levels of Variable4.4. May Be Combined With Quantitative May Be Combined With Quantitative
Variable (1Variable (1stst Order or 2 Order or 2ndnd Order Model) Order Model)
EPI809/Spring 2008EPI809/Spring 2008 3535
Dummy-Variable Model Dummy-Variable Model WorksheetWorksheet
Case, i Yi X1i X2i
1 1 1 1
2 4 8 0
3 1 3 1
4 3 5 1
: : : :
Case, i Yi X1i X2i
1 1 1 1
2 4 8 0
3 1 3 1
4 3 5 1
: : : :
XX22 levels: 0 = Group 1; 1 = Group 2. levels: 0 = Group 1; 1 = Group 2.
Run regression with Run regression with YY, , XX11, , XX22
EPI809/Spring 2008EPI809/Spring 2008 3636
Interpreting Dummy-Variable Interpreting Dummy-Variable Model EquationModel Equation
EPI809/Spring 2008EPI809/Spring 2008 3737
Interpreting Dummy-Variable Interpreting Dummy-Variable Model EquationModel Equation
Given:Given: Starting sStarting salary of calary of college graollege grad'd'ssGPAGPA
iiif Femaleif Female
f Malef Male
YY XX XXYYXX
XX
ii ii ii
00 11 11 22 22
11
220011
EPI809/Spring 2008EPI809/Spring 2008 3838
Interpreting Dummy-Variable Interpreting Dummy-Variable Model EquationModel Equation
Given:Given: Starting sStarting salary of calary of college graollege grad'd'ssGPAGPA
Males (Males (
):):
YY XX XXYYXX
YY XX XX
ii ii ii
ii ii ii
XX
00 11 11 22 22
11
00 11 11 22 00 11 11(0)(0)
22 00
iiif Femaleif Femalef Malef Male
XX 220011
EPI809/Spring 2008EPI809/Spring 2008 3939
Interpreting Dummy-Variable Interpreting Dummy-Variable Model EquationModel Equation
Same slopesSame slopesSame slopesSame slopes
Given:Given: Starting sStarting salary of calary of college graollege grad'd'ssGPAGPA
Males (Males (
):):
YY XX XXYYXX
YY XX XX
ii ii ii
ii ii ii
XX
00 11 11 22 22
11
00 11 11 22 00 11 11(0)(0)
22 00
iiif Femaleif Femalef Malef Male
XX 220011
YY XX XXii ii ii 00 11 11 22 00 11 11(1)(1)
Females (Females ( ):): X X 2 2 11))22
EPI809/Spring 2008EPI809/Spring 2008 4040
Dummy-Variable Model Dummy-Variable Model RelationshipsRelationships
YY
XX1100
00
Same Slopes 1
00
0 0 + + 22
^̂
^̂ ^̂
^̂
Females
Males
EPI809/Spring 2008EPI809/Spring 2008 4141
Dummy-Variable Model Dummy-Variable Model ExampleExample
EPI809/Spring 2008EPI809/Spring 2008 4242
Dummy-Variable Model Dummy-Variable Model ExampleExample
Computer OComputer Output:utput:
f Malef Maleif Femaleif Female
ii
YY XX XX
XX
ii ii ii
33 55 77
0011
11 22
22
EPI809/Spring 2008EPI809/Spring 2008 4343
Dummy-Variable Model Dummy-Variable Model ExampleExample
Computer OComputer Output:utput:
Males (Males (
):):
YY XX XX
YY XX XX
ii ii ii
ii ii ii
XX
33 55 77
33 55 77(0)(0) 33 55
11 22
11 11
22 00
f Malef Maleif Femaleif Femaleii
XX 001122
EPI809/Spring 2008EPI809/Spring 2008 4444
Dummy-Variable Model Dummy-Variable Model ExampleExample
Same slopesSame slopesSame slopesSame slopes
Computer OComputer Output:utput:
Males (Males (
):):
YY XX XX
YY XX XX
ii ii ii
ii ii ii
XX
33 55 77
33 55 77(0)(0) 33 55
11 22
11 11
22 00
f Malef Maleif Femaleif Femaleii
XX 001122
Females Females YY XX XXii ii ii 33 55 77(1)(1) (3 + 7)(3 + 7) 5511 11
):):(X(X 22 11
EPI809/Spring 2008EPI809/Spring 2008 4545
Sample SAS codes for fitting linear Sample SAS codes for fitting linear regressions with interactions and regressions with interactions and
higher order termshigher order terms
PROCPROC GLM GLM data=complex; data=complex;Class Class gendergender; ; model model salary salary = = gpa gendergpa gender gpa*gendergpa*gender;;RUN; RUN;
EPI809/Spring 2008EPI809/Spring 2008 4646
ConclusionConclusion
1.1. Explained the Linear Multiple Regression ModelExplained the Linear Multiple Regression Model
2.2. Tested Overall SignificanceTested Overall Significance
3.3. Described Various Types of ModelsDescribed Various Types of Models
4.4. Evaluated Portions of a Regression ModelEvaluated Portions of a Regression Model
5.5. Interpreted Linear Multiple Regression Computer Interpreted Linear Multiple Regression Computer OutputOutput
6.6. Described Stepwise RegressionDescribed Stepwise Regression
7.7. Explained Residual AnalysisExplained Residual Analysis
8.8. Described Regression PitfallsDescribed Regression Pitfalls