epi 809/spring 2008 1 chapter 11 regression and correlation methods

EPI 809/Spring 2008EPI 809/Spring 2008 11

Chapter 11Chapter 11

Regression and Correlation Regression and Correlation methodsmethods

Learning ObjectivesLearning Objectives

1.1. Describe the Linear Regression ModelDescribe the Linear Regression Model

2.2. State the Regression Modeling StepsState the Regression Modeling Steps

3.3. Explain Ordinary Least SquaresExplain Ordinary Least Squares

4.4. Compute Regression CoefficientsCompute Regression Coefficients

5.5. Understand and check model assumptionsUnderstand and check model assumptions

6.6. Predict Response VariablePredict Response Variable

7.7. Comments of SAS OutputComments of SAS Output

Learning Objectives… Learning Objectives…

8.8. Correlation ModelsCorrelation Models

9.9. Link between a correlation model and a Link between a correlation model and a regression modelregression model

10.10. Test of coefficient of CorrelationTest of coefficient of Correlation

ModelsModels

What is a Model?What is a Model?

1.1. Representation of Representation of

Some PhenomenonSome Phenomenon

Non-Math/Stats ModelNon-Math/Stats Model

1.1. Representation of Representation of

Some PhenomenonSome Phenomenon

Non-Math/Stats ModelNon-Math/Stats Model

What is a Math/Stats Model?What is a Math/Stats Model?

1.1. Often Describe Relationship between Often Describe Relationship between VariablesVariables

2.2. TypesTypes- Deterministic Models (no randomness)Deterministic Models (no randomness)

- Probabilistic Models (with randomness)Probabilistic Models (with randomness)

Deterministic ModelsDeterministic Models

1.1. Hypothesize Exact RelationshipsHypothesize Exact Relationships2.2. Suitable When Prediction Error is NegligibleSuitable When Prediction Error is Negligible3.3. Example: Body mass index (BMI) is measure of Example: Body mass index (BMI) is measure of

body fat basedbody fat based

Metric Formula: BMI = BMI = Weight in KilogramsWeight in Kilograms (Height in Meters) (Height in Meters)22

Non-metric Formula: BMI = BMI = Weight (pounds)x703Weight (pounds)x703 (Height in inches)(Height in inches)22

Probabilistic ModelsProbabilistic Models

1.1. Hypothesize 2 ComponentsHypothesize 2 Components• DeterministicDeterministic• Random ErrorRandom Error

2.2. Example: Systolic blood pressure of newborns Example: Systolic blood pressure of newborns Is 6 Times the Age in days + Random ErrorIs 6 Times the Age in days + Random Error

• SBPSBP = 6xage(d) = 6xage(d) + + • Random Error May Be Due to Factors Random Error May Be Due to Factors

Other Than age in days (e.g. Birthweight)Other Than age in days (e.g. Birthweight)

Types of Types of Probabilistic ModelsProbabilistic Models

ProbabilisticModels

RegressionModels

CorrelationModels

OtherModels

ProbabilisticModels

RegressionModels

CorrelationModels

OtherModels

Regression ModelsRegression Models

Types of Types of Probabilistic ModelsProbabilistic Models

ProbabilisticModels

RegressionModels

CorrelationModels

OtherModels

ProbabilisticModels

RegressionModels

CorrelationModels

OtherModels

Regression ModelsRegression Models

Relationship between one Relationship between one dependentdependent variablevariable and and explanatory variable(s)explanatory variable(s)

Use equation to set up relationshipUse equation to set up relationship• NumericalNumerical Dependent (Response) Variable Dependent (Response) Variable• 1 or More Numerical or Categorical Independent 1 or More Numerical or Categorical Independent

(Explanatory) Variables(Explanatory) Variables

Used Mainly for Prediction & EstimationUsed Mainly for Prediction & Estimation

Regression Modeling Steps Regression Modeling Steps

1.1. Hypothesize Deterministic Hypothesize Deterministic ComponentComponent

• Estimate Unknown ParametersEstimate Unknown Parameters

2.2. Specify Probability Distribution of Specify Probability Distribution of Random Error TermRandom Error Term

• Estimate Standard Deviation of ErrorEstimate Standard Deviation of Error

3.3. Evaluate the fitted ModelEvaluate the fitted Model 4.4. Use Model for Prediction & Use Model for Prediction &

Estimation Estimation

Model SpecificationModel Specification

Specifying the deterministic Specifying the deterministic componentcomponent

1.1. Define the dependent variable and Define the dependent variable and independent variableindependent variable

2.2. Hypothesize Nature of RelationshipHypothesize Nature of Relationship Expected Effects (i.e., Coefficients’ Signs)Expected Effects (i.e., Coefficients’ Signs) Functional Form (Linear or Non-Linear)Functional Form (Linear or Non-Linear) InteractionsInteractions

Model Specification Model Specification Is Based on TheoryIs Based on Theory

1.1. Theory of Field (e.g., Theory of Field (e.g., Epidemiology)Epidemiology)

2.2. Mathematical TheoryMathematical Theory 3.3. Previous ResearchPrevious Research 4.4. ‘Common Sense’‘Common Sense’

Thinking Challenge: Thinking Challenge: Which Is More Logical?Which Is More Logical?

Years since seroconversion

CD+ counts

Years since seroconversion

CD+ counts

OB/GYN Study OB/GYN Study

Types of Types of Regression ModelsRegression Models

RegressionModels

Simple

1 Explanatory1 ExplanatoryVariableVariable

RegressionModels

2+ Explanatory2+ ExplanatoryVariablesVariables

Simple Multiple

RegressionModels

Linear

Simple Multiple

RegressionModels

LinearNon-

Linear

Simple Multiple

RegressionModels

LinearNon-

Linear

Simple Multiple

Linear

RegressionModels

LinearNon-

Linear

Simple Multiple

Linear

Non-Linear

Linear Regression Linear Regression ModelModel

RegressionModels

LinearNon-

Linear

2+ ExplanatoryVariables

Simple

Non-Linear

Multiple

Linear

1 ExplanatoryVariable

RegressionModels

LinearNon-

Linear

2+ ExplanatoryVariables

Simple

Non-Linear

Multiple

Linear

1 ExplanatoryVariable

Y = mX + b

b = Y-intercept

Changein Y

Change in X

m = Slope

Linear EquationsLinear Equations

YY XXii ii ii 00 11

Linear Regression ModelLinear Regression Model

1.1. Relationship Between Variables Is Relationship Between Variables Is a Linear Functiona Linear Function

Dependent Dependent (Response) (Response) VariableVariable(e.g., CD+ c.)(e.g., CD+ c.)

Independent Independent (Explanatory) Variable (Explanatory) Variable (e.g., Years s. serocon.)(e.g., Years s. serocon.)

Population Population SlopeSlope

Population Population Y-InterceptY-Intercept

Random Random ErrorError

Population & Sample Population & Sample Regression ModelsRegression Models

PopulationPopulation

Unknown Relationship

PopulationPopulation

Y Xi i i 0 1

PopulationPopulation Random SampleRandom Sample

Y Xi i i 0 1

PopulationPopulation Random SampleRandom Sample

Y Xi i i 0 1

Y Xi i i 0 1Y Xi i i 0 1

Population Linear Regression Population Linear Regression ModelModel

iXYE 10 iXYE 10

ObservedObservedvaluevalue

Observed valueObserved value

ii = Random error= Random error

Sample Linear Regression Sample Linear Regression ModelModel

Y Xi i 0 1 Y Xi i 0 1

Unsampled Unsampled observationobservation

ii = Random = Random

errorerror

Observed valueObserved value

Estimating Parameters:Estimating Parameters:Least Squares MethodLeast Squares Method

0204060

0 20 40 60

Scatter plotScatter plot

1.1. Plot of All (Plot of All (XXii, , YYii) Pairs) Pairs

2.2. Suggests How Well Model Will FitSuggests How Well Model Will Fit

Thinking ChallengeThinking Challenge

How would you draw a line through the How would you draw a line through the points? How do you determine which line points? How do you determine which line ‘fits best’? ‘fits best’?

0204060

0 20 40 60

Thinking ChallengeThinking ChallengeHow would you draw a line through the How would you draw a line through the points? How do you determine which line points? How do you determine which line ‘fits best’?‘fits best’?

0204060

0 20 40 60

YSlope changed

Intercept unchanged

Thinking ChallengeThinking Challenge

How would you draw a line through the How would you draw a line through the points? How do you determine which line points? How do you determine which line ‘fits best’?‘fits best’?

0204060

0 20 40 60

Slope unchanged

Intercept changed

Thinking ChallengeThinking ChallengeHow would you draw a line through the How would you draw a line through the points? How do you determine which line points? How do you determine which line ‘fits best’?‘fits best’?

0204060

0 20 40 60

YSlope changed

Intercept changed

Least SquaresLeast Squares

1.1. ‘Best Fit’ Means Difference Between ‘Best Fit’ Means Difference Between Actual Y Values & Predicted Y Values Are Actual Y Values & Predicted Y Values Are a Minimum. a Minimum. ButBut Positive Differences Off- Positive Differences Off-Set Negative onesSet Negative ones

1.1. ‘Best Fit’ Means Difference Between ‘Best Fit’ Means Difference Between Actual Y Values & Predicted Y Values is a Actual Y Values & Predicted Y Values is a Minimum. Minimum. ButBut Positive Differences Off-Set Positive Differences Off-Set Negative ones. Negative ones. So square errors!So square errors!

iii YY

1.1. ‘Best Fit’ Means Difference Between ‘Best Fit’ Means Difference Between Actual Y Values & Predicted Y Values Are Actual Y Values & Predicted Y Values Are a Minimum. a Minimum. ButBut Positive Differences Off- Positive Differences Off-Set Negative. So square errors!Set Negative. So square errors!

2.2. LS Minimizes the Sum of the LS Minimizes the Sum of the Squared Differences (errors) (SSE)Squared Differences (errors) (SSE)

iii YY

Least Squares GraphicallyLeast Squares Graphically

Y X2 0 1 2 2 Y X2 0 1 2 2

Y Xi i 0 1 Y Xi i 0 1

LS minimizes ii

Coefficient EquationsCoefficient Equations

Prediction equationPrediction equation

Sample slopeSample slope

Sample Y - interceptSample Y - intercept

ii xy 10ˆˆˆ

yyxxSS

xy 10 ˆˆ

Derivation of Parameters (1)Derivation of Parameters (1)

Least Squares (L-S): Least Squares (L-S):

Minimize squared errorMinimize squared error

xy 10 ˆˆ

i i iy x

ny n n x

i i ii i

Derivation of Parameters (1)Derivation of Parameters (1)

Least Squares (L-S): Least Squares (L-S):

Minimize squared errorMinimize squared error

x y y x x

i i i i

x x x x y y

x x x x x x y y

Computation TableComputation Table

Xi Yi Xi2 Yi

2 XiYi

X1 Y1 X12 Y1

2 X1Y1

X2 Y2 X22 Y2

2 X2Y2

: : : : :

Xn Yn Xn2 Yn

2 XnYn

Xi2 Yi

2 XiYi

Xi Yi Xi2 Yi

2 XiYi

X1 Y1 X12 Y1

2 X1Y1

X2 Y2 X22 Y2

2 X2Y2

: : : : :

Xn Yn Xn2 Yn

2 XnYn

Xi2 Yi

2 XiYi

Interpretation of CoefficientsInterpretation of Coefficients

1.1. Slope (Slope (11)) Estimated Estimated YY Changes by Changes by 11 for Each 1 Unit for Each 1 Unit

Increase in Increase in XX• If If 11 = 2, then = 2, then YY Is Expected to Increase by 2 for Is Expected to Increase by 2 for

Each 1 Unit Increase in Each 1 Unit Increase in XX

1.1. Slope (Slope (11)) Estimated Estimated YY Changes by Changes by 11 for Each 1 Unit for Each 1 Unit

Increase in Increase in XX• If If 11 = 2, then = 2, then YY Is Expected to Increase by 2 for Is Expected to Increase by 2 for

Each 1 Unit Increase in Each 1 Unit Increase in XX

2.2. Y-Intercept (Y-Intercept (00)) Average Value of Average Value of YY When When XX = 0 = 0

• If If 00 = 4, then Average = 4, then Average YY Is Expected to Be Is Expected to Be

4 When 4 When XX Is 0 Is 0

Parameter Estimation ExampleParameter Estimation Example Obstetrics:Obstetrics: What is the What is the relationshiprelationship between between

Mother’s Estriol level & Birthweight using the Mother’s Estriol level & Birthweight using the following data?following data?

EstriolEstriol BirthweightBirthweight

(mg/24h)(mg/24h) (g/1000)(g/1000)

11 1122 1133 2244 2255 44

0 1 2 3 4 5 6

Scatterplot Scatterplot Birthweight vs. Estriol level Birthweight vs. Estriol level

Birthweight

Estriol level

Parameter Estimation Solution Parameter Estimation Solution TableTable

Xi Yi Xi2 Yi

2 XiYi

1 1 1 1 1

2 1 4 1 2

3 2 9 4 6

4 2 16 4 8

5 4 25 16 20

15 10 55 26 37

Xi Yi Xi2 Yi

2 XiYi

1 1 1 1 1

2 1 4 1 2

3 2 9 4 6

4 2 16 4 8

5 4 25 16 20

15 10 55 26 37

Parameter Estimation SolutionParameter Estimation Solution

10.0370.02ˆˆ

Coefficient Interpretation Coefficient Interpretation SolutionSolution

1.1. Slope (Slope (11)) Birthweight (Birthweight (YY) Is Expected to Increase by .7 ) Is Expected to Increase by .7

Units for Each 1 unit Increase in Estriol (Units for Each 1 unit Increase in Estriol (XX))

1.1. Slope (Slope (11)) Birthweight (Birthweight (YY) Is Expected to Increase by .7 ) Is Expected to Increase by .7

Units for Each 1 unit Increase in Estriol (Units for Each 1 unit Increase in Estriol (XX))

2.2. Intercept (Intercept (00)) Average Birthweight (Average Birthweight (YY) Is -.10 Units When ) Is -.10 Units When

Estriol level (Estriol level (XX) Is 0) Is 0• Difficult to explainDifficult to explain• The birthweight should always be positiveThe birthweight should always be positive

SAS codes for fitting a simple linear SAS codes for fitting a simple linear regressionregression

DataData BW; /*Reading data in SAS*/ BW; /*Reading data in SAS*/ input input estriol birthwestriol birthw@@;@@; cards;cards; 11 11 2 2 1 1 33 22

44 2 2 5 5 44 ; ; runrun;;

PROC REGPROC REG data=BW data=BW; /*Fitting linear regression ; /*Fitting linear regression models*/models*/

model model birthwbirthw==estriolestriol;; runrun; ;

Parameter EstimatesParameter Estimates

Parameter Standard

Variable DF Estimate Error t Value Pr > |t|

Intercept 1 -0.10000 0.63509 -0.16 0.8849

Estriol 1 0.70000 0.19149 3.66 0.0354

Parameter Estimation Parameter Estimation SAS Computer OutputSAS Computer Output

0^̂ 1

Parameter Estimation Thinking Parameter Estimation Thinking ChallengeChallenge

You’re a Vet epidemiologist for the county You’re a Vet epidemiologist for the county cooperative. You gather the following data:cooperative. You gather the following data:

Food (lb.)Food (lb.) Milk yield (lb.)Milk yield (lb.) 4 4 3.03.0 6 6 5.55.51010 6.56.51212 9.09.0

What is the What is the relationshiprelationship between cows’ food intake and milk yield?between cows’ food intake and milk yield?

0 5 10 15

Scattergram Scattergram Milk Yield vs. Food intake*Milk Yield vs. Food intake*

M. Yield (lb.)M. Yield (lb.)M. Yield (lb.)M. Yield (lb.)

Food intake (lb.)Food intake (lb.)Food intake (lb.)Food intake (lb.)

Parameter Estimation Solution Parameter Estimation Solution Table*Table*

Xi Yi Xi2 Yi

2 XiYi

4 3.0 16 9.00 12

6 5.5 36 30.25 33

10 6.5 100 42.25 65

12 9.0 144 81.00 108

32 24.0 296 162.50 218

Xi Yi Xi2 Yi

2 XiYi

4 3.0 16 9.00 12

6 5.5 36 30.25 33

10 6.5 100 42.25 65

12 9.0 144 81.00 108

32 24.0 296 162.50 218

Parameter Estimation Solution*Parameter Estimation Solution*

80.0865.06ˆˆ

Coefficient Interpretation Coefficient Interpretation Solution*Solution*

1.1. Slope (Slope (11)) Milk Yield (Milk Yield (YY) Is Expected to Increase ) Is Expected to Increase

by .65 lb. for Each 1 lb. Increase in Food by .65 lb. for Each 1 lb. Increase in Food intake (intake (XX))

1.1. Slope (Slope (11)) Milk Milk Yield (Yield (YY) Is Expected to Increase ) Is Expected to Increase

by .65 lb. for Each 1 lb. Increase inby .65 lb. for Each 1 lb. Increase in Food Food intakeintake ( (XX))

2.2. Y-Intercept (Y-Intercept (00)) Average Milk yield (Average Milk yield (YY) Is Expected to Be 0.8 ) Is Expected to Be 0.8

lb. When Food intake (lb. When Food intake (XX) Is 0) Is 0

epi 809/spring 2008 1 chapter 11 regression and correlation methods

linear regression model

regression models relationship

explanatory variable

correlation slide

counts slide

randomness slide

birthweight slide

correlation models

Documents

epi powerpoint

epi 809/spring 2008 1 probability distribution of random...

1z0 809 dumps

epi info -...

epi-use country version/ epi-use localizaciones de nómina

professional letter€¦ · web viewneshaminy district...

bureau of land management17 809 . b ok 74 . cc-n. book ....

epi calculation. contents energy performance index aggregate...

809 juan evangelio

arts 804-809

presentation epi

manual epi

epi thermal

confined spaces, chapter 296-809, wac 296-809 wac table of...

science 809 - lifepac

hobbes 809

covid- 19 epi dashboard: epi week 40 (10/10/2021)

epi 809/spring 20081 two main complications of analysis of...

epi 809 / spring 2008 chapter 9 nonparametric statistics

epi 809 / spring 2008 final review epi 809 / spring 2008...