inferential methods in regression and correlationxuanyaoh/stat350/xyapr13lec28.pdfinferential...
TRANSCRIPT
![Page 1: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/1.jpg)
Inferential Methods in Regression and Correlation
Chapter 11
Lecture 27 4/15/12 1
![Page 2: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/2.jpg)
Announcements
• Make-up for Final Exam
– fill out the form TWO WEEKS AHEAD of time (by Friday, Apr 20th),
– scheduled ahead of time, – taken at the testing center…
4/15/12 Lecture 27 2
![Page 3: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/3.jpg)
Back to Ch.3 (Linear Regression):
• Recall Simple Linear Regression: – Visualization, by scatterplot of response vs. predictor – Fit a line in the data when you see a linear trend – Minimizing the errors using LS method – Get estimates of slope and intercept accordingly – Random residuals
• In this chapter, we introduce – The normal error regression model – Considering “regression” as a sample that we want to draw
inference on – Backward elimination to choose the “effective” variables in
MLR.
Lecture 27 4/15/12 3
![Page 4: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/4.jpg)
Motivation
• Remember when we used to estimate µ? • What did we do?
– Used confidence intervals to give a guess where µ falls
– Used hypothesis testing to check specific hypotheses for µ
• Treat the regression line similarly – Need to understand the sample distribution
again!
X
Lecture 27 4/15/12 4
![Page 5: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/5.jpg)
Normal Error Regression Model
Lecture 27 4/15/12 5
![Page 6: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/6.jpg)
Normal Error Regression Model
Lecture 27 4/15/12 6
![Page 7: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/7.jpg)
A Toy Example • Suppose we use Age to predict Blood Pressure
– Which is X? Y? • Draw a picture… (for the bivariate data: Age and BP)
• For any fixed x, the dependent y has a normal distribution – The mean of y falls on the “population regression line” – P(3< y<5) = ? Using the Normal Error Regression Model…
• Another way to say the same thing is just: ei ~ N(0, σ)
Lecture 27 4/15/12 7
![Page 8: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/8.jpg)
Estimating the slope and intercept
Lecture 27 4/15/12 8
![Page 9: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/9.jpg)
Why This Model?
Lecture 27 4/15/12 9
![Page 10: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/10.jpg)
Only estimates!!!
• Of course, these are only sample estimates – If we took a different sample, we would get different estimates – Need to use these estimates a and b to draw inference about the
“real” slope and intercept, α and β
• Need to know the sampling distribution… – No problem. – We already know it’s normal, the important statement is this
again: ei ~ N(0, σ)
• Meanings of a and b:
– Intercept: a, the value of predicted y when x = 0. – Slope: b, the amount of increase (or change) in y when 1 unit
increase in x.
Lecture 27 4/15/12 10
![Page 11: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/11.jpg)
Estimating the error variance • From the model, we know that ei ~ N(0, σ) • To estimate σ we use the residuals • After the slope and intercept is estimated, the
residuals are calculated as:
• SSE is calculated as: • It is used to estimate σ by:
• Notice: n – 2 is the df for SSE. Why n – 2?
2 2ˆ( )i ie y y= −∑ ∑)
MSEnSSEse =−
=2
Lecture 27 4/15/12 11
![Page 12: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/12.jpg)
Sampling distribution of the slope, b
Lecture 27 4/15/12 12
![Page 13: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/13.jpg)
Confidence interval for β • The interval is:
b ± (t crit) * sb ,
Lecture 27 4/15/12 13
![Page 14: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/14.jpg)
Testing Hypotheses for β • H0: β = β0 • Ha: β ≠ β0 • Notice:
• Test statistics is: , based on t distribution with df = n – 2
• Usually want to test H0: β = 0 – Why?
*
b
btsβ−
=
Lecture 27 4/15/12 14
![Page 15: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/15.jpg)
Example 1
Lecture 27 4/15/12 15
![Page 16: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/16.jpg)
Testing Linear Relationship in the SLR
Chapter 11
Lecture 27 4/15/12 16
![Page 17: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/17.jpg)
Example 2 —Height and Weight
• The following data set gives the average heights and weights for American women aged 30-39 (source: The World Almanac and Book of Facts, 1975). Total observations 15.
Lecture 27 4/15/12 17
![Page 18: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/18.jpg)
Example—Height and Weight
Lecture 27 4/15/12 18
![Page 19: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/19.jpg)
SAS output—Height and Weight
Lecture 27 4/15/12 19
![Page 20: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/20.jpg)
Using the SAS output for inference
• Construct a 95% confidence interval for β
• Test whether or not there is a significant linear relationship (H0: β = 0)
Lecture 27 4/15/12 20
![Page 21: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/21.jpg)
SAS Code
proc reg data=example; model weight = height / alpha=0.05 clb; plot weight*height; run;
• Note the ‘clb’ option will produce a
(1-0.05) = 95% confidence interval for b in addition to the hypothesis test
Lecture 27 4/15/12 21
![Page 22: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/22.jpg)
Review: Coefficient of Determination
Lecture 27
• Also, just the square of the correlation, r. • Multiplying r2 by 100 gives the percent of variation attributed to the linear regression between Y and X. Percent of variation explained. • Notice when SSR is large (or SSE small), we have explained a large amount of the variation in Y
4/15/12 22
![Page 23: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/23.jpg)
Review: Standard Deviation about the regression line
• Given by:
n – 2 comes from the degrees of freedom! • Roughly speaking
– It is the typical amount by which an observation varies about the regression line
• Also called “root MSE” or the square root of the Mean Square Error
2−=nSSEse
Lecture 27 4/15/12 23
![Page 24: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/24.jpg)
SAS Code
proc reg data=example; model weight = height / alpha=0.05 clb;
àoutput out=fit r=res p=pred; plot weight*height;
run; • This creates a new dataset called “fit” • It contains:
– ei, residuals in a variable called “res” – , predicted values in a variable called “pred”
iyLecture 27 4/15/12 24
![Page 25: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/25.jpg)
Review: Residual Plots
• The residuals can be used to assess the appropriateness of a linear regression model.
• Specifically, a residual plot, plotting the residuals against x gives a good indication of whether the model is working – The residual plot should not have any pattern but
should show a random scattering of points – If a pattern is observed, the linear regression
model is probably not appropriate.
Lecture 27 4/15/12 25
![Page 26: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/26.jpg)
Examples—good
Lecture 27 4/15/12 26
![Page 27: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/27.jpg)
Examples— linearity violation
Lecture 27 4/15/12 27
![Page 28: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/28.jpg)
Examples— constant variance violation
Lecture 27 4/15/12 28
![Page 29: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/29.jpg)
Tests of Linear Relationship • H0: β = 0 ó H0: ρ = 0 • Ha: β ≠ 0 Ha: ρ ≠ 0
• Test statistics is:
• Where r is the sample correlation coefficient • Still based on t distribution with df = n – 2
• Remark: A third way to test the linear relationship – construct the confidence interval for β.
212r
nrt−
−=
Lecture 27 4/15/12 29
![Page 30: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/30.jpg)
Test of Linear Relationship, using ρ (optional)
Lecture 27 4/15/12 30
![Page 31: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/31.jpg)
Example 3
Lecture 27 4/15/12 31
![Page 32: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/32.jpg)
ANOVA Approach to Linear Regression
Lecture 27 4/15/12 32
![Page 33: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/33.jpg)
ANOVA Table Source DF SS MS
Model (Regression)
1 SSM = (or SSR)
SSM/1 = MSM (or
MSR) Error n – 2 SSE =
(or SSResid) SSE/n – 2
= MSE
Total n – 1 SST = SSM + SSE =
Lecture 27 4/15/12 33
![Page 34: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/34.jpg)
ANOVA approach to Linear Regression
Lecture 27 4/15/12 34
![Page 35: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/35.jpg)
Revisit Example 3
Lecture 27 4/15/12 35
![Page 36: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/36.jpg)
SAS codes and Output
Lecture 27 4/15/12 36
![Page 37: Inferential Methods in Regression and Correlationxuanyaoh/stat350/xyApr13Lec28.pdfInferential Methods in Regression and Correlation Chapter 11 4/15/12 Lecture 27 1 Announcements •](https://reader033.vdocuments.us/reader033/viewer/2022050312/5f74eaa48a1e6f59025f6922/html5/thumbnails/37.jpg)
After Class…
• Review today’s Lecture Notes, Section 11.1 (optional part: “Exponential Regression”) and 11.2
• Read Section 11.4.
• Lab #7, Wed, Apr 18th. Try the template codes at first … • Hw#11, Monday after next week.
Lecture 27 4/15/12 37