chapter 12. simple linear regression and...
Post on 07-Feb-2018
252 Views
Preview:
TRANSCRIPT
Chapter 12. Simple Linear Chapter 12. Simple Linear Chapter 12. Simple Linear Chapter 12. Simple Linear Regression and CorrelationRegression and CorrelationRegression and CorrelationRegression and Correlation
12.1 The Simple Linear Regression Model12.1 The Simple Linear Regression Model12.1 The Simple Linear Regression Model12.1 The Simple Linear Regression Model12.2 Fitting the Regression Line12.2 Fitting the Regression Line12.2 Fitting the Regression Line12.2 Fitting the Regression Line12.3 Inferences on the Slope Rarameter 12.3 Inferences on the Slope Rarameter 12.3 Inferences on the Slope Rarameter 12.3 Inferences on the Slope Rarameter ββββ1111
NIPRLNIPRLNIPRLNIPRL 1
12.3 Inferences on the Slope Rarameter 12.3 Inferences on the Slope Rarameter 12.3 Inferences on the Slope Rarameter 12.3 Inferences on the Slope Rarameter ββββ111112.4 Inferences on the Regression Line12.4 Inferences on the Regression Line12.4 Inferences on the Regression Line12.4 Inferences on the Regression Line12.5 Prediction Intervals for Future Response Values12.5 Prediction Intervals for Future Response Values12.5 Prediction Intervals for Future Response Values12.5 Prediction Intervals for Future Response Values12.6 The Analysis of Variance Table12.6 The Analysis of Variance Table12.6 The Analysis of Variance Table12.6 The Analysis of Variance Table12.7 Residual Analysis12.7 Residual Analysis12.7 Residual Analysis12.7 Residual Analysis12.8 Variable Transformations12.8 Variable Transformations12.8 Variable Transformations12.8 Variable Transformations12.9 Correlation Analysis12.9 Correlation Analysis12.9 Correlation Analysis12.9 Correlation Analysis12.10 Supplementary Problems12.10 Supplementary Problems12.10 Supplementary Problems12.10 Supplementary Problems
12.1 The Simple Linear Regression Model12.1 The Simple Linear Regression Model12.1 The Simple Linear Regression Model12.1 The Simple Linear Regression Model12.1.1 Model Definition and Assumptions(1/5)12.1.1 Model Definition and Assumptions(1/5)12.1.1 Model Definition and Assumptions(1/5)12.1.1 Model Definition and Assumptions(1/5)
• With the simple linear regression modelyi=β0+β1xi+εithe observed value of the dependent variable yi is composed of a linear function β0+β1xi of the explanatory variable xi, together with an error termerror termerror termerror term εi. The error terms ε1,…,εn are generally taken to be
NIPRLNIPRLNIPRLNIPRL 2
error termerror termerror termerror term εi. The error terms ε1,…,εn are generally taken to be independent observations from a N(0,σ2) distribution, for some error error error error variancevariancevariancevariance σ2. This implies that the values y1,…,yn are observations from the independent random variablesYi ~ N (β0+β1xi, σ2)as illustrated in Figure 12.1
12.1.1 Model Definition and Assumptions(2/5)12.1.1 Model Definition and Assumptions(2/5)12.1.1 Model Definition and Assumptions(2/5)12.1.1 Model Definition and Assumptions(2/5)
NIPRLNIPRLNIPRLNIPRL 3
12.1.1 Model Definition and Assumptions(3/5)12.1.1 Model Definition and Assumptions(3/5)12.1.1 Model Definition and Assumptions(3/5)12.1.1 Model Definition and Assumptions(3/5)• The parameter β0 is known as the intercept parameter, and the
parameter β0 is known as the intercept parameterintercept parameterintercept parameterintercept parameter, and the parameter β1 is known as the slope parameterslope parameterslope parameterslope parameter. A third unknown parameter, the error variance σ2, can also be estimated from the data set. As illustrated in Figure 12.2, the data
NIPRLNIPRLNIPRLNIPRL 4
in Figure 12.2, the data values (xi , yi ) lie closer to the liney = β0+β1xas the error variance σ2
decreases.
12.1.1 Model Definition and Assumptions(4/5)12.1.1 Model Definition and Assumptions(4/5)12.1.1 Model Definition and Assumptions(4/5)12.1.1 Model Definition and Assumptions(4/5)• The slope parameterslope parameterslope parameterslope parameter β1 is of particular interest since it indicates how
the expected value of the dependent variable depends upon the explanatory variable x, as shown in Figure 12.3
• The data set shown in Figure 12.4 exhibits a quadratic (or at least nonlinear) relationship between the two variables, and it would make no sense to fit a straight line to the data set.
NIPRLNIPRLNIPRLNIPRL 5
12.1.1 Model Definition and Assumptions(5/5)12.1.1 Model Definition and Assumptions(5/5)12.1.1 Model Definition and Assumptions(5/5)12.1.1 Model Definition and Assumptions(5/5)
• Simple Linear Regression ModelSimple Linear Regression ModelSimple Linear Regression ModelSimple Linear Regression ModelThe simple linear regressionsimple linear regressionsimple linear regressionsimple linear regression modelyi = β0 + β1xi + εi
fit a straight line through a set of paired data observations (x1,y1),…,(xn, yn). The error terms ε1,…,εn are taken to be
NIPRLNIPRLNIPRLNIPRL 6
(x1,y1),…,(xn, yn). The error terms ε1,…,εn are taken to be independent observations from a N(0,σ2) distribution. The three unknown parameters, the intercept parameterintercept parameterintercept parameterintercept parameter β0 , the slope slope slope slope parameterparameterparameterparameter β1, and the error varianceerror varianceerror varianceerror variance σ2, are estimated from the data set.
12.1.2 Examples(1/2)12.1.2 Examples(1/2)12.1.2 Examples(1/2)12.1.2 Examples(1/2)• Example 3 : Car Plant Electricity UsageCar Plant Electricity UsageCar Plant Electricity UsageCar Plant Electricity Usage
The manager of a car plant wishes to investigate how the plant’s electricity usage depends upon the plant’s production.
The linear model
will allow a month’s electrical0 1y xβ β= +
NIPRLNIPRLNIPRLNIPRL 7
will allow a month’s electricalusage to be estimated as afunction of the month’s pro-duction.
12.1.2 Examples(2/2)12.1.2 Examples(2/2)12.1.2 Examples(2/2)12.1.2 Examples(2/2)
NIPRLNIPRLNIPRLNIPRL 8
12.2 Fitting the Regression Line12.2 Fitting the Regression Line12.2 Fitting the Regression Line12.2 Fitting the Regression Line12.2.1 Parameter Estimation(1/4)12.2.1 Parameter Estimation(1/4)12.2.1 Parameter Estimation(1/4)12.2.1 Parameter Estimation(1/4)
0 1 1 1 The regression line is fitted to the data points ( , ), , ( , )
by finding the line that is "closest" to the data points in some sense.
As Figure 12.14 illustrates, the fitted line is
n ny x x y x yβ β= +� K
�
2
1 0 1
chosen to be the line that
the sum of the squares of these vertical deviations
( ( ))
and this is referred to as
n
i i i
minimizes
Q y xβ β== ∑ − +
NIPRLNIPRLNIPRLNIPRL 9
and this is referred to as
the fit.least squares
12.2.1 Parameter Estimation(2/4)12.2.1 Parameter Estimation(2/4)12.2.1 Parameter Estimation(2/4)12.2.1 Parameter Estimation(2/4)
2
1
2
0 1
1
2
ˆ ˆ With normally distributed error terms, and are maximum
likelihood estimates.
( ) The joint density of the error terms , , is
1 .
2
This likelihoo
n
i
i
n
n
e
ε
σ
β β
ε ε
πσ
=−
∑
�
Q K
2 2
d is maximized by minizing
( ( ))y x Qε β β∑ = ∑ − + =
NIPRLNIPRLNIPRLNIPRL 10
2 2
0 1
1 0 1
0
1 0 1
1
0 1 1
1 0 1 1 1
( ( ))
2( ( )) and
normal equati
2 ( ( ))
the
ˆ ˆ and
ˆ ˆ
on
s
i i i
n
i i i
n
i i i i
n
i i i
n n
i i i i i i
y x Q
Qy x
Qx y x
y n x
x y x
ε β β
β ββ
β ββ
β β
β β
=
=
=
= = =
∑ = ∑ − + =
∂= − ∑ − +
∂
∂= − ∑ − +
∂
⇒
∑ = + ∑
∑ = ∑ + ∑
�
n 2
ix
12.2.1 Parameter Estimation(3/4)12.2.1 Parameter Estimation(3/4)12.2.1 Parameter Estimation(3/4)12.2.1 Parameter Estimation(3/4)�
� � �
n
1 1 11 n 2 n 2
i=1 i=1
1 10 1 1
2 2 2
1 1
2
( )( )
n ( )
and then
where
( )
( )
n n
i i i i i i i XY
i i XX
n n
i i i i
n n
XX i i i i
n
n x y x y S
x x S
y xy x
n n
S x x x nx
x
β
β β β
= = =
= =
= =
∑ − ∑ ∑= =
∑ − ∑
∑ ∑= − = −
= ∑ − = ∑ −
∑
�
NIPRLNIPRLNIPRLNIPRL 11
22 1
1
1 1
( )
and
( )( )
nn i ii i
n n
XY i i i i i i
xx
n
S x x y y x y nxy
==
= =
∑= ∑ −
= ∑ − − = ∑ −
� �*
1 11
*
*
0 1
( )( )
For a specific value of the explanatory variable , this equation
ˆ provides a fitted value | for the dependent variable , as
illustrated in Figure 12.15.
n nn i i i ii i i
x
x yx y
n
x
y x yβ β
= ==
∑ ∑= ∑ −
= +
�
12.2.1 Parameter Estimation(4/4)12.2.1 Parameter Estimation(4/4)12.2.1 Parameter Estimation(4/4)12.2.1 Parameter Estimation(4/4)
�
2 The error variance can be estimated by considering the
deviations between the observed data values and their fitted
values . Specifically, the sum of squares for error SSE is
defined t
i
i
y
y
σ�
� � �
� �
2 2
11 1 0
2
1 0 1 1 1
o be the sum of the squares of these deviations
SSE ( ) ( ( ))n n
i i i i i i
n n n
i i i i i i i
y y y x
y y x y
β β
β β
= =
= = =
= ∑ − = ∑ − +
= ∑ − ∑ − ∑
NIPRLNIPRLNIPRLNIPRL 12
�
1 0 1 1 1
2
and the estimate of the error variance is
SSE
2
i i i i i i i
nσ
= = =
=−
12.2.2 Examples(1/5)12.2.2 Examples(1/5)12.2.2 Examples(1/5)12.2.2 Examples(1/5)
• Example 3 : Car Plant Electricity UsageCar Plant Electricity UsageCar Plant Electricity UsageCar Plant Electricity Usage
12
1
12
For this example 12 and
4.51 4.20 58.62
2.48 2.53 34.15
i
i
i
n
x
y
=
=
= + + =
= + + =
∑
∑
L
L
NIPRLNIPRLNIPRLNIPRL 13
1
122 2 2
1
122 2 2
1
12
1
2.48 2.53 34.15
4.51 4.20 291.2310
2.48 2.53 98.6967
(4.51 2.48) (4.20 2.53) 169.2532
i
i
i
i
i
i
i i
i
y
x
y
x y
=
=
=
=
= + + =
= + + =
= + + =
= × + + × =
∑
∑
∑
∑
L
L
L
L
12.2.2 Examples(2/5)12.2.2 Examples(2/5)12.2.2 Examples(2/5)12.2.2 Examples(2/5)
NIPRLNIPRLNIPRLNIPRL 14
12.2.2 Examples(3/5)12.2.2 Examples(3/5)12.2.2 Examples(3/5)12.2.2 Examples(3/5)
� 1 1 11
2 2
1 1
2
The estimates of the slope parameter and the intercept parameter :
( )( )
( )
(12 169.2532) (58.62 34.15)0.49883
(12 291.2310) 58.62
34.15
n n n
i i i i
i i i
n n
i i
i i
n x y x y
n x x
β = = =
= =
−=
−
× − ×= =
× −
∑ ∑ ∑
∑ ∑
58.62
NIPRLNIPRLNIPRLNIPRL 15
� �0 1
34.15(0.49883
12y xβ β= − = − ×
� �
$
0 1
5.5
58.62) 0.4090
12
The fitted regression line :
0.409 0.499
| 0.409 (0.499 5.5) 3.1535
y x x
y
β β
=
= + = +
= + × =
12.2.2 Examples(4/5)12.2.2 Examples(4/5)12.2.2 Examples(4/5)12.2.2 Examples(4/5)Using the model for production values outside this range is known
as and may give inacce uxtrapolatio resra un te lts.
x
NIPRLNIPRLNIPRLNIPRL 16
12.2.2 Examples(5/5)12.2.2 Examples(5/5)12.2.2 Examples(5/5)12.2.2 Examples(5/5)
�
� �
�
2
0 12
1 1 1
2
98.6967 (0.4090 34.15) (0.49883 169.2532)0.0299
10
0.0299 0.1729
n n n
i i i i
i i i
y y x y
n
β βσ
σ
= = =
− −=
−− × − ×
= =
⇒ = =
∑ ∑ ∑
NIPRLNIPRLNIPRLNIPRL 17
12.3 Inferences on the Slope Parameter 12.3 Inferences on the Slope Parameter 12.3 Inferences on the Slope Parameter 12.3 Inferences on the Slope Parameter ββββ111112.3.1 Inference Procedures(1/4)12.3.1 Inference Procedures(1/4)12.3.1 Inference Procedures(1/4)12.3.1 Inference Procedures(1/4)Inferences on the Slope Parameter Inferences on the Slope Parameter Inferences on the Slope Parameter Inferences on the Slope Parameter ββββ1111
� � � �
2
1
1 1 / 2, 2 1 1 / 2, 2 1
ˆ , ).
A two-sided confidence interval with a confidence level 1 for the slope
parameter in a simple linear regression model is
( . .( ), . .( ))
XX
n n
S
t s e t s eα α
σβ β
α
β β β β β
1
− −
Ν(
−
∈ − × + ×
� �
�
NIPRLNIPRLNIPRLNIPRL 18
1 1 / 2, 2 1 1 / 2, 2 1( . .( ), . .( ))
wh
n nt s e t s eα αβ β β β β− −∈ − × + ×
��
��
��
��
/ 2, 2 / 2, 2
1 1 1
, 2 , 2
1 1 1 1
ich is
( , )
One-sided 1 confidence level confidence intervals are
( , ) and ( , )
n n
XX XX
n n
XX XX
t t
S S
t t
S S
α α
α α
σ σβ β β
α
σ σβ β β β
− −
− −
∈ − +
−
∈ −∞ + ∈ − ∞
�
12.3.1 Inference Procedures(2/4)12.3.1 Inference Procedures(2/4)12.3.1 Inference Procedures(2/4)12.3.1 Inference Procedures(2/4)
�
�
0 1 1 1 1
1
1 1
The two-sided hypotheses
: versus :
for a fixed value of interest are tested with the -statistic
A
XX
H b H b
b t
bt
S
β β
β
σ
•
= ≠
− =
NIPRLNIPRLNIPRLNIPRL 19
The -value is
-value 2 ( | |)
where the random
p
p P X t
= × >
/ 2, 2
variable has a -distribution with 2 degrees of freedom.
A size test rejects the null hypothesis if | | .n
X t n
t tαα −
−
>
12.3.1 Inference Procedures(3/4)12.3.1 Inference Procedures(3/4)12.3.1 Inference Procedures(3/4)12.3.1 Inference Procedures(3/4)
0 1 1 1 1
, 2
The one-sided hypotheses
: versus :
have a -value
-value ( )
and a size test rejects the null hypothesis if .
A
n
H b H b
p
p P X t
t tα
β β
α −
•
≥ <
= <
< −
NIPRLNIPRLNIPRLNIPRL 20Slki Lab.Slki Lab.Slki Lab.Slki Lab.
The one-sided hypotheses
H
•
0 1 1 1 1
, 2
: versus :
have a -value
-value ( )
and a size test rejects the null hypothesis if .
A
n
b H b
p
p P X t
t tα
β β
α −
≤ >
= >
>
12.3.1 Inference Procedures(4/4)12.3.1 Inference Procedures(4/4)12.3.1 Inference Procedures(4/4)12.3.1 Inference Procedures(4/4)• An interesting point to notice is that for a fixed value of the error
variance σ2, the variance of the slope parameter estimate decreases as the value of SXX increases. This happens as the values of the explanatory variable xi become more spread out, as illustrated in Figure 12.30. This result
NIPRLNIPRLNIPRLNIPRL 21
is intuitively reasonable since a greater spread in the values xi provides a greater “leverage” for fitting the regression line, and therefore the slope parameter estimate should be more accurate.
�1β
12.3.2 Examples(1/2)12.3.2 Examples(1/2)12.3.2 Examples(1/2)12.3.2 Examples(1/2)• Example 3 : Car Plant Electricity UsageCar Plant Electricity UsageCar Plant Electricity UsageCar Plant Electricity Usage
��
122
2122 1
1
1
( )58.62
291.2310 4.872312 12
0.1729. .( ) 0.0783
4.8723
i
iXX i
i
XX
x
S x
s eS
σβ
=
=
= − = − =
⇒ = = =
∑∑
NIPRLNIPRLNIPRLNIPRL 22
�
�( )
0 1
1
1
The -statistic for testing : 0
0.498836.37
0.0783. .
The two-sided -value
value 2 ( 6.37) 0
t H
ts e
p
p P X
β
β
β
=
= =
− = × > �
12.3.2 Examples(2/2)12.3.2 Examples(2/2)12.3.2 Examples(2/2)12.3.2 Examples(2/2)
� � � �
( )
0.005,10
1 1 1 1 1
With 3.169, a 99% two-sided confidence interval for the
slope parameter
( critical point . .( ), critical point . .( ))
0.49883 3.169 0.0783, 0.49883 3.169 0.0783
0.251,
t
s e s eβ β β β β
=
∈ − × + ×
= − × + ×
= ( )0.747
NIPRLNIPRLNIPRLNIPRL 23
12.4 Inferences on the Regression Line12.4 Inferences on the Regression Line12.4 Inferences on the Regression Line12.4 Inferences on the Regression Line12.4.1 Inference Procedures(1/2)12.4.1 Inference Procedures(1/2)12.4.1 Inference Procedures(1/2)12.4.1 Inference Procedures(1/2)
Inferences on the Expected Value of the Dependent VariableInferences on the Expected Value of the Dependent VariableInferences on the Expected Value of the Dependent VariableInferences on the Expected Value of the Dependent Variable
� �
*
0 1
*
* *
A 1 confidence level two-sided confidence interval for , the
expected value of the dependent variable for a particular value of the ex-
planatory variable, is
( . .(
x
x
x x t s eα
α β β
β β β β −
− +
+ ∈ + − × � � *),xβ β+
NIPRLNIPRLNIPRLNIPRL 24
� �0 1 0 1 / 2, 1 ( . .(nx x t s eαβ β β β −+ ∈ + − × � �
� � � �
� � �
0 1
* *
0 1 / 2, 2 0 1
* 2*
0 1
),
. .( ))
where
1 ( ) . .( )
n
XX
x
x t s e x
x xs e x
n S
α
β β
β β β β
β β σ
−
+
+ + × +
−+ = +
12.4.1 Inference Procedures(2/2)12.4.1 Inference Procedures(2/2)12.4.1 Inference Procedures(2/2)12.4.1 Inference Procedures(2/2)
� � � �
� � � �
* * *
0 1 0 1 , 2 0 1
* * *
0 1 0 1 , 1 0 1
One-sided confidence intervals are
( , . .( ))
and
( . .( ), )
n
n
x x t s e x
x x t s e x
α
α
β β β β β β
β β β β β β
−
−
+ ∈ −∞ + + × +
+ ∈ + − × + ∞
NIPRLNIPRLNIPRLNIPRL 25
� � � �0 1 0 1 , 1 0 1
*
0 1
( . .( ), )
Hypothesis tests on can be performed by comparing the -statistic
(
nx x t s e x
x t
t
αβ β β β β β
β β
β
−+ ∈ + − × + ∞
+
=� �
� �
* *
0 1 0 1
*
0 1
) ( )
. .( )
with a -distribution with 2 degrees of freedom.
x x
s e x
t n
β β β
β β
+ − +
+
−
12.4.2 Examples(1/2)12.4.2 Examples(1/2)12.4.2 Examples(1/2)12.4.2 Examples(1/2)
• Example 3 : Car Plant Electricity UsageCar Plant Electricity UsageCar Plant Electricity UsageCar Plant Electricity Usage
� � �( )2* * 2
*
0 1
*
0.025,10 0 1
* 2
1 1 ( 4.885). .( ) 0.1729
12 4.8723
With 2.228, a 95% confidence interval for
1 ( 4.885)
XX
x x xs e x
n S
t x
x
β β σ
β β
− −+ = + = × +
= +
−
NIPRLNIPRLNIPRLNIPRL 26
* 2* *
0 1
*
1 ( 4.885)(0.409 0.499 2.228 0.1729 ,
12 4.8723
10.409 0.499 2.228 0.179
1
xx x
x
β β−
+ ∈ + − × × +
+ + × ×* 2
*
0 1
( 4.885))
2 4.8723
At 5
5 (0.409 (0.499 5) 0.113,0.409 (0.499 5) 0.113)
(2.79,3.02)
x
x
β β
−+
=
+ ∈ + × − + × +
=
12.4.2 Examples(2/2)12.4.2 Examples(2/2)12.4.2 Examples(2/2)12.4.2 Examples(2/2)
NIPRLNIPRLNIPRLNIPRL 27
12.5 Prediction Intervals for Future Response Values12.5 Prediction Intervals for Future Response Values12.5 Prediction Intervals for Future Response Values12.5 Prediction Intervals for Future Response Values12.5.1 Inference Procedures(1/2)12.5.1 Inference Procedures(1/2)12.5.1 Inference Procedures(1/2)12.5.1 Inference Procedures(1/2)
• Prediction Intervals for Future Response ValuesPrediction Intervals for Future Response ValuesPrediction Intervals for Future Response ValuesPrediction Intervals for Future Response Values*
*
A 1 confidence level two-sided prediction interval for | , a future value
of the dependent variable for a particular value of the explanatory variable,
is
xy
x
α−
NIPRLNIPRLNIPRLNIPRL 28
� � �*
* 2*
0 1 / 2, 1
is
1 ( )| 1nx
X
x xy x t
n Sαβ β σ−
−∈ ( + − + +
� � �* 2
*
0 1 / 2, 2
,
1 ( )1
X
n
XX
x xx t
n Sαβ β σ−
−+ + + + )
12.5.1 Inference Procedures(2/2)12.5.1 Inference Procedures(2/2)12.5.1 Inference Procedures(2/2)12.5.1 Inference Procedures(2/2)
� � �*
* 2*
0 1 , 2
One-sided confidence intervals are
1 ( ) | ( , 1 )
and
nxXX
x xy x t
n Sαβ β σ−
−∈ −∞ + + + +
NIPRLNIPRLNIPRLNIPRL 29
� � �*
* 2*
0 1 , 1
and
1 ( ) | ( 1 , )nx
XX
x xy x t
n Sαβ β σ−
−∈ + − + + ∞
12.5.2 Examples(1/2)12.5.2 Examples(1/2)12.5.2 Examples(1/2)12.5.2 Examples(1/2)
• Example 3 : Car Plant Electricity UsageCar Plant Electricity UsageCar Plant Electricity UsageCar Plant Electricity Usage
*
*
0.025,10
* 2*
With 2.228, a 95% confidence interval for |
13 ( 4.885)| (0.409 0.499 2.228 0.1729 ,
xt y
xy x
=
−∈ + − × × +
NIPRLNIPRLNIPRLNIPRL 30
*
* 2*
*
5
| (0.409 0.499 2.228 0.1729 ,12 4.8723
13 ( 4.885)0.409 0.499 2.228 0.179 )
12 4.8723
At 5
| (0.409 (0.499 5) 0.401,0.409
xy x
xx
x
y
∈ + − × × +
−+ + × × +
=
∈ + × − (0.499 5) 0.401)
(2.50,3.30)
+ × +
=
12.5.2 Examples(2/2)12.5.2 Examples(2/2)12.5.2 Examples(2/2)12.5.2 Examples(2/2)
NIPRLNIPRLNIPRLNIPRL 31
12.6 The Analysis of Variance Table12.6 The Analysis of Variance Table12.6 The Analysis of Variance Table12.6 The Analysis of Variance Table12.6.1 Sum of Squares Decomposition(1/5)12.6.1 Sum of Squares Decomposition(1/5)12.6.1 Sum of Squares Decomposition(1/5)12.6.1 Sum of Squares Decomposition(1/5)
NIPRLNIPRLNIPRLNIPRL 32
12.6.1 Sum of Squares Decomposition(2/5)12.6.1 Sum of Squares Decomposition(2/5)12.6.1 Sum of Squares Decomposition(2/5)12.6.1 Sum of Squares Decomposition(2/5)
NIPRLNIPRLNIPRLNIPRL 33
12.6.1 Sum of Squares Decomposition(3/5)12.6.1 Sum of Squares Decomposition(3/5)12.6.1 Sum of Squares Decomposition(3/5)12.6.1 Sum of Squares Decomposition(3/5)
SourceSourceSourceSource Degrees of freedomDegrees of freedomDegrees of freedomDegrees of freedom Sum of squaresSum of squaresSum of squaresSum of squares Mean squaresMean squaresMean squaresMean squares FFFF----statisticstatisticstatisticstatistic pppp----valuevaluevaluevalueRegressionRegressionRegressionRegression
ErrorErrorErrorError1111
NNNN----2222SSRSSRSSRSSRSSESSESSESSE
MSR=SSRMSR=SSRMSR=SSRMSR=SSR=MSE=SSE/(=MSE=SSE/(=MSE=SSE/(=MSE=SSE/(nnnn----2222))))
FFFF=MSR/MSE=MSR/MSE=MSR/MSE=MSR/MSE P( FP( FP( FP( F1,n1,n1,n1,n----2 2 2 2 > F )> F )> F )> F )
TotalTotalTotalTotal nnnn----1111
�2
σ
NIPRLNIPRLNIPRLNIPRL 34
F I G U R E 12.41 F I G U R E 12.41 F I G U R E 12.41 F I G U R E 12.41 Analysis of variance table for simpleAnalysis of variance table for simpleAnalysis of variance table for simpleAnalysis of variance table for simple
linear regression analysislinear regression analysislinear regression analysislinear regression analysis
12.6.1 Sum of Squares Decomposition(4/5)12.6.1 Sum of Squares Decomposition(4/5)12.6.1 Sum of Squares Decomposition(4/5)12.6.1 Sum of Squares Decomposition(4/5)
NIPRLNIPRLNIPRLNIPRL 35
12.6.1 Sum of Squares Decomposition(5/5)12.6.1 Sum of Squares Decomposition(5/5)12.6.1 Sum of Squares Decomposition(5/5)12.6.1 Sum of Squares Decomposition(5/5)Coefficient of Determination RCoefficient of Determination RCoefficient of Determination RCoefficient of Determination R2222
2
1
The total variability in the dependent variable, the total sum of squares
SST ( )
can be partitioned into the variability explained by the regression line,
t
n
i iy y=
•
= ∑ −
�
�
2
1he regression sum of squares SSR ( )
and the variability about the regression line, the error sum of squares
n
i iy y= = ∑ −
NIPRLNIPRLNIPRLNIPRL 36
� 2
1 SSE ( ) .
The proportion of the tot
n
i i iy y= = ∑ −
•
2
coefficient of determination
al variability accounted for by the regression line is
the
SSR SSE 1 1
SSESST SST1
SSR
which takes a value between zero and one.
R
= = − =+
12.6.2 Examples(1/1)12.6.2 Examples(1/1)12.6.2 Examples(1/1)12.6.2 Examples(1/1)
• Example 3 : Car Plant Electricity UsageCar Plant Electricity UsageCar Plant Electricity UsageCar Plant Electricity Usage
MSR 1.212440.53
MSE 0.0299F = = =
NIPRLNIPRLNIPRLNIPRL 37
2 SSR 1.21240.802
SST 1.5115R = = =
12.7 Residual Analysis12.7 Residual Analysis12.7 Residual Analysis12.7 Residual Analysis12.7.1 Residual Analysis Methods(1/7)12.7.1 Residual Analysis Methods(1/7)12.7.1 Residual Analysis Methods(1/7)12.7.1 Residual Analysis Methods(1/7)
• The residualsresidualsresidualsresiduals are defined to be
so that they are the differences between the observed values of the dependent variable and the corresponding fitted values .
• A property of the residuals
� , 1i i ie y y i n= − ≤ ≤
0n e∑ =
iy�iy
NIPRLNIPRLNIPRLNIPRL 38
• Residual analysis can be used to– Identify data points that are outliersoutliersoutliersoutliers,– Check whether the fitted model is appropriateappropriateappropriateappropriate,– Check whether the error variance is constantconstantconstantconstant, and– Check whether the error terms are normallynormallynormallynormally distributed.
1 0n
i ie=∑ =
12.7.1 Residual Analysis Methods(2/7)12.7.1 Residual Analysis Methods(2/7)12.7.1 Residual Analysis Methods(2/7)12.7.1 Residual Analysis Methods(2/7)• A nice random scatter plot such as the one in Figure 12.45
⇒ there are no indications of any problems with the regression analysis
• Any patterns in the residual plot or any residuals with a large absolute value alert the experimenter to possible problems with the fitted regression model.
NIPRLNIPRLNIPRLNIPRL 39
12.7.1 Residual Analysis Methods(3/7)12.7.1 Residual Analysis Methods(3/7)12.7.1 Residual Analysis Methods(3/7)12.7.1 Residual Analysis Methods(3/7)• A data point (xi, yi ) can be considered to be an outlier if it does not appear
to predict well by the fitted model.• Residuals of outliers have a large absolute value, as indicated in Figure
12.46. Note in the figure that is used instead of • [For your interest only] ˆ
ies
.ie 2
2( )1( ) (1 ) .ii
XX
x xVar e
n Ss
-= - -
NIPRLNIPRLNIPRLNIPRL 40
12.7.1 Residual Analysis Methods(4/7)12.7.1 Residual Analysis Methods(4/7)12.7.1 Residual Analysis Methods(4/7)12.7.1 Residual Analysis Methods(4/7)• If the residual plot shows positive
and negative residuals grouped together as in Figure 12.47, then a linear model is not appropriate. As Figure 12.47 indicates, a nonlinear model is needed for such a data set.
NIPRLNIPRLNIPRLNIPRL 41
such a data set.
12.7.1 Residual Analysis Methods(5/7)12.7.1 Residual Analysis Methods(5/7)12.7.1 Residual Analysis Methods(5/7)12.7.1 Residual Analysis Methods(5/7)• If the residual plot shows a “funnel
shape” as in Figure 12.48, so that the size of the residuals depends upon the value of the explanatory variable x, then the assumption of a constant error variance σ2 is not valid.
NIPRLNIPRLNIPRLNIPRL 42
valid.
12.7.1 Residual Analysis Methods(6/7)12.7.1 Residual Analysis Methods(6/7)12.7.1 Residual Analysis Methods(6/7)12.7.1 Residual Analysis Methods(6/7)• A normal probability plot ( a normal score plot) of the residuals
– Check whether the error terms εi appear to be normally distributed.• The normal score of the i th smallest residual
– 1
3
81
4
i
n
−
− Φ
+
NIPRLNIPRLNIPRLNIPRL 43
• The main body of the points in a normal probability plot lie approximately on a straight line as in Figure 12.49 is reasonable
• The form such as in Figure 12.50 indicates that the distribution is not normal
4
12.7.1 Residual Analysis Methods(7/7)12.7.1 Residual Analysis Methods(7/7)12.7.1 Residual Analysis Methods(7/7)12.7.1 Residual Analysis Methods(7/7)
NIPRLNIPRLNIPRLNIPRL 44
12.7.2 Examples(1/2)12.7.2 Examples(1/2)12.7.2 Examples(1/2)12.7.2 Examples(1/2)• Example : Nile River FlowrateNile River FlowrateNile River FlowrateNile River Flowrate
NIPRLNIPRLNIPRLNIPRL 45
12.7.2 Examples(2/2)12.7.2 Examples(2/2)12.7.2 Examples(2/2)12.7.2 Examples(2/2)
$
�
�
5
3.88
| 0.470 (0.836 3.88) 2.77
4.01 2.77 1.24
1.243.75
0.1092
i i i
i
x
y
e y y
e
σ
=
= − + × =
⇒ = − = − =
= =
NIPRLNIPRLNIPRLNIPRL 46
�
�
�
0.1092
6.13
5.67 ( 0.470 (0.836 6.13)) 1.02
1.023.07
0.1092
i i i
i
x
e y y
e
σ
σ
=
= − = − − + × =
= =
12.8 Variable Transformations12.8 Variable Transformations12.8 Variable Transformations12.8 Variable Transformations12.8.1 Intrinsically Linear Models(1/4)12.8.1 Intrinsically Linear Models(1/4)12.8.1 Intrinsically Linear Models(1/4)12.8.1 Intrinsically Linear Models(1/4)
NIPRLNIPRLNIPRLNIPRL 47
12.8.1 Intrinsically Linear Models(2/4)12.8.1 Intrinsically Linear Models(2/4)12.8.1 Intrinsically Linear Models(2/4)12.8.1 Intrinsically Linear Models(2/4)
NIPRLNIPRLNIPRLNIPRL 48
12.8.1 Intrinsically Linear Models(3/4)12.8.1 Intrinsically Linear Models(3/4)12.8.1 Intrinsically Linear Models(3/4)12.8.1 Intrinsically Linear Models(3/4)
NIPRLNIPRLNIPRLNIPRL 49
12.8.1 Intrinsically Linear Models(4/4)12.8.1 Intrinsically Linear Models(4/4)12.8.1 Intrinsically Linear Models(4/4)12.8.1 Intrinsically Linear Models(4/4)
NIPRLNIPRLNIPRLNIPRL 50
12.8.2 Examples(1/5)12.8.2 Examples(1/5)12.8.2 Examples(1/5)12.8.2 Examples(1/5)• Example : Roadway Base AggregatesRoadway Base AggregatesRoadway Base AggregatesRoadway Base Aggregates
NIPRLNIPRLNIPRLNIPRL 51
12.8.2 Examples(2/5)12.8.2 Examples(2/5)12.8.2 Examples(2/5)12.8.2 Examples(2/5)
NIPRLNIPRLNIPRLNIPRL 52
12.8.2 Examples(3/5)12.8.2 Examples(3/5)12.8.2 Examples(3/5)12.8.2 Examples(3/5)
NIPRLNIPRLNIPRLNIPRL 53
12.8.2 Examples(4/5)12.8.2 Examples(4/5)12.8.2 Examples(4/5)12.8.2 Examples(4/5)
NIPRLNIPRLNIPRLNIPRL 54
12.8.2 Examples(5/5)12.8.2 Examples(5/5)12.8.2 Examples(5/5)12.8.2 Examples(5/5)
NIPRLNIPRLNIPRLNIPRL 55
12.9 Correlation Analysis12.9 Correlation Analysis12.9 Correlation Analysis12.9 Correlation Analysis12.9.1 The Sample Correlation Coefficient12.9.1 The Sample Correlation Coefficient12.9.1 The Sample Correlation Coefficient12.9.1 The Sample Correlation Coefficient
Sample Correlation CoefficientSample Correlation CoefficientSample Correlation CoefficientSample Correlation Coefficient
1
2 2
The for a set of paired data observations
( , ) is
( )( )
( ) ( )
sample correlation coeffici tne
i i
n
i i iXY
n nXX YY
r
x y
x x y ySr
S S x x y y
=∑ − −= =
∑ − ∑ −
NIPRLNIPRLNIPRLNIPRL 56
2 2
1 1
1
2 2 2 2
1 1
( ) ( )
It measures the st gr n te
i
n nXX YY i i i i
n
i i i
n n
i i i
S S x x y y
x y nxy
x nx y ny
= =
=
= =
∑ − ∑ −
∑ −=
∑ − ∑ −
h of association between two variables and can
be thought of as an estimate of the correlation between the two associated
random variable and .
linear
X Y
ρ
0
Under the assumption that the and random variables have a bivariate
normal distribution, a test of the null hypothesis
: 0
can be performed by comparing the -statistic
X Y
H
t
ρ =
NIPRLNIPRLNIPRLNIPRL 57
2
can be performed by comparing the -statistic
2
1
with a -dis
t
r nt
r
t
−=
−
0 1
tribution with 2 degrees of freedom. In a regression framework,
this test is equivalent to testing : 0.
n
H β−
=
NIPRLNIPRLNIPRLNIPRL 58
NIPRLNIPRLNIPRLNIPRL 59
12.9.2 Examples(1/1)12.9.2 Examples(1/1)12.9.2 Examples(1/1)12.9.2 Examples(1/1)• Example : Nile River FlowrateNile River FlowrateNile River FlowrateNile River Flowrate
2 0.871 0.933r R= = =
NIPRLNIPRLNIPRLNIPRL 60
top related