10. simple regression
TRANSCRIPT
-
8/11/2019 10. Simple Regression
1/35
Simple Linear Regression
-
8/11/2019 10. Simple Regression
2/35
-
8/11/2019 10. Simple Regression
3/35
Simple Linear Regression
Regression analysis can be used to develop anequation showing how the variables are related.
Managerial decisions often are based on therelationship between two or more variables.
The variables being used to predict the value of thedependent variable are called the independent
variables and are denoted by x.
The variable being predicted is called the dependentvariable and is denoted by y.
-
8/11/2019 10. Simple Regression
4/35
Simple Linear Regression
The relationship between the two variables isapproximated by a straight line.
Simple linear regression involves one independentvariable and one dependent variable.
Regression analysis involving two or moreindependent variables is called multiple regression.
-
8/11/2019 10. Simple Regression
5/35
Simple Linear Regression Model
y = b 0 + b 1x +e
where: b 0 and b 1 are called parameters of the model,e is a random variable called the error term.
The simple linear regression model is:
The equation that describes how y is related to x andan error term is called the regression model.
-
8/11/2019 10. Simple Regression
6/35
Simple Linear Regression Equation
The simple linear regression equation is:
E(y) is the expected value of y for a given x value.
b 1 is the slope of the regression line.
b 0 is the y intercept of the regression line.
Graph of the regression equation is a straight line.
E(y) = b 0 + b 1x
-
8/11/2019 10. Simple Regression
7/35
Estimated Simple Linear Regression Equation
The estimated simple linear regression equation
0 1
y b b x
is the estimated value of y for a given x value.
y
b1 is the slope of the line.
b0 is the y intercept of the line.
The graph is called the estimated regression line.
-
8/11/2019 10. Simple Regression
8/35
Estimation Process
Regression Modely = b 0 + b 1x +e
Regression EquationE(y) = b 0 + b 1x
Unknown Parameters b 0, b 1
Sample Data:x yx1 y1
. .
. . xn yn
b0 and b
1
provide estimates of b 0 and b 1
EstimatedRegression Equation
Sample Statisticsb0, b1
0 1
y b b x
-
8/11/2019 10. Simple Regression
9/35
Least Squares Method
Least Squares Criterion
min ( y yi i
)2
where:
yi = observed value of the dependent variablefor the ith observation
yi = estimated value of the dependent variablefor the ith observation
-
8/11/2019 10. Simple Regression
10/35
Slope for the Estimated Regression Equation
1 2
( )( )( )
i i
i
x x y yb
x x
Least Squares Method
where:xi = value of independent variable for ith
observation
_y = mean value for dependent variable
_x = mean value for independent variable
yi = value of dependent variable for ithobservation
-
8/11/2019 10. Simple Regression
11/35
y-Intercept for the Estimated Regression Equation
Least Squares Method
0 1b y b x
-
8/11/2019 10. Simple Regression
12/35
Kataria Auto periodically has a special week-longsale. As part of the advertising campaign Kataria runsone or more television commercials during theweekend preceding the sale. Data from a sample of 5previous sales are shown on the next slide.
Simple Linear Regression
Example: Kataria Auto Sales
-
8/11/2019 10. Simple Regression
13/35
Simple Linear Regression
Example: Kataria Auto Sales
Number ofTV Ads ( x)
Number ofCars Sold ( y)
1
3213
14
24181727
S x = 10 S y = 1002x 20y
-
8/11/2019 10. Simple Regression
14/35
Estimated Regression Equation
10 5y x
1 2
( )( ) 20 5( ) 4
i i
i
x x y yb
x x
0 1 20 5(2) 10b y b x
Slope for the Estimated Regression Equation
y-Intercept for the Estimated Regression Equation
Estimated Regression Equation
-
8/11/2019 10. Simple Regression
15/35
Coefficient of Determination
Relationship Among SST, SSR, SSE
where:SST = total sum of squares SSR = sum of squares due to regression SSE = sum of squares due to error
SST = SSR + SSE
2( )iy y2( )iy y
2( )i iy y
-
8/11/2019 10. Simple Regression
16/35
The coefficient of determination is:
Coefficient of Determination
where:
SSR = sum of squares due to regressionSST = total sum of squares
r 2 = SSR/SST
-
8/11/2019 10. Simple Regression
17/35
Coefficient of Determination
r 2 = SSR/SST = 100/114 = .8772
The regression relationship is very strong; 87.72%of the variability in the number of cars sold can be
explained by the linear relationship between thenumber of TV ads and the number of cars sold.
-
8/11/2019 10. Simple Regression
18/35
Sample Correlation Coefficient
21 )of (sign r br xy
xyr 1(sign of ) Coefficient of Determination b
where: b1 = the slope of the estimated regressionequation xbb y 10
-
8/11/2019 10. Simple Regression
19/35
21 )of (sign r br xy
The sign of b1 in the equation is +.
10 5y x
= + .8772xyr
Sample Correlation Coefficient
r xy = +.9366
-
8/11/2019 10. Simple Regression
20/35
Assumptions About the Error Term e
1. The error e is a random variable with mean of zero.2. The variance of e , denoted by 2, is the same for
all values of the independent variable.
3. The values of e are independent.
4. The error e is a normally distributed randomvariable.
-
8/11/2019 10. Simple Regression
21/35
Testing for Significance
To test for a significant regression relationship, wemust conduct a hypothesis test to determine whetherthe value of b 1 is zero.
Two tests are commonly used:t Test and F Test
Both the t test and F test require an estimate of 2,the variance of e in the regression model.
-
8/11/2019 10. Simple Regression
22/35
An Estimate of 2
Testing for Significance
2102 )()
(SSE iiii xbb y y y
where:
s 2 = MSE = SSE/( n 2)
The mean square error (MSE) provides the estimateof 2, and the notation s2 is also used.
-
8/11/2019 10. Simple Regression
23/35
Testing for Significance
An Estimate of
2SSE
MSEn
s
To estimate we take the square root of 2. The resulting s is called the standard error of
the estimate.
-
8/11/2019 10. Simple Regression
24/35
Hypotheses
Test Statistic
Testing for Significance: t Test
0 1: 0H b
1: 0aH b
1
1
b
bts
where1 2
( )
b
i
ss
x xS
-
8/11/2019 10. Simple Regression
25/35
Rejection Rule
Testing for Significance: t Test
where:t is based on a t distributionwith n - 2 degrees of freedom
Reject H 0 if p-value < or t < -t or t > t
-
8/11/2019 10. Simple Regression
26/35
1. Determine the hypotheses.
2. Specify the level of significance.
3. Select the test statistic.
= .05
4. State the rejection rule. Reject H 0 if p-value < .05or | t| > 3.182 (with3 degrees of freedom)
Testing for Significance: t Test
0 1: 0H b 1: 0aH b
1
1
b
bts
-
8/11/2019 10. Simple Regression
27/35
Testing for Significance: t Test
5. Compute the value of the test statistic.
6. Determine whether to reject H 0.t = 4.541 provides an area of .01 in the uppertail. Hence, the p-value is less than .02. (Also,t = 4.63 > 3.182.) We can reject H 0.
1
1 5 4.631.08b
bts
-
8/11/2019 10. Simple Regression
28/35
Confidence Interval for b 1
H 0 is rejected if the hypothesized value of b 1 is notincluded in the confidence interval for b 1.
We can use a 95% confidence interval for b 1 to testthe hypotheses just used in the t test.
-
8/11/2019 10. Simple Regression
29/35
The form of a confidence interval for b 1 is:
Confidence Interval for b 1
11 /2 bb t s
where is the t value providing an area
of /2 in the upper tail of a t distributionwith n - 2 degrees of freedom
2/ t
b1 is thepoint
estima tor
is themarginof error
1/2 bt s
-
8/11/2019 10. Simple Regression
30/35
Confidence Interval for b 1
Reject H 0 if 0 is not included inthe confidence interval for b 1.
0 is not included in the confidence interval.Reject H 0
= 5 +/- 3.182(1.08) = 5 +/- 3.4412/1 b st b or 1.56 to 8.44
Rejection Rule
95% Confidence Interval for b 1
Conclusion
-
8/11/2019 10. Simple Regression
31/35
Hypotheses
Test Statistic
Testing for Significance: F Test
F = MSR/MSE
0 1: 0H b
1: 0aH b
-
8/11/2019 10. Simple Regression
32/35
Rejection Rule
Testing for Significance: F Test
where:F is based on an F distribution with1 degree of freedom in the numerator andn - 2 degrees of freedom in the denominator
Reject H 0 if p-value F
-
8/11/2019 10. Simple Regression
33/35
1. Determine the hypotheses.
2. Specify the level of significance.
3. Select the test statistic.
= .05
4. State the rejection rule. Reject H 0 if p-value < .05
or F > 10.13 (with 1 d.f.in numerator and3 d.f. in denominator)
Testing for Significance: F Test
0 1: 0H b
1: 0aH b
F = MSR/MSE
-
8/11/2019 10. Simple Regression
34/35
Testing for Significance: F Test
5. Compute the value of the test statistic.
6. Determine whether to reject H 0.
F = 17.44 provides an area of .025 in the uppertail. Thus, the p-value corresponding to F = 21.43is less than 2(.025) = .05. Hence, we reject H 0.
F = MSR/MSE = 100/4.667 = 21.43
The statistical evidence is sufficient to concludethat we have a significant relationship between thenumber of TV ads aired and the number of cars sold.
S C i b h
-
8/11/2019 10. Simple Regression
35/35
Some Cautions about theInterpretation of Significance Tests
Just because we are able to reject H 0: b
1 = 0 and
demonstrate statistical significance does not enableus to conclude that there is a linear relationshipbetween x and y.
Rejecting H 0: b 1 = 0 and concluding that therelationship between x and y is significant doesnot enable us to conclude that a cause-and-effectrelationship is present between x and y.