1

Douglas C. MontgomeryProgettazione e analisi degli esperimenti

© 2006 McGraw-Hill

CAPITOLO 10(versione estesa)

Accostamento di modelli per regressione

Metodi statistici e probabilistici per l’ingegneriaCorso di Laurea in Ingegneria Civile

A.A. 2009-10

Facoltà di Ingegneria, Università di PadovaDocente: Dott. L. Corain

2

6-1 Introduction To Empirical Models

6-1 Introduction To Empirical Models

6-1 Introduction To Empirical Models

3

6-1 Introduction To Empirical Models

Based on the scatter diagram, it is probably reasonable to assume that the mean of the random variable Y is related to x by the following straight-line relationship:

where the slope and intercept of the line are called regression coefficients.The simple linear regression model is given by

where ε is the random error term.

We think of the regression model as an empirical model.

Suppose that the mean and variance of ε are 0 and σ2, respectively, then

The variance of Y given x is

6-1 Introduction To Empirical Models

• The true regression model is a line of mean values:

where β1 can be interpreted as the change in the mean of Y for a unit change in x.• Also, the variability of Y at a particular value of xis determined by the error variance, σ2.• This implies there is a distribution of Y-values at each x and that the variance of this distribution is the same at each x.

6-1 Introduction To Empirical Models

4

6-1 Introduction To Empirical Models

6-1 Introduction To Empirical Models

A Multiple Regression Model:

6-1 Introduction To Empirical Models

5

6-1 Introduction To Empirical Models

6-2 Simple Linear Regression

6-2.1 Least Squares Estimation• The case of simple linear regression considers a single regressor or predictor x and a dependent or response variable Y.

• The expected value of Y at each level of x is a random variable:

• We assume that each observation, Y, can be described by the model

6-2 Simple Linear Regression6-2.1 Least Squares Estimation

• Suppose that we have n pairs of observations (x1, y1), (x2, y2), …, (xn, yn).

•The method of least squares is used to estimate the parameters, β0 and β1 by minimizing the sum of the squares of the vertical deviations in Figure 6-6.

6

6-2 Simple Linear Regression6-2.1 Least Squares Estimation

• Using Equation 6-8, the n observations in the sample can be expressed as

• The sum of the squares of the deviations of the observations from the true regression line is

6-2 Simple Linear Regression6-2.1 Least Squares Estimation

6-2 Simple Linear Regression6-2.1 Least Squares Estimation

7

6-2 Simple Linear Regression6-2.1 Least Squares Estimation

6-2 Simple Linear Regression6-2.1 Least Squares Estimation

6-2 Simple Linear Regression6-2.1 Least Squares Estimation

Notation

n

xxxxS

n

iin

ii

n

iixx

2

1

1

2

1

2)(

−=−=∑

∑∑ =

==

n

yxyxyyxxS

n

ii

n

iin

iii

n

iiixy

−=−−=∑∑

∑∑ ==

==

11

11))((

8

6-2 Simple Linear Regression6-2.1 Least Squares Estimation

6-2 Simple Linear Regression6-2.1 Least Squares Estimation

6-2 Simple Linear Regression6-2.1 Least Squares Estimation

9

6-2 Simple Linear Regression6-2.1 Least Squares Estimation

6-2 Simple Linear Regression6-2.1 Least Squares Estimation

6-2 Simple Linear Regression6-2.1 Least Squares Estimation

10

6-2 Simple Linear RegressionRegression Assumptions and Model Properties

6-2 Simple Linear Regression

Regression Assumptions and Model Properties

6-2 Simple Linear Regression

Regression and Analysis of Variance

11

6-2 Simple Linear Regression

Regression and Analysis of Variance

6-2 Simple Linear Regression

6-2.2 Testing Hypothesis in Simple Linear Regression

Use of t-TestsSuppose we wish to test

An appropriate test statistic would be

6-2 Simple Linear Regression

6-2.2 Testing Hypothesis in Simple Linear Regression

Use of t-Tests

We would reject the null hypothesis if

12

6-2 Simple Linear Regression

6-2.2 Testing Hypothesis in Simple Linear Regression

Use of t-TestsSuppose we wish to test

An appropriate test statistic would be

6-2 Simple Linear Regression

6-2.2 Testing Hypothesis in Simple Linear Regression

Use of t-Tests

We would reject the null hypothesis if

6-2 Simple Linear Regression

6-2.2 Testing Hypothesis in Simple Linear Regression

Use of t-TestsAn important special case of the hypotheses of Equation 6-23 is

These hypotheses relate to the significance of regression.

Failure to reject H0 is equivalent to concluding that there is no linear relationship between x and Y.

13

6-2 Simple Linear Regression

6-2.2 Testing Hypothesis in Simple Linear Regression

Use of t-Tests

6-2 Simple Linear Regression

6-2.2 Testing Hypothesis in Simple Linear Regression

Use of t-Tests

6-2 Simple Linear Regression

The Analysis of Variance Approach

14

6-2 Simple Linear Regression

6-2.2 Testing Hypothesis in Simple Linear Regression

The Analysis of Variance Approach

6-2 Simple Linear Regression

6-2.3 Confidence Intervals in Simple Linear Regression

6-2 Simple Linear Regression

6-2.3 Confidence Intervals in Simple Linear Regression

15

6-2 Simple Linear Regression

6-2 Simple Linear Regression

6-2.4 Prediction of New Observations

6-2 Simple Linear Regression

6-2.4 Prediction of New Observations

16

6-2 Simple Linear Regression

6-2.5 Checking Model Adequacy• Fitting a regression model requires several

assumptions.

1. Errors are uncorrelated random variables with mean zero;

2. Errors have constant variance; and,

3. Errors be normally distributed.

• The analyst should always consider the validity of these assumptions to be doubtful and conduct analyses to examine the adequacy of the model

6-2 Simple Linear Regression

6-2.5 Checking Model Adequacy

• The residuals from a regression model are ei = yi - ŷi , where yiis an actual observation and ŷi is the corresponding fitted value from the regression model.

• Analysis of the residuals is frequently helpful in checking the assumption that the errors are approximately normally distributed with constant variance, and in determining whether additional terms in the model would be useful.

6-2 Simple Linear Regression

6-2.5 Checking Model Adequacy

17

6-2 Simple Linear Regression

6-2.5 Checking Model Adequacy

6-2 Simple Linear Regression

6-2.5 Checking Model Adequacy

6-2 Simple Linear Regression

6-2.5 Checking Model Adequacy

18

6-2 Simple Linear Regression

6-2.5 Checking Model Adequacy

6-2 Simple Linear Regression

6-2.6 Correlation and Regression

The sample correlation coefficient between X and Y is

6-2 Simple Linear Regression

6-2.6 Correlation and Regression

The sample correlation coefficient is also closely related to the slope in a linear regression model

19

6-2 Simple Linear Regression

6-2.6 Correlation and Regression

It is often useful to test the hypotheses

The appropriate test statistic for these hypotheses is

Reject H0 if |t0| > tα/2,n-2.

6-3 Multiple Regression6-3.1 Estimation of Parameters in Multiple Regression

6-3 Multiple Regression6-3.1 Estimation of Parameters in Multiple Regression

• The least squares function is given by

• The least squares estimates must satisfy

20

6-3 Multiple Regression6-3.1 Estimation of Parameters in Multiple Regression• The least squares normal equations are

• The solution to the normal equations are the least squares estimators of the regression coefficients.

6-3 Multiple Regression

6-3 Multiple Regression

21

6-3 Multiple Regression

6-3 Multiple Regression

6-3 Multiple Regression

22

6-3 Multiple Regression

6-3.1 Estimation of Parameters in Multiple Regression

6-3 Multiple Regression

6-3.2 Inferences in Multiple RegressionTest for Significance of Regression

6-3 Multiple Regression6-3.2 Inferences in Multiple RegressionInference on Individual Regression Coefficients

•This is called a partial or marginal test

23

6-3 Multiple Regression6-3.2 Inferences in Multiple Regression

Confidence Intervals on the Mean Response and Prediction Intervals

6-3 Multiple Regression6-3.2 Inferences in Multiple Regression

Confidence Intervals on the Mean Response and Prediction Intervals

6-3 Multiple Regression6-3.2 Inferences in Multiple Regression

Confidence Intervals on the Mean Response and Prediction Intervals

24

6-3 Multiple Regression6-3.2 Inferences in Multiple RegressionA Test for the Significance of a Group of Regressors

6-3 Multiple Regression6-3.3 Checking Model AdequacyResidual Analysis

6-3 Multiple Regression6-3.3 Checking Model AdequacyResidual Analysis

25

6-3 Multiple Regression6-3.3 Checking Model AdequacyResidual Analysis

6-3 Multiple Regression6-3.3 Checking Model AdequacyResidual Analysis

6-3 Multiple Regression6-3.3 Checking Model AdequacyResidual Analysis

26

6-3 Multiple Regression6-3.3 Checking Model AdequacyInfluential Observations

6-3 Multiple Regression6-3.3 Checking Model AdequacyInfluential Observations

6-3 Multiple Regression6-3.3 Checking Model Adequacy

27

6-3 Multiple Regression6-3.3 Checking Model AdequacyMulticollinearity

6-4 Other Aspects of Regression6-4.1 Polynomial Models

6-4 Other Aspects of Regression6-4.1 Polynomial Models

28

6-4 Other Aspects of Regression6-4.1 Polynomial Models

6-4 Other Aspects of Regression6-4.1 Polynomial Models

6-4 Other Aspects of Regression6-4.1 Polynomial Models

29

6-4 Other Aspects of Regression6-4.2 Categorical Regressors

• Many problems may involve qualitative or categorical variables.

• The usual method for the different levels of a qualitative variable is to use indicator variables.

• For example, to introduce the effect of two different operators into a regression model, we could define an indicator variable as follows:

6-4 Other Aspects of Regression6-4.2 Categorical Regressors

6-4 Other Aspects of Regression6-4.2 Categorical Regressors

30

6-4 Other Aspects of Regression6-4.3 Variable Selection ProceduresBest Subsets Regressions

6-4 Other Aspects of Regression6-4.3 Variable Selection Procedures

Backward Elimination

6-4 Other Aspects of Regression6-4.3 Variable Selection Procedures

Forward Selection

31

6-4 Other Aspects of Regression6-4.3 Variable Selection Procedures

Stepwise Regression