Linear Regression and Correlation

Describing Relationship between Two Variables

When we study the relationship between two variables we refer to the data as bivariate.

One graphical technique we use to show the relationship between variables is called a scatter diagram.

To draw a scatter diagram we need two variables. We scale one variable along the horizontal axis (X-axis) of a graph and the other variable along the vertical axis (Y-axis).

Describing Relationship between Two Variables – Scatter Diagram Examples

Regression Analysis - Introduction Correlation Analysis is the study of the relationship

between variables. It is also defined as group of techniques to measure the association between two variables.

Scatter Diagram is a chart that portrays the relationship between the two variables. It is the usual first step in correlations analysis.

The Dependent Variable is the variable being predicted or estimated.

The Independent Variable provides the basis for estimation. It is the predictor variable.

Scatter Diagram ExampleThe sales manager of Copier Sales of America, which has a large sales force throughout

the United States and Canada, wants to determine whether there is a relationship between the number of sales calls made in a month and the number of copiers sold that month. The manager selects a random sample of 10 representatives and determines the number of sales calls each representative made last month and the number of copiers sold.

The Coefficient of Correlation, r

It shows the direction and strength of the linear relationship between two interval or ratio-scale variables

It can range from -1.00 to +1.00.

Values of -1.00 or +1.00 indicate perfect and strong correlation.

Values close to 0.0 indicate weak correlation.

Negative values indicate an inverse relationship and positive values indicate a direct relationship.

The Coefficient of Correlation (r) is a measure of the strength of the relationship between two variables.The Coefficient of Correlation (r) is a measure of the strength of the relationship between two variables.

Coefficient of Determination ( r2 )

Interpretation:

Coefficient of Determination ( r2 ) provides the amount of the variation in the dependent variable (Y) explained by the variation in the independent variable (X).

2222

yynxxn

yxxynr

Correlation Coefficient - Example

EXAMPLE

Using the Copier Sales of America data which a scatterplot is shown below, compute the correlation coefficient and coefficient of determination.

How do we interpret a correlation of 0.759? First, it is positive and closer to 1.0. So we see there is a strong direct relationship between the number of sales calls and the number of copiers sold. The value of coefficient of determination of 57.6% indicates the percentage of variation explained in “the number of copiers sold” by the variation in “the number of calls made by the sales representatives”.

2222

yynxxn

yxxynr

%6.57576.02 r

r = 0.759

Regression AnalysisIn regression analysis we use the independent variable (X) to estimate the dependent variable (Y). The relationship between the variables is linear. Both variables must be at least interval scale. The least squares criterion is used to determine the equation. REGRESSION EQUATION An equation that expresses the linear relationship between two variables.

LEAST SQUARES PRINCIPLE Determining a regression equation by minimizing the sum of the squares of the vertical distances between the actual Y values and the predicted values of Y.

bn XY X Y

n X X

aY

nbX

n

( ) ( )( )

( ) ( )

2 2

Regression Equation - Example

Recall the example involving Copier Sales of America. The sales manager gathered information on the number of sales calls made and the number of copiers sold for a random sample of 10 sales representatives. Use the least squares method to determine a linear equation to express the relationship between the two variables.

What is the expected number of copiers sold by a representative who made 20 calls?

6316.42

)20(1842.19476.18

1842.19476.18

:isequation regression The

^

^

^

^

Y

Y

XY

bXaY

bn XY X Y

n X X

aY

nbX

n

( ) ( )( )

( ) ( )

2 2

b = 1.1842

SIMPLE REGRESSION – ANOTHER EXAMPLE

Suppose a store conducts an analysis of five-month data to determine the effect of advertising on sales revenue. The data are shown below. Use the least squares method to determine a linear equation to express the relationship between the two variables.

Advertising Exp.($100s) Sales Rev.($1000s)(X) (Y)

1 1 2 1 3 2 4 2 5 4

SIMPLE REGRESSION – ANOTHER EXAMPLE

X Y X2 XY1 1 1 12 1 4 23 2 9 64 2 16 85 4 25 20

15X 10Y 552X 37XY

35

15 X 25

10 Y

SIMPLE REGRESSION – ANOTHER EXAMPLE

103702 ..a

XbYa

X..Y 7010

22 XXn

YXXYnb b = 0.7