1Jerrell T.Stracener – Ph.D.

Covariance & Correlation

Covariance – Cov(X,Y)

Covariance between X and Y is a measure of the association between two random variables, X & Y If positive, then both move up or down together If negative, then if X is high, Y is low, vice versa

YXXY YXEYXCov ),(

Correlation Between X and Y

Covariance is dependent upon the units of X & Y [Cov(aX,bY)=abCov(X,Y)] Correlation, Corr(X,Y), scales covariance by the standard deviations of X & Y so that it lies between 1 & –1

2

1

)()(

),(

YVarXVar

YXCov

YX

XYXY

More Correlation & Covariance

If X,Y =0 (or equivalently X,Y =0) then X and Y are linearly unrelated

If X,Y = 1 then X and Y are said to be perfectly positively correlated

If X,Y = – 1 then X and Y are said to be perfectly negatively correlated Corr(aX,bY) = Corr(X,Y) if ab>0 Corr(aX,bY) = –Corr(X,Y) if ab<0

Properties of Expectations

E(a)=a, Var(a)=0

E(X)=X, i.e. E(E(X))=E(X)

E(aX+b)=aE(X)+b

E(X+Y)=E(X)+E(Y)

E(X-Y)=E(X)-E(Y)

E(X- X)=0 or E(X-E(X))=0

E((aX)2)=a2E(X2)

More Properties

Var(X) = E(X2) – x2

Var(aX+b) = a2Var(X)

Var(X+Y) = Var(X) +Var(Y) +2Cov(X,Y)

Var(X-Y) = Var(X) +Var(Y) - 2Cov(X,Y)

Cov(X,Y) = E(XY)-xy

If (and only if) X,Y independent, then Var(X+Y)=Var(X)+Var(Y), E(XY)=E(X)E(Y)

7Jerrell T.Stracener – Ph.D.

Covariance of X and Y

Let X and Y be random variables with joint mass function p(x,y) if X & Y are discrete random variables or with joint probability density function f(x, y) if X & Y are continuous random variables. The covariance of X and Y is

if X and Y are discrete, and

if X and Y are continuous.

yxpyxYXEx y

yxYXXY ,

dxdyyxfyxYXE yxYXXY ,

8Jerrell T.Stracener – Ph.D.

Covariance of X and Y

The covariance of two random variables X and Y with means X and Y , respectively is given by

YXXY XYE

9Jerrell T.Stracener – Ph.D.

Correlation Coefficient

Let X and Y be random variables with covariance XY and standard deviation X and Y , respectively. The correlation coefficient of X and Y is

YX

XYXY

10Jerrell T.Stracener – Ph.D.

Theorem

If X and Y are random variables with joint probability distribution f(x, y), then

XYYXbYaX abba 222222

11Jerrell T.Stracener – Ph.D.

Theorem

If X and Y are independent random variables, then

22222YXbYaX ba

12Jerrell T.Stracener – Ph.D.

Correlation Analysis

A statistical analysis used to obtain a quantitative measure of

the strength of the linear relationship between a dependent

variable and one or more independent variables

13Jerrell T.Stracener – Ph.D.

Correlation – Scatter Diagram

Visual Relationship Between X and Y