1 covariance, covariance and variance rules, and correlation covariance the covariance of two random...
TRANSCRIPT
1
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Covariance
The covariance of two random variables X and Y, often written sXY, is defined to be the expected value of the product of their deviations from their population means.
YXXY YXEYX ,cov
2
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Covariance
If two variables are independent, their covariance is zero. To show this, start by rewriting the covariance as the product of the expected values of its factors.
000
,cov
YYXX
YX
YX
YX
EYEEXE
YEXE
YXEYX
YXXY YXEYX ,cov
If X and Y are independent,
3
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Covariance
We are allowed to do this because (and only because) X and Y are independent (see the earlier sequence on independence.
000
,cov
YYXX
YX
YX
YX
EYEEXE
YEXE
YXEYX
YXXY YXEYX ,cov
If X and Y are independent,
000
,cov
YYXX
YX
YX
YX
EYEEXE
YEXE
YXEYX
The expected values of both factors are zero because E(X) = mX and E(Y) = mY. E(mX) = mX and E(mY) = mY because mX and mY are constants. Thus the covariance is zero.
4
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Covariance
YXXY YXEYX ,cov
If X and Y are independent,
5
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Covariance rules
There are some rules that follow in a perfectly straightforward way from the definition of covariance, and since they are going to be used frequently in later chapters it is worthwhile establishing them immediately. First, the addition rule.
1. If Y = V + W,
WXVXYX ,cov,cov,cov
2. If Y = bZ, where b is a constant,
ZXbYX ,cov,cov
6
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Covariance rules
Next, the multiplication rule, for cases where a variable is multiplied by a constant.
1. If Y = V + W,
WXVXYX ,cov,cov,cov
Finally, a primitive rule that is often useful.
7
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
0,cov YX3. If Y = b, where b is a constant,
2. If Y = bZ, where b is a constant,
ZXbYX ,cov,cov
Covariance rules
1. If Y = V + W,
WXVXYX ,cov,cov,cov
8
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Covariance rules
The proofs of the rules are straightforward. In each case the proof starts with the definition of cov(X, Y).
1. If Y = V + W,
WXVXYX ,cov,cov,cov
Since Y = V + W, mY = mV + mW
WXVX
WXVXE
WVXE
YXEYX
WXVX
WVX
YX
,cov,cov
][][
,cov
Proof
9
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Covariance rules
We now substitute for Y and re-arrange.
Since Y = V + W, mY = mV + mW
WXVX
WXVXE
WVXE
YXEYX
WXVX
WVX
YX
,cov,cov
][][
,cov
Proof
1. If Y = V + W,
WXVXYX ,cov,cov,cov
Since Y = V + W, mY = mV + mW
10
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Covariance rules
WXVX
WXVXE
WVXE
YXEYX
WXVX
WVX
YX
,cov,cov
][][
,cov
This gives us the result.
Proof
1. If Y = V + W,
WXVXYX ,cov,cov,cov
Since Y = bZ, mY = bmZ
11
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Covariance rules
2. If Y = bZ, where b is a constant,
ZXbYX ,cov,cov
Next, the multiplication rule, for cases where a variable is multiplied by a constant. The Y terms have been replaced by the corresponding bZ terms.
Proof
ZXb
ZXbE
bbZXE
YXEYX
ZX
ZX
YX
,cov
,cov
Since Y = bZ, mY = bmZ
12
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Covariance rules
b is a common factor and can be taken out of the expression, giving us the result that we want.
ZXb
ZXbE
bbZXE
YXEYX
ZX
ZX
YX
,cov
,cov
Proof
2. If Y = bZ, where b is a constant,
ZXbYX ,cov,cov
The proof of the third rule is trivial.
Since Y = b, mY = b
0
0
,cov
E
bbXE
YXEYX
X
YX
13
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Covariance rules
3. If Y = b, where b is a constant,
0,cov YX
Proof
14
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Example use of covariance rules
Here is an example of the use of the covariance rules. Suppose that Y is a linear function of Z and that we wish to use this to decompose cov(X, Y). We substitute for Y (first line) and then use covariance rule 1 (second line).
ZXb
ZbXbX
ZbbXYX
,cov
,cov,cov
][,cov,cov
2
21
21
Suppose Y = b1 + b2Z
cov(X, b1) = 0, using covariance rule 3. The multiplicative factor b2 may be taken out of the second term, using covariance rule 2.
Suppose Y = b1 + b2Z
15
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
ZXb
ZbXbX
ZbbXYX
,cov
,cov,cov
][,cov,cov
2
21
21
Example use of covariance rules
16
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Variance rules
Corresponding to the covariance rules, there are parallel rules for variances. First the addition rule.
1. If Y = V + W,
WVWVY ,cov2varvarvar
17
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Variance rules
Next, the multiplication rule, for cases where a variable is multiplied by a constant.
1. If Y = V + W,
WVWVY ,cov2varvarvar
2. If Y = bZ, where b is a constant,
ZbY varvar 2
18
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Variance rules
A third rule to cover the special case where Y is a constant.
1. If Y = V + W,
WVWVY ,cov2varvarvar
2. If Y = bZ, where b is a constant,
ZbY varvar 2
3. If Y = b, where b is a constant,
0var Y
Finally, it is useful to state a fourth rule. It depends on the first three, but it is so often of practical value that it is worth keeping it in mind separately.
19
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Variance rules
1. If Y = V + W,
WVWVY ,cov2varvarvar
2. If Y = bZ, where b is a constant,
ZbY varvar 2
3. If Y = b, where b is a constant,
0var Y
VY varvar 4. If Y = V + b, where b is a constant,
WVWV
WWVWWVVV
WVWWVV
YWYV
YWVYYY
,cov2varvar
,cov,cov,cov,cov
][,cov][,cov
,cov,cov
],[cov,covvar
20
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Variance rules
),(cov
))((
)()(var 2
XX
XXE
XEX
XX
X
The proofs of these rules can be derived from the results for covariances, noting that the variance of Y is equivalent to the covariance of Y with itself.
1. If Y = V + W,
WVWVY ,cov2varvarvar
Proof
21
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Variance rules
We start by replacing one of the Y arguments by V + W.
1. If Y = V + W,
WVWVY ,cov2varvarvar
Proof
WVWV
WWVWWVVV
WVWWVV
YWYV
YWVYYY
,cov2varvar
,cov,cov,cov,cov
][,cov][,cov
,cov,cov
],[cov,covvar
22
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Variance rules
We then use covariance rule 1.
1. If Y = V + W,
WVWVY ,cov2varvarvar
Proof
WVWV
WWVWWVVV
WVWWVV
YWYV
YWVYYY
,cov2varvar
,cov,cov,cov,cov
][,cov][,cov
,cov,cov
],[cov,covvar
23
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Variance rules
We now substitute for the other Y argument in both terms and use covariance rule 1 a second time.
1. If Y = V + W,
WVWVY ,cov2varvarvar
Proof
WVWV
WWVWWVVV
WVWWVV
YWYV
YWVYYY
,cov2varvar
,cov,cov,cov,cov
][,cov][,cov
,cov,cov
],[cov,covvar
24
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Variance rules
WVWV
WWVWWVVV
WVWWVV
YWYV
YWVYYY
,cov2varvar
,cov,cov,cov,cov
][,cov][,cov
,cov,cov
],[cov,covvar
This gives us the result. Note that the order of the arguments does not affect a covariance expression and hence cov(W, V) is the same as cov(V, W).
1. If Y = V + W,
WVWVY ,cov2varvarvar
Proof
25
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Variance rules
Zb
ZZb
bZbZYYY
var
,cov
,cov,covvar
2
2
The proof of the variance rule 2 is even more straightforward. We start by writing var(Y) as cov(Y, Y). We then substitute for both of the iYi arguments and take the b terms outside as common factors.
Proof
2. If Y = bZ, where b is a constant,
ZbY varvar 2
3. If Y = b, where b is a constant,
26
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Variance rules
0,covvar bbY
0var Y
The third rule is trivial. We make use of covariance rule 3. Obviously if a variable is constant, it has zero variance.
Proof
4. If Y = V + b, where b is a constant,
27
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Variance rules
V
bVbVbVY
var
,cov2varvarvarvar
VY varvar
The fourth variance rule starts by using the first. The second term on the right side is zero by variance rule 3. The third is also zero by covariance rule 3.
Proof
The intuitive reason for this result is easy to understand. If you add a constant to a variable, you shift its entire distribution by that constant. The expected value of the squared deviation from the mean is unaffected.
00 V + b
VmV
mV + b
28
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Variance rules
Proof
4. If Y = V + b, where b is a constant,
VY varvar
VY varvar
cov(X, Y) is unsatisfactory as a measure of association between two variables X and Y because it depends on the units of measurement of X and Y.
29
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Correlation
22YX
XYXY
population correlation coefficient
A better measure of association is the population correlation coefficient because it is dimensionless. The numerator possesses the units of measurement of both X and Y.
30
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Correlation
22YX
XYXY
population correlation coefficient
The variances of X and Y in the denominator possess the squared units of measurement of those variables.
31
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Correlation
22YX
XYXY
population correlation coefficient
However, once the square root has been taken into account, the units of measurement are the same as those of the numerator, and the expression as a whole is unit free.
32
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Correlation
22YX
XYXY
population correlation coefficient
If X and Y are independent, rXY will be equal to zero because sXY will be zero.
33
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Correlation
22YX
XYXY
population correlation coefficient
If there is a positive association between them, sXY, and hence rXY, will be positive. If there is an exact positive linear relationship, rXY will assume its maximum value of 1. Similarly, if there is a negative relationship, rXY will be negative, with minimum value of –1.
34
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Correlation
22YX
XYXY
population correlation coefficient
If X and Y are independent, rXY will be equal to zero because sXY will be zero.
35
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Correlation
22YX
XYXY
population correlation coefficient
If there is a positive association between them, sXY, and hence rXY, will be positive. If there is an exact positive linear relationship, rXY will assume its maximum value of 1.
36
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Correlation
22YX
XYXY
population correlation coefficient
Similarly, if there is a negative relationship, rXY will be negative, with minimum value of –1.
37
COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Correlation
22YX
XYXY
population correlation coefficient
Copyright Christopher Dougherty 2012.
These slideshows may be downloaded by anyone, anywhere for personal use.
Subject to respect for copyright and, where appropriate, attribution, they may be
used as a resource for teaching an econometrics course. There is no need to
refer to the author.
The content of this slideshow comes from Section R.4 of C. Dougherty,
Introduction to Econometrics, fourth edition 2011, Oxford University Press.
Additional (free) resources for both students and instructors may be
downloaded from the OUP Online Resource Centre
http://www.oup.com/uk/orc/bin/9780199567089/.
Individuals studying econometrics on their own who feel that they might benefit
from participation in a formal course should consider the London School of
Economics summer school course
EC212 Introduction to Econometrics
http://www2.lse.ac.uk/study/summerSchools/summerSchool/Home.aspx
or the University of London International Programmes distance learning course
EC2020 Elements of Econometrics
www.londoninternational.ac.uk/lse.
2012.10.31