1

ECE 450 - Lecture #9 Part 2 Overview

• Bivariate Moments

– Mean or Expected Value of Z = g(X, Y)

– Correlation and Covariance of 2 RV’s

• Functions of 2 RV’s: Z = g(X, Y); finding fZ(z)

– Method 1: First find F(z), by definition; then differentiate to

find f(z);

– Method 2: Method of Auxiliary Variables

2

Expected Value of Z = g(X, Y)

For continuous RV’s: E(Z) = E[g(X, Y)]

For discrete RV’s:

E(Z) = E[g(X, Y)]

dydx)y,x(f)y,x(g XY

dz)z(fz Z

(often hard

to find)

)yYxXPr()y,x(g k,jk

kjj

3

Property/Example: Expected Value

• Say Z = X + Y

• E(Z) = E(X + Y) =

• Note: Expectation is linear!

• Similarly, E(aX + bY + c)= ____________________

dydx)y,x(f)yx( XY

dydx)y,x(fydydx)y,x(fx XYXY

)Y(E)X(Edy)y(fydx)x(fx YX

4

Correlation of RV’s

• Definition: The correlation of RV’s X and Y is: E(______).

• Calculation: E(XY) =

• Concept: Correlation is a measure of similarity –

– (Large) positive correlation X, Y tend to be both negative or both positive, on the average

– (Large, in abs. value) negative correlation X, Y tend to be opposite in sign, on the average

• Note E(XY) E(X) E(Y), in general

• Problem with correlation as a measure of similarity: the number may be “large” just due to the fact the RV’s take large values.

• Ex: correlation between gpa’s and entry-level salaries would be larger if we measured salaries in cents rather than dollars.

dydx)y,x(fxy XY

5

Covariance of RV’s

• Definition: The covariance of RV’s X and Y is

cov(X, Y)

• Calculation:

• Property #1: cov(X, Y) = E(____) – E(___) E(___)

– Mean of the product minus the product of the means

– Similar to the expression for variance

– Covariance is a (partially) normalized measure of similarity between RV’s

)}YY()XX{(E

dydx)y,x(f)Yy()Xx()Y,Xcov( XY

6

Proof of Property #1

Cov(X, Y) =

= E(___) - “term2” – “term3” + E(__) E(__)

where terms 2 and 3 are computed on the next page:

dydx)y,x(f)Yy()Xx( XY

dydx)y,x(f)YXYxyXxy( XY

dydx)y,x(fyXdydx)y,x(fxy XYXY

dydx)y,x(fYXdydx)y,x(fYx XYXY

7

Proof of Property #1, continued

So far, cov(X, Y) = E(_____) - “term2” – “term3” + E(____) E(____)

where term2

and where term3 =

cov(X, y)

dydxyxfyXdydxyxfyX XYXY ),(),(

YXdyyfyXdyydxyxfX YXY )(),(

dydxyxfxYdydxyxfYx XYXY ),(),(

XYdxxxfYdxxdyyxfY XXY )(),(

YXXYEYXXYYXXYE )()(

8

More About Covariances

• Var(X Y) = Var(X) + Var(Y) 2 cov(X, Y)

• Definition: 2 RV’s X and Y are uncorrelated if

cov(X, Y) = 0.

• Property: X, Y independent X, Y uncorrelated

(not, in general, conversely)

9

Correlation Coefficient

• Defn: The correlation coefficient between 2 RV’s X and Y

is

r = (often denoted: r)

• Measures the degree to which X and Y are statistically

related in a linear sense.

• Property 1: -1 r 1

• Property 2: If Y = aX + b, where a 0

then r = 1 (r = 1 if a > 0)

YX

)Y,Xcov(

Note: the parameter r (or r) in the pdf for joint Gaussian RV’s

is the correlation coefficient for X and Y.

10

Correlation Intuition

re: Correlation Coefficient, r

• Say we run an experiment in which we measure the outcomes of RV’s

X and Y, and plot the resulting points in the x-y plane:

Case 1: r .9 Case 2: r -.9

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

-1 -0.75 -0.5 -0.25 0 0.25 0.5 0.75 1

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

-1 -0.75 -0.5 -0.25 0 0.25 0.5 0.75 1

(highly correlated in the positive

sense; nearly fall on a line with

positive slope.

(highly correlated in the negative

sense); nearly fall on a line with

negative slope.

11

Correlation Intuition, continued

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

-1 -0.75 -0.5 -0.25 0 0.25 0.5 0.75 1

Case 3: r 0 Case 4: r 0

(little correlation; little dependence

of any kind between X and Y)

-5 -4 -3 -2 -1 0 1 2 3 4 5

-4

-3

-2

-1

0

1

2

3

4

(little correlation; little linear

dependence, but obviously there

exists a strong dependence

between X and Y)

12

Example 1

Givens: E(X) = 0, E(Y) = 2, var(X) = 4, var(Y) = 1, rXY = .4;

W = X + Y; Z = 2X + 3Y

Find

a) The mean of W and the mean of Z;

E(W) = E(X + Y) = _____ + _____ = ___ + ___ = ___

E(Z) = E(2X + 3Y) = 2 _____ + 3 ____ = 2(___) + 3(___) =

6

b) The variance of W and the variance of Z;

var(W) = var(X + Y) = var(X) + var(Y) + 2 cov(X, Y)

(need this)

13

Example 1, continued

Thus, var(W) = var(X + Y) = var(X) + var(Y) + 2 cov(X, Y)

= 4 + 1 + 2(.8) = 6.6

And, var(Z) = var(U + V), where U = 2X, V = 3Y

= var(U) + var(V) + 2 cov(U, V)

var(U) = var(2X) = 4 var(X) = 4(4) = 16;

var(V) = var(3Y) = 9 var(Y) = 9(1) = 9;

cov(U, V) = E(UV) – E(U) E(V)

= E[ (2X) (3Y) ] – 2E(X) 3E(Y)= 6 E(XY)

8.)1)(2(4.r)Y,Xcov()Y,Xcov(

r YXXYYX

XY

(need

this)

14

Example 1, continued

To find E(XY): cov(XY) = E(XY) – E(X) E(Y)

.8 = E(XY) – (0) (2) E(XY) = .8

Thus, cov(U, V) = 6 E(XY) = 6 (.8) = 4.8

Hence, var(Z) = var(U) + var(V) + 2 cov(U, V)

= 16 + 9 + 2(4.8) = 34.6

c) The correlation coefficient, rWZ, of W and Z.

First we will find cov(W, Z).

15

Example 1, continued

Cov(W, Z) = E(WZ) – E(W) E(Z)

E(WZ) = E[(X+Y)(2X + 3Y)] = E[2X2 + 5XY + 3Y2]

= 2 E(X2) + 5 E(XY) + 3 E(Y2)

var(X) = E(X2) – (E(X))2

4 = E(X2) – 0 E(X2) = 4

var(Y) = E(Y2) – (E(Y))2

1 = E(Y2) – (2)2 E(Y2) = 5

Thus, E(WZ) = 2 E(X2) + 5 E(XY) + 3 E(Y2)

= 2 (4) + 5(.8) + 3 (5) = 27

Therefore, Cov(W, Z) = 27 – (2)(6) = 27 – 12 = 15

16

Example 2 (continued from Lecture 8)

• Recall fXY(x, y) = 6 (1 – x – y), on a triangle:

• So far: fX(x) = 3(1 - x)2 0 x < 1

0 else

fY(y) = 3(1 – y)2 0 y < 1

else

• Find (a) cov(X, Y) and (b) rXY.

• (a) Solution: we need E(XY), E(X), and E(Y) since

cov(X, Y) = E(XY) – E(X) E(Y)

17

Example 2, continued

• E(X) =

• E(Y) =

• E(XY) =

=

1

0

2X 4

1dx)x1(x3dx)x(fx

1

0

2y 4

1dy)y1(y3dy)y(fy

dydx)y,x(fxy XY

dydx)yx1(xy61

0 1x

y

y = -x + 1

1

0x

x1

0y

22 dxdy)xyyxxy(6

18

Example 2, continued

• Inner Integral:

• Thus (evaluating the outer integral),

E(XY) =

cov(X, Y) = E(XY) – E(X) E(Y) = (1/20) – (1/4) (1/4)

= (1/20) – (1/16) = -1/80

1

0x

3

201dx)x1(x

x1

0y

322

6

)x1(xdy)xyyxxy(

19

Example 2, continued

b) Find rXY

To get var(X), we need E(X2); to get var(Y), we need E(Y2)

E(X2) =

Thus, var(X) = var(Y) = E(X2) – (E(X))2 =(1/10) – (1/16) =3/80

rXY = = -1/3

)Y(E10

1dx)x1(x3dx)x(fx 21

0

22X

2

YXYXXY

)Y(E)X(E)XY(E)Y,Xcov(r

(need var(X), var(Y)

for denominator)

80/3

801

)Y,Xcov(

YX

20

Functions of 2 RV’s – Method 1

• Let Z = g(X, Y), where X and Y are RV’s

• FZ(z) = Pr(Z z) = Pr(g(X, Y) z)

FZ(z) =

• Then: fZ(z) =

dydxyxf

zR

XY ),(

)(

Defines a subset, say R(z), of the

xy-plane, meeting this condition

)(zFdz

dZ

21

Example

• Let Z = X2 + Y2, and

fXY(x, y) =

• So FZ(z) = Pr(Z z) =

• Polar Coordinates:

2

22

2 2exp

2

1

yx

dydx)y,x(f

zyx

XY22

r2 r dr dq

)z(Ue1ddr2

rexp

2

1r)z(F )2/(z

2

2

2

2

0

z

0rz

2

q

q

eu du

Find fZ(z).

22

Example, continued

• Thus, fz(z) =

=

dz

dzF

dz

dZ )(

)(1 )2/( 2

zUe z

2)2/(z)2/(z

2

1e)z(U)z(e1

22

product

rule

)z(Ue2

1 )2/(z2

2

23

Method 2: Auxiliary Variables(not shown in Cooper & McGillem)

• Given 2 inputs X, Y, and 1 output, Z = g(X, Y);

• Find fZ(z).

• Method of Auxiliary Variable: create a new (auxiliary)

variable, W = h(X, Y) = h(X).

g(x, y)X

Y ZModel of actual

problem

Zg(x, y)X

Y

h(x, y) W(aux.)

Aux. Variable

Model for

same problem

24

Method 2: Auxiliary Variables

- 4-step Approach -

1. Solve the output equations “backwards” for X and Y

2. Find the Jacobian of the transformation:

3. fZW(z, w) =

(analogous to fY(y) = , for transformation of a single RV)

4. fZ(z) =

w,z

XY

)y,x(J

)y,x(f

dw)w,z(fZW (getting rid of the auxiliary

variable)

dx

dy

)x(fX

x-1

25

Method 2, Example 1:

Let Z = X + Y; find fZ(z)

1. Solve the output equations “backwards” for X and Y:

Y = Z – X = _______ (in terms of Z, W)

X = W

g(x, y)X

Y

h(x, y) W = X(aux.)

Z = X + Y

26

Method 2, Example 1, continued:

(Z = X + Y; aux: W = X)

2. Find the Jacobian of the transformation, J(x, y):

J(x, y) =

3. fZW(z, w) =

4. fZ(z) = (**)

)wz,w(f|1|

)wz,w(f

)y,x(J

)y,x(fXY

XY

w,z

XY

101

11

dy

dw

dx

dw

dy

dz

dx

dz

det"old"d

"new"ddet

dw)wz,w(fdw)w,z(f XYZW

27

Method 2, Example 1 – Going Further

• Special Case of Interest: Z = X + Y where X, Y

independent

• Repeating the (**) result from p. 25:

fZ(z) =

• If X and Y are independent, this becomes:

fZ(z) =

d)z(f)(f

yYX

______________

d)z,(fdw)wz,w(f XYXY

28

Example 1, continued

Summary: If Z = X+Y, and if X and Y are independent

RV’s, then

fZ(z) = fX(z) * fY(z)

(This generalizes to the sum of an arbitrary # of

independent RV’s)

Specific Example: If X and Y are both U(0, 1), and if X

and Y are independent, find the pdf of Z = X + Y.

29

Example 1 – Specific Example, continued

Solving by Graphical Convolution:

0 1

fX(x)

x0 1

fY(y)

y

0 2

fZ(z)

z

1(Convolution

details will be

reviewed during

class.)

Check: area = 1

30

Other Things to Recall about Convolution

• Important for Discrete RV’s:

(x) * f(x) = f(x)

(x - a) * f(x) = f(x - a)

• In class application/example:

– Discrete RV X takes values 0 and 5 with equal

probability

– Discrete RV Y takes values 1 and -1 with equal

probability

– Find the pdf of RV Z = X + Y if X and Y are

independent.

Discrete RV X takes values 0 and 5 with equal probability

Discrete RV Y takes values 1 and -1 with equal probability

Find the pdf of RV Z = X + Y

_______________________________________________

_______________________________________________

Sketch of pdf’s:

31

32

Method 2, Example 2: Let Z = XY;

use aux. variable W = X; find fZ(z)

1. Solve the output equations “backwards” for X and Y:

X = W; Y = Z/W

2. Find the Jacobian of the transformation:

J(x, y) =

3. fZW(z, w) =w

)w/z,w(f

)y,x(J

)y,x(f XY

w,z

XY

x01

xy

dy

dw

dx

dw

dy

dz

dx

dz

det"old"d

"new"ddet

4. fZ(z) =