lecture 5 - başkent Üniversitesiosezgin/lecture 5.pdf · lecture 5 joint, marginal, conditional...
TRANSCRIPT
![Page 1: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/1.jpg)
LECTURE 5
JOINT, MARGINAL, CONDITIONAL DISTRIBUTION
![Page 2: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/2.jpg)
MULTIVARIATE RANDOM VARIABLES
• In many applications there will be more than one random variable
• Often used to study the relationship among characteristics and the prediction of one based on the other(s)
• Three types of distributions: – Joint: Distribution of outcomes across all
combinations of variables levels – Marginal: Distribution of outcomes for a single
variable – Conditional: Distribution of outcomes for a single
variable, given the level(s) of the other variable(s)
1 2,X ,..., XkX
![Page 3: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/3.jpg)
JOINT DISCRETE DISTRIBUTION
Let X1, X2, …, Xk denote k discrete random variables, then
p(x1, x2, …, xk )
is joint probability function of X1, X2, …, Xk if
1
12. , , 1n
n
x x
p x x
11. 0 , , 1np x x
1 13. , , , ,n nP X X A p x x
1, , nx x A
![Page 4: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/4.jpg)
Example
• For e.g.: Tossing two fair dice 36 possible sample points
• Let X: sum of the two dice;
Y: difference of the two dice
– For (3,3), X=6 and Y=0.
– For both (4,1) and (1,4), X=5, Y=3.
![Page 5: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/5.jpg)
• Joint pmf (pdf) of (x,y)
x
y
2 3 4 5 6 7 8 9 10 11 12
0 1/36 1/36 1/36 1/36 1/36 1/36
1 1/18 1/18 1/18 1/18 1/18
2 1/18 1/18 1/18 1/18
3 1/18 1/18 1/18
4 1/18 1/18
5 1/18
Empty cells are equal to 0. e.g.P(X=7,Y≤4)=f(7,0)+f(7,1)+f(7,2)+f(7,3)+f(7,4)=0+1/18+0+1/18+0=1/9
![Page 6: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/6.jpg)
Example
• A bridge hand (13 cards) is selected from a deck of 52 cards.
• X = the number of spades in the hand.
• Y = the number of hearts in the hand.
• In this example we will define:
• p(x,y) = P[X = x, Y = y]
(Extended Hypergeometric Distribution)
![Page 7: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/7.jpg)
Note:
The possible values of X are 0, 1, 2, …, 13
The possible values of Y are also 0, 1, 2, …, 13
and X + Y ≤ 13.
, ,p x y P X x Y y
13 13 26
13
52
13
x y x y
The total number of
ways of choosing the
13 cards for the hand
The number of
ways of choosing
the x spades for the
hand
The number of
ways of choosing
the y hearts for the
hand
The number of ways
of completing the hand
![Page 8: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/8.jpg)
p(x,y)
0 1 2 3 4 5 6 7 8 9 10 11 12 13
0 0.0000 0.0002 0.0009 0.0024 0.0035 0.0032 0.0018 0.0006 0.0001 0.0000 0.0000 0.0000 0.0000 0.0000
1 0.0002 0.0021 0.0085 0.0183 0.0229 0.0173 0.0081 0.0023 0.0004 0.0000 0.0000 0.0000 0.0000 -
2 0.0009 0.0085 0.0299 0.0549 0.0578 0.0364 0.0139 0.0032 0.0004 0.0000 0.0000 0.0000 - -
3 0.0024 0.0183 0.0549 0.0847 0.0741 0.0381 0.0116 0.0020 0.0002 0.0000 0.0000 - - -
4 0.0035 0.0229 0.0578 0.0741 0.0530 0.0217 0.0050 0.0006 0.0000 0.0000 - - - -
5 0.0032 0.0173 0.0364 0.0381 0.0217 0.0068 0.0011 0.0001 0.0000 - - - - -
6 0.0018 0.0081 0.0139 0.0116 0.0050 0.0011 0.0001 0.0000 - - - - - -
7 0.0006 0.0023 0.0032 0.0020 0.0006 0.0001 0.0000 - - - - - - -
8 0.0001 0.0004 0.0004 0.0002 0.0000 0.0000 - - - - - - - -
9 0.0000 0.0000 0.0000 0.0000 0.0000 - - - - - - - - -
10 0.0000 0.0000 0.0000 0.0000 - - - - - - - - - -
11 0.0000 0.0000 0.0000 - - - - - - - - - - -
12 0.0000 0.0000 - - - - - - - - - - - -
13 0.0000 - - - - - - - - - - - - -
13 13 26
13
52
13
x y x y
![Page 9: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/9.jpg)
Example Suppose that we observe an experiment that has k possible outcomes {O1, O2, …, Ok } independently n times.
Let p1, p2, …, pk denote probabilities of O1, O2, …, Ok respectively.
Let Xi denote the number of times that outcome Oi occurs in the n repetitions of the experiment.
Then the joint probability function of the random variables X1, X2, …, Xk is (The Multinomial distribution)
1 2
1 1 2
1 2
! , ,
! ! !kxx x
n k
k
np x x p p p
x x x
![Page 10: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/10.jpg)
Note:
is the probability of a sequence of length n containing
x1 outcomes O1
x2 outcomes O2
…
xk outcomes Ok
1 2
1 2kxx x
kp p p
![Page 11: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/11.jpg)
1 21 2
!
! ! ! kk
nn
x x xx x x
is the number of ways of choosing the positions for the x1
outcomes O1, x2 outcomes O2, …, xk outcomes Ok
1 21
31 2
k
k
n x x xn n x
x xx x
1 1 2
1 1 2 1 2 3 1 2 3
! !!
! ! ! ! ! !
n x n x xn
x n x x n x x x n x x x
1 2
!
! ! !k
n
x x x
![Page 12: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/12.jpg)
is called the Multinomial distribution
1 2
1 1 2
1 2
! , ,
! ! !kxx x
n k
k
np x x p p p
x x x
1 2
1 2
1 2
kxx x
k
k
np p p
x x x
![Page 13: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/13.jpg)
Suppose that a treatment for back pain has three possible outcomes:
O1 - Complete cure (no pain) – (30% chance)
O2 - Reduced pain – (50% chance)
O3 - No change – (20% chance)
Hence p1 = 0.30, p2 = 0.50, p3 = 0.20.
Suppose the treatment is applied to n = 4 patients suffering back pain and let X = the number that result in a complete cure, Y = the number that result in just reduced pain, and Z = the number that result in no change.
Find the distribution of X, Y and Z.
4! , , 0.30 0.50 0.20 4
! ! !
x y zp x y z x y z
x y z
![Page 14: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/14.jpg)
Table: p(x,y,z) z
x y 0 1 2 3 4
0 0 0 0 0 0 0.0016
0 1 0 0 0 0.0160 0
0 2 0 0 0.0600 0 0
0 3 0 0.1000 0 0 0
0 4 0.0625 0 0 0 0
1 0 0 0 0 0.0096 0
1 1 0 0 0.0720 0 0
1 2 0 0.1800 0 0 0
1 3 0.1500 0 0 0 0
1 4 0 0 0 0 0
2 0 0 0 0.0216 0 0
2 1 0 0.1080 0 0 0
2 2 0.1350 0 0 0 0
2 3 0 0 0 0 0
2 4 0 0 0 0 0
3 0 0 0.0216 0 0 0
3 1 0.0540 0 0 0 0
3 2 0 0 0 0 0
3 3 0 0 0 0 0
3 4 0 0 0 0 0
4 0 0.0081 0 0 0 0
4 1 0 0 0 0 0
4 2 0 0 0 0 0
4 3 0 0 0 0 0
4 4 0 0 0 0 0
![Page 15: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/15.jpg)
MARGINAL DISCRETE DISTRIBUTIONS
Let X1, X2, …, Xq, Xq+1 …, Xk denote k discrete random variables with joint probability function
p(x1, x2, …, xq, xq+1 …, xk )
1
12 1 1 , , , ,q n
q q n
x x
p x x p x x
then the marginal joint probability function
of X1, X2, …, Xq is
![Page 16: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/16.jpg)
• If the pair (X1,X2) of discrete random variables has the joint pmf p(x1,x2), then the marginal pmfs of X1 and X2 are
2 1
1 1 1 2 2 2 1 2, and p ,x x
p x p x x x p x x
![Page 17: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/17.jpg)
Example A die is rolled n = 5 times
X = the number of times a “six” appears.
Y = the number of times a “five” appears.
0 1 2 3 4 5 p X (x )
0 0.1317 0.1646 0.0823 0.0206 0.0026 0.0001 0.4019
1 0.1646 0.1646 0.0617 0.0103 0.0006 0 0.4019
2 0.0823 0.0617 0.0154 0.0013 0 0 0.1608
3 0.0206 0.0103 0.0013 0 0 0 0.0322
4 0.0026 0.0006 0 0 0 0 0.0032
5 0.0001 0 0 0 0 0 0.0001
p Y (y ) 0.4019 0.4019 0.1608 0.0322 0.0032 0.0001
![Page 18: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/18.jpg)
Example Assume that the random variables X and Y have the joint probability mass function given as
f(x ,y)= λx+y e -2λ x = 0 , 1 , 2 ,.. x!y! y=0,1,2,…… Find the marginal distribution of X
∑ ∞
ƒ(x)= ∑ λx+y e-2λ = x! y! = λx e -λ x!
[Using e λ = λt ] t = 0 t !
λY
Y! Y=0
λ x
X! e-2λ
∑ ∞
![Page 19: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/19.jpg)
Example Let the joint distribution of X and Y be given as
f(x , y) = x + y x = 0,l,2,3 y = 0,1,2, 30
y x 0 1 2 3
0 0 1/30 1/15 1/10
1 1/30 1/15 1/10 2/15
2 1/15 1/10 2/15 1/6
1/10 1/5 3/10 2/5
Y 0 1 2
f(y) 1/5 1/3 7/15
Find the marginal distribution function of X and Y. Marginal of X Similarly, f(y) = (3+2x)/15 Or , since the joint is the marginal of y
x 0 1 2 3
f(x) 1/10 1/5 3/10 2/5
10
1]33[
30
1)]2()1()0[(
30
1
30),()(
2
0
2
0
xxxxx
yxyxfxf
yy
![Page 20: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/20.jpg)
JOINT DISCRETE CUMULATIVE DISTRIBUTION FUNCTION
• F(x1,x2) is a cdf iff
.xX,...,xXPx,...,x,xF kk11k21
. x x ,,,lim,lim
d.c and ba ,0,,,,),(
1,,lim
.,0,,lim
.,0,,lim
2 121210h
210h
21
21
1121
2221
2
1
2
1
andxxFhxxFxhxF
caFdaFcbFdbFdXcbXaP
FxxF
xxFxxF
xxFxxF
xx
x
x
![Page 21: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/21.jpg)
CONDITIONAL DISCRETE DISTRIBUTION
Let X1, X2, …, Xq, Xq+1 …, Xk denote k discrete random variables with joint probability function
p(x1, x2, …, xq, xq+1 …, xk )
1
1 11 1
1 1
, , , , , ,
, ,
k
q q kq q k
q k q k
p x xp x x x x
p x x
then the conditional joint probability function
of X1, X2, …, Xq given Xq+1 = xq+1 , …, Xk = xk is
![Page 22: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/22.jpg)
Let X and Y denote two discrete random variables
with joint probability function
p(x,y) = P[X = x, Y = y]
Then
pX |Y(x|y) = P[X = x|Y = y] is called the conditional
probability function of X given Y
= y and
pY |X(y|x) = P[Y = y|X = x] is called the conditional
probability function of Y given
X = x
![Page 23: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/23.jpg)
Note
X Yp x y P X x Y y
, ,
Y
P X x Y y p x y
P Y y p y
Y Xp y x P Y y X x
, ,
X
P X x Y y p x y
P X x p x
and
![Page 24: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/24.jpg)
Example A die is rolled n = 5 times
X = the number of times a “six” appears.
Y = the number of times a “five” appears.
0 1 2 3 4 5 p X (x )
0 0.1317 0.1646 0.0823 0.0206 0.0026 0.0001 0.4019
1 0.1646 0.1646 0.0617 0.0103 0.0006 0 0.4019
2 0.0823 0.0617 0.0154 0.0013 0 0 0.1608
3 0.0206 0.0103 0.0013 0 0 0 0.0322
4 0.0026 0.0006 0 0 0 0 0.0032
5 0.0001 0 0 0 0 0 0.0001
p Y (y ) 0.4019 0.4019 0.1608 0.0322 0.0032 0.0001
x
y
![Page 25: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/25.jpg)
The conditional distribution of X given Y = y.
0 1 2 3 4 5
0 0.3277 0.4096 0.5120 0.6400 0.8000 1.0000
1 0.4096 0.4096 0.3840 0.3200 0.2000 0.0000
2 0.2048 0.1536 0.0960 0.0400 0.0000 0.0000
3 0.0512 0.0256 0.0080 0.0000 0.0000 0.0000
4 0.0064 0.0016 0.0000 0.0000 0.0000 0.0000
5 0.0003 0.0000 0.0000 0.0000 0.0000 0.0000
pX |Y(x|y) = P[X = x|Y = y]
x
y
![Page 26: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/26.jpg)
0 1 2 3 4 5
0 0.3277 0.4096 0.2048 0.0512 0.0064 0.0003
1 0.4096 0.4096 0.1536 0.0256 0.0016 0.0000
2 0.5120 0.3840 0.0960 0.0080 0.0000 0.0000
3 0.6400 0.3200 0.0400 0.0000 0.0000 0.0000
4 0.8000 0.2000 0.0000 0.0000 0.0000 0.0000
5 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000
The conditional distribution of Y given X = x.
pY |X(y|x) = P[Y = y|X = x]
x
y
![Page 27: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/27.jpg)
Example The joint probability mass function of X and Y is given by
f(1,1) = 1 f(1,2) = 1 f(2,1)= 1 f(2,2)= 1 8 4 8 2
1.Compute the conditional mass function of X given Y = i, i =1,2 2.Compute P(XY ≤ 3) = f(1, 1) + f(1, 2) + f(2, 1) = 1/2 3. P(X/Y> l) = f(2, 1)= 1/8
Y x 1 2 Sum
1 1/8 1/8 2/8
2 1/4 1/2 6/8
sum 3/8 5/8 1
Marginal of y The conditional mass f n of X/Y =1 The conditional of X/Y=2
y 1 2 f(y) 2/8 6/8
x 1 2 f(x\y=1) 1/2 1/2
y 1 2 Sum
f(x\y=2) 1/3 2/3 1
![Page 28: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/28.jpg)
JOINT CONTINUOUS DISTRIBUTION
Let X1, X2, …, Xk denote k continuous random variables, then
f(x1, x2, …, xk )
is joint density function of X1, X2, …, Xk if
1 12. , , , , 1n nf x x dx dx
11. , , 0nf x x
1 1 13. , , , , , ,n n nP X X A f x x dx dx
A
![Page 29: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/29.jpg)
Example
• Assume that joint pdf of random variable X and Y is the joint cdf is
( , ) 4xy 0 x 1, 0 y 1f x y
2 2
1 2 1 2
0 0
( , ) 4 0 x 1, 0 y 1
yx
F x y t t dt dt x y
![Page 30: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/30.jpg)
MARGINAL CONTINUOUS DISTRIBUTION
Let X1, X2, …, Xq, Xq+1 …, Xk denote k continuous random variables with joint probability density function
f(x1, x2, …, xq, xq+1 …, xk )
12 1 1 1 , , , ,q q n q nf x x f x x dx dx
then the marginal joint probability function
of X1, X2, …, Xq is
![Page 31: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/31.jpg)
• If the pair (X1,X2) of discrete random variables has the joint pdf f(x1,x2), then the marginal pdfs of X1 and X2 are
.,, 1212222111 dxxxfxf and dxxxfxf
![Page 32: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/32.jpg)
Example Joint density f(x,y) for X and Y:
Marginal density function for X:
otherwise0
1y01,x03y)(2x5
2y)f(x,
5
3x
5
41
05
1
5
2
5
2
2
1
0
y3xy2
dy)y3x2(
dy)y,x(f)x(g
![Page 33: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/33.jpg)
• If X1, X2,…,Xk are independent from each other, then the joint pdf can be given as
And the joint cdf can be written as
k21k21 xf...xfxfx,...,x,xf
k21k21 xF...xFxFx,...,x,xF
INDEPENDENCE OF RANDOM VARIABLES
![Page 34: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/34.jpg)
Let X1, X2, …, Xq, Xq+1 …, Xk denote k continuous random variables with joint probability density function
f(x1, x2, …, xq, xq+1 …, xk )
then the conditional joint probability function
of X1, X2, …, Xq given Xq+1 = xq+1 , …, Xk = xk is
1
1 11 1
1 1
, , , , , ,
, ,
k
q q kq q k
q k q k
f x xf x x x x
f x x
CONTINUOUS CONDITIONAL DISTRIBUTIONS
![Page 35: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/35.jpg)
• If X1 and X2 are continuous random variables with joint pdf f(x1,x2), then the conditional pdf of X2 given X1=x1 is defined by
• For independent rvs,
elsewhere. 0 f that such ,0xx,xf
x,xfxxf 11
1
2112
.
.
121
212
xfxxf
xfxxf
![Page 36: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/36.jpg)
• If X and Y are random variables with joint pdf and marginal pdf’s
and if X and Y are independent then
( , )f x y ( ), ( )f x f y
( , ) ( ) ( | ) ( ) ( | )f x y f x f y x f y f x y
( | ) ( )
( | ) ( )
f x y f x
f y x f y
![Page 37: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/37.jpg)
Suppose that a rectangle is constructed by first choosing
its length, X and then choosing its width Y.
Its length X is selected form an exponential distribution
with mean = 1/ = 5. Once the length has been chosen
its width, Y, is selected from a uniform distribution form
0 to half its length.
Find the joint distribution function.
Example
![Page 38: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/38.jpg)
, X Y Xf x y f x f y x
151
5 for 0
x
Xf x e x
1 if 0 2
2Y X
f y x y xx
1 15 51 2
5 5
1= if 0 2, 0
2
x x
xe e y x x
x
![Page 39: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/39.jpg)
Example
Let X, Y, Z denote 3 jointly distributed random variable with joint density function then
2 0 1,0 1,0 1, ,
0 otherwise
K x yz x y zf x y z
Find the value of K.
Determine the marginal distributions of X, Y and Z.
Determine the joint marginal distributions of
X, Y
X, Z
Y, Z
![Page 40: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/40.jpg)
1 1 1
2
0 0 0
1 , ,f x y z dxdydz K x yz dxdydz
Determining the value of K.
11 1 1 13
0 0 0 00
1
3 3
x
x
xK xyz dydz K yz dydz
11 12
0 00
1 1 1
3 2 3 2
y
y
yK y z dz K z dz
12
0
1 1 71
3 4 3 4 12
z zK K K
12if
7K
![Page 41: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/41.jpg)
1 1
2
1
0 0
12, ,
7f x f x y z dydz x yz dydz
The marginal distribution of X.
11 122 2
0 00
12 12 1
7 2 7 2
y
y
yx y z dz x z dz
12
2 2
0
12 12 1 for 0 1
7 4 7 4
zx z x x
![Page 42: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/42.jpg)
1
2
12
0
12, , ,
7f x y f x y z dz x yz dz
The marginal distribution of X,Y.
12
2
0
12
7 2
z
z
zx z y
212 1 for 0 1,0 1
7 2x y x y
![Page 43: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/43.jpg)
The marginal distribution of X,Y.
2
12
12 1, for 0 1,0 1
7 2f x y x y x y
Thus the conditional distribution of Z given X = x,Y = y
is 2
212
12, , 7
12 1,
7 2
x yzf x y z
f x yx y
2
2
for 0 11
2
x yzz
x y
![Page 44: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/44.jpg)
The marginal distribution of X.
2
1
12 1 for 0 1
7 4f x x x
Thus the conditional distribution of Y , Z given X = x is
2
21
12, , 7
12 1
7 4
x yzf x y z
f xx
2
2
for 0 1,0 11
4
x yzy z
x
![Page 45: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/45.jpg)
Example (Bivariate Normal Distribution)
• A pair of continuous rvs X and Y is said to have a bivariate normal distribution if it has a joint pdf of the form
2
2
y
2
y
1
x2
1
x22
21
yyx2
x
12
1exp
12
1y,xf
.11,0,0,y,x 21
![Page 46: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/46.jpg)
If ,,,,BVN~Y,Xyx
2
2
2
1
2
2
2
1,N~Y and ,N~X
yx
, then
and is the correlation coefficient btw X and Y. 1. Conditional on X=x,
222X
1
2y 1),x(N~xY
2. Conditional on Y=y,
221Y
2
1x 1),y(N~yX
![Page 47: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/47.jpg)
CONDITIONAL EXPECTATION
• For X, Y discrete random variables, the conditional expectation of Y given X = x is
and the conditional variance of Y given X = x is where these are defined only if the sums converges
absolutely. • In general, y
XY xypyhxXYhE || |
22
|
2
||
|||
xXYExXYE
xypxXYEyxXYVy
XY
y
XY xypyxXYE || |
![Page 48: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/48.jpg)
• For X, Y continuous random variables, the conditional expectation of Y given X = x is
and the conditional variance of Y given X = x is
• In general,
dyxyfyxXYE XY || |
22
|
2
||
|||
xXYExXYE
dyxyfxXYEyxXYV XY
dyxyfyhxXYhEy
XY || |
![Page 49: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/49.jpg)
• If X and Y are jointly distributed random variables then,
• If X and Y are independent random variables then,
• If X and Y are jointly distributed random variables then,
E[E(Y|X)]=E(Y)
E(Y|X)=E(Y)
E(X|Y)=E(X)
XVar(Y)=E [Var(Y|X)]+Var [E(X | Y)]X
![Page 50: LECTURE 5 - Başkent Üniversitesiosezgin/LECTURE 5.pdf · LECTURE 5 JOINT, MARGINAL, CONDITIONAL DISTRIBUTION . MULTIVARIATE RANDOM VARIABLES •In many applications there will be](https://reader033.vdocuments.us/reader033/viewer/2022053012/5f0f22247e708231d442a76a/html5/thumbnails/50.jpg)
MOMENT GENERATING FUNCTION OF A JOINT DISTRIBUTION
• The E(e^tX) and M’(0) approaches both work
• Can get E(XY), Cov(X,Y)
MX ,Y (t1,t2) E[e t1X t2Y ]
E[X nY m ]dn m
dnt1dmt2MX ,Y (t1,t2) |t1 t2 0