prbability theory and mathematical statistics lecture 05: probability distributions ... · 2019. 3....

Introduction Multivariate Distributions Marginal Distributions Conditional Distributions ending

Prbability Theory and Mathematical StatisticsLecture 05: Probability Distributions and

Probability Densities IIMultivariate Probabilities

Chih-Yuan Hung

School of Economics and ManagementDongguan University of Technology

March 27, 2019


Introduction

In the one r.v. case, we know that many different randomvariables can be defined over one and the same sample space.

Rolling two dice, we can consider many different r.v.s

Consider the following example


Example

Two caplets are selected at random from a bottle containing 3aspirin, 2 sedative, and 4 laxative caplets. If X and Y are,respectively, the numbers of aspirin and sedative caplets includedamong the 2 caplets drawn from the bottle, find the probabilitiesassociated with all possible pairs of values of X and Y .


It is preferable to express the probabilities by means of a functionwith the values f (x , y) = P(X = x ,Y = y) for any pair of valuesof (x , y) within the range of X and Y .The example above, we can have

f (x .y) =(3x)(

2y)(

42−x−y)

(92)for x = 0, 1, 2; y = 0, 1, 2; 0 ≤ x+ y ≤ 2


Joint Probability Distribution

Definition

If X and Y are discrete random variables, the function given byf (x , y) = P(X = x ,Y = y) for each pair of values (x , y) withinthe range of X and Y is called the joint probability distributionof X and Y .


Joint Probability Distribution

Theorem (7)

A bivariate function can serve as the joint probability distributionof a pair of discrete random variables X and Y if and only if itsvalues, f (x , y), satisfy the conditions

1 f (x , y) ≥ 0 for each pair of values (x , y) within its domain;

2 ∑x

∑yf (x , y) = 1, where the double summation extends over all

possible pairs (x , y) within its domain.


Example 13

Determine the value of k for which the function given by

f (x , y) = kxy , for x = 1, 2, 3; y = 1, 2, 3

can serve as a joint probability distribution.Solution: Substituting the various values of x and y , we get

yf (x , y) 1 2 3

x1 k 2k 3k2 2k 4k 6k3 3k 6k 9k

To satisfy the first condition of Theorem 7, the constant k must benonnegativeTo satisfy the second condition,

k + 2× 2k + 2× 3k + 4k + 2× 6k + 9k = 1,

which is k = 136 .


If we want to know that the probability that the values of tworandom variables are less than or equal to some real numbers xand y .

Definition (Joint Distribution Function)

If X and Y are discrete random variables, the function given by

F (x , y) = P(X ≤ x ,Y ≤ y) = ∑s≤x

∑t≤y

f (s, t)

for −∞ < x < ∞;−∞ < y < ∞

where f (s, t) is the value of the joint probability distribution of Xand Y at (s, t), is called the joint distribution function, or thejoint cumulative distribution of X and Y .


Exercise 48 (c)

Theorem

If F (x , y) is the value of the joint distribution function of twodiscrete random variables X and Y at (x, y), show that If a < band c < d , then F (a, c) ≤ F (b, d).

Proof.

By definition,

F (a, c) = ∑x≤a

∑y≤c

f (x , y)

F (b, d) = ∑x≤b

∑y≤d

f (x , y)

When a < b and c < d , we have


Proof.

F (b, d) = ∑x≤b

∑y≤d

f (x , y)

= ∑x≤b

(∑y≤c

f (x , y) + ∑c<y≤d

f (x , y)

)= ∑

x≤b∑y≤c

f (x , y) + ∑x≤b

∑c<y≤d

f (x , y)

= ∑x≤a

∑y≤d

f (x , y) + ∑a<x≤b

∑y≤d

f (x , y) + ∑x≤b

∑c<y≤d

f (x , y)

=F (a, c) + some summation of f (x , y)

Since f (x , y) ≥ 0 for all x in the range of X and y in the range ofY , we have

F (a, c) ≤ F (b, d)


Multivariate Distributions

With reference to Example 12,

F (1, 1) =P(x ≤ 1, y ≤ 1)

=f (0, 0) + f (0, 1) + f (1, 0) + f (1, 1)

=1

6+

2

9+

1

3+

1

6

=8

9

Note that, the joint distribution is defined for all real numbers.

Using the same example,

F (−2, 1) = P(X ≤ −2,Y ≤ 1) = 0

andF (3.7, 4.5) = P(X ≤ 3.7,Y ≤ 4.5) = 1.


Continuous Cases

Definition

A bivariate function with values f (x , y) defined over the xy -planeis called a joint probability density function of the continuousrandom variables X and Y if and only if

P(X ,Y ) ∈ A =∫∫A

f (x , y)dxdy

for any given A in the xy -plane


Theorem (8)

A bivariate function can serve as a joint probability densityfunction of a pair of continuous random variables X and Y if itsvalues, f (x , y), satisfy the conditions

1. f (x , y) ≥ 0 for −∞ < x < ∞ −∞ < y < ∞

2.∫ ∞

−∞

∫ ∞

−∞f (x , y)dxdy = 1


Example (15)

Given the joint probability density function

f (x , y) =

{35x(y + x) for 0 < x < 1, 0 < y < 2

0 elsewhere

of two random variables X and Y , find P [(X ,Y ) ∈ A], where A isthe region {(x , y)|0 < x < 1

2 , 1 < y < 2}


Solution

P [(X ,Y ) ∈ A] =P

(0 < x <

1

2, 1 < y < 2

)=∫ 2

1

∫ 12

0

3

5x(y + x)dxdy

=3

5

∫ 2

1

(x2y

2+

x3

3

)∣∣∣∣ 120

dy

=3

5

∫ 2

1

(y

8+

1

24

)dy

=3

5

(y2

16+

y

24

)∣∣∣∣21

=11

80


Joint Distribution Function

Definition (9)

If X and Y are continuous random variables, the function given by

F (x , y) = P(X ≤ x ,Y ≤ y) =∫ y

−∞

∫ x

−∞f (s, t)dxdy

for −∞ < x < ∞; −∞ < y < ∞where f (s, t) is the jointprobability density of X and Y at (s, t), is called the jointdistribution function of X and Y .

Note that partial differentiation in Definition 9 leads to

f (x , y) =∂2

∂x∂yF (x , y)


Example (16)

If the joint probability density of X and Y is given by

f (x , y) =

{x + y for 0 < x < 1, 0 < y < 1

0 elsewhere

find the joint distribution function of these two random variables.


Solution

The region of A ∈ {(x , y)|0 < x < 1, 0 < y < 1} can be dividedby four parts


Solution

For 0 < x < 1 and 0 < y < 1, we get

F (x , y) =∫ y

0

∫ x

0s + tdsdt =

1

2xy(x + y)

for x > 1 and 0 < y < 1, we get

F (x , y) =∫ y

0

∫ 1

0s + tdsdt =

1

2y(y + 1)

for 0 < x < 1 and y > 1, we get

F (x , y) =∫ 1

0

∫ x

0s + tdsdt =

1

2x(x + 1)

and for x > 1 and y > 1, we get

F (x , y) = 1


Solution

Summarize these,

F (x , y) =

0 for x ≤ 0ory ≤ 012xy(x + y) for 0 < x < 1, 0 < y < 112y(1 + y) for x ≥ 1, 0 < y < 112x(1 + x) for 0 < x < 1, y ≥ 1

1 for x ≥ 1, y ≥ 1


Example (17)

Find the joint probability density of the two random variables Xand Y whose joint distribution function is given by

F (x , y) =

{(1− e−x )(1− e−y ) for x > 0, y > 0

0 elsewhere

Also use the joint probability density to determineP(1 < X < 3, 1 < Y < 2).


Solution

Since partial differentiation yields

∂2

∂x∂yF (x , y) = e−(x+y )

for x > 0 and y > 0 and 0 elsewhere, we find the joint probabilitydensity of X an Y is given by

f (x , y) =

{e−(x+y ) for x > 0 and y > 0

0 elsewhere

Thus, the integration yields∫ 3

1

∫ 2

1e−(x+y )dxdy =(e−1 − e−3)(e−1 − e−2)

=e−2 − e−3 − e−4 + e−5

=0.074


Geometric Presentation of pdf and CDF

We are finding the volume under the surface of probability density


Generalization of Multivariate Probability

The values of the joint probability distribution of n discrete randomvariables X1,X2, ..., and Xn are given by

f (x1, x2, ..., xn) = P(X1 = x1,X2 = x2, ...,Xn = xn)

And the distribution function is given by

F (x1, x2, ..., xn) = P(X1 ≤ x1,X2 ≤ x2, ...,Xn ≤ xn)

for −∞ < x1 < ∞,−∞ < x2 < ∞, ...,−∞ < xn < ∞


Example (18)

If the joint probability distribution of three discrete randomvariables X, Y, and Z is given by

f (x , y , z) =(x + y)z

63for x = 1, 2; y = 1, 2, 3; z = 1, 2

find P(X = 2,Y + Z ≤ 3)

Solution

P(X = 2,Y + Z ≤ 3) =f (2, 1, 1) + f (2, 1, 2) + f (2, 2, 1)

=3

63+

6

63+

4

63

=13

63


Generalization of Multivariate Density

The joint distribution function is given by

F (x1, x2, ..., xn) =∫ xn

−∞...∫ x2

−∞

∫ x1

−∞f (t1, t2, ..., tn)dt1dt2...dtn

for −∞ < x1 < ∞,−∞ < x2 < ∞, ...,−∞ < xn < ∞. Also thedensity function is given by

f (x1, x2, ..., xn) =∂n

∂x1∂x2...∂xnF (x1, x2, ..., xn)


Example

If the trivariate probability density of X1,X2, and X3 is given by

f (x1, x2, x3) =

{(x1 + x2)e−x3 for 0 < x1 < 1, 0 < x2 < 1, x3 > 0

0 elsewhere

find P [(x1, x2, x3) ∈ A], where A is the regin

{(x1, x2, x3)|0 < x1 <1

2,

1

2< x2 < 1, x3 < 1}


Solution

P [(x1, x2, x3) ∈ A] =P

(0 < x1 <

1

2,

1

2< x2 < 1, x3 < 1

)=∫ 1

0

∫ 1

12

∫ 12

0(x1 + x2)e

−x3dx1dx2dx3

=∫ 1

0

∫ 1

12

(1

8+

x22)e−x3dx2dx3

=∫ 1

0

1

4e−x3dx3

=1

4(1− e−1)

=0.158


Marginal Distributions

Consider the following example:

Example (20)

In Example 12 we derived the joint probability distribution of tworandom variables X and Y , the number of aspirin caplets and thenumber of sedative caplets included among two caplets drawn atrandom from a bottle containing three aspirin, two sedative, andfour laxative caplets.Find the probability distribution of X alone and that of Y alone.


Solution

The results of Example 12 are shown in the following table,together with the marginal totals, that is, the totals of therespective rows and columns:

xf (x , y) 0 1 2

y

0 16

13

112

712

1 29

16

728

2 136

136

512

12

112


Marginal Distribution

Definition

If X and Y are discrete random variables and f (x , y) is the valueof their joint probability distribution at (x , y), the function given by

g(x) = ∑y

f (x , y)

for each x within the range of X is called the marginaldistribution of X. Correspondingly, the function given by

h(y) = ∑x

f (x , y)

for each y within the range of Y is called the marginaldistribution of Y.


Marginal Density

Definition

If X and Y are continuous random variables and f (x , y) is thevalue of their joint probability density at (x , y), the function givenby

g(x) =∫ ∞

−∞f (x , y)dy

is called the marginal density of X. Correspondingly, the functiongiven by

h(y) =∫ ∞

−∞f (x , y)dx

is called the marginal density of Y.


Example (21)

Given the joint probability density

f (x , y) =

{23 (x + 2y) for 0 < x < 1, 0 < y < 1

0 elsewhere

find the marginal densities of X and Y .

Solution

g(x) =∫ ∞

−∞f (x , y)dy =

∫ 1

0

2

3(x + 2y)dy =

2

3(x + 1)

for 0 < x < 1 and g(x) = 0 elsewhere. Likewise,

h(y) =∫ ∞

−∞f (x , y)dx =

∫ 1

0

2

3(x + 2y)dx =

1

3(1 + 4y)

for 0 < x < 1 and g(x) = 0 elsewhere.


Joint Marginal Distribution

When we are dealing with more than two random variables, we canspeak of the joint marginal distributions of several of therandom variables.

Example (22)

Considering again the trivariate probability density of Example 19,

f (x1, x2, x3) =

(x1 + x2)e−x3 for 0 < x1 < 1, 0 < x2 < 1,

x3 > 0

0 elsewhere

find the joint marginal density of X1 and X3 and the marginaldensity of X1 alone.


Solution

Performing the necessary integration, we find that the jointmarginal density of X1 and X3 is given by

m(x1, x3) =∫ 1

0(x1 + x2)e

−x3dx2 =(x1 +

1

2

)e−x3

for 0 < x1 < 1 and x3 > 0 and m(x1, x3) = 0 elsewhere. Usingthis result, we find that the marginal density of X1 alone is given by

g(x1) =∫ ∞

0

∫ 1

0f (x1, x2, x3)dx2dx3

=∫ ∞

0m(x1, x3)dx3

=∫ ∞

0

(x1 +

1

2

)e−x3dx3 = x1 +

1

2

for 0 < x1 < 1 and g(x1) = 0 elsewhere.


Conditional Distributions

Review of conditional probability. For Event A and B in samplespace S ,

P(A|B) = P(A∩ B)

P(B)

Suppose A and B are X = x and Y = y ,

P(X = x |Y = y) =P(X = x ,Y = y)

P(Y = y)=

f (x , y)

h(y)


Definition

Conditional Distribution If f (x , y) is the value of the jointprobability distribution of the discrete random variables X and Yat (x , y) and h(y) is the value of the marginal distribution of Y aty , the function given by

f (x |y) = f (x , y)

h(y)h(y) 6= 0

for each x within the range of X is called the conditionaldistribution of X given Y = y. Correspondingly, if g(x) is thevalue of the marginal distribution of X at x , the function given by

w(y |x) = f (x , y)

g(x)g(x) 6= 0

for each y within the range of Y is called the conditionaldistribution of Y given X = x.


Example (23)

With reference to Examples 12 and 20, find the conditionaldistribution of X given Y = 1.

Solution

Substituting the appropriate values from the table in Example 20,we get

f (0|1) =29718

=4

7

f (1|1) =16718

=3

7

f (2|1) = 0718

= 0


Definition

Conditional Density If f (x , y) is the value of the joint density ofthe continuous random variables X and Y at (x , y) and h(y) isthe value of the marginal density of Y at y , the function given by

f (x |y) = f (x , y)

h(y)h(y) 6= 0

for −∞ < x < ∞, is called the conditional density of X given Y= y. Correspondingly, if g(x) is the value of the marginal densityof X at x , the function given by

w(y |x) = f (x , y)

g(x)g(x) 6= 0

for −∞ < y < ∞, is called the conditional density of Y given X= x.


Example (24)

With reference to Example 21, find the conditional density of Xgiven Y = y , and use it to evaluate P(X ≤ 1

2 |Y = 12).

Solution

Using the results obtained on the previous page, we have

f (x |y) = f (x , y)

h(y)=

23 (x + 2y)13 (1 + 4y)

=2x + 4y

1 + 4y

for 0 < x < 1 and f (x |y) = 0 elsewhere.


Solution

Now,

f

(x

∣∣∣∣12)=

2x + 4 · 121 + 4 · 12

=2x + 2

3

Thus,

P

(X ≤ 1

2

∣∣∣∣Y =1

2

)=∫ 1

2

0

2x + 2

3dx

=5

12


Geometric Presentation


Example (25)

Given the joint probability density

f (x , y) =

{4xy for 0 < x < 1, 0 < y < 1,

0 elsewhere

find the marginal densities of X and Y and the conditional densityof X given Y = y .


Solution

Performing the necessary integrations, we ge

g(x) =∫ ∞

−∞f (x , y)dy =

∫ 1

04xydy

=2xy2∣∣∣∣1y=0

= 2x

for 0 < x < 1, and g(x) = 0 elsewhere; also

h(y) =∫ ∞

−∞f (x , y)dx =

∫ 1

04xydx

=2x2y

∣∣∣∣1x=0

= 2y

for 0 < y < 1, and h(y) = 0 elsewhere.


Solution

Then, substituting into the formula for a conditional density, we get

f (x |y) = f (x , y)

h(y)=

4xy

2y= 2x

for 0 < x < 1, and f (x |y) = 0 elsewhere.

When we are dealing with two or more random variables, questionsof independence are usually of great importance.Here, f (x |y) = 2x does not depend on the given value of Y = y ,but this is clearly not the case in Example 24

f (x |y) = 2x + 4y

1 + 4y

Thus we have the analogy definition of independent randomvariables and its probability distributions.


INDEPENDENCE OF DISCRETE RANDOM VARIABLES

Definition

If f (x1, x2, ..., xn) is the value of the joint probability distributionof the discrete random variables X1,X2, ...,Xn at (x1, x2, ..., xn)and fi (xi ) is the value of the marginal distribution of Xi at xi fori = 1, 2, ..., n, then the n random variables are independent if andonly if

f (x1, x2, ..., xn) = f1(x1) · f2(x2) · ... · fn(xn)

for all (x1, x2, ..., xn) within their range.


Example (26)

Considering n independent flips of a balanced coin, let Xi be thenumber of heads (0 or 1) obtained in the ith flip for i = 1, 2, ..., n.Find the joint probability distribution of these n random variables.

Solution

fi (xi ) =1

2

n random variable are independent, the joint probabilitydistribution is given by

f (x1, x2, ..., xn) =f1(x1) · f2(x2) · ... · fn(xn)

=1

2

1

2...

1

2=

(1

2

)n

where xi = 0 or 1 for i = 1, 2, ..., n.


Example (27)

Given the independent random variables X1, X2, and X3 with theprobability densities

f1(x1) =

{e−x1 for x1 > 0

0 elsewhere

f2(x2) =

{2e−2x2 for x2 > 0

0 elsewhere

f3(x3) =

{3e−3x3 for x3 > 0

0 elsewhere

find their joint probability density, and use it to evaluate theprobability P(X1 + X2 ≤ 1,X3 > 1).


Solution

The joint probability density:

f (x1, x2, x3) =f1(x1) · f2(x2) · f3(x3)=e−x1 · 2e−2x2 · 3e−3x3

=6e−x1−2x2−3x3

for x1 > 0, x2 > 0, x3 > 0, and f (x1, x2, x3) = 0 elsewhere. Thus,

P(X1 + X2 ≤ 1,X3 > 1) =∫ ∞

1

∫ 1

0

∫ 1−x2

06e−x1−2x2−3x3dx1dx2dx3

=(1− 2e−1 + e−2)e−3

=0.02


Homework

Work with your partner (in group)

hand in the homework to the editor group on duty 3 before17:00, Saturday.

Group editor on duty shall organize the final answers and sendthe file of final answer to [email protected] before nextTuesday

HW Chapter 3: 69, 97, 99, 101, 103, 105, 107, 109.


Questions??

prbability theory and mathematical statistics lecture 05: probability distributions ... · 2019. 3....

Documents