winter 2006ee384x1 review of probability theory review session 1 ee384x

Winter 2006 EE384x 1

Review of Probability Theory

Review Session 1EE384X

Winter 2006 EE384x 2

A random experiment with set of outcomes

Random variable is a function from set of outcomes to real numbers

Random Variable

Winter 2006 EE384x 3

Example

Indicator random variable:

A : A subset of is called an event

Winter 2006 EE384x 4

CDF and PDF Discrete random variable:

The possible values are discrete (countable)

Continuous random variable: The rv can take a range of values in R

Cumulative Distribution Function (CDF):

PDF and PMF:

Winter 2006 EE384x 5

Expectation and higher moments

Expectation (mean):

if X>0 :

Variance:

Winter 2006 EE384x 6

Two or more random variables

Joint CDF:

Covariance:

Winter 2006 EE384x 7

Independence

For two events A and B:

Two random variables

IID : Independent and Identically Distributed

Winter 2006 EE384x 8

Useful Distributions

Winter 2006 EE384x 9

Bernoulli Distribution

The same as indicator rv:

IID Bernoulli rvs (e.g. sequence of coin flips)

Winter 2006 EE384x 10

Binomial Distribution Repeated Trials:

Repeat the same random experiment n times. (Experiments are independent of each other)

Number of times an event A happens among n trials has Binomial distribution (e.g., number of heads in n coin tosses, number of arrivals in n time

slots,…)

Binomial is sum of n IID Bernouli rvs

Winter 2006 EE384x 11

Mean of Binomial

Note that:

Winter 2006 EE384x 12

1 2 3 4 5 6 7 8 9 10 11 120

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

Binomial - Example

n=4

n=40

n=10

n=20

p=0.2

Winter 2006 EE384x 13

Binomial – Example (ball-bin)

There are B bins, n balls are randomly dropped into bins.

: Probability that a ball goes to bin i : Number of balls in bin i after n drops

Winter 2006 EE384x 14

Multinomial Distribution Generalization of Binomial Repeated Trails (we are interested in

more than just one event A) A partition of into A1,A2,…,Al

Xi shows the number of times Ai occurs

among n trails.

Winter 2006 EE384x 15

Poisson Distribution

Used to model number of arrivals

Winter 2006 EE384x 16

0 5 10 150

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

Poisson Graphs

=10

=.5

=1

=4

Winter 2006 EE384x 17

Poisson as limit of Binomial

Poisson is the limit of Binomial(n,p) as

Let

Winter 2006 EE384x 18

0 1 2 3 4 5 6 7 8 9 100

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

Poisson and Binomial

Poisson(4)

n=5,p=4/5

n=20, p=.2

n=10,p=.4

n=50,p=.08

Winter 2006 EE384x 19

Geometric Distribution

Repeated Trials: Number of trials till some event occurs

Winter 2006 EE384x 20

Exponential Distribution

Continuous random variable Models lifetime, inter-arrivals,…

Winter 2006 EE384x 21

Minimum of Independent Exponential rvs

: Independent Exponentials

Winter 2006 EE384x 22

Memoryless property

True for Geometric and Exponential Dist.:

The coin does not remember that it came up tails l times

Root cause of Markov Property.

Winter 2006 EE384x 23

Proof for Geometric

Winter 2006 EE384x 24

Moment Generating Function (MGF) For continuous rvs (similar to Laplace

xform)

For Discrete rvs (similar to Z-transform):

Characteristic Function

Winter 2006 EE384x 25

Characteristic Function

Can be used to compute mean or higher moments:

If X and Y are independent and T=X+Y

Winter 2006 EE384x 26

Useful CFs

Bernoulli(p) :

Binomial(n,p) :

Multinomial:

Poisson: