probability distribution summaryeacademic.ju.edu.jo/ghazi.alsukkar/material/probability... ·...

of 18

Probability Distribution Summary Dr. Mohammed Hawa

October 2007

There are many probability distributions that have been studied over the years. Each of which was developed based on observations of a particular physical phenomenon. The table below shows some of these distributions:

Probability distributions

Univariate Multivariate

Discrete:

Benford • Bernoulli • binomial • Boltzmann • categorical • compound

Poisson • discrete phase-type • degenerate • Gauss-Kuzmin • geometric • hypergeometric • logarithmic • negative binomial • parabolic fractal •

Poisson • Rademacher • Skellam • uniform • Yule-Simon • zeta • Zipf • Zipf-Mandelbrot

Ewens •

multinomial •

multivariate

Polya

Continuous:

Beta • Beta prime • Cauchy • chi-square • Dirac delta function • Coxian •

Erlang • exponential • exponential power • F • fading • Fermi-Dirac •

Fisher's z • Fisher-Tippett • Gamma • generalized extreme value •

generalized hyperbolic • generalized inverse Gaussian • Half-Logistic • Hotelling's T-square • hyperbolic secant • hyper-exponential •

hypoexponential • inverse chi-square (scaled inverse chi-square) • inverse Gaussian • inverse gamma (scaled inverse gamma) • Kumaraswamy •

Landau • Laplace • Lévy • Lévy skew alpha-stable • logistic • log-normal •

Maxwell-Boltzmann • Maxwell speed • Nakagami • normal (Gaussian) •

normal-gamma • normal inverse Gaussian • Pareto • Pearson • phase-type •

polar • raised cosine • Rayleigh • relativistic Breit-Wigner • Rice • shifted

Gompertz • Student's t • triangular • truncated normal • type-1 Gumbel •

type-2 Gumbel • uniform • Variance-Gamma • Voigt • von Mises •

Weibull • Wigner semicircle • Wilks' lambda

Dirichlet •

Generalized

Dirichlet

distribution .

inverse-Wishart

• Kent • matrix

normal •

multivariate

normal •

multivariate

Student • von

Mises-Fisher •

Wigner quasi •

Wishart

In this article we will summarize the characteristics of some popular probability distributions, which you are likely to encounter later in your study of communication engineering. We will begin with discrete distributions then move to continuous distributions.

A - Discrete Random Variables:

•••• Bernoulli distribution

The Bernoulli random variable takes value 1 with success probability p and value 0 with failure probability q = 1 - p. Applications:

• Coin toss.

• Success/Failure experiments.

of 18

Probability mass function

Cumulative distribution function

Parameters success probability (p is real)

Support

Probability mass function (pmf)

Cumulative distribution function (CDF)

Mean

Median N/A

Mode

Variance

Skewness

Excess kurtosis

Probability-generating function (PGF) 1 − � + �� Moment-generating function (MGF) 1 − � + �� Characteristic function 1 − � + ��

•••• Binomial distribution

The binomial random variable is the sum of n independent, identically distributed Bernoulli random variables, each of which yields success with probability p. In fact, when n = 1, the binomial random variable is a Bernoulli random variable. Applications:

• Obtaining k heads in n tossings of a coin.

• Receiving k bits correctly in n transmitted bits.

• Batch arrivals of k packets from n inputs at an ATM switch.

of 18



Parameters number of trials (integer)

success probability (real)

Support


Cumulative distribution function (CDF) �� 1 − ��

��

Mean

Median one of

Mode

Variance

Skewness

Excess kurtosis

Probability-generating function (PGF) �1 − � + ��

Moment-generating function (MGF)

Characteristic function

•••• Geometric distribution

This is the distribution of time slots between the successes of consecutive independent Bernoulli trails. Like its continuous analogue (the exponential distribution), the geometric distribution is memoryless, which is an important property in certain applications.

Applications:

• To represent the number of dice rolls needed until you roll a six.

• Queueing theory and discrete Markov chains.

0 5 10 15 20 25 300

0.05

0.1

0.15

0.2

0.25

0.3

p = 0.5 and n = 20

p = 0.7 and n = 20

p = 0.5 and n = 40

0 5 10 15 20 25 300

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

p = 0.5 and n = 20

p = 0.7 and n = 20

p = 0.5 and n = 40

Parameters

Support



Mean

Median

Mode

Variance

Skewness

Excess kurtosis

Probability

Moment-


•••• Poisson

The Poisson distributionfixed period of time Applications:

• The number of phone calls at a

• The number of times a

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1


Parameters



Variance

Skewness

kurtosis

Probability-generating function

-generating function


Poisson distribution

Poisson distributionfixed period of time

Applications:

The number of phone calls at a

The number of times a

1 2 3




generating function (PGF)

generating function (MGF


distribution

Poisson distribution describes the probability fixed period of time assuming

The number of phone calls at a

The number of times a web

4 5 6 7



(PGF)

MGF)

describes the probability assuming these events occur at

The number of phone calls at a call center

web server is accessed per minute.

7 8 9 10

p = 0.2

p = 0.5

p = 0.8

1

�ln1

1

1

describes the probability these events occur at

call center per minute.

is accessed per minute.


1 − �1 − ��

� − ln�2�ln�1 − �� 1

��1 − �1 − ��

��1 − �1 − ��

describes the probability that a number these events occur at random with a rate

per minute.


0 1 20

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1


success probability (

� �

�

�

� ��

a number k of events occurwith a rate


3 4 5

Page


success probability (

of events occur .

6 7 8

p = 0.2

p = 0.5

p = 0.8

of 18


success probability (real)

of events occur in a

9 10

p = 0.2

p = 0.5

p = 0.8

in a

of 18

• The number of spelling mistakes one makes while typing a single page.

• The number of mutations in a given stretch of DNA after radiation.

• The number of unstable nuclei that decayed within a given period of time in a piece of radioactive substance.

• The number of stars in a given volume of space.



Parameters rate (real)

Support



where Γ��, !� is the Incomplete gamma function

Mean

Median

Mode

Variance

Skewness

Excess kurtosis

Probability-generating function (PGF) �"�#�$� Moment-generating function (MGF) �"%&'�$( Characteristic function �"%&)'�$(

•••• Zipf distribution

The term Zipf's law refers to frequency distributions of rank data. Originally, Zipf's law stated that, in natural language utterances, the frequency of any word is roughly inversely proportional to its rank in the frequency table. So, the most frequent word will occur approximately twice as often as the second most frequent word, which occurs twice as often as the fourth most frequent word, etc.

0 2 4 6 8 10 12 14 16 18 200

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

λ = 1

λ = 4

λ = 10

0 2 4 6 8 10 12 14 16 18 200

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

λ = 1

λ = 4

λ = 10

N=10 (

defined at integer values of k. The connecting lines do

Parameters

Support


Cumulative distribution function(CDF)

Mean

Moment-


B - Continuous Random Variables

•••• Exponential

The exponential distribution between counterpart of the memoryless. Applications:

• Similar to that of Poisson distribution (e.g., call, the distance between mutations on a DNA strand


(log-log scale


not indicate continuity.

Parameters





Continuous Random Variables

Exponential distribution

The exponential distribution between successive counterpart of the geometric distributionmemoryless.

Applications:

Similar to that of Poisson distribution (e.g., the distance between mutations on a DNA strand


log scale). Note that the function is only





generating function (MGF)



distribution

The exponential distribution successive Poisson arrivals.

geometric distribution



Note that the function is only




(MGF)


distribution

The exponential distribution represents the probability distribution of the Poisson arrivals.

geometric distribution


Note that the function is only


N=10.

values of k. The connecting lines do not indicate

Continuous Random Variables:

represents the probability distribution of the The exponential distribution is the continuous

geometric distribution, and is the only continuous distribution that is

Similar to that of Poisson distribution (e.g., the time it takes before your next telephone the distance between mutations on a DNA strand


N=10. Note that the function is only defined at integer


exponent

number of elements

(rank)

where *harmonic number


, and is the only continuous distribution that is

the time it takes before your next telephone the distance between mutations on a DNA strand, etc).


Note that the function is only defined at integer


continuity.

exponent (real)

number of elements

(rank)

*+,, is the �th generalized harmonic number *



the time it takes before your next telephone , etc).

Page




continuity.

number of elements (integer)

th generalized

*+,, - ∑ �+��$

represents the probability distribution of the time intervalsThe exponential distribution is the continuous


the time it takes before your next telephone

of 18




th generalized $�/

time intervalsThe exponential distribution is the continuous



time intervals The exponential distribution is the continuous



Parameters

Support

Probability density function


Mean

Median

Mode

Variance

Skewness

Excess kurtosis

Moment-


•••• Uniform

The uniform of the same length are equally probable

Applications:

• To test

• In simulation software packages, in which uniformlynumbers


Parameters



Variance

Skewness

kurtosis



Uniform distribution

uniform distribution is often abbreviated of the same length are equally probable

Applications:

test a statistic for

In simulation software packages, in which uniformlynumbers are generated



Probability density function (pdf)




distribution

distribution is often abbreviated of the same length are equally probable

statistic for the simple null hypothesis.

In simulation software packages, in which uniformlyare generated first



(pdf)


MGF)

distribution is often abbreviated of the same length are equally probable.

simple null hypothesis.

In simulation software packages, in which uniformlyfirst and then converted to other types of distributions.


0��

1 −

distribution is often abbreviated U(a,.


In simulation software packages, in which uniformlyand then converted to other types of distributions.


rate or

�"� − ��"�

, b). In this distribution,


In simulation software packages, in which uniformly-distributed and then converted to other types of distributions.



rate or inverse scale

In this distribution,

distributed pseudoand then converted to other types of distributions.


Page


inverse scale (real)

In this distribution, all time

pseudo-random and then converted to other types of distributions.


of 18

time intervals

random and then converted to other types of distributions.


of 18

Parameters lower/upper limits

Support



Mean

Median

Mode any value in

Variance

Skewness

Excess kurtosis

Moment-generating function (MGF)

The raw moments are:


•••• Triangular distribution

The Triangular distribution is used as a subjective description of a population for which there is only limited sample data. Applications:

• Used in business decision making and project management to describe the time to completion of a task.



Parameters lower limit

upper limit

mode

Support



Mean

Median

Mode

Variance

Skewness

Excess kurtosis

Moment-


•••• Normal

The normal distribution

The and a variancedensity resembles a Applications:

• Many physical phenomena (like distribution.

• Many statistical tests are based on the assumption of normal distribution

• Normal distributiondiscrete families of distributions

• Measurement errors are often assumed to be normally distributed

• Financial variables normal distribution.

• Light intensity from a single source is usually assumed to be normally distributed.



Variance

Skewness

kurtosis



Normal distribution

normal distribution

The standard normal variance of one. It is often called the

resembles a

Applications:

Many physical phenomena (like distribution.

any statistical tests are based on the assumption of normal distribution

Normal distributiondiscrete families of distributions

Measurement errors are often assumed to be normally distributed

Financial variables normal distribution.

Light intensity from a single source is usually assumed to be normally distributed.





distribution

normal distribution is also called the

standard normal of one. It is often called the

resembles a bell.

Many physical phenomena (like


Normal distribution arisesdiscrete families of distributions


Financial variables such as stock values or commodity prices normal distribution.


(pdf)


MGF)

is also called the

standard normal distributionof one. It is often called the

Many physical phenomena (like noise


s as the limiting distribution of several continuous and discrete families of distributions by the


such as stock values or commodity prices


is also called the Gaussian distribution

distribution is the normal distribution with a of one. It is often called the bell curve

noise) can be approximated well by the normal


as the limiting distribution of several continuous and by the central limit theorem




Gaussian distribution

is the normal distribution with a bell curve because the gra

) can be approximated well by the normal


as the limiting distribution of several continuous and central limit theorem




Gaussian distribution, and is denoted by

is the normal distribution with a because the gra



as the limiting distribution of several continuous and central limit theorem.

Measurement errors are often assumed to be normally distributed.

such as stock values or commodity prices can be mo


Page

, and is denoted by

is the normal distribution with a meanbecause the graph of its probability


any statistical tests are based on the assumption of normal distribution.

as the limiting distribution of several continuous and

can be modeled by the


of 18

, and is denoted by

mean of zero probability


as the limiting distribution of several continuous and

deled by the


, and is denoted by

of zero probability

The green line is the standard normal distribution

Parameters

Support



Mean

Median

Mode

Variance

Skewness

Excess kurtosis

Moment-


•••• Log-normal

If Y is a random variable with a normal distribution, then distribution; likewise, if Applications:

• The long



Parameters



Variance

Skewness

kurtosis



normal distribution

is a random variable with a normal distribution, then distribution; likewise, if

Applications:

he long-term return rate on a stock investment can be







distribution

is a random variable with a normal distribution, then distribution; likewise, if X is log

term return rate on a stock investment can be



(pdf)


MGF)

distribution

is a random variable with a normal distribution, then is log-normally distributed, then



12

12

is a random variable with a normal distribution, then normally distributed, then



Colors match the i

2 3 0

2

is a random variable with a normal distribution, then X =normally distributed, then ln

term return rate on a stock investment can be modeled as log


Colors match the image

location / mean

squared scale

= exp(Y) has a logln(X) is normally distributed.

modeled as log

Page


mage to the left

/ mean (real)

scale / variance

) has a log-normal ) is normally distributed.

modeled as log-normal.

of 18


/ variance (real)

normal ) is normally distributed.

normal.

Parameters

Support



Mean

Median

Mode

Variance

Skewness

Excess kurtosis

Moment-

•••• Gamma A gamma distributed random variable is denoted by special case of the Gamma distribution, this parameter is a real number.

Applications:

• The number of telephone calls which might be made at the same time to center


Parameters



Variance

Skewness

kurtosis


Gamma (ErlangA gamma distributed random variable is denoted by special case of the Gamma distribution, this parameter is a real number.

Applications:

he number of telephone calls which might be made at the same time to center.


1=µ




Erlang) distributionA gamma distributed random variable is denoted by special case of the gamma distributionGamma distribution, this parameter is a real number.

he number of telephone calls which might be made at the same time to


(pdf)


�5

MGF) The

6�

) distributionA gamma distributed random variable is denoted by

amma distribution where the shape parameter Gamma distribution, this parameter is a real number.


standard deviation

5

The MGF does not exist

� - ��57�898⁄

) distribution A gamma distributed random variable is denoted by

where the shape parameter Gamma distribution, this parameter is a real number.



mean

standard deviation

does not exist, but 2⁄

A gamma distributed random variable is denoted by Γ�;, 0�where the shape parameter

Gamma distribution, this parameter is a real number.



1=µ

standard deviation

but moments do

� �. The Erlang distribution is a where the shape parameter k is an integer. In the


Page


do, and are given by:

The Erlang distribution is a is an integer. In the

he number of telephone calls which might be made at the same time to a switching

of 18

, and are given by:


switching


Parameters

Support


Cumulative distribution function(CDF)

Mean

Median

Mode

Variance

Skewness

Excess kurtosis

Moment-


•••• chi-square distribution

The chi-square distribution (

where k = Applications:

• Used in theoretical

• Used in statistical significance testsranks.


Parameters



Variance

Skewness

kurtosis



square distribution

square distribution (

= n/2 and λ

Applications:

in the common chitheoretical distribution

in statistical significance testsranks.






square distribution

square distribution (χ

λ = 1/2.

the common chi-square tests for goodness of fit of observed distribution.

in statistical significance tests


(pdf)

0��$�Γ�;�

where the gamma function

becomes

Cumulative distribution function < !��"�

� Γ�

no simple closed form

MGF)

square distribution

χ2 distribution) is a special case of the

square tests for goodness of fit of observed

in statistical significance tests. One example is Friedman's analysis of variance by

shape

rate (

��"��

the gamma function

becomes Γ�;� - �;�$��=>!�;�


for

for

distribution) is a special case of the


One example is Friedman's analysis of variance by


shape (real)

rate (real)

the gamma function Γ�;�� − 1�! if ; is


distribution) is a special case of the




� � - < !��$�D�

is an integer.

distribution) is a special case of the gamma distribution



Page


��=>! .

gamma distribution

square tests for goodness of fit of observed data to a


of 18


gamma distribution

to a



Parameters

Support



Mean

Median

Mode

Variance

Skewness

Excess kurtosis

Moment-


•••• Rayleigh distribution

The Rayleigh distribution multi-path Rayleigh distribution


Parameters



Variance

Skewness

kurtosis



Rayleigh distribution

The Rayleigh distribution path fading.






generating function (MGF)



The Rayleigh distribution has been used to model attenuation of The Chi, Rice and

Rayleigh distribution.



(pdf)


(MGF)

has been used to model attenuation of , Rice and Weibull


<� 2⁄�

approximately

has been used to model attenuation of Weibull distribution


degrees of freedom

!�2�$��=>!Γ ;2

approximately

if

for

has been used to model attenuation of distributions



degrees of freedom

>!

has been used to model attenuation of wirelesss are all generalization


Page


wireless signals facing generalization


of 18


signals facing generalizations of the

signals facing of the

Parameters

Support



Mean

Median

Mode

Variance

Skewness

Excess kurtosis

Moment-


•••• Student's t distribution

The Student’s t distribution adistributed population when the sample size is small Applications:

• It is the basis of the popular Student's tbetween two sample means.

Parameters

Support

Parameters



Variance

Skewness

kurtosis



Student's t distribution

The Student’s t distribution adistributed population when the sample size is small

Applications:

It is the basis of the popular Student's t

between two sample means.


Parameters






The Student’s t distribution arises in the problem of estimatidistributed population when the sample size is small

It is the basis of the popular Student's t

between two sample means.


(pdf)


MGF)


rises in the problem of estimatidistributed population when the sample size is small

It is the basis of the popular Student's t-tests for the statistical significance of the diffe


deg. of freedom (

rises in the problem of estimatidistributed population when the sample size is small

tests for the statistical significance of the diffe

deg. of freedom (real

rises in the problem of estimatidistributed population when the sample size is small.



real)

ng the mean of a normally



Page




of 18


tests for the statistical significance of the difference




Mean

Median

Mode

Variance

Skewness

Excess kurtosis

•••• Cauchy

Applications:

• In spectroscopylines which are broadened by collision

• In nuclearrelativistic Cauchy distribution

The green line is the standard Cauchy distribution

Parameters

Support



Probability density (pdf)

Cumulative distribution (CDF)

Variance

Skewness

kurtosis

Cauchy distribution

Applications:

spectroscopy lines which are broadened by collision

nuclear and particle physicselativistic Cauchy distribution



Parameters



Cumulative distribution

0 for

0

0

0 for

distribution

the Cauchy distribution is the description of the line shape of spectral lines which are broadened by collision

particle physicselativistic Cauchy distribution





0 for K 3 1, otherwise undefined

, otherwise un

0 for K 3 3

Cauchy distribution is the description of the line shape of spectral lines which are broadened by collision

particle physics, the energy profile of a elativistic Cauchy distribution.



(pdf)


, otherwise undefined

, otherwise un

Cauchy distribution is the description of the line shape of spectral lines which are broadened by collisions.

, the energy profile of a




Cauchy distribution is the description of the line shape of spectral

, the energy profile of a resonance


Colors match the pdf above

location

scale (real)

where

hypergeometric function


resonance is described by



location (real)

(real)

Page

where is the



is described by



of 18

is the



is described by the


of 18

•••• Pareto distribution

The Pareto distribution is characterized by a curved long tail when plotted on a linear scale, thus is it used to describe long-range dependent phenomena. Applications:

• Used to describe the allocation of wealth among individuals.

• Describes frequencies of words in longer texts (a few words are used often, lots of words are used infrequently).

• File size distribution of Internet traffic which uses the TCP protocol (many smaller files, few larger ones).

• Clusters of Bose-Einstein condensate near absolute zero.

• The values of oil reserves in oil fields (a few large fields, many small fields)

• The length distribution in jobs assigned to supercomputers (a few large ones, many small ones).

• The standardized price returns on individual stocks.

• Areas burnt in forest fires.


xm = 1


xm = 1

Parameters location (real)

shape (real)

Support



Mean for k > 1

Median

Mode

Variance for k > 2

of 18

Skewness for k > 3

Excess kurtosis for k > 4

•••• Laplace distribution



Parameters location (real)

scale (real)

Support



Mean

Median

Mode

Variance

Skewness

Excess kurtosis

Moment-generating function (MGF) for


of 18

Notes: The expected value for a discrete random variable is:

M - NOPQ - ��

For a continuous random variable:

M - NOPQ - R �S��>�D

�D

The variance of a random variable is a measure of its statistical dispersion, indicating how its possible values are spread around the expected value. For a random variable X, the variance is:

If the random variable is discrete the variance is equivalent to:

VarOPQ - �� − M�2��

��$

The mode is the most frequent value assumed by a random variable, or occurring in a sampling of a random variable. The mode is not necessarily unique, since the same maximum frequency may be attained at different values. The median is the number separating the higher half of a sample or a probability distribution, from the lower half. The median of a finite list of numbers can be found by arranging all the observations from lowest value to highest value and picking the middle one. If there is an even number of observations, the median is not unique. Skewness is a measure of the asymmetry of the probability density function. Kurtosis (from the Greek word kurtos) is a measure of the peakedness of the probability density function.

probability distribution summaryeacademic.ju.edu.jo/ghazi.alsukkar/material/probability... ·...

Documents