probability distributions - stony brook...

02/07/07 PHY310: Statistical Data Analysis 1

Road Map

The Gausssian Describing Distributions

Expectation ValueVariance

Basic Distributions Generating Random Numbers

PHY310: Lecture 05

Probability Distributions


The Gaussian or Normal DistributionAlso known as a Bell Curve. It's important since when you add enough continuous random

variables (i.e. almost any measurement) the sum becomes a Gaussian.

gx ; ,2dx =

1

22e−x−2

22

dx

The central value is given by μ.Called the “mean”

The width is given by σThe variance is σ²The standard deviation is σ


The Expectation ValueThe “Expectation Value” of a P.D.F. is the value of a random variable that you expect to measure

Also called the “Population Mean”, or just the “Mean”The expectation value is written “E[x]”

This is an abbreviation that means “The expectation value of x”Depends on the P.D.F. and isn't a function of x.

If a random variable x has a P.D.F. f(x), then the expectation value is:

E[x ] =∫−∞

∞

dx xf x

If you want to find the expectation value of q(x) (q is a function of x):

E[qx] =∫−∞

∞

dxqx f x

The average, <x>, and the expectation value, E[x], are differentThe average is an estimator of the expectation.


Example: Gaussian Expectation ValueE[x ] =∫

−∞

∞

dx xgx ; ,2

⇔E[x ] = ∫−∞

∞

dxx

22e−x−2

22

⇔E[x ] =

mcgrew@boxer:macros$ maximaMaxima 5.10.0 http://maxima.sourceforge.netUsing Lisp GNU Common Lisp (GCL) GCL 2.6.7 (aka GCL)Distributed under the GNU Public License. See the file COPYING.Dedicated to the memory of William Schelter.This is a development version of Maxima. The function bug_report()provides bug reporting information.

(%i1) integrate((x/sqrt(2*%pi*sigma^2))*exp(-(x-mu)^2/(2*sigma^2)),x,minf,inf);

Is sigma positive or negative?positive; mu sigma(%o1) ---------- abs(sigma)(%i2) quit();mcgrew@boxer:macros$


The Variance

The “Variance” describes the width of a distributionAlso called the “Population Variance”The variance of x is written as “V[x]”

V [x ]=E [x−E [x ]2]

V [x ]=E[x2−2xE[x ]E[x ]2]

V [x ]=E[x2]−2E[x ]E[x ]E[x ]2

V [x ]=E[x2]−E[x ]2

If k is a constant, then E[k] = k

Multiply out the argument

Simplify


Example: Gaussian VarianceE[x2

] = ∫−∞

∞

dx x2gx ; ,2

⇔E[x2] =∫

−∞

∞

dxx2

22e−x−2

22

⇔E[x2] =

2

2

mcgrew@boxer:macros$ maxima

Maxima 5.10.0 http://maxima.sourceforge.netUsing Lisp GNU Common Lisp (GCL) GCL 2.6.7 (aka GCL)Distributed under the GNU Public License. See the file COPYING.Dedicated to the memory of William Schelter.This is a development version of Maxima. The function bug_report()provides bug reporting information.(%i1) integrate((x^2/sqrt(2*%pi*%sigma^2))*exp(-(x-%mu)^2/(2*%sigma^2)),x,minf,inf);Is %sigma positive or negative?positive; 3 2 2 sqrt(2) sqrt(%pi) %sigma + 2 sqrt(2) sqrt(%pi) %mu %sigma(%o1) ------------------------------------------------------------- 2 sqrt(2) sqrt(%pi) abs(%sigma)(%i2) quit();mcgrew@boxer:macros$

V [x ]=E[x2]−E[x ]2

V [x ]=2

2−

2

V [x ]=2


(How I Really Do Integrals)


Multi-Dimensional Correlations

Multi-dimensional p.d.f.s can have internal correlations between variables

Not all correlations are linear

Variables with a Linear Correlation Variables with a Non-Linear Correlation


The Multi-Dimensional Variance:The Covariance

The “Covariance” describes the width of a multi-dimensional distribution like f(x

1, x

2, ... , x

n)

Cov [x i , x j ] = E[x i−ix j− j] = E[x i x j ]−i j

The Covariance is usually written as an n x n matrix

Vij = Cov[x

i, x

j]

The “Correlation Coefficient” is a dimensionless measure of the correlation between two random variables

ij =Cov [x i , x j ]

V [x i ]V [x j ]=

V ij

xy


Binomial Distribution (Discrete)

P n;N ,p =N!

n!N−n!pn

1−pN−n

If the probability of a single positive outcome is p, what is the probability of n positiveoutcomes in a total of N measurements?

A “|” is too hard to get in math mode, so I'm going touse “;” from now on!

Equivalently: If the probability of a single positive measurement is p, and a negative measurement is q=1-p, what is the probability of n positive and m=N-n negative outcomes in of a total of N measurements?

P n,m;N ,p=N!

n!m!pnqm

The Expectation Value is E[n] = Np E[n]=∑n=0

∞

nN!

n!N−n!pn

1−pN−n=Np

The Variance is V[n] = Np(1-p) = Npq

The Standard Deviation is n=V [n]=Np1−p


Binomial ExampleIf there were 8 children born at the Hospital today:

What do we know (assume)?It's 50/50 a given child will be a girlThere were exactly 8 children born.

The point is that the number of “measurements” is fixed.

What is the expected distribution of boys and girls?

What is the expected number of girls?

What is the expected variation in the number of girls?

P n;N ,p=N!

n!N−n!pn

1−pN−nP n;8,0.5=

8!n!8−n!

0.5n0.58−n

E[n ]=Np=8×0.5=4

V [n]=Np1−p=8×0.5×1−0.5=2 ; n=V [n]=2


Multinomial Distribution (Discrete)

P n1 , ... ,nm ;N , p1 , ... , pm=N!

n1! ...nm!p1n1 ...pm

nm

When there are several possible outcomes for a single measurement; What is the probability of n positive outcomes in a total of N measurements? e.g. What is the probability of drawing a spade and a diamond from a deck of cards

The “Trinomial” can be used for any multinomial distribution where you are only interested in pairs of variables (e.g. n, m, everything else)

P n,m;N ,p ,q=N!

n!m! N−n−m!pnqm

1−p−qN−n−m

Trinomial expectation values are E[n] = Np, E[m] = Nq

Trinomial variances are V[n] = Np(1-p), V[m] = Nq(1-q)

Trinomial covariance is Cov[n,m] = -NpqNotice the negative correlation!


Poisson Distribution (Discrete)The Poisson Distribution is the limit of the Binomial for μ = Np constant as N → ∞ and p → 0.

It describes processes like the number of radioactive decays per unit time.

P n; = ne−

n!

The expectation value is E[n] = μ

The variance is V[n] = μ

The standard deviation is


Poisson ExampleHow many cosmic rays enter a detector in 10 seconds if the mean rate is 20 decays/second?

At 20 decays/second for 10 seconds, we expect 200 cosmic rays.This is a Poisson process so the uncertainty on the expectation is

CR=200

How can we tell this is a Poisson process?Poisson statistics apply when you are counting events, but there isn't an upper limit on the number you might measure (e.g. not binomial)

The number of events could be zeroThe number of events could be hung

The expected number might be less than one.Since we live in a quantum world, many (most) processes are fundamentally Poissonian.

CHEAP TRICK: When the Poisson mean is large (>15) a Gaussian (at integer values) is a reallygood approximation for a Poissonian. And, it's much easier to calculate.


Continuous Distributions

The Uniform DistributionThis is what most computer pseudo-random number generators approximate

The Exponential DistributionUsually seen when you measure the time until a random process happens

The inverse of Poisson: you measure seconds/count instead of counts/second

f x ; ,={1

−, x

0, otherwise

E[x] = (α+β)/2

V[x] = (β-α)²/12

σ = (β-α)/√12

f t ;=1e−t /

E[x] = λ

V[x] = λ²

σ = λ


Gaussian Distribution (Continuous)How do you get a Gaussian?

It's the limiting distribution if you take a sum of continuous random variables (add figure showing the sum of uniform variables).If you added enough discrete variables together they become “continuous”

If you add enough random variables together, you get a Gaussian

gx ; ,2dx =

1

22e−x−

2

22

dx

The expectation value is E[x] = μ.The variance is V[x] = σ²The standard deviation is σ


Binomial Approaches Gaussian

As the number of measurements gets large, the binomial distributions gets more and more like a Gaussian


Chi-Squared (χ²) DistributionChi-squared is the distribution of the sum of squares of Gaussian distributed random variables.

When you compare measurements to the average, the deviation is described by chi-squared.

The expectation value is the number of degrees of freedomThe variance is two times the d.o.f.The standard deviation is sqrt(2 d.o.f.)We will usually use the cumulative distribution for Chi-Squared

Can be calculated with the ROOT TMATH::Prob(z,dof) function...


Deciding Which Distribution to Use

You are counting things (boys born today)If you have a fixed number of trials use the binomial distribution

e.g. There were eight children born in the hospital, how many were boys?If you don't have a fixed number use the Poisson distribution

e.g. How many children were born in NY City?

You have a continuous distributionIf you are measuring the “distance” between discrete events (time or space), use the exponential distribution.

e.g. How long do you have to wait until a C.R. muon arrives?e.g. The time between radioactive decays?

Almost everything else is GaussianAnything that depends on a sum of multiple random processes will have a Gaussian distribution.


Pseudo-Random NumbersMC's depend on a source of random numbers, but computers are deterministicPseudo-random number generators are algorithms that sequences of apparently uncorrelated numbers

Be very careful, the numbers are correlated, and are repeatableThere are about a half dozen good generatorsMost computer generators return a uniform distribution between 0 and 1

Example of a simple generator: The Multiplicative Linear Congruential Generator (The MLC Generator)

Start with an initial integer, n1, called the seed

Generate a sequence of new integers, ni+1

= (A ni + 1) mod M

The integers will lie between 0 and M-1MLC Generators are common and dangerous

Very FastConsecutive pairs of integers fall on a finite number of hyperplanes

I'll use the term “random” (not “pseudo-random”) unless I'm making a point


Generating Non-Uniform Distributions

Start with a sequence of uniform random numbers and get a sequence of non-uniform random numbersThe “Transform Method”

You need to solve the equationUsually can't be doneFunctions where solution exists are usually tabulated in librariesMore details available in “Statistical Data Analysis”

The “Accept/Reject” MethodAlways works!But, can be inefficient

∫−∞

x r

dx 'f x ' = r


Basic AcceptanceWorks for functions over a finite interval

To generate a random value from the p.d.f. f(x) over an interval [α,β]Generate a uniform random value, x, between α and β.Generate a second random value, y, between 0 and 1If y < f(x) use x, otherwise reject x and start over.


General Acceptance

The basic acceptance method only works over a fixed interval.A more general method uses the ratio of two functions.

f(x) – The P.D.F. you want to generateg(x) – A P.D.F. that you have a transform generator for

Must satisfy εf(x) < g(x) for all x, and ε>0

The algorithm:Generate x according to g(x)Generate a uniform y between 0 and g(x)if y < εf(x), accept x, otherwise reject x and start over


General Acceptance Example

Generate random numbers for f x = Ae−x2

2 sin22x

double x, g, y, f;x = gRandom->Gaus();g = exp(-x*x/2);y = gRandom->Uniform(0.0,g);f = exp(-x*x/2)*sin(2*x)*sin(2*x);if (y<f) h2->Fill(x);

Generate a Gaussian distributed number, xGenerate a number, y, between 0 and g(x)

Check if y is less than f(x)

probability distributions - stony brook...

Documents