reference: some basic distributions excellent reference for probability distributions is: evans, ......

Reference: some basic distributions

An excellent reference for probability distributions is: Evans, Merran, Nicholas Hastings, and Brian Peacock. 2000.Statistical Distributions, 3rd Edition. John Wiley & Sons. New York.

Continuous

Expected, or meanE@ZD = Ÿ z f HzL „ z

VarianceVARHZL = E@HZ - E@ZDL2D = Ÿ Hz - E@zDL2 f HzL „ z

For continuous variables, the probability density function is the probability of the value z given the parameters

ü Uniform

Uniform Distribution:a uniform distribution on the interval [a,b] where a < bprobability density function:

f HzL = 1ÅÅÅÅÅÅÅÅÅÅb-a , where a § z § bmean: b+aÅÅÅÅÅÅÅÅÅÅ2variance: Hb-aL2

ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅ12

ü r

dunif Hz, min = a, max = bL

# generate 1 uniform random number over the range 1.4 to 2.4runif H1, 1.4, 2.4L

ü mathematica

f@z_, a_, b_D :=1

b − a

H∗ random number over range 0,1 ∗LRandom@D

0.0407265

H∗ random number of specified range ∗L

MLEtools.nb 1

Random@Real, 83, 6<D

4.87348

ü Normal

Normal Distribution:has two parameters: m is the location parameter, in this case the mean; s is the scale parameter, in this case the standarddeviationprobability density function:

f HzL = 1ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅè!!!!!!!!!!!!!!2 p s2 expI- Hz-mL2

ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅ2 s2 Mmean: mvariance: s2

note: plotting f HzL will give you the familiar bell curve

ü r

probability density function

dnorm Hz, mean, sdL# plot the pdfcurve Hdnorm Hx, 4, 1L, 0, 7L

random number generation, where n is the number random numbers to generation. In other words, n = 1 will create 1random number.

rnorm Hn, mean, sdL

ü mathematica

f@z_, m_, σ_D :=1

è!!!!!!!!!!!!!2 π σ2

ExpA−Hz − mL2

2 σ2E

Plot@f@z, 4, 1D, 8z, 0, 7<D;

1 2 3 4 5 6 7

0.1

0.2

0.3

0.4

MLEtools.nb 2

Show that pdf integrates to 1

Integrate@f@z, 1, 1D, 8z, −∞, ∞<D

1

<< Statistics`ContinuousDistributions`H∗ normal random number with mean of 3 and standard deviation 2 ∗LRandom@NormalDistribution@3, 2DD

1.87451

ü Lognormal

Lognormal Distribution:has two parameters: m is the location parameter; s is the shape parameterprobability density function:

f HzL = 1ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅz è!!!!!!!!!!!!!!2 p s2

expI- HlnHzL-logHmLL2ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅ2 s2 M, where z > 0, m >0, and s >0

mean: m expI s2ÅÅÅÅÅÅÅ2 M

median: ms2 is the variance of the ln(z)variance: m2 expHs2L HexpHs2L - 1Lnote: This describes a variable in which the logarithm of the variable is a normal distribution. Whereas the normal distribu-tion is additive, the lognormal distribution is multiplicative.

ü r

dlnorm Hz, meanlog = m, sdlog = σL

R uses a slightly different form of the lognormal model. Use the following form so that m is defined in the same way asdescribed in the f(z) equation above.

dlnorm Hz, log H mL, σL

# generate random numberrlnorm H1, meanlog = 1, sdlog = 1.2L

ü mathematica

f@z_, m_, σ_D :=1

z è!!!!!!!!!!!!!

2 π σ2 ExpA−

HLog@zD − Log@mDL2

2 σ2E

MLEtools.nb 3

Plot@f@z, 1.7, 1D, 8z, 0, 20<D;

5 10 15 20

0.1

0.2

0.3


Integrate@f@z, 1, 1D, 8z, 0, ∞<D

1

Derive expected value

Integrate@z f@z, m, σD, 8z, 0, ∞<, Assumptions → 8σ > 0, m > 0, σ ∈ Reals, m ∈ Reals<Dσ22 m

Derive variance

Integrate@Hz − Integrate@z f@z, m, σD, 8z, 0, ∞<, Assumptions → 8σ > 0, m > 0, σ ∈ Reals, m ∈ Reals<DL2

f@z, m, σD, 8z, 0, ∞<, Assumptions → 8σ > 0, m > 0, σ ∈ Reals, m ∈ Reals<Dσ2 I−1 + σ2M m2

<< Statistics`ContinuousDistributions`H∗ lognormal random number with the first number being the Log@mD

and the second number being σ. In the case below, m = 3 and σ = 2 ∗LRandom@LogNormalDistribution@Log@3D, 2DD

6.94065

MLEtools.nb 4

ü Exponential

Exponential Distribution:has three parameters: m is the location parameter; s is the shape parameterprobability density function:

f HzL = a Exp@-a zD, where z > 0mean: 1ÅÅÅÅavariance: 1ÅÅÅÅÅÅÅa2

note: Continuous distribution analog of Poisson. Can be used for dispersal distance. Widely used in Baysian Analysis

ü r

dexp Hz, rate = aL

# generate random numberrexp H1, rate = aL

ü mathematica

f@z_, a_D := a Exp@−a zD

Plot@f@z, 0.5D, 8z, 0, 15<, PlotRange → AllD;

2 4 6 8 10 12 14

0.1

0.2

0.3

0.4

0.5

<< Statistics`ContinuousDistributions`H∗ exponential random number with a = 0.5 ∗LRandom@[email protected]

2.1514

MLEtools.nb 5

ü Gamma

Gamma Distribution:has two parameters: a is the rate parameter; n is the shape parameterprobability density function:

f HzL = anÅÅÅÅÅÅÅÅÅÅÅGHnL Exp@-a zD zn-1, where z > 0

mean: nÅÅÅÅavariance: nÅÅÅÅÅÅÅa2

note: can have a long tail. When n = 1, the gamma distribution becomes the exponential distribution. When n = degrees offreedom / 2 and a = 2, the gamma distribution becomes the chi-square distribution.

ü r

dgamma Hz, shape = n , rate = aL

# generate random numberrgamma H1, shape = n, rate = aL

ü mathematica

f@z_, a_, n_D :=an

Gamma@nD Exp@−a zD zn−1

Plot@f@z, 1, 1D, 8z, 0, 12<, PlotRange → AllD;

2 4 6 8 10 12

0.2

0.4

0.6

0.8

1

MLEtools.nb 6

Plot@f@z, 1, 2D, 8z, 0, 12<, PlotRange → AllD;

2 4 6 8 10 12

0.05

0.1

0.15

0.2

0.25

0.3

0.35

Plot@f@z, 0.5, 2D, 8z, 0, 12<, PlotRange → AllD;

2 4 6 8 10 12

0.025

0.05

0.075

0.1

0.125

0.15

0.175


Integrate@z f@z, a, nD, 8z, 0, ∞<, Assumptions → 8a > 0, n > 0, a ∈ Reals, n ∈ Reals<Dna

Derive variance

Integrate@Hz − Integrate@z f@z, a, nD, 8z, 0, ∞<, Assumptions → 8a > 0, n > 0, a ∈ Reals, n ∈ Reals<DL2

f@z, a, nD, 8z, 0, ∞<, Assumptions → 8a > 0, n > 0, a ∈ Reals, n ∈ Reals<Dna2

<< Statistics`ContinuousDistributions`H∗ gamma random number with n = 2 and a = 0.5 ∗LRandom@[email protected] , 1ê0.5DD

2.82494

MLEtools.nb 7

ü Chi-Square

Note: this section is a bit incomplete

Chi-Square Distribution: A squared normal distribution is a chi-square distribution.

probability density function for 1 degree of freedom:f HzL = 1ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅè!!!!!!!!!!!!!!!!!2 p s2 z

expH- zÅÅÅÅÅÅÅÅÅÅÅ2 s2 L

probability density function for n degrees of freedom:f HzL = 1ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅsn 2Hnê2L GH nÅÅÅÅ2 L zHnê2L-1 expH- zÅÅÅÅÅÅÅÅÅÅÅ2 s2 L, for z ¥ 0

mean: n s2

variance: 2 n s4

notes:- n can also be thought of as a shape parameter in the context of a general distribution- often assume unit variance, s = 1 - GH 1ÅÅÅÅ2 L =

è!!!!p

ü r

dchisq HL

ü mathematica

f@z_, σ_D :=1

è!!!!!!!!!!!!!!!!!2 π σ2 z

ExpA−z

2 σ2E

Plot@f@z, 2.1D, 8z, 0, 7<D;

1 2 3 4 5 6 7

0.25

0.5

0.75

1

1.25

1.5

1.75


MLEtools.nb 8

Integrate@f@z, 2.1D, 8z, 0, ∞<D

1.

<< Statistics`ContinuousDistributions`H∗ Chi−Square distributed random number with 1 degree of freedom ∗LRandom@ChiSquareDistribution@1DD

0.111756

Add non-centrality, or location, parameter with a single degree of freedom, although no scale parametermean: 1 + m

Two forms are given below

f@z_, μ_D := ExpA−z + μ

2E

1

2 J z

μN

−14

BesselIA−1ê2,è!!!!!!!

μ z E

f@z_, μ_D := Hypergeometric0F1A 1

2,

1

4 z μE

1è!!!!!!!!!!!!

2 π z ExpA−

z + μ

2E

NIntegrate@ f@z, 2.0D, 8z, 0, 100000<D

1.

NIntegrate@z f@z, 2.0D, 8z, 0, 100000<D

3.

Integrate@z f@z, μD, 8z, 0, ∞<, Assumptions → 8μ > 0, μ ∈ Reals<D

1 + μ

Hypergeometric0F1A 1

2,

1

4 z μE

CoshAè!!!!!!!z μ E

1

2 IExpAè!!!!!!!

z μ E + ExpA−è!!!!!!!

z μ EM 1

è!!!!!!!!!!!!2 π z

ExpA−z + μ

2E

IExpAè!!!!!!!z μ E + ExpA−

è!!!!!!!z μ EM

1

2 è!!!!!!!!!!!!

2 π z ExpA−

z + μ

2E

1

2 è!!!!!!!!!!!!

2 π z ExpAè!!!!!!!

z μ −z + μ

2E +

1

2 è!!!!!!!!!!!!

2 π z ExpA−

è!!!!!!!z μ −

z + μ

2E

MLEtools.nb 9

ü Bimodal Normal

Normal Distribution:has two parameters: m is the location parameter, in this case the mean; s is the scale parameter, in this case the standarddeviationprobability density function:

f HzL = 1ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅè!!!!!!!!!!!!!!2 p s2 expI- Hz-mL2

ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅ2 s2 Mmean: mvariance: s2

note: plotting f HzL will give you the familiar bell curve

Here I extend that to a bimodal normal distribution. I am assuming that I can just divide each unimodal pdf by 2, but itwould be nice to confirm this

ü r

# assuming equal weighting to each modeHdnorm Hz, m1, s1L + dnorm Hz, m2, s2LLê2

# weighting c for modesH1êH1 + exp H−cLLL ∗ dnorm Hz, m1, s1L + H1êH1 + exp HcLLL dnorm Hz, m2, s2L

ü mathematica

f@z_, m1_, σ1_, m2_, σ2_D :=1

2 è!!!!!!!!!!!!!!!!

2 π σ12 ExpA−

Hz − m1L2

2 σ12E +

1

2 è!!!!!!!!!!!!!!!!

2 π σ22 ExpA−

Hz − m2L2

2 σ22E

Plot@f@z, 5, 1, 2, 0.7D, 8z, 0, 8<D;

2 4 6 8

0.05

0.1

0.15

0.2

0.25


Integrate@f@z, 5, 1, 2, 0.7D, 8z, −∞, ∞<D

1.

MLEtools.nb 10

Therefore the support function is

H∗ bimodal normal distribution ∗Lsf@z_, m1_, σ1_, m2_, σ2_D :=

LogA2 è!!!!!!!!!

2 π E − LogA 1è!!!!!!!!!!

σ12 ExpA−

Hz − m1L2

2 σ12E +

1è!!!!!!!!

σ22 ExpA−

Hz − m2L2

2 σ22EE;

bimodalRandom@m1_, σ1_, m2_, σ2_D := BlockA8indef, x<,

indef@x_RealD := NIntegrate@bimodalPDF@z, m1, σ1, m2, σ2D, 8z, −∞, x<D;

x ê. FindRootARandom@D indef@xD, 9x,m1 + m2

2∗ 0.5,

m1 + m2

2∗ 1.0=EE

Perhaps I should not assume an equal distribution of each mode, in which case I add the parameter c which is between [0,1]and gives the percent of each mode in generating the overall distribution

f@z_, m1_, σ1_, m2_, σ2_, c_D :=c

è!!!!!!!!!!!!!!!!2 π σ12

ExpA−Hz − m1L2

2 σ12E +

1 − cè!!!!!!!!!!!!!!!!

2 π σ22 ExpA−

Hz − m2L2

2 σ22E

Plot@f@z, 5, 1, 2, 0.7, 0.3D, 8z, 0, 8<D;

2 4 6 8

0.1

0.2

0.3

0.4


Integrate@f@z, 5, 1, 2, 0.7, 0.3D, 8z, −∞, ∞<D

1.

H∗ bimodal normal distribution ∗Lsf@z_, m1_, σ1_, m2_, σ2_, c_D :=

LogAè!!!!!!!!!2 π E − LogA c

è!!!!!!!!!!σ12

ExpA−Hz − m1L2

2 σ12E +

1 − cè!!!!!!!!

σ22 ExpA−

Hz − m2L2

2 σ22EE;

MLEtools.nb 11

ü Beta

Beta Distribution:This distribution is good for modeling fractions because it is constrained to the domain between 0 and 1. It has two parame-ters: both p and q are shape parameters with p>0 and q>0, but z is restrict to the range (0,1)

probability density function:f HzL = zp-1H1-zLq-1

ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅBHp,qL , B is the beta function, B(p,q) = Ÿ01up-1H1 - uLq-1

„ u)

f HzL = zp-1H1-zLq-1ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅŸ0

1up-1H1-uLq-1 „u

mean: pÅÅÅÅÅÅÅÅÅÅÅp+q

variance: p qÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅHp+qL2 Hp+q+1L

ü r

dbeta Hz, shape1 = p, shape2 = qL

# random number generationrbeta H1, shape1 = p, shape2 = qL

ü mathematica

f@z_, p_, q_D :=zp−1 H1 − zLq−1

Beta@p, qD

Plot@f@z, 2.2, 3.6D, 8z, 0, 1<D;

0.2 0.4 0.6 0.8 1

0.5

1

1.5

2


H∗ beta distribution ∗Lsf@z_, p_, q_D := Log@Beta@p, qDD − Hp − 1L Log@zD − Hq − 1L Log@H1 − zLD

The Beta distribution can be transformed to be in terms of a mean parameter and variance parameter instead of p and q.This can be more useful because one is often interested in modeling the mean.

MLEtools.nb 12

substiution = 9q →μ

σ2 Hμ H1 − μL − σ2L

1 − μ

μ, p −>

μ

σ2 Hμ H1 − μL − σ2L=;

f@z_, μ_, σ_D :=H1 − zL−1+

H1−μL IH1−μL μ−σ2Mσ2 z−1+

μ IH1−μL μ−σ2Mσ2

BetaA μ HH1−μL μ−σ2Lσ2 , H1−μL HH1−μL μ−σ2L

σ2 E;

sf@z_, μ_, σ_D :=ikjjjj1 −

H−1 + μL H−μ + μ2 + σ2Lσ2

y{zzzz Log@H1 − zLD +

ikjjjj

−μ2 + μ3 + σ2 + μ σ2

σ2

y{zzzz Log@ zD + LogABetaA−

μ H−μ + μ2 + σ2Lσ2

,H−1 + μL H−μ + μ2 + σ2L

σ2EE ;

where m œ [0,1]and s2 < mH1 - mL

mean: mvariance: s2

The Beta distribution can be transformed to be in terms of a mean parameter and dispersion parameter, p = s(1-m) and q =sm. This just scales p and q without the specific constraints on the variance version. Here m œ (0,1), s œ (0,¶).

ü Bimodal Beta

extend the Beta Distribution by adding 2 pdf's together and dividing by 2probability density function:

f HzL = zp-1H1-zLq-1ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅ2 BHp,qL + zr-1H1-zLs-1

ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅ2 BHr,sL , B is the beta function, B(p,q) = Ÿ01up-1H1 - uLq-1

„ u)mean: variance:

ü mathematica

f@z_, v_, w_, x_, y_D :=zv−1 H1 − zLw−1

2 Beta@v, wD +zx−1 H1 − zLy−1

2 Beta@x, yD

[email protected], 2.2, 3.6, 2.2, 3.6D

1.8448

MLEtools.nb 13

Plot@f@z, 1.3, 3.6, 4.1, 1.6D, 8z, 0, 1<D;

0.2 0.4 0.6 0.8 1

0.4

0.6

0.8

1.2

Integrate@f@z, 1.3, 3.6, 4.1, 1.6D, 8z, 0, 1<D

1.


H∗ bimodal beta distribution ∗L

sf@z_, v_, w_, x_, y_D := −LogA zv−1 H1 − zLw−1


2 Beta@x, yD E

Discrete

Expected, or meanE@ZD = ⁄z z fz

VarianceVARHZL = E@HZ - E@ZDL2D = ⁄z Hz - E@zDL2 fz

where Z is a random variable, z is a particular instance of that variable, fz is the frequency of occurance of z.

For discrete variables, the probability distribution function is the probability of the value z given the parameters

MLEtools.nb 14

ü Binomial

Binomial Distribution:has two parameters: N is the number of times the trial was conducted in which one of two outcomes can occur. p is theprobability of one outcome, and z is the total number of times that outcome occurs. For example, flip a coin 10 times andcount the number of heads. The result would be a binomial distribution, and if the coin is fair, p = 0. 5. Whereas the otherdiscrete distributions are used to model discrete observations of continuous processes, the binomial models discrete pro-cesses.probability distribution function:

Pr 8Z = z< = I N!ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅHN-zL! z! M pzH1 - pLN-z

mean: N pvariance: N p H1 - pL

ü r

dbinom Hz, n, pLplot Hdbinom Hseq H1 : 20L, 20, 0.3LL

# random number generationrbinom H1, n, pL

# see function description belowcurve H1 êH1 + exp H−xLL, −10, 10L

ü mathematica

f@z_, n_, p_D :=n!

Hn − zL! z! pz H1 − pLn−z

ListPlot@Table@8z, f@z, 20, 0.25D<, 8z, 0, 20<D, PlotStyle → [email protected];

5 10 15 20

0.05

0.1

0.15

0.2

Sums to 1

Plus @@ Table@f@z, 20, 0.3D, 8z, 0, 20<D

1.

MLEtools.nb 15

<< Statistics`DiscreteDistributions`H∗ binomial random number with n = 3 and p = 0.5 ∗LRandom@BinomialDistribution@3 , 0.5DD

1

Here is a useful function for modeling a probability. The funciton is constrained to a range between 0 and 1 for all realvalues of f HxL, both positive and negative. This function is derived by the following logic:

1) Probabilities œ [0,1]. An odds ratio is a ratio of the probability of sucess over the probability of failure, and odds ratios œ [0,¶). Thereforeln(odds ratios) œ (-¶,¶).

2) For a binary model, only two outcomes are examined, therefore the Pr{failure} = 1 - Pr{success}.lnI Pr 8success<ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅPr 8failure< M = f HxLPr 8success<ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅPr 8failure< = expH f HxLL

Pr 8success<ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅ1-Pr 8success< = expH f HxLLPr 8success< = expH f HxLLÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅexpH f HxLL+1

Pr 8success< = 1ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅ1+expH- f HxLL

parameters = 8a → −4, b → 0.7<;f@x_D := a + b x ê. parameters;

PlotA 1

1 + Exp@−f@xDD ê. parameters, 8x, 0, 10<, PlotRange → 80, 1<E;

2 4 6 8 10

0.2

0.4

0.6

0.8

1

The Hill function is also useful for modeling probabilities as it contains interpretable parameters: a is the point at which theprobability equals 0.50 and b is the "slope" at the 0.50 point.

MLEtools.nb 16

parameters = 8a → 5, b → 4<;

PlotA 1

1 + H axLb

ê. parameters, 8x, 0, 10<, PlotRange → 80, 1<E;

2 4 6 8 10

0.2

0.4

0.6

0.8

1

ü Multinomial

Multinomial Distribution:This is an extension of the binomial to m possible outcomes each with a probability pi.probability distribution function:

Pr 8Zm = zm< = I N!ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅz1! z2! . . . zm! M p1z1 p2

z2 . . . pmzm

mean: N pm

variance: N pm H1 - pmL

ü mathematica

f@z1_, z2_, n_, p1_, p2_D :=n!

z1! z2! Hn − z1 − z2L! p1z1 p2z2 H1 − p1 − p2Ln−z1−z2

ListPlot@Table@8z, f@z, 3, 20, 0.25, 0.25D<, 8z, 0, 20<D, PlotStyle → [email protected];

5 10 15 20

0.005

0.01

0.015

0.02

0.025

Sums to 1

MLEtools.nb 17

Plus @@ Flatten@Table@f@z, i, 5, 0.25, 0.25D, 8i, 0, 5<, 8z, 0, 5<DD

1.

<< Statistics`MultiDiscreteDistributions`

H∗ multinomial random vector for an event that occurs 10 times;the probability vector follows ∗L

Random@MultinomialDistribution@10, 80.2, 0.2, 0.3, 0.3<DD

81, 3, 2, 4<

ü r

dmultinom HL

# random number generation

ü Poisson

Poisson Distribution:has one parameter that can really be broken into two parameters: r is the rate constant and t is the time.One description of a poisson distribution: Imagine a process with a constant probability of occurance r. The event canoccur at any time, and repeatedly. However, if we measure at discrete time intervals, and count the number of times theevent has occurred for discrete entities, the result will be a Poisson Distribution. Imagine rain falling onto floor tiles. Ifyou count the number of drops that hit each tile in one minute, the result would be distribute as a Poisson distribution.probability distribution function:

Pr 8Z = z< = e-r tHr tLzÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅz!

mean: r tvariance: r tnote: if evenly spaced time intervals, set t equal to 1

ü r

dpois Hz, lambdaL

# random number generationrpois H1, lambdaL

ü mathematica

f@z_, λ_D :=Exp@−λD λz

z!

MLEtools.nb 18

ListPlot@Table@8z, f@z, 3D<, 8z, 0, 15<D, PlotStyle → [email protected];

2 4 6 8 10 12 14

0.05

0.1

0.15

0.2

Sums to 1

Plus @@ Table@f@z, 20, 0.3D, 8z, 0, 100<D

1.

<< Statistics`DiscreteDistributions`H∗ poisson random number with λ = 3.26 ∗LRandom@[email protected]

2

ü Negative binomial

Negative Binomial Distribution:The negative binomial is similar to the poisson except that the data are overdispersed. As a consequence, the variance willalways be larger than the mean. The negative binomial can be derived assuming the rate constant in the poisson is nolonger constant but is gamma distributed. As k Ø ¶, the distribution approaches a Poisson. k is the shape parameter.probability distribution function:

Pr 8Z = z< = GHk+zLÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅGHkL z! H kÅÅÅÅÅÅÅÅÅÅÅk+m Lk H mÅÅÅÅÅÅÅÅÅÅÅk+m Lz

where G is the gamma function, GHnL = Ÿ0¶ExpH-tL tn-1 „ t

mean: mvariance: m + m2

ÅÅÅÅÅÅÅÅk

Alternative form: used to describe the number the number of failures that occur before n sucesses have occured, where theprobability of success is p.

ü r

# note, "mu =" is needed to specify this particular parameterizationdnbinom Hz, size = k, mu = mL

rnbinom H1, size = k, mu = mL

MLEtools.nb 19

ü mathematica

f@z_, k_, m_D :=Gamma@k + zDGamma@kD z!

ikjj k

k + my{zz

k

J m

k + mN

z

ListPlot@Table@8z, f@z, 10, 3.2D<, 8z, 0, 15<D, PlotStyle → [email protected];

2 4 6 8 10 12 14

0.05

0.1

0.15

0.2

ListPlot@Table@8z, f@z, 0.26, 10D<, 8z, 0, 100<D, PlotStyle → [email protected];

20 40 60 80 100

0.005

0.01

0.015

0.02

Sums to 1

Plus @@ Table@f@z, 10, 3D, 8z, 0, 100<D êê N

1.


Sum@z f@z, k, mD, 8z, 0, ∞<D

m

Derive variance

Sum@Hz − Sum@z f@z, k, mD, 8z, 0, ∞<DL2 f@z, k, mD, 8z, 0, ∞<D êê Simplify

m Hk + mLk

MLEtools.nb 20

Mathematica does not contain a built-in random number generator for this form of the negative binomial. While not veryelegant, the function below can be used to randomly generate numbers from a negative binomial distribution.

NegBiRandom@m_, k_D := 9

funtable = TableA Gamma@k + xDGamma@kD x!

ikjj k

k + my{zz

k

J m

k + mN

x, 8x, 0, 250<E;

CDF = Table@Sum@funtable@@iDD, 8i, endsum<D, 8endsum, 1, Length@funtableD<D;x = Random@D;i = 1;While@x > CDF@@iDD,

i++;D;i − 1=@@1DD

NegBiRandom@10, 3D

5

H∗ negative binomial distribution ∗LClear@sfD;sf@z_, m_ ?NumberQ, k_ ?NumberQD := If@m > 0 && k > 0, −Log@Gamma@k + zDD + Log@z!D +

Log@Gamma@kDD − k Log@kD + k Log@k + mD − z Log@mD + z Log@k + mD, 10000.0D;SetAttributes@sf, ListableD;

ü Generalized Poisson

The generalized Poisson distribution (Consul and Jain. 1973. Technometrics. 15: 791-799) can be used to represent aPoisson process in which the data are over-dispersed. It is also known as Lagrangian Poisson Distribution (Johnson, Kotz,and Kemp. 1992. Univariate Discrete Distributions). Despite the claims of the original authors, this distribution cannot beused for under-dispersed data because the sum of the probabilities do not add to 1.

This distribution explicity models an increase in the rate of moving to subsequent categories and can represent data withvery long right-tails as compared to the negative binomial. This distribution worked very well for the distribution of pollendeposited on flowers (Castellanos et al. 2003. Evolution 57:2742-2752.) When l2 = 0 this distribution becomes the Poissondistribution.

Pr 8Z = z< = e-Hl1+z l2 L l1Hl1+z l2Lz-1ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅz! , where 0 < l1 and 0 § l2 < 1

mean: l1ÅÅÅÅÅÅÅÅÅÅÅÅÅ1-l2

variance: l1ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅH1-l2L3

f@z_, λ1_, λ2_D :=λ1 Hλ1 + z λ2Lz−1 Exp@−Hλ1 + z λ2LD

z!

Consul and Jain (1973) list the following method for estimating l1and l2from the mean and variance, but I don't trust theseas the the best estimates, just a close estimate.

l2 = 1 - "###############meanÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅvariance

MLEtools.nb 21

l1 = mean H1 - l2L

ü Beta-Binomial

Beta Binomial Distribution:This distribution assumes that p in the binomial distribution has a beta distribution with parameters v and w (see betadistribution above).

probability distribution function:Pr 8Z = z< = I N!ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅHN-zL! z! M BHv+z,N+w-zLÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅBHv,wL , where B(v,w) is the beta function

mean: variance:

The Beta distribution can be transformed to be in terms of a mean parameter and dispersion parameter, p = s(1-m) and q =sm. This just scales p and q without the specific constraints on the variance version. Here m œ (0,1), s œ (0,¶). The m, sversion is the one used for sfBetaBinomial in Mathematica. This form has been used to describe correlated instances of theBinomial Distribution, although more mechanistic distributions for these cases exist.

ü r

library HrmutilL# this is based on a parameterization so that m is the mean and s is# an overdispersion parameters. As s approaches infintiy,# the Beta − Binomial collapses to a Biomial Distributiondbetabinom Hz, N, m, sL

ü mathematica

f@z_, n_, m_, s_D :=n!

Hn − zL! z!

Beta@s m + z, n + s H1 − mL − zDBeta@s m, s H1 − mLD

f@z_, n_, v_, w_D :=n!

Hn − zL! z!

Beta@v + z, n + w − zDBeta@v, wD

ListPlot@Table@8z, f@z, 20, 2.0, 3.2D<, 8z, 0, 20<D, PlotStyle → [email protected];

5 10 15 20

0.02

0.04

0.06

0.08

MLEtools.nb 22

Sums to 1

Plus @@ Table@f@z, 20, 2.0, 3.2D, 8z, 0, 20<D

1.

H∗ Beta binomial distribution ∗Lsf@z_, n_, v_, w_D :=

Log@Hn − zL!D + Log@z!D − Log@n!D + Log@Beta@v, wDD − Log@Beta@v + z, n + w − zDD

The beta binomial is derived by multiply the pdf of the beta distribution by the pdf of the binomial, and then integratingfrom across the the full range of z for the beta binomial, in this case [0,1]

H∗ pdf for beta distribution ∗L

f@z_, p_, q_D :=zp−1 H1 − zLq−1

Beta@p, qD

H∗ pdf for binomial distribution ∗Lf@z_, n_, p_D :=

n!


IntegrateA n!


pv−1 H1 − pLw−1

Beta@v, wD ,

8p, 0, 1<, Assumptions → 8v > 0, w > 0, z ≥ 0, n ≥ 0<E

1Beta@v, wD Hn − zL! z!

Jn! IfAz < n + w, Gamma@n + w − zD Gamma@v + zDGamma@n + v + wD , Integrate@H1 − pL−1+n+w−z p−1+v+z,

8p, 0, 1<, Assumptions → v > 0 && w > 0 && n ≥ 0 && z ≥ 0 && z ≥ n + wDEN

Mathematica doesn't do a good job of pulling out the beta function, but this simplifies to

n!

Hn − zL! z!

Beta@z + vD Beta@w + n − zDBeta@v, wD

ü Bimodal Beta-Binomial

Using the same assumptions from the derivation of the unimodal beta binomial distribution, multiply the bimodal betadistribition by the binomial distribution and integrate p from 0 to 1

H∗ pdf for binomial distribution ∗Lf@z_, n_, p_D :=

n!


H∗ pdf for bimodal beta ∗L

f@z_, v_, w_, x_, y_D :=zv−1 H1 − zLw−1


2 Beta@x, yD

MLEtools.nb 23

H∗ multiply both distributions by each other ∗Ln!


ikjjjj


2 Beta@v, wD +px−1 H1 − pLy−1

2 Beta@x, yDy{zzzz

n!



2 Beta@v, wD +n!


px−1 H1 − pLy−1

2 Beta@x, yDn!

Hn − zL! z!

pz+v−1 H1 − pLw+n−z−1

2 Beta@v, wD +n!

Hn − zL! z!

pz+x−1 H1 − pLy+n−z−1

2 Beta@x, yDn!

Hn − zL! z! ikjjjj


2 Beta@v, wD +pz+x−1 H1 − pLy+n−z−1

2 Beta@x, yDy{zzzz

‡0

1 n!

Hn − zL! z! ikjjjj


2 Beta@v, wD +pz+x−1 H1 − pLy+n−z−1

2 Beta@x, yDy{zzzz p

n!

Hn − zL! z! ‡

0

1 pz+v−1 H1 − pLw+n−z−1

2 Beta@v, wD p +n!

Hn − zL! z! ‡

0

1 pz+x−1 H1 − pLy+n−z−1

2 Beta@x, yD p

n!

Hn − zL! z!

Beta@z + v, w + n − zD2 Beta@v, wD +

n!

Hn − zL! z!

Beta@z + x, y + n − zD2 Beta@x, yD

n!

Hn − zL! z! ikjj Beta@z + v, w + n − zD

2 Beta@v, wD +Beta@z + x, y + n − zD

2 Beta@x, yDy{zz

H∗ pdf for bimodal beta−binomal distribution ∗L

f@z_, n_, v_, w_, x_, y_D :=n!

Hn − zL! z! ikjj Beta@z + v, w + n − zD

2 Beta@v, wD +Beta@z + x, y + n − zD

2 Beta@x, yDy{zz

Bimodal Beta-Binomial Distribution:has 5 parameters: shape parameters for one node are v and w; shape parameters for the other node are x and y; n is the totalnumber of individuals in the set and z is the number of success in that set.

Note: this distribution and the unimodal beta-binomial assume a different draw from beta for the p of each set, but itassumes the same p for all the individuals within a set. This statement is based on Skellam. 1948. Journal of the RoyalStatistical Society. Series B, 10: 257-261.

probability density function:f HzL = zp-1H1-zLq-1

ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅBHp,qL , B is the beta function, B(p,q) = Ÿ01up-1H1 - uLq-1

„ u)

f HzL = zp-1H1-zLq-1ÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅŸ0

1up-1H1-uLq-1 „u

mean: variance:

MLEtools.nb 24

ListPlot@Table@f@z, 30, 1.2, 3.6, 3.6, 1.2D, 8z, 0, 30<D,PlotStyle → [email protected]<, PlotRange → 80, 0.04<D;

5 10 15 20 25 30

0.01

0.02

0.03

0.04

PlotA zv−1 H1 − zLw−1


2 Beta@x, yD ê. 8v −> 1.2, w −> 3.6, x −> 3.6, y −> 1.2<, 8z, 0, 1<E;

0.2 0.4 0.6 0.8 1

0.2

0.4

0.6

0.8

1

1.2

Plus @@ Table@f@z, 30, 1.2, 3.6, 3.6, 1.2D, 8z, 0, 30<D

1.


H∗ bimodal beta−binomial distribution ∗Lsf@z_, n_, v_, w_, x_, y_D :=

Log@Hn − zL!D + Log@z!D − Log@n!D − LogA Beta@z + v, w + n − zD2 Beta@v, wD +

Beta@z + x, y + n − zD2 Beta@x, yD E

ü Bimodal Binomial

f@z_, n_, p_, q_D :=n!

2 Hn − zL! z! pz H1 − pLn−z +

n!

2 Hn − zL! z! qz H1 − qLn−z

MLEtools.nb 25

ListPlot@Table@8z, f@z, 20, 0.15, 0.45D<, 8z, 0, 20<D, PlotStyle → [email protected];

5 10 15 20

0.02

0.04

0.06

0.08

0.1

0.12

Sums to 1

Plus @@ Table@f@z, 20, 0.3D, 8z, 0, 20<D

1.

H∗ bimodal binomial ∗Lsf@z_, n_, p_, q_D := Log@2 Hn − zL!D + Log@z!D − Log@n!D − Log@pz H1 − pLn−z + qz H1 − qLn−zD

MLEtools.nb 26

reference: some basic distributions excellent reference for probability distributions is: evans, ......

Documents