moment generating functions - university of saskatchewanlaverty/s241/s241 lectures pdf/10... ·...

Moment Generating Functions

0

0.1

0.2

0.3

0.4

0 5 10 15

1

b a

a b

f x

x0

0.1

0.2

0.3

0.4

0 5 10 15

1

b a

a b

f x

x

1

b a

a b

f x

x

Continuous Distributions

The Uniform distribution from a to b

1

0 otherwise

a x bf x b a

The Normal distribution

(mean m, standard deviation s)

2

221

2

x

f x em

s

s

0

0.1

0.2

-2 0 2 4 6 8 10

The Exponential distribution

0

0 0

xe xf x

x

Weibull distribution with parameters a and b.

Thus 1x

F x eba

b

1and 0x

f x F x x e xba

b ba

The Weibull density, f(x)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0 1 2 3 4 5

(a = 0.5, b = 2)

(a = 0.7, b = 2)

(a = 0.9, b = 2)

The Gamma distribution

Let the continuous random variable X have

density function:

1 0

0 0

xx e xf x

x

aa

a

Then X is said to have a Gamma distribution

with parameters a and .

Expectation of functions of

Random Variables

X is discrete

i i

x i

E g X g x p x g x p x

E g X g x f x dx

X is continuous

Moments of Random Variables

k

k E Xm

-

if is discrete

if is continuous

k

x

k

x p x X

x f x dx X

The kth moment of X.

the kth central moment of X

0 k

k E Xm m

-

if is discrete

if is continuous

k

x

k

x p x X

x f x dx X

m

m

where m = m1 = E(X) = the first moment of X .

Rules for expectation

Rules:

1. where is a constantE c c c

2. where , are constantsE aX b aE X b a b

20

23. var X E Xm m

22 2

2 1E X E X m m

24. var varaX b a X

Moment generating functions

Moment Generating function of a R.V. X

if is discrete

if is continuous

tx

xtX

Xtx

e p x X

m t E e

e f x dx X

Examples

1. The Binomial distribution (parameters p, n)

tX tx

X

x

m t E e e p x

0

1n

n xtx x

x

ne p p

x

0 0

1n n

x n xt x n x

x x

n ne p p a b

x x

1nn ta b e p p

0,1,2,!

x

p x e xx

The moment generating function of X , mX(t) is:

2. The Poisson distribution (parameter )

tX tx

X

x

m t E e e p x 0 !

xtx

x

e ex

0 0

using ! !

t

xt x

e u

x x

e ue e e e

x x

1tee

0

0 0

xe xf x

x


3. The Exponential distribution (parameter )

0

tX tx tx x

Xm t E e e f x dx e e dx

0 0

t xt x e

e dxt

undefined

tt

t

2

21

2

x

f x e


4. The Standard Normal distribution (m = 0, s = 1)

tX tx

Xm t E e e f x dx

2 22

1

2

x tx

e dx

2

21

2

xtxe e dx

2 2 2 22 2

2 2 21 1

2 2

x tx t x tx t

Xm t e dx e e dx

We will now use the fact that 2

221

1 for all 0,2

x b

ae dx a ba

We have

completed

the square

22 2

2 2 21

2

x tt t

e e dx e

This is 1

1 0

0 0

xx e xf x

x

aa

a


4. The Gamma distribution (parameters a, )

tX tx

Xm t E e e f x dx

1

0

tx xe x e dxa

a

a

1

0

t xx e dx

aa

a

We use the fact

1

0

1 for all 0, 0a

a bxbx e dx a b

a

1

0

t x

Xm t x e dxa

a

a

1

0

t xtx e dx

tt

a aaa

a

a

Equal to 1

Properties of

Moment Generating Functions

1. mX(0) = 1

0, hence 0 1 1tX X

X Xm t E e m E e E

v) Gamma Dist'n Xm tt

a

2

2iv) Std Normal Dist'n t

Xm t e

iii) Exponential Dist'n Xm tt

1

ii) Poisson Dist'n te

Xm t e

i) Binomial Dist'n 1n

t

Xm t e p p

Note: the moment generating functions of the following

distributions satisfy the property mX(0) = 1

2 33212. 1

2! 3! !

kkXm t t t t t

k

m mmm

We use the expansion of the exponential function:

2 3

12! 3! !

ku u u u

e uk

tX

Xm t E e

2 32 31

2! 3! !

kkt t t

E tX X X Xk

2 3

2 312! 3! !

kkt t t

tE X E X E X E Xk

2 3

1 2 312! 3! !

k

k

t t tt

km m m m

0

3. 0k

k

X X kk

t

dm m t

dtm

Now

2 33211

2! 3! !

kkXm t t t t t

k

m mmm

2 1321 2 3

2! 3! !

kkXm t t t kt

k

m mmm

2 13

1 22! 1 !

kkt t tk

m mm m

1and 0Xm m

242 3

2! 2 !

kkXm t t t t

k

mmm m

2and 0Xm m

continuing we find 0k

X km m

i) Binomial Dist'n 1n

t

Xm t e p p

Property 3 is very useful in determining the moments of a

random variable X.

Examples

1

1n

t t

Xm t n e p p pe

1

0 0

10 1n

Xm n e p p pe np m m

2 1

1 1 1n n

t t t t t

Xm t np n e p p e p e e p p e

2

2

1 1 1

1 1

nt t t t

nt t t

npe e p p n e p e p p

npe e p p ne p p

2 2

21np np p np np q n p npq m

1

ii) Poisson Dist'n te

Xm t e

1 1t te e tt

Xm t e e e

1 1 2 121

t t te t e t e tt

Xm t e e e e

1 2 12 2 1

t te t e tt t

Xm t e e e e

1 2 12 3t te t e tte e e

1 3 1 2 13 23t t te t e t e t

e e e

0 1 0

1 0e

Xm e

m

0 01 0 1 02 2

2 0e e

Xm e e

m

3 0 2 0 0 3 2

3 0 3 3t

Xm e e em

To find the moments we set t = 0.

iii) Exponential Dist'n Xm tt

1

X

d tdm t

dt t dt

2 2

1 1t t

3 3

2 1 2Xm t t t

4 4

2 3 1 2 3Xm t t t

5 54

2 3 4 1 4!Xm t t t

1

!kk

Xm t k t

Thus

2

1

10Xmm m

3

2 2

20 2Xmm

1 !

0 !kk

k X k

km km

2 33211

2! 3! !

kkXm t t t t t

k

m mmm

The moments for the exponential distribution can be calculated in an

alternative way. This is note by expanding mX(t) in powers of t and

equating the coefficients of tk to the coefficients in:

2 31 11

11Xm t u u u

tt u

2 3

2 31

t t t

Equating the coefficients of tk we get:

1 ! or

!

kkk k

k

k

mm

2

2t

Xm t e

The moments for the standard normal distribution

We use the expansion of eu. 2 3

0

1! 2! 3! !

k ku

k

u u u ue u

k k

2 2 2

22

2

2 3

2 2 2

21

2! 3! !

t

kt t t

tXm t e

k

2 4 6 212 2 3

1 1 11

2 2! 2 3! 2 !

k

kt t t t

k

We now equate the coefficients tk in:

2 22211

2! ! 2 !

k kk kXm t t t t t

k k

m mmm

If k is odd: mk = 0.

2 1

2 ! 2 !

k

kk k

mFor even 2k:

2

2 !or

2 !k k

k

km

1 2 3 4 2

2! 4!Thus 0, 1, 0, 3

2 2 2!m m m m

Summary

Moments

Moment generating functions

Moments of Random Variables

k

k E Xm

-

if is discrete

if is continuous

k

x

k

x p x X

x f x dx X

The moment generating function

if is discrete

if is continuous

tx

xtX

Xtx

e p x X

m t E e

e f x dx X

Examples

1 0,1,2, ,n xx

np x p p x n

x


1n n

t t

Xm t e p p e p q

0,1,2,!

x

p x e xx


1te

Xm t e

0

0 0

xe xf x

x


undefined

X

tm t t

t

2

21

2

x

f x e


2

2t

Xm t e

1 0

0 0

xx e xf x

x

aa

a


Xm tt

a

6. The Chi-square distribution (degrees of freedom n)

(a n/2, 1/2

21

2 2

112

2

0

0 0

xx e x

f x

x

n

n

n

2

2

12

12

1 2Xm t tt

n

n

1. mX(0) = 1

2 33212. 1

2! 3! !

kkXm t t t t t

k

m mmm

0

3. 0k

k

X X kk

t

dm m t

dtm

Properties of Moment Generating Functions

1i.e. 0Xm m

20Xm m

30 , etcXm m

Let lX (t) = ln mX(t) = the log of the moment generating function

Then 0 ln 0 ln1 0X Xl m

The log of Moment Generating Functions

1

X

X X

X X

m tl t m t

m t m t

2

2

X X X

X

X

m t m t m tl t

m t

1

0 0

0

X

X

X

ml

mm m

2

2 2

2 12

0 0 0 0

0

X X X

X

X

m m ml

mm m s

Thus lX (t) = ln mX(t) is very useful for calculating the mean and variance of a random variable

1. 0Xl m

2 2. 0Xl s

Examples


1n n

t t

Xm t e p p e p q

ln ln t

X Xl t m t n e p q

1 t

X tl t n e p

e p q

10Xl n p np

p qm

2

t t t t

Xt

e p e p q e p e pl t n

e p q

2

20X

p p q p pl n npq

p qs


1te

Xm t e

ln 1t

X Xl t m t e

t

Xl t e

t

Xl t e

0Xlm

2 0Xls


undefined

X

tm t t

t

ln ln ln if X Xl t m t t t

11

Xl t tt

2

2

11 1Xl t t

t

2

2

1 1Thus 0 and 0X Xl lm s


2

2t

Xm t e

2

2ln t

X Xl t m t

, 1X Xl t t l t

2Thus 0 0 and 0 1X Xl lm s


Xm tt

a

ln ln lnX Xl t m t ta

1

Xl tt t

aa

2

21 1Xl t t

t

aa

2

2Hence 0 and 0X Xl l

a am s

6. The Chi-square distribution (degrees of freedom n)

21 2Xm t tn

ln ln 1 22

X Xl t m t tn

1

22 1 2 1 2

Xl tt t

n n

2

2

21 1 2 2

1 2Xl t t

t

nn

2Hence 0 and 0 2X Xl lm n s n

Summary of Discrete Distributions

Name

probability function p(x)

Mean

Variance

Moment

generating

function MX(t)

Discrete

Uniform Nxp

1)( x=1,2,...,N

2

1N

12

12 N

1

1

t

tNt

e

e

N

e

Bernoulli

0

1)(

xq

xpxp

p pq q + pet

Binomial xNxqp

x

Nxp

)(

Np Npq (q + pet)N

Geometric p(x) =pqx-1 x=1,2,...

p

1

2p

q

t

t

qe

pe

1

Negative

Binomial kxk qp

k

xxp

1

1)(

x = k, k+1,...

p

k

2p

kq

k

t

t

qe

pe

1

Poisson p(x) =

x

x! e- x=1,2,...

e(et-1)

Hypergeometric

p(x) =

A

x

N-A

n-x

N

n

n

A

N n

A

N

1-A

N

N-n

N-1

not useful

Name

probability

density function f(x)

Mean

Varianc

e

Moment generating

function MX(t)

Continuous

Uniform

otherwise

bxaabxf

0

1

)(

a+b

2

(b-a)2

12

tab

ee atbt

][

Exponential

00

0

x

xexf

x

1

1

2

t

for t

<

Gamma

00

0)(

1

x

xexxf

xaa

a

a

a

2

t

a

for t <

c2

n d.f. f(x) =

(1/2)n/2

(n/2) xn/21e-(1/2)x x ≥ 0

0 x < 0

n 2n

1

1-2t

n/2

for t < 1/2

Normal f(x) =

2

2

2

1

2

1s

m

s

x

exf e-(x-

m)2/2s2

m s2 etm+(1/2)t2s2

Weibull

f(x) =

x1 e-x/ x ≥ 0

0 x < 0

2/( )+1

2/

( )+2

-[ ]( )+1

2

not

avail.

Continuous distributions

moment generating functions - university of saskatchewanlaverty/s241/s241 lectures pdf/10... ·...

Documents