SOME GENERAL PROBLEMS*

Problem A certain lion has three possible states of activity each night; they are very active (denoted by 1), moderately active (denoted by 2), and lethargic (lacking energy) (denoted by 3). Also, each night this lion eats people; it eats i people with probability p(i|), ={1, 2, 3} . Of course, the probability distribution of the number of people eaten depends on the lions activity state . The numeric values are given in the following table. *

Problem*If we are told X=x0 people were eaten last night, how should we estimate the lions activity state (1, 2 or 3)?

i01234p(i|1)00.050.050.80.1p(i|2)0.050.050.80.10p(i|3)0.90.080.0200

SolutionOne reasonable method is to estimate as that in for which p(x0|) is largest. In other words, the that provides the largest probability of observing what we did observe. : the MLE of based on X

(Taken from Dudewicz and Mishra, 1988, Modern Mathematical Statistics, Wiley)

*

ProblemConsider the Laplace distribution centered at the origin and with the shape parameter , which for all x has the p.d.f.

Find MME and MLE of .

*

ProblemLet X1,,Xn be independent r.v.s each with lognormal distribution, ln N(,2). Find the MMEs of ,2*

STATISTICAL INFERENCEPART IIIBETTER OR BEST ESTIMATORS, FISHER INFORMATION, CRAMER-RAO LOWER BOUND (CRLB)*

*RECALL: EXPONENTIAL CLASS OF PDFSIf the pdf can be written in the following formthen, the pdf is a member of exponential class of pdfs. (Here, k is the number of parameters)

*EXPONENTIAL CLASS and CSSRandom Sample from Regular Exponential Classis a css for .

*RAO-BLACKWELL THEOREMLet X1, X2,,Xn have joint pdf or pmf f(x1,x2,,xn;) and let S=(S1,S2,,Sk) be a vector of jss for . If T is an UE of () and (S)=E(TS), then(S) is an UE of () .(S) is a fn of S, so it is free of .Var((S) ) Var(T) for all .(S) is a better unbiased estimator of () .

RAO-BLACKWELL THEOREMNotes:(S)=E(TS) is at least as good as T. For finding the best UE, it is enough to consider UEs that are functions of a ss, because all such estimators are at least as good as the rest of the UEs. *

ExampleHogg & Craig, Exercise 10.10X1,X2~Exp()Find joint p.d.f. of ss Y1=X1+X2 for and Y2=X2.Show that Y2 is UE of with variance .Find (y1)=E(Y2|Y1) and variance of (Y1).

*

THE MINIMUM VARIANCE UNBIASED ESTIMATORRao-Blackwell Theorem: If T is an unbiased estimator of , and S is a ss for , then (S)=E(TS) isan UE of , i.e.,E[(S)]=E[E(TS)]= and with a smaller variance than Var(T).

*

*LEHMANN-SCHEFFE THEOREMLet Y be a css for . If there is a function Y which is an UE of , then the function is the unique Minimum Variance Unbiased Estimator (UMVUE) of . Y css for . T(y)=fn(y) and E[T(Y)]=.T(Y) is the UMVUE of . So, it is the best unbiased estimator of .

*THE MINIMUM VARIANCE UNBIASED ESTIMATORLet Y be a css for . Since Y is complete, there could be only a unique function of Y which is an UE of . Let U1(Y) and U2(Y) be two function of Y. Since they are UEs, E(U1(Y)U2(Y))=0 imply W(Y)=U1(Y)U2(Y)=0 for all possible values of Y. Therefore, U1(Y)=U2(Y) for all Y.

ExampleLet X1,X2,,Xn ~Poi(). Find UMVUE of .Solution steps:Show that is css for .

Find a statistics (such as S*) that is UE of and a function of S.Then, S* is UMVUE of by Lehmann-Scheffe Thm.

*

NoteThe estimator found by Rao-Blackwell Thm may not be unique. But, the estimator found by Lehmann-Scheffe Thm is unique. *

*RECALL: EXPONENTIAL CLASS OF PDFSIf the pdf can be written in the following formthen, the pdf is a member of exponential class of pdfs. (Here, k is the number of parameters)

*EXPONENTIAL CLASS and CSSRandom Sample from Regular Exponential Classis a css for .If Y is an UE of , Y is the UMVUE of .

*EXAMPLESLet X1,X2,~Bin(1,p), i.e., Ber(p).

This family is a member of exponential family of distributions.is a CSS for p.is UE of p and a function of CSS.is UMVUE of p.

*EXAMPLESX~N(,2) where both and 2 is unknown. Find a css for and 2 .

*FISHER INFORMATION AND INFORMATION CRITERIAX, f(x;), , xA (not depend on ).Definitions and notations:

*FISHER INFORMATION AND INFORMATION CRITERIA The Fisher Information in a random variable X:The Fisher Information in the random sample:Lets prove the equalities above.

*FISHER INFORMATION AND INFORMATION CRITERIA

*FISHER INFORMATION AND INFORMATION CRITERIA

*FISHER INFORMATION AND INFORMATION CRITERIA The Fisher Information in a random variable X:The Fisher Information in the random sample:Proof of the last equality is available on Casella & Berger (1990), pg. 310-311.

*CRAMER-RAO LOWER BOUND (CRLB)Let X1,X2,,Xn be sample random variables.Range of X does not depend on .Y=U(X1,X2,,Xn): a statistic; doesnt contain .Let E(Y)=m().

Let prove this!

*CRAMER-RAO LOWER BOUND (CRLB)-1Corr(Y,Z)1

0 Corr(Y,Z)21

Take Z=(x1,x2,,xn;)Then, E(Z)=0 and V(Z)=In() (from previous slides).

*CRAMER-RAO LOWER BOUND (CRLB)Cov(Y,Z)=E(YZ)-E(Y)E(Z)=E(YZ)

*CRAMER-RAO LOWER BOUND (CRLB)E(Y.Z)=m(), Cov(Y,Z)=m(), V(Z)=In()

The Cramer-Rao Inequality(Information Inequality)

*CRAMER-RAO LOWER BOUND (CRLB)CRLB is the lower bound for the variance of an unbiased estimator of m().When V(Y)=CRLB, Y is the MVUE of m().For a r.s., remember that In()=n I(), so,

*ASYMPTOTIC DISTRIBUTION OF MLEs : MLE of X1,X2,,Xn is a random sample.

*EFFICIENT ESTIMATORT is an efficient estimator (EE) of if T is UE of , and,Var(T)=CRLBT is an efficient estimator (EE) of its expectation, m(), if its variance reaches the CRLB.An EE of m() may not exist.The EE of m(), if exists, is unique.The EE of m() is the unique MVUE of m().

*ASYMPTOTIC EFFICIENT ESTIMATORY is an asymptotic EE of m() if

*EXAMPLESA r.s. of size n from X~Poi(). Find CRLB for any UE of .Find UMVUE of .Find an EE for .Find CRLB for any UE of exp{-2}. Assume n=1, and show that is UMVUE of exp{-2}. Is this a reasonable estimator?

*EXAMPLEA r.s. of size n from X~Exp(). Find UMVUE of , if exists.

SummaryWe covered 3 methods for finding good estimators (possibly UMVUE):Rao-Blackwell Theorem (Use a ss T, an UE U, and create a new statistic by E(U|T))Lehmann-Scheffe Theorem (Use a css T which is also UE)Cramer-Rao Lower Bound (Find an UE with variance=CRLB)*

ProblemsLet be a random sample from gamma distribution, Xi~Gamma(2,). The p.d.f. of X1 is given by:

a) Find a complete and sufficient statistic for .b) Find a minimal sufficient statistic for .c) Find CRLB for the variance of an unbiased estimator of .d) Find a UMVUE of .

*

ProblemsSuppose X1,,Xn are independent with densityfor >0a) Find a complete sufficient statistic.b) Find the CRLB for the variance of unbiased estimators of 1/.c) Find the UMVUE of 1/ if there is one.

*

*************************