chapter2. minimum variance unbiased...
TRANSCRIPT
Chapter2. Minimum variance unbiased
estimation
Student: Işfan Geraldina ,1st year of Master in Communication Networks
01.11.2010
Coordinating professor: Sl. Dr. Eng. Corina Naforniţă
Summary
Minimum variance criterion
Existence of the minimum variance unbiased estimator
Finding the minimum variance unbiased estimator
Extension to a vector parameter
Conclusions
References
3
In searching for optimal estimators we need to
adopt some optimality criterion. A natural one is the
mean square error (MSE):
(*)
This measures the average mean squared deviation of
the estimator from the true value.
Minimum variance criterion
4
Adoption of this natural criterion leads to
unrealizable estimators, ones that cannot be written
solely as a function of the data.
Dem:
That shows that the MSE is composed of errors due
to the variance of the estimator as well as the bias.
5
Generally, any criterion which depends on the
bias will lead to an unrealizable estimator.
From a practical viewpoint the minimum MSE estimator
needs to be abandoned.
An alternative approach is to constrain the bias to be zero
and find the estimator which minimizes the variance.
Such an estimator is called the minimum variance
unbiased (MVU) estimator.
Note: The MSE of an unbiased estimator is just the
variance.
6
Existence of the Minimum Variance
Unbiased Estimator
Minimum variance unbiased (MVU) estimator : is an
unbiased estimator that has lower variance than any
other unbiased estimator for all possible values of the
parameter.
The question arises as to whether a MVU estimator
exists, i.e., an unbiased estimator with minimum
variance for all θ.
Two possible situations are described in the next figure.
7
If there are three unbiased estimators that exist and
whose variances are shown in a), θ3 is the MVU
estimator.
In the other figure, b), there is no MVU estimator
since for θ<θ0 , θ2 is better, while for θ>θ0 , θ3 is
better.
Fig 1.1 Possible dependence of estimator variance with θ
8
Finding the Minimum Variance
Unbiased Estimator
Despite the fact that the MVUE doesn't always exist, in
many cases of interest it does exist, and we need
methods for finding it. Unfortunately, there is no 'turn
the crank' algorithm for finding MVUE's.
There are, instead, a variety of techniques that can
sometimes be applied to find the MVUE.
9
These methods include:
1. Compute the Cramer-Rao Lower Bound, and check the
condition for equality.
2. Apply the Rao-Blackwell Theorem.
3. Restrict the class of estimators to be not only unbiased
but also linear. Then find the minimum variance
estimator within this restricted class.
1. and 2. may produce the MVU estimator, while 3. will yield it only
if the MVU estimator is linear in the data.
10
If θ = [θ1 θ2 …θp]T is a vector of unknown parameters,
then we say that an estimator
is unbiased if:
(**)
for i= 1,2,…p.
Extension to a Vector Parameter
11
By defining we can equivalently
define an unbiased estimator to have the property
for every θ conteined within the space defined in (**).
A MVU estimator has the additional property that
is minimum among all unbiased estimators.
12
Conclusions
MSE of an estimator is one of many ways to quantify
the difference between an estimator and the true
value of the quantity being estimated. MSE is an
expectation.
MVUE is an unbiased estimator that has lower
variance than any other unbiased estimator for all
possible values of the parameter.
13
An efficient estimator doesn’t need to exist, but if it
does, it’s the MVUE.
Since the mean square error (MSE) of an
estimator is
MSE(θ) = var(θ) + bias(θ)2
the MVUE minimizes MSE among unbiased
estimators .
In some cases biased estimators have lower MSE
because they have a smaller variance than does any
unbiased estimator.
14
References
Fundamentals of Statistical Signal Processing :
Estimation Theory, Steven M.Key, University of Rhode
Island.
Prediction and Improved Estimation in Linear Models, J.
Wiley, New York, 1977.
Unbiased estimators and Their Applications,
V.G Voinov, M.S. Nikulin, Kluwer Academic Publishers.
15
Thank you for your attention !