maximum likelihood estimation of parameters of

8/6/2019 Maximum Likelihood Estimation of Parameters Of

1/21


2/21

INTRODUCTION

In most physical situations additional

filtering is introduced by the medium or

the system through which signals are

transmitted.

A digital signal transmitted over the

wires and cables of the telephone plant

is smeared out in time by the distributedcapacitance of lines.


3/21

Conceptually we denote this by saying

that some input signal x(t) is convertedto another signal y(t) after passing

through the physical system.

This is indicated by the below figure.


4/21

If we are attempting to the input x(t) , we

must know the characteristics of the

medium.

For simple systems or media , the

transfer function or impulse response

can be determined quite readily in astraight forward way.

But in much more complicated situations

i.e an entire telephone system over

which signals must be conveyed or

water through which underwater signals

must travel.


5/21

These physical considerations are not

as readily applied in determining the

characteristics of the medium.

In addition , the medium or signal may

be changing its characteristics with time.

Therefore the method we discuss here,as an application of maximum-likelihood

estimation , is to assume a model of the

system under study, with certain

parameters unknown and use

measurements to determine these

parameters.


6/21

Here we take representation of a linear

system of a non recursive digital filter. Therefore digital signal processing are

therefore interested in discrete-timemodels.

In which signal samples are measuredonly at prescribed instants of time ratherthan all times.

Our objective is then to estimate thefilter(system) coefficients.


7/21

Specifically, if the input-signal samples

are Xj ,the samples at the output of thesystem under consideration are given by

Yj=(n=0 to m) h(n).Xj-n

We thus are interested in determining

the m+ 1 coefficients hn , n=0..m that

characterize the system.

Roughly speaking the number of

coefficients needed m+1 in this case,isdetermined by the expected spread or

dispersion introduced by the system.


8/21

As an example, consider the solitary

pulse Xo introduced at time t=0 in fig

Fig

The output is shown spread out over

seven time slots. One would then expect

the model to need at least sevencoefficients for appropriate

characterization.


9/21

In many physical situations the systemimpulse response builds up to a peak

value some time units after the impulse

is applied and then decays back to zero.

For simplicity we shall assume that the

spread or dispersive effect of the

medium is equally distributed about the

peak value


10/21

The output samples can then equallywell be written with time now measured

with reference to the output samples

and 2N+1=m+1 the number of

coefficients needed.

Yj=(n= -N to N)hn*Xj-n.

Fig below compares output with input

time for that example.


11/21

Comparision of input with output

and non recursive filter


12/21

One problem with measuring the desired

coefficients directly is that the Yj samplesnormally appear corrupted with noise.

This noise may be introduced during signaltransmission or it may representinaccuracies in the measurements or the

model representation itself. The actualreceived information is thus given by

Yj=(n= -N to N)hn*Xj-n+nj

with the desired coefficients now to bedetermined from noisy measurements.


13/21

How does one now estimate the h(n)s fromthe measured samples Yj?

One common method is that of sending aknown signal sequence Xj , measuring theerror Yj-Xj introduced by the medium and

the additive noise, and using the sequenceof errors to find the h(n)s.

For example taking pseudorandom pulsesequence,this is a relatively long sequenceof binary pulses , + or 1.One per time slot

or sampling interval that can be made toapproximate truly random binary symbols.


14/21

The output signal sequence is then

Yj=(n=-N to N) h(n).Aj-n + nj

If there were no dispersion but simplyinnate time delay N units along the mediumoutput would be hoAj+nj.

The rest of the terms in Yj representdistortion due to the medium.

Since we assume Aj known ,we canmeasure the error j=Yj-Aj and use it to

estimate the medium coefficients.


15/21

Therefore the error for N coefficients is

given by j=Yj-Aj=h(-N)*Aj+N + .+(h0-

1)+..+h(N)*Aj-N+nj

j=(n=-N to N) h(n)*Aj-n+nj

With hn indicating that ho=ho-1.Assume that we now measure K such

terms in succession , say e1,e2,.ek anduse these measured errors to estimate the

hn coefficients.


16/21

Using Maximum-likelihood estimation we find h(n)s.

Specifically with the noise samples assumedgaussian and independent , the ensemble of Ksamples e1,e2,.ek , denoted by vector e andconditioned on the coefficients h-N..

h0-1.hN, to be estimated , is itself jointlygaussian.

The density function f( /h-N) being given by theproduct of the individual density functions.

Function f(/h-N.)=exp[-(j=1 to k).(j-(n)hn*Aj-n)2/2^2]/(2pi^2).


17/21

The maximum-likelihood estimate is now

found by setting the derivative of thelogarithm of f() equal to zero.

d/dx(ln f(/h-N.))=0=(j=1 to N)(j-hn*Aj-n)Aj-i i=-N..N

The hat notation is again used to denotethe maximum-likelihood estimate.

Rewriting this set of equations, we have

(j=1 to k)Aj-i*j=(j=1 to k)(n=-N toN)hn*Aj-n*Aj-I i=-NN


18/21

We thus have to solve this set of 2N+1

equations simultaneously to find the

estimates hn.

They look rather formidable but can be

put in less forbidding form by definingthe coefficients

gi=(j=1 to k)Aj-j

Rin=(j= 1 to k)Aj-n*Aj-i.


19/21

The set of equations is then written much

more simply as gi=(n =-N to N)hn*Rin i=-N..N

Taking the above equation in vector formwe get

g=Rh^ Therefore h^=R-1g

The above vector equations can be easilymanipulated but can be costly to solve ifthe order of the matrices(here(2N+1)*(2N+1) is large.


20/21

Iterative techniques have beendeveloped for solving such equations ona computer.

Here we simply point out another

estimate for the hns developed for themaximum-likelihood approach justconsidered,that obviates the need forsimultaneous solution of 2N+1

equations.i.e going to suboptimumestimation procedures.


21/21

maximum likelihood estimation of parameters of

Documents