adaptive iir filtering - pudn.comread.pudn.com/downloads74/ebook/269736/adaptive iir...
TRANSCRIPT
5th IEE Adaptive Signal Processing Professional Development Course
March 2006, Birmingham, UK
Bob StewartDept. of Electronic and Electrical Engineering
University of StrathclydeGlasgow G1 1XW
Scotland, UK
Adaptive IIR Filtering
Introduction to Adaptive IIR Filters .1
• In this section we will introduce adaptive IIR filters.
• Intuitive comment is made as to why adaptive IIR filters are not assimple and straightforward as FIR filters.
• The adaptive IIR LMS algorithms will be defined.
• Adaptive RLS algorithms will be presented.
• Applications ranging from speech coding to active noise control will bepresented.
• Classic decision feedback architectures (DFE) will be presented andshown to be forms of non-linear adaptive IIR filter.
• SystemView simulations of adaptive IIR algorithms will bedemonstrated.
Notes:For linear adaptive filters, two types can be identified: FIR and IIR.
Within these two classes, we can distinguish between RLS and LMS based algorithms.
Adaptive Algorithms
FIR IIR(non-recursive) (recursive)
Fein
tuch
’s LM
S
Equation ErrorOutput Error
Sta
ndar
dN
orm
alis
edFi
ltere
d-X
Filte
red-
Erro
rS
ignE
rror/D
ata
RPE
New
ton
Dec
isio
nFe
edba
ck
LMS
New
ton
Nor
mal
ised
Leas
tMea
n4t
h
QR
-IIR
RLS
PLR
New
ton
Blo
ckN
on-L
inea
r
Sta
ndar
dQ
RK
alm
an
LMSRLS
QR
-IIR
RLS
PLR
LMSRLS LMSRLS
Adaptive Filtering 2
• For a number of applications, adaptive IIR filters may have acompuatational and modelling advantage. Consider the inverse sys id:
• Using adaptive FIR filter, the inverse has many weights:
• Using adaptive IIR filter, the inverse may have only two weights:
s(k) 1 0.5
y(k)Σ
+
-e(k)Adaptive
Filter G(z)
d(k)
x(k)
Adaptive AlgorithmH z( ) 1 0.5z 1–+=
G z( ) H 1– z( ) 1 0.5z 1–– 0.25z 2– 0.125z 3–– 0.0625z 4– …+ + += =
G z( ) H 1– z( ) 11 0.5z 1–+-------------------------- = =
Notes:When using an FIR filter to find theinverse of the above system, around20-30 weights would be required toinvert the filter.
This is clearly a minimum phase systemas the zero of the transfer function(when ) is inside the unitcircle at . Hence the inverse ofthe system could have a single poleinside the unit circle if inverse modellingwas performed using an IIR filter.
Using an adaptive FIR filter, the inverse can be calcuated as the z-transform of the unknown system impulseresponse. About 20-30 weights should be sufficient to invert this system.
However, if the adaptive filter was in fact an IIR filter, then only two weights would be required to produced theinverse!
H z( ) 1 0.5z 1–+=
sample, k
h k( )
0 1 2 3 4
1
0.5
Real
Imag
0.5 1-0.5-1
0.5
1
-0.5
1
z-plane
1 0.5z 1–+ 0=z 0.5–=
1 0.5z 1–+ 11 0.5z 1–+
0.5z 1––
1 0.5z 1– 0.25z 2– 0.125z 3– …+–+–
0.5z 1–– 0.25z 2––0.25z 2–
0.25z 2– 0.125z 3–+
0.125z 3––
G z( ) H 1– z( ) 11 0.5z 1–+--------------------------- 1 0.5z 1– 0.25z 2– 0.125z 3– …+–+–= = =
sample, k
g k( )
0
1
0.5
-0.51 2
34
PolynomialDivision
....and so on
Adaptive IIR Filter - Output Error 3
• General Closed Loop Adaptive IIR Signal Processor:
“The aim is to adapt the digital filter such that the input signal x(k) isfiltered to produce y(k) which when subtracted from desired signal d(k),will minimise the power of the error signal e(k).” .
input y(k)
Σ
+
-
desired
e(k)
d(k)
x(k)signal
signal
error signal
Output signal
Σ
Adaptive Algorithm
+
+
Adaptive FIR Filter
Adaptive FIR Filter
Adaptive IIR Filter
z-1
A(z)
B(z)
Notes:One key aspect of the adaptive FIR filter was that the minimum mean squared error (MMSE) solution is unique,i.e. the performance surface had exactly one minimum which was the best or optimal solution in an MMSEsense.
For adaptive IIR filters the performance surface no longer has just one minimum, and there may exist more thanone solution (i.e. overdetermined in some sense) giving multiple “optimal” solutions).
Note that in the above figure the block denoted adaptive FIR filter in both feedforward and feedback sections isessentially an FIR filter block with adaptive weights:
As the filter is connected in a feedback loop, this forms the recursive filter weights.
Hence in the feedback section we explicitly show the delay that is required in the feedback loop (otherwise thereis a delayless loop!). The signal flow graph is presented in this was as later we will be considering another typeof adaptive IIR, namely the equation error IIR. Presenting in this style makes things somewhat easier.
We specifically refer to this architecture as the “output” error formulation. Later we will also view the “equationerror” IIR (which is in fact not an IIR at all!)
Adaptive FIR Filterx(k) y(k)
x(k)
y(k)
z-1 z-1 z-1
B z( )
Adaptive IIR Filter Signal Flow Graph 4
• Example with 4 feedforward, and 3 feedback weights:
a0 a1 a2
b3
a3
b1b2
y(k)Σ
+
-e(k)
d(k)
Output signal
x(k)
Adaptive Algorithm
Adaptive IIR Filterz-1 z-1 z-1
z-1 z-1 z-1
Y z( )X z( )------------ A z( )
B z( )------------
a0 k( ) a1 k( )z 1– a2 k( )z 2– a3 k( )z 3–+ + +1 b1 k( )z 1– b2 k( )z 2– b3 k( )z 3–+ + +
------------------------------------------------------------------------------------------------------= =
Notes:The general input output equation for an IIR filter with feedforward weights and feedback weights is:
In vector notation
,
where the weight and data vectors are respectively:
and
N M 1–
y k( ) anx k n–( )
n 0=
N 1–
∑ bmy k m–( )
m 1=
M 1–
∑+=
y k( ) aTxk bTyk 1–+ wTuk= =
wT aT bT,[ ] a0 a1 a2 … aN 1– b1 b2 … bM 1–, , , , , , , ,[ ]= =
ukT xk
T yk 1–T,[ ]=
x k( ) x k 1–( ) x k 2–( ) … x k N– 1+( ) y k 1–( ) y k 2–( )… y k M– 1+( ), , , , , , ,[ ]=
System Identification of Recursive System 5
• The optimal solution would appear to be
• Of course there are many “optimal” solutions obtainable simply bymultiplying numerator and denominator by pole zero pairs:
G z( ) A z( )B z( )------------ N z( )
D z( )------------= =
Σ
+
-Σ
Adaptive Algorithm
+
+
Adaptive IIR Filter
Unknown System
N z( )D z( )------------
WhiteNoise A z( )
B z( )
G z( )
z-1
G z( ) A z( )B z( )------------ N z( ) 1 qz 1–+( )
D z( ) 1 qz 1–+( )---------------------------------------= =
Notes:The point being made here is simple. Adaptive IIR filters are NOT as straightforward as adaptive FIR filters! Wesimply demonstrate that whereas the optimal (best) adaptive FIR solution is unique, this is not the case foradaptive IIR as demonstrated by showing that there are multiple solutions.
Similarly in the notes of Slide 2 we noted two different solutions, one based on feedforward weights only, andone based on feedback weights only.
For example, consider again the inverse system identification in Slide 2 where the inverse of was being calculated. If we used an IIR with say 20 feedforward weights and 20 feedback weights, then an exactsolution that would give an error of zero would be:
, i.e.
However another solution would be the result of the polynomial division in the notes below Slide 2:
, i.e.
with no significant coefficients on the denominator, i.e. all feedback weights are zero.
There are of course many other solutions formed if there are pairs of cancelling poles and zeroes, e.g.
is another “valid” solution.
In the slides that follow we will again note that there is not a unique solution, and that there are also otherproblems whereby
H z( ) 1 0.5z 1–+=
G z( ) 1 0z 1– 0z 2– … 0z 19–+ + + +1 0.5z 1– 0z 2– … 0z 19–+ + + +--------------------------------------------------------------------------------- 1
1 0.5z 1–+---------------------------= = G z( )H z( ) 1=
G z( ) 1 0.5z 1–– 0.25z 2– … 0.191 -5×10– z 19–+ +1 0z 1– 0z 2– … 0z 19–+ + + +
------------------------------------------------------------------------------------------------------------------= 2 19– 0.191 -5×10 ...= G z( )H z( ) 1≈
G z( ) 11 0.5z 1–+---------------------------
1 0.5z 1–+( )1 0.5z 1–+( )
-------------------------------- 1 0.5z 1–+( )1 z 1– 0.25z 2–+ +( )
---------------------------------------------------= =
Minimising the MSE 6
• Assuming signals and are ergodic and wide sensestationary (WSS), then from Slide 3:
and
and hence
x k( ) d k( )
MSE E e2 k( )[ ] 1M----- e2 k( )
k 0=
M 1–
∑= = , for large M
e2 k( ) d k( ) y k( )–( )2 d k( ) wTuk–( )2= =
d2 k( ) wT u k( )uT k( )[ ]w 2d k( )wTu k( )–+=
E e2 k( )[ ] E d2 k( )[ ] wTE u k( )u k( )T[ ]w 2wTE d k( )u k( )[ ]–+= E d2 k( )[ ] wTRuuw 2wTpdu–+=
Notes:Minimising the mean square error requires that the assumptions of and being ergodic and wide sensestationary. This is the same assumption as made for the adaptive FIR LMS filters.
We will note that unlike for the FIR filter, no one step linear Wiener-Hopf type solution exists. In fact the solutionfor this IIR filter, or ARMA (autoregressive moving average) filter is non-linear and virtually impossible to solveby analysis for other than trivial systems.
The presentation of the IIR filter is done using typical adaptive filter signal flow graphs and signal labelling.There are other forms of general recursive system identification known as Box-Jenkins and ARMA(autoregressive moving average). In many senses both Box-Jenkins and ARMA modelling are part of the moregeneral subset of adaptive IIR filtering, however there application is restricted to system identification and oftenmakes more detailed reference to any measurement noise that may be present on the signal. For adaptiveIIR filtering the problem may not be one of simply system identification. It could be noise cancellation, echocancellation, inverse system identification and so on. Hence we refrain from specifically using the restrictivenames of Box-Jenkins and ARMA in favour of the general name, “adaptive IIR filtering”.
Note that if the filter simplifies to an FIR filter (i.e. it has no poles), then the mean squared error (MSE) reducesto:
where is the autocorrelation matrix, and the cross correlation vector. Thisequation is quadratic in and can be solved relatively easily to find the minimium mean squared error.
In the equation on the above slide note that is in fact a function of past values of since all past outputswere created using .
x k( ) d k( )
d k( )
E e2 k( )[ ] E d2 k( )[ ] wTE x k( )xT k( )[ ]w 2wTE d k( )x k( )[ ]–+=
E d2 k( )[ ] wTRw 2wTp–+=
R E xkxkT[ ]= p E xkd k( )[ ]=
w
Ruu ww
The IIR “Wiener-Hopf”? 7
• Note that the square matrix of dimension is given by:
and the vector is:
• Recall for the adaptive FIR filter that the performance surface wasquadratic on and a “hyperparaboloid” with one minimum. Howeverfor this IIR performance surface note that and are functions of
.
Ruu N M 1–+
RuuE x k( )xT k( )[ ] E x k( )yT k 1–( )[ ]
E y k 1–( )xT k( )[ ] E y k 1–( )yT k 1–( )[ ]
Rxx Rxy
RTxy Ryy
= =
N M 1–+ pdu
pduE d k( )x k( )[ ]
E d k( )y k 1–( )[ ]
pdx
pdy
= =
wRyy Rxy
w
Notes:Note that is simply the correlation matrix normally used in the Wiener Hopf solution.
Recall that the performance surface is a plot of against filter weights.
For an FIR filter (no feedback), the classic Wiener-Hopf solution is realised by noting that the equation:
is quadratic in the vector w. Hence there is only one minimum value denoted MMSE (minimum mean squareerror) and which occurs at, wopt.
The MMSE is found from setting the (partial derivative) gradient vector, , to zero:
The solution to this equation is linear.
If the FIR filter has two weights the performance surface isa paraboloid in 3 dimensions::
If the filter has more than three weights then we cannot draw the performacesurface in three dimensions, however, mathematically there is only oneminimum point which occurs when the gradient vector (with respect to w) iszero. A quadratic performance surface with more than three dimensions isoften called a hyperparaboloid.
Rxx
E e2 k( )[ ]
E e2 k( )[ ] E d2 k( )[ ] wTRxxw 2wTp–+=
∇
∇w∂∂ E e2 k( )[ ] 2Rxxw 2p– 0= = =
wopt⇒ Rxx1– p=
MMSE
w0
w1
w0(opt)
w1(opt)
MSE, ζwT w0 w1,[ ]=
Multimodal Performance Surface 8
• Therefore unlike for an adaptive FIR filter, for the IIR filter theperformance surface equation:
is NOT quadratic and a straighforward gradient based solution doesnot exist.
• The performance surface is now multimodal (i.e. localminima) and in general the gradient function:
is non-linear and non-trivial to solve since is a function of (c.f.for adaptive FIR filter).
E e2 k( )[ ] E d2 k( )[ ] wTRuuw 2wTpdu–+=
E e2 k( )[ ]
∇ k( )w∂∂ E e2 k( )[ ]=
Ruu w
Notes:For an adaptive FIR filter, the minimum mean square error is found simply by locating the “bottom” of thehyperparaboloid. Mathematically this is done by finding the zero gradient vector using vector calculustechniques.
The adaptive IIR filter multimodal performance surface, basically means that there are now local minima aswell as the desired single global minimum:
Therefore although we can still look for a zero gradient point on the performance surface of the adaptive IIRfilter, it cannot be guaranteed that the point found is the global minimum, and it may in fact be a local minimum.Hence the term “stuck in local minima” is often used.
MMSE
MSE
wwopt
Local Minima
GlobalMinima
Gradient IIR Filtering Techniques 9
• Following the strategy for deriving the FIR LMS algorithm, an adaptivegradient based IIR LMS algorithm can be derived.
• Note however that the performance surface has local minima, andtherefore the solution achieved via this gradient descent technique maybe a local minimum (sub-optimal) rather than the global minimum.
• However the general IIR LMS solution has proved useful, and the localminima problem is less “severe” than first anticipated.
• Therefore the adaptive IIR LMS is NOT guaranteed to converge to theglobal minimum
Notes:Recall that the LMS is derived by estimating the mean squared error performance surface gradient,
by the instantaneous squared error surface gradient:
This simplification makes the problem mathematically tractable.
∇kE e2 k( )[ ]∂
wk∂--------------------------=
∇ˆ
ke2 k( )∂
wk∂-----------------=
Full Gradient IIR LMS 10
• From the FIR LMS derivation the weight vector update is:
and
• Therefore:
where & are used for notational convenience.
wk 1+ wk µ ∇ˆ
k–( )+=
∇ˆ
ke2 k( )∂w k( )∂
----------------- 2– e k( ) y k( )∂w k( )∂
---------------= =
y k( )∂wk∂
-------------- y k( )∂a0 k( )∂
----------------- y k( )∂a1 k( )∂
----------------- … y k( )∂aN 1– k( )∂
------------------------- y k( )∂b1 k( )∂
----------------- …, , , , , y k( )∂bM 1– k( )∂
-------------------------T
=
α0 k( ) α1 k( ) … αN 1– k( ) β1 k( ) … βM 1– k( ), , , , , ,[ ]=
αn k( ) βm k( )
Notes:
Previously for the FIR LMS the instantaneous gradient vector was simply . and
However for the IIR LMS, the gradient vector will more complex due to the fact that past outputs must bedifferentiated , all of which are functions of the weight vector.
The step size is given by the parameter .
2– µe k( )x k( ) y k( )∂w k( )∂
--------------- x k( )=
µ
Full Gradient IIR LMS....cont’d11
• It is (reasonably) straightforward to show that:
• The resulting algorithm is termed the full gradient IIR LMS, and isperhaps best represented via a signal flow graph......
αn k( ) y k( )∂an∂ k( )
----------------- x k n–( ) bj k( ) y k j–( )∂an∂ k j–( )
------------------------j 1=
M 1–
∑+≈=
βm k( ) y k( )∂bm∂ k( )
------------------ y k m–( ) bj k( ) y k j–( )∂bm∂ k j–( )
-------------------------j 1=
M 1–
∑+≈=
Notes:In derving the above equations we have implicitly made the assumption
to simplify the mathematics and achieve a definable adaptive filter.
We can write both and values in a recursive form:
leading to the signal flow graphs:
is z-domain representation of the current values of the feedbackweights.
Note that changes every update.
y k j–( )∂an∂ k j–( )
------------------------- y k j–( )∂an∂ k( )
---------------------=
α β
αn k( ) x k n–( ) bj k( )αn k j–( )
j 1=
M 1–
∑+=
βm k( ) y k m–( ) bj k( )βm k j–( )
j 1=
M 1–
∑+=
Σ+
+
Bk z( )
αn k( )x k n–( )Σ
++
Bk z( )
βm k( )y k m–( )
Bk z( ) b1z 1– b2z 2– … bM 1– z M– 1++ + +=
Bk z( )
Full Gradient IIR LMS SFG 12
Σ+
+y(k)
Σ+
-
d(k)
e(k)
LMSA LMSB
e(k)α0 k( ) αN 1– k( )
. . .
. . . β1 k( ) βM k( )
. . .
. . .
z-1
z-1
αN 1– k( )
z-1
z-1
β1 k( )
βM 1– k( )
β2 k( )
y(k-1)
y(k-2)
y(k-M+1)
y(k-M)
x(k)
x(k-1)
x(k-N+2)
x(k-N+1)
x(k)
Σ+
+
11 Bk z( )–------------------------
11 Bk z( )–------------------------
11 Bk z( )–------------------------
α0 k( )
α1 k( )
11 Bk z( )–------------------------
11 Bk z( )–------------------------
11 Bk z( )–------------------------
Bk z( )
Ak z( )
z-1
z-1
Notes:SFG - Signal Flow Graph
On each time iteration, the current feedback weights are used with the current and past values of inputsignal and with past outputs to produce the and in order to update the LMS equation.
It should be clear that:
Bk z( )x k( ) y k( ) αn k( ) βm k( )
Σ+
+
Bk z( )1
1 Bk z( )–------------------------
Simplified Gradient IIR LMS 13
• If it is assumed that the IIR filter weights are “slowly adapting” thennoting that is calculated using data sample and ....
....and similarly that is calculated using data sample and
the simplified gradient IIR LMS can be realised such that we assume:
• Hence can be “reasonably” approximately calculated using datasample and ......
.....and similarly can be calculated using data sample and
αn k( ) x k n–( ) Bk z( )
βm k( ) y k m–( )Bk z( )
Bk z( ) Bk 1– z( ) … Bk M– 1+ z( ) … Bk N– 1+ z( )≈ ≈ ≈ ≈ ≈
αn k( )x k n–( ) Bk n– z( )
βm k( ) y k m–( )Bk m– z( )
Notes:The slowly varying weights assumption can be somewhat forced by the algorithm designer by choosing a verysmall step size for the algorithm!
Note that although the weight update here uses simple LMS, more involved algorithms such as an RPE(recursive predictor error) could be used (see later notes). The RPE uses an estimate of the correlation matrixto accelerate the convergence. See also Shynk’s paper for more details.
Another form of IIR algorithm exists called the SHARF (simplified hyperstable recursive filter) which in a sensesmooths the error signal with an adaptive filter in order to try and stabilise the IIR LMS. See Widrow’s text fordetails.
See also Shynk’s paper for information on the filtered-error IIR algorithm.
Simplified IIR LMS SFG 14
Σ+
+y(k)
Σ+
-
d(k)
e(k)
LMSA LMSB
e(k)α0 k( ) αN 1– k( )
. . .
. . . β1 k( ) βM k( )
. . .
. . .
α0 k( )
αN 1– k( )
β2 k( )
z-1
y(k-1)x(k)
x(k)
z-1
z-1
z-1
α1 k( )
β0 k( )
βM 1– k( )
z-1
z-1
z-1
β1 k( )
11 Bk z( )–------------------------
Bk z( )
Ak z( )
11 Bk z( )–------------------------
z-1
Notes:The simplified gradient algorithm reduces the level of computation considerably.
For the full gradient technique the computation required at each new sample is of the order multiply-accumulates (MACs) per iteration whereas for the simplified gradient the complexity reduces to around
MACs per iteration.
NM M2+
2 N M+( ) 2M+
Feintuch’s IIR LMS 15
• Feintuch made a further simplification, where in Slide 11, it wasassumed all derivatives of past outputs with respect to current weightsare zero:
• The justification for Feintuch is simplification, and the “proof” is analgorithm that does in fact converge in many situations.
αn k( ) y k( )∂an∂ k( )
----------------- x k n–( ) bj k( ) y k j–( )∂an∂ k j–( )
------------------------
j 1=
M 1–
∑+≈=
x k n–( )=
0
βm k( ) y k( )∂bm∂ k( )
------------------ y k m–( ) bj k( ) y k j–( )∂an∂ k j–( )
------------------------
j 1=
M 1–
∑+≈=
y k m–( )=
0
Notes:When it first appeared in 1976 Feintuch’s algorithm caused some debate in the signal processing literature.Within the control literature there is a similar technique refered to as Landau’s method. A number of commentswere to the effect that this algorithm could not and should not work, whereas some others were very positive bynoting that not only did it converge in certain circumstances, but it was also more inherently stable than the fullgradient and simplified gradient due to a tendency to adapt poles away from the unit circle towards the origin ofthe z-plane.
This stabilising property is a result of the PLR (pseudo linear regression) assumption (i.e. all past outputderivatives with respect to current weights are zero) and is well known in both signal processing and controlliterature. Therefore not only is Feintuch simpler than the simplified gradient algorithm, it is also more likely tobe stable, with the adapted poles less likely to adapt out of the unit circle.
Feintuch’s algorithm is still widely used and anaylsed within the signal processing literature.
The computational complexity is around MACs per iteration.2 N M+( )
Feintuch’s Algorithms SFG 16
Σ+
+y(k)
Σ+
-
d(k)
e(k)
LMSA LMSB
e(k)α0 k( ) αN 1– k( )
. . .
. . . β1 k( ) βM k( )
. . .
. . .
α0 k( )
αN 1– k( )
β2 k( )
z-1
y(k-1)x(k)
z-1
z-1
z-1
α1 k( )
β0 k( )
βM 1– k( )
z-1
z-1
z-1
β1 k( )
Bk z( )
Ak z( )z-1
x(k)
Notes:Note that the computational complexity of this algorithm is now only .
In typical applications Feintuch will outperform the FIR LMS if the system being identified has high levels ofrecursion within it. However just using Feintuch to identify a non-recursive system, very little advantage will beseen.
For a typical system, up to 10 or even 20 poles might be used with Feintuch’s algorihm.
A filtered-error algorithm may also be used whereby the error signal is smoothed prior to input into theadaptive algorithm. The possible advantage here is that the IIR algorithm can be assisted in convergening underSPR (strictly positive real) conditions. For certain SPR conditions a PLR algorithm will converge to an arbitrarypoint on the performance surface. Using a filtered-error can often alleviate this problem. A simple filtered erroris given by:
the coefficients should represent a simple low pass filtering function.
O N M+( )
ef k( ) cje k j–( )
j 0=
P
∑=
cj
Feintuch’s IIR LMS 17
• Feintuch’s IIR LMS can be drawn in a more “familiar” form to moreclosely resemble the well known FIR LMS SFG:
• Note that if the number of recursive weights is zero, then the SFGsimplifies to the FIR LMS.The algorithm requires MACs per sample.
y(k)Σ
+
-e(k)
d(k)
x(k)Σ+
+
Adaptive FIR Filter
Adaptive FIR Filter
Adaptive IIR Filter
z-1
a k 1+( ) a k( ) 2µe k( )x k( )+= b k 1+( ) b k( ) 2µe k( )y k 1–( )+=
2 N M+( )
Notes:In vector notation
,
where the weight and data vectors are respectively:
and
and
y k( ) aTx k( ) bTy k( )+=
bT b1 b2 … bM 1–, , ,[ ]= aT a0 a1 a2 … aN 1–, , , ,[ ]=
x k( ) x k( ) x k 1–( ) … x k N– 1+( ), , ,[ ]= y k 1–( ) y k 1–( ) y k 2–( ) … y k M– 1+( ), , ,[ ]=
Performance of IIR LMS Algorithms 18
• Feintuch is the simplest to implement of the three algorithms in termsof MACs per iteration.
• The full gradient IIR LMS is rarely implemented due to its high level ofcomputation required. The performance advantages for this high levelof computation are also noted to be minimal.
• The simplified gradient IIR LMS is also rarely implemented with,again, minimal performance improvement seen over thecomputationally simpler Feintuch’s IIR LMS.
• Situations where the full gradient and simplified gradient IIR LMS’shave improved performance over Feintuch’ IIR LMS are usually inenvironments where the system being modelled has very high levels offeedback. However these scenarios are rare!
Notes:Feintiuch’s LMS is therefore simplest to implement, and likely to be the most stable, given the inherent stabilsingnature of PLR type systems.
Note that some form of “stability monitoring” may be performed when implementing an IIR LMS algorithm. Thisbasically means that at every iteration the poles of the filter are calculated and if any are outside the unit circlethe adaption is stopped and restarted. In general this form of stability monitoring is not convenient for real timeDSP implementation.
Alternatively some recent publications have suggested implementing the adaptive filter as a cascade of 2ndorder filter sections whereby the stability of each section is easy to calculate. If any sections become unstablethen this particular section is “zeroed” and the adaptiion restarted.
Stability and Convergence 19
• Feintuch can be shown to be more stable than the full gradient andsimplified gradient IIR LMS’s due to an inherent ability not to adaptpoles outside of the unit circle.
• Step-size selection for IIR LMS algorithms is less well defined than forFIR LMS.
One practical rule of thumb is to choose the feedforward step size asfor an FIR filter, and set the feedback step size to one tenth of thisvalue!
• Feintuch Algorithm System Identification
• Feintuch Algorithm Inverse System Identification
Notes:Observe the OHP demonstrations and note here the main comments from the simulations.
Newton/Recursive Predictor Error 20
• This algorithm uses an estimate of the correlation matrix, toaccelerate convergence.
• Computational overhead is high in updating the correlation matrix oneach iteration.
y(k)Σ
+
-e(k)
d(k)
x(k)Σ+
+
Adaptive FIR Filter
Adaptive FIR Filter
Adaptive IIR Filter
z-1
A(z)
B(z)
w k( ) w k 1–( ) µR 1– k( )z k( )e k( )+=
R k( )
Notes:The output of the filter is and z(k) is defined as
where x(k) and are the input and output data vectors:
w(k) is where a(k) and b(k) are the feedforward and feedback filter weights
The adaptive filter weights are updated as:
where is recursively updated as:
The a priori (before weights updated) error is: . is a forgetting factor typically between 0.9and 1Initial values: , and (hence ).
The basis of the RPE algorithms is similar to the basis of the Gauss-Newton familty of FIR LMS algorithmswhereby an estimate of the data correlation matrix is used to accelerate convergence
y k( ) zT k( )w k 1–( )= z k( ) xT k( ) yT k 1–( )T
=
y k 1–( )
x k( ) x k( ) x k 1–( ) … x k N– 1+( )T
= y k 1–( ) y k 1–( ) y k 2–( ) … y k M– 1+( )T
=
w k( ) aT k( ) bT k( )T
=
a k( ) a0 k( ) a1 k( ) … aN 1– k( )T
= b k( ) b1 k( ) b2 k( ) … bM 1– k( )T
=
w k( ) w k 1–( ) µR 1– k( )z k( )e k k 1–( )+=
R 1– k( )
R 1– k( ) 1λ--- R 1– k 1–( ) R 1– k 1–( )z k( )zT k( )R 1– k 1–( )
λ zT k( )R 1– k 1–( )z k( )+--------------------------------------------------------------------------------–=
e k( ) d k( ) y k( )–= λ
z 0( ) 0= w 0( ) 0= R 0( ) δI= R 1– 0( ) 1δ---I=
Adaptive QR-IIR Filter 21
• A QR-IIR filter can be implemented by essentially setting up a twochannel QR filter, and feeding back the output to be the input to one ofthe channels:
Σ
+-
d(k)
y(k)
QRx(k) e(k|k-1)
e(k|k)
÷γ(k)
(⋅)½-
x1(k)
x2(k)
input
desired
signal
signal error signal
Output signal
Notes:A two channel QR-RLS algorithm. The output of the filter is:
where
x1(k) and x2(k) are the first and second channel input signals respectively:
R(k) and p(k) are obtained as:
The a priori (before weights updated) and a posteriori (after weights updated) errors are respectively:
and The output γ(k) is obtained as:
γ(k) is also known as the conversion factor, as it converts the a priori estimation error e(k|k-1), obtained with theold filter weights, into the a posteriori estimation error e(k), obtained with the updated filter weights. Note that
, for all k.
The QR-IIR is a Pseudo-linear-regressive algorithm.
y k( ) zT k( ) R 1– k( )p k( )[ ]= z k( ) x1T k( ) x2
T k( )T
=
x1 k( ) x1 k( ) x1 k 1–( ) … x1 k N1– 1+( )T
= x2 k( ) x2 k( ) x2 k 1–( ) … x2 k N2– 1+( )T
=
R k( ) p k( )
OT *Q
λR k 1–( ) λp k 1–( )
zT k( ) d k( )=
e k k 1–( ) d k( ) zT k( ) R 1– k 1–( )p k 1–( )[ ]–= e k k( ) d k( ) y k( )–=
γ k( )e k k 1–( )
e k k( )--------------------------
1 2⁄–=
0 γ k( ) 1≤ ≤
QR IIR SFG - Given’s Rotations 22
• For RLS IIR filters the “Feintuch” assumption is more formally stated asthe PLR (pseudo-linear regression).
• The RLS-IIR filter, based on a QR decomposition can be representedas the classic QR triangular array signal flow graph (SFG):
Σ
d k( )
e k( )
-+
y k( )x k( )
Givens Generation
Givens Rotation
y k( ) d k( ) e k( )–=
3 feedforward and2 feedback weights.
Notes:The QR decomposition for adaptive FIR filtering was extensively presented earlier. Note the similarity of theabove SFG to that for an FIR filter. In some sense the IIR-QR can be thought of as a two channel QR where,the second channel is the output of the first fedback!
More details on the QR-IIR filter can be found in:
M.Harteneck, R.W. Stewart, I. Proudler, J. McWhirter. An effective QRD based approach to adaptive IIRfiltering. Proceedings of IEEE ICASSP, Atlanta, USA, May 1996.
M.Harteneck, R.W. Stewart. Acoustic echo cancellation using a pseudo-linear regression and QRdecomposition. Proceedings of IEEE ISCAS, Atlanta, USA, May 1996.
M.Harteneck, R.W. Stewart, I. Proudler, J. McWhirter. An algorithmically engineered fast multichannel adaptivefilter based on the QR-RLS. Institute of Mathmatics Conference, Mathematics in Signal Processing, Warwick,December 1997.
Development of the QR-IIR is jointly being undertaken by DRA Malvern and University of Strathclyde.
QR-IIR Algorithm Properties23
• To date the QR-IIR has proved to be an excellent algorithm which canoutperform IIR-LMS and RPE IIR LMS gradient based strategies.
• All “traditional” QR algorithm features are exploitable:
• direct residual extraction;
• fast algorithm implementation;
• multichannel implementation.
• Fast QR-IIR algorithms can also be dervied in the same style as for fastQR-FIR.
Notes:
Adaptive IIR Filter - Equation Error 24
• One alternative form of the adaptive IIR is the equation error, which infact is not actually a truely recursive system!
• This adaptive filter actually has no feedback weights (poles) and asimple LMS feedforward-type adaptive algorithm can be derived.
y(k)Σ
+
-e(k)
d(k)
x(k)Σ
Adaptive Algorithm
+
+
Adaptive FIR Filter
Adaptive FIR Filter
Adaptive IIR Filter
z-1
No outputfeedback
Notes:There exists a type of “pseudo” IIR filter usually called the equation error, whereby the adaptive filter is actuallytwo FIR filters and the desired signal has been used to approximate the signal . This can be seen bycomparing Slide 3 and Slide 24.
The resulting architecture is not actually recursive and consists of two FIR LMS filters. The equation errorsolution is only of use when the system is used in zero noise conditions, otherwise the solution is biased!
Note that instead of feeding back we feedback . The loose mathematical justification is that if thesystem were adapted with a truely zero error, then Hence starting of with approximated as
should allow adaption in conditions of zero noise.
The equation error adaptive IIR performance surface is hyperparaboloid and therefore has only one minimum.
Adaptive equation error systems actually form the basis of the widely used decision feedback equaliser (DFE).
d k( ) y k( )
y k( ) d k( )d k( ) y k( )= y k( )
d k( )
Equation Error LMS Algorithm 25
• Assuming signals and are ergodic and wide sensestationary (WSS), then from the LMS weight vector update is:
and
From:
• Therefore the equation error LMS update is:
where
x k( ) d k( )
w k 1+( ) w k( ) µ ∇ˆ
k–( )+= ∇ˆ
ke2 k( )∂w k( )∂
----------------- 2– e k( ) y k( )∂w k( )∂
---------------= =
y k( ) aT k( )x k( ) bT k( )d k 1–( )+ aT k( ) bT k( )x k( )
d k 1–( )= =
y k( )∂wk∂
-------------- y k( )∂a k( )∂
-------------- y k( )∂b k( )∂
--------------,T
xT k( ) dT k 1–( ),[ ] uT k( )= = =
w k 1+( ) w k( ) 2µe k( )u k( )+= wT k( ) aT k( ) bT k( )=
Notes:The equation error adaptive IIR performance surface is hyperparaboloid and therefore has only one minimum.
Equation Error Signal Flow Graph 26
• Unlike the output error adaptive IIR, the equation has only oneminimum.
(Note this is not in fact an IIR filter in the sense of having filter poles!)
y(k)Σ
+
-e(k)
d(k)
x(k)Σ+
+
Adaptive FIR Filter
Adaptive FIR Filter
Adaptive IIR Filter
z-1
a k 1+( ) a k( ) 2µe k( )x k( )+= b k 1+( ) b k( ) 2µe k( )d k 1–( )+=
Notes:Redrawing the SFG for an inverse system identification example indicates that there is not recursive filter part(i.e. poles) of the output error IIR.
.
In conditions of no observation noise the solution is unbiased.
Σ
+
-Σ+
+
Unknown System A z( )
B z( )z-1
Σ
Noise
Noise
Σ
Noise
Σ+
Σ+Unknown
System A z( )
B z( )z-1
-+
Adaptive Equalisation 27
• The feedforward/decision feedback (FF-DFE) equaliser is essentiallyan equation error adaptive IIR filter when in training mode.
Adaptive Filter
Σ+
−Mobile
ChannelPRBS
Training
PRBS Training
time
Implicit connectiontime
Σ+
z-1d k( )
y k( ) e k( )
M z( )
Adaptive Filter
+
Westwood, Los Angeles, USA Eastwood, Glasgow, UK
Notes:Note that this system when operating in training mode is essentially an equation error (EE) IIR filter. Althoughthe name IIR would suggest feedback, the EE-IIR is actually non-recursive as is fedback, rather than .
More discussion on equalisers can be found in the DSP Communications notes.
d k( ) y k( )
FF-DFE Decision Directed Mode (DDM) 28
• The feedforward/decision feedback equaliser is essentially an outputerror adaptive IIR filter when connected in decision feedback mode.
Adaptive Filter
Σ+
−Mobile
ChannelPRBS
Training
time
Σ+
d k( )
y k( )
e k( )
M z( )
Adaptive Filter
+
z-1
+1
−1
Slicer
Eastwood, Glasgow, UKWestwood, Los Angeles, USA
Notes:When the system switches to DDM, the adaptive filter then begins to resemble an output error IIR filter. Analysisis not straightforward given the non-linear slicer element in the feedback.
Adaptive LMS equalisers are widely used in data communications ranging from telephone channel modem(V32, V34, V90) to cable modems and ADSL / HDSL modems and equalisation for mobile channels. AdaptiveDFEs are widely used within these applications when the channel may have spectral zeroes, or noiseenhancement is a concern.
In the above examples we have only used 2 symbol data (0 and 1, represented as -1 and +1). In many realworld systems there will be many more symbols and hence the “slicer” represented above is somewhat morecomplicated.
For QAM based communications the data will be complex. In these applications the DFE and adaptive IIR LMS(equation and output error) will of course be complex.
Other Application Examples 29
• Adaptive IIR filters have very relevant application in any applicationwhere recursion is to be identified:
• Some forms of echo cancellation (ISDN)
• Acoustic systems
• Active noise control
• Speech compression - adaptive predictors.
• If however an adaptive IIR is being to use to attempt to model a non-recursive system using fewer weights than an adaptive FIR, then this isunlikely to produce useful results.
Notes:IIR filters should be used in applications where there is true recursion to be modelled. If there is not, then wewould recommend simply using an FIR filter with a “long enough” impulse response.
Active Noise Control 30
• Filtered-U IIR LMS based algorithms for real time noise cancellationcan be derived based on a slight variation of Feintuch’s IIR LMS:.
Errormicrophone
Reference microphone
He(f)
NOISE
Q(f)
T(f)
Loudspeaker
d(k)
+Filter
Zeroes a
y(k)x(k)
Heˆ
z( )
Σ+
Filter Poles b
f(k)
g(k)
bk 1+ bk 2– µekgk=ak 1+ ak 2– µekfk=
Heˆ
z( )
Notes:In the active noise control architecture the output of the filter passes through the acoustic environment beforeproducing the error signal by summing with the noise. Hence the adaptive algorithm must be rederived toproduce the filtered-U LMS. The filtered-U is the IIR equivalent of the filtered-X LMS for FIR filters.
Active noise control is based on the simple physical principle of destructive interference:
Applications include noise reduction inside aircraft, cars, factories, and so on.
Note that because the error signal in the above architecture is actually:
then the IIR LMS algorithm is modified slightly to realise the filtered-U LMS whereby the impulse response ofthe loudspeaker, room acoustics and microphone is necessarily taken into account. A derivation of the filtered-U can be found in the “draft” A-Z. For further details on filtered-U algorithms, please see:
L.J. Eriksson. "Development of the filtered-U algorithm for active noise control." Journal of the Acoustical Society ofAmerica, Vol. 89 (No.1), pp 27-265, 1991
R.W. Stewart, R. Duncan, and S. Weiss. "Multichannel adaptive active noise cancellation using the DSP56001." IEEEInternl Conf on Acoustics, Speech and Signal Proc., Vol I, pp. 51-54, April 1993.
ANCLoud-
speaker
NOISE
Quiet Zone:(destructive interference)
Anti-phasenoise
Periodicnoise
ek dk yk*he––
Two Channel ANC System 31
•
c11
c12
c11
c12
c21
c22
c21
c22
c11
c12
c21
c22
Secondary Loudspeakers
Error Microphones
y1(k)
y2(k) d2(k), NOISE
d1(k), NOISE
e2(k)
e1(k)
a12
LMSa12
b12
LMSb12
a11
LMSb11
b11
LMSb11
DAC
DAC
Amp
Amp
AD
CA
mp
AD
CA
mp
Figure 2: Example of a 2 channel adaptive IIR based ANC system implemented on the testbed
"Reference" Filter
ReferenceMicrophoneinput
"Reference" Filter
ReferenceMicrophoneinput
DSP56002
ADC
ADC
Notes:Strathclyde/ have developed a multichannel active noise control testbed . The system has been widely used,and the results below are shown for active noise control of car engine noise, and air ducting noise.
0 5 10 15 20 25 30 35 40 45 50
Time (ms)
Am
plitu
de 20001200
400-400
-1200-2000/2000
1200400
-400-1200-2000
0
0
0 100 200 300 400 500 600 700 800 900 1000
Frequency (Hz)
Mag
nitu
de (d
B)
0-8
-16-24-32
-40/0-8
-16-24-32-40
TIme Analysis
Power Spectra
Before ANC
After ANC
Before ANC
After ANC
0 5 10 15 20 25 30 35 40 45 50
Time (ms)
Mag
nitu
de (u
nits
)
25001500
500-500
-1500-2500/2500
1500500
-500-1500-2500
0
0
0 100 200 300 400 500 600 700 800 900 1000
Frequency (Hz)
Mag
nitu
de (d
B)
0-8
-16-24-32
-40/0-8
-16-24-32-40
TIme Analysis
Power Spectra
20 25 30
Before ANC
After ANC
Before ANC
After ANC
ANC Inside a Car ANC for Air Ducting
Adaptive Differential PCM (ADPCM) 32
• Adaptive differential pulse code modulation (ADPCM) achievescompression using an adaptive step size, and an adaptive IIR predictorfilter.
• The ITU G-721 and G726 (more recently) recommendation definesADPCM for 32 kbits/s and lower bit rate data transmission.
Adaptive IIR
Quantizer
Adaptive IIR
Σ∆(n)
xd(n)
x(n)Channel
xd(n)
Σ+
+Quantizer
Step size adapter
Step size adapter
Σ
+
+
Notes:ADPCM uses a 2 pole, four zero IIR filter for prediction purposes. Given that practically every PC has anADPCM compress facility this makes ADPCM one of the most implemented algorithms around. Hence to saythat adaptive IIR filters are rarely implemented is quite untrue given ADPCM is on every Windows PC and manydigital telephone systems such as DECT (Digital European Cordless Telephone).
Adaptive IIR Conclusions 33
• This presentation has introduced adaptive IIR LMS algorihms.
• We derived the most widely used adaptive IIR algorithm - Feintuch’sLMS.
• Convergence and stability have been demonstrated via real worldsimulation.
• Pseudo linear regression (PLR) strategies (RLS) were presented.
• The relation between adaptive equalisers and adaptive IIR filters washighlighted.
• A number of real world applications were highlighted.
Notes: