part 4: iir filters – optimization approach …andreas/iscastutorial/part 4-iir...
TRANSCRIPT
![Page 1: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/1.jpg)
Part 4: IIR Filters – Optimization Approach
Tutorial ISCAS 2007
Copyright © 2007 Andreas AntoniouVictoria, BC, Canada
Email: [email protected]
July 24, 2007
Frame # 1 Slide # 1 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 2: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/2.jpg)
Introduction
IIR filters that would satisfy prescribed specifications can bedesigned by using a variety of optimization algorithms.
Five general steps are involved, as follows:
1. Formulate a suitable objective (cost) function.2. Deduce the gradient of the objective function and, if
necessary, the Hessian matrix.3. Choose an appropriate weighting function.4. Select a suitable optimization algorithm.5. Run the optimization algorithm for increasing filter orders
until an order is found that will satisfy the requiredspecifications.
Note: The material for this module is taken from Antoniou, DigitalSignal Processing: Signals, Systems, and Filters,Chap. 16.
Frame # 2 Slide # 2 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 3: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/3.jpg)
Problem Formulation
� An Nth-order IIR filter can be represented by adiscrete-time transfer function of the form
H (z) = H0
J∏j=1
a0j + a1j z + z2
b0j + b1j z + z2
where aij and bij are real coefficients, J = N/2, and H0 is apositive multiplier constant.
Frame # 3 Slide # 3 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 4: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/4.jpg)
Problem Formulation
� An Nth-order IIR filter can be represented by adiscrete-time transfer function of the form
H (z) = H0
J∏j=1
a0j + a1j z + z2
b0j + b1j z + z2
where aij and bij are real coefficients, J = N/2, and H0 is apositive multiplier constant.
� As presented, H (z) would be of even order; however, anodd-order H (z) can be obtained by letting
a0j = b0j = 0
for one value of j .
Frame # 3 Slide # 4 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 5: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/5.jpg)
Problem Formulation Cont’d
The amplitude response of an arbitrary IIR filter can bededuced as
M (x, ω) = |H (ejωT )|where ω is the frequency and
x = [a01 a11 b01 b11 · · · b1J H0]T
is a column vector with 4J + 1 elements.
Frame # 4 Slide # 5 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 6: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/6.jpg)
Problem Formulation Cont’d
Straightforward analysis gives the amplitude response as
M (x, ω) = H0
J∏j=1
Nj (ω)
Dj (ω)
where
Nj (ω) = [1 + a20j + a2
1j + 2a1j (1 + a0j ) cos ωT + 2a0j cos 2ωT ] 12
and
Dj (ω) = [1 + b20j + b2
1j + 2b1j (1 + b0j ) cos ωT + 2b0j cos 2ωT ] 12
for j = 1, 2, . . . , J.
Frame # 5 Slide # 6 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 7: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/7.jpg)
Problem Formulation Cont’d
� If M0(ω) is the specified amplitude response, theapproximation error can be expressed as
e(x, ω) = M (x, ω) − M0(ω)
Frame # 6 Slide # 7 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 8: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/8.jpg)
Problem Formulation Cont’d
� If M0(ω) is the specified amplitude response, theapproximation error can be expressed as
e(x, ω) = M (x, ω) − M0(ω)
� By sampling e(x, ω) at frequencies ω1, ω2, . . . , ωK , thecolumn vector
E(x) = [e1(x) e2(x) . . . eK (x)]T
can be formed where
ei (x) = e(x, ωi )
for i = 1, 2, . . . , K .
Frame # 6 Slide # 8 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 9: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/9.jpg)
Problem Formulation Cont’d
Gai
n
ω1 ω2 ωK
M(x, ω)
M0(ω)
e(x, ωi)
ω, rad/s ωi
Frame # 7 Slide # 9 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 10: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/10.jpg)
Problem Formulation Cont’d
� An IIR filter can be designed by finding a point x = �x such
thatei (
�x ) ≈ 0 for i = 1, 2, . . . , K
Frame # 8 Slide # 10 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 11: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/11.jpg)
Problem Formulation Cont’d
� An IIR filter can be designed by finding a point x = �x such
thatei (
�x ) ≈ 0 for i = 1, 2, . . . , K
� Such a point can be obtained by minimizing the Lp norm ofE(x), which is defined as
�(x) = Lp(x) = ||E(x)||p =[
K∑i=1
|ei (x)|p]1/p
where p is a positive integer.
Frame # 8 Slide # 11 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 12: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/12.jpg)
Problem Formulation Cont’d
For filter design, the most important norms are the L2 and L∞norms which are defined as
L2(x) =[
K∑i=1
|ei (x)|2]1/2
and L∞(x) = limp→∞
{K∑
i=1
|ei (x)|p}1/p
= �
E (x) limp→∞
{K∑
i=1
[|ei (x)|�
E (x)
]p}1/p
= �
E (x)
where�
E (x) = max1 ≤ i ≤ K
|ei (x)|.
Frame # 9 Slide # 12 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 13: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/13.jpg)
Problem Formulation Cont’d
� In summary, an IIR filter with a amplitude response thatapproaches a specified amplitude response M0(ω) can bedesigned by solving the optimization problem
minimizex
�(x)
Frame # 10 Slide # 13 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 14: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/14.jpg)
Problem Formulation Cont’d
� In summary, an IIR filter with a amplitude response thatapproaches a specified amplitude response M0(ω) can bedesigned by solving the optimization problem
minimizex
�(x)
� If�(x) = L2(x)
a least-squares solution is obtained and if
�(x) = L∞(x)
the outcome will be a so-called minimax solution.
Frame # 10 Slide # 14 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 15: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/15.jpg)
Gradient of Objective Function
For an objective function defined in terms of the Lp norm, thegradient of the objective function can be deduced as
∇�k (x) ={
K∑i=1
[|ei (x)|�
E (x )
]p}(1/p)−1 K∑i=1
[|ei (x)|�
E (x )
]p−1
∇|ei (x)|
where∇|ei (x)| = [sgn ei (x)]∇ei (x)
with
sgnei (x) ={
1 if ei (x) ≥ 0
−1 otherwise
Frame # 11 Slide # 15 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 16: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/16.jpg)
Gradient of Objective Function Cont’d
The elements of ∇ei (x) can be deduced as follows bydifferentiating the amplitude response:
∂ei (x)
∂a0l= a0l + a1l cos ωi T + cos 2ωi T
[Nl (ωi )]2· M (x, ωi )
∂ei (x)
∂a1l= a1l + (1 + a0l ) cos ωi T
[Nl (ωi )]2· M (x, ωi )
∂ei (x)
∂b0l= −b0l + b1l cos ωi T + cos 2ωi T
[Dl (ωi )]2· M (x, ωi )
∂ei (x)
∂b1l= −b1l + (1 + b0l ) cos ωi T
[Dl (ωi )]2· M (x, ωi )
∂ei (x)
∂H0= 1
H0· M (x, ωi )
for l = 1, 2, . . . , J and i = 1, 2, . . . , KFrame # 12 Slide # 16 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 17: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/17.jpg)
Optimization Algorithms
� The optimization problem under consideration can besolved by using a variety of algorithms, such as thesteepest-descent or Newton algorithm or one of severalquasi-Newton algorithms.
Frame # 13 Slide # 17 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 18: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/18.jpg)
Optimization Algorithms
� The optimization problem under consideration can besolved by using a variety of algorithms, such as thesteepest-descent or Newton algorithm or one of severalquasi-Newton algorithms.
� Quasi-Newton algorithms offer a number of importantadvantages as follows:
– They do not require the Hessian matrix.
– Can be used with inexact line searches which lead toimproved efficiency.
– Are robust.
– Offer fast convergence.
Frame # 13 Slide # 18 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 19: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/19.jpg)
Quasi-Newton Algorithms
A generic quasi-Newton Algorithm is as follows:
1. Input x0 and ε. Set S0 = In and k = 0. Compute g0.
Frame # 14 Slide # 19 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 20: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/20.jpg)
Quasi-Newton Algorithms
A generic quasi-Newton Algorithm is as follows:
1. Input x0 and ε. Set S0 = In and k = 0. Compute g0.
2. Set dk = −Sk gk and find αk , the value of α that minimizesf (xk + α dk ), using a line search.
Frame # 14 Slide # 20 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 21: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/21.jpg)
Quasi-Newton Algorithms
A generic quasi-Newton Algorithm is as follows:
1. Input x0 and ε. Set S0 = In and k = 0. Compute g0.
2. Set dk = −Sk gk and find αk , the value of α that minimizesf (xk + α dk ), using a line search.
3. Set δk = α k dk and xk+1 = xk + δk , and computefk+1 = f (xk+1).
Frame # 14 Slide # 21 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 22: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/22.jpg)
Quasi-Newton Algorithms
A generic quasi-Newton Algorithm is as follows:
1. Input x0 and ε. Set S0 = In and k = 0. Compute g0.
2. Set dk = −Sk gk and find αk , the value of α that minimizesf (xk + α dk ), using a line search.
3. Set δk = α k dk and xk+1 = xk + δk , and computefk+1 = f (xk+1).
4. If ‖δk ‖2 < ε, then output�x = x k+1, f (
�x ) = fk+1 and stop.
Frame # 14 Slide # 22 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 23: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/23.jpg)
Quasi-Newton Algorithms
A generic quasi-Newton Algorithm is as follows:
1. Input x0 and ε. Set S0 = In and k = 0. Compute g0.
2. Set dk = −Sk gk and find αk , the value of α that minimizesf (xk + α dk ), using a line search.
3. Set δk = α k dk and xk+1 = xk + δk , and computefk+1 = f (xk+1).
4. If ‖δk ‖2 < ε, then output�x = x k+1, f (
�x ) = fk+1 and stop.
5. Compute gk+1 and set γk = gk+1 − gk .
Frame # 14 Slide # 23 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 24: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/24.jpg)
Quasi-Newton Algorithms
A generic quasi-Newton Algorithm is as follows:
1. Input x0 and ε. Set S0 = In and k = 0. Compute g0.
2. Set dk = −Sk gk and find αk , the value of α that minimizesf (xk + α dk ), using a line search.
3. Set δk = α k dk and xk+1 = xk + δk , and computefk+1 = f (xk+1).
4. If ‖δk ‖2 < ε, then output�x = x k+1, f (
�x ) = fk+1 and stop.
5. Compute gk+1 and set γk = gk+1 − gk .
6. Compute Sk+1 = Sk + Ck .
Frame # 14 Slide # 24 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 25: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/25.jpg)
Quasi-Newton Algorithms
A generic quasi-Newton Algorithm is as follows:
1. Input x0 and ε. Set S0 = In and k = 0. Compute g0.
2. Set dk = −Sk gk and find αk , the value of α that minimizesf (xk + α dk ), using a line search.
3. Set δk = α k dk and xk+1 = xk + δk , and computefk+1 = f (xk+1).
4. If ‖δk ‖2 < ε, then output�x = x k+1, f (
�x ) = fk+1 and stop.
5. Compute gk+1 and set γk = gk+1 − gk .
6. Compute Sk+1 = Sk + Ck .
7. Check Sk+1 for positive definiteness and if it is found to benonpositive definite force it to become positive definite.
Frame # 14 Slide # 25 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 26: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/26.jpg)
Quasi-Newton Algorithms
A generic quasi-Newton Algorithm is as follows:
1. Input x0 and ε. Set S0 = In and k = 0. Compute g0.
2. Set dk = −Sk gk and find αk , the value of α that minimizesf (xk + α dk ), using a line search.
3. Set δk = α k dk and xk+1 = xk + δk , and computefk+1 = f (xk+1).
4. If ‖δk ‖2 < ε, then output�x = x k+1, f (
�x ) = fk+1 and stop.
5. Compute gk+1 and set γk = gk+1 − gk .
6. Compute Sk+1 = Sk + Ck .
7. Check Sk+1 for positive definiteness and if it is found to benonpositive definite force it to become positive definite.
8. Set k = k + 1 and go to step 2.
Frame # 14 Slide # 26 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 27: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/27.jpg)
Quasi-Newton Algorithms Cont’d
Notes:� x 0 is the initial point.� ε is a termination tolerance.� I n is the n × n identity matrix.� d k is the direction vector at the start of the k th iteration.� gk is the gradient at the start of the k th iteration.� Sk is an approximation to the inverse Hessian at the start
of the k th iteration.� Ck is a correction matrix.�
�x is the minimum point and f (
�x ) is the minimum value of
the objective function.� Efficiency can be achieved by using Fletcher’s inexact line
search in Step 2.
Frame # 15 Slide # 27 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 28: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/28.jpg)
Quasi-Newton Algorithms Cont’d
� Several quasi-Newton algorithms are available. They differfrom one another in the formula used to update matrix Sk+1
in Step 7 of the generic quasi-Newton algorithm presented.
Frame # 16 Slide # 28 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 29: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/29.jpg)
Quasi-Newton Algorithms Cont’d
� Several quasi-Newton algorithms are available. They differfrom one another in the formula used to update matrix Sk+1
in Step 7 of the generic quasi-Newton algorithm presented.
� The two most important algorithms of this family are theDavidon-Fletcher-Powell (DFP) algorithm in which
Sk+1 = Sk + δk δTk
γ Tk δk
− Sk γk γ Tk Sk
γ Tk Sk γk
and the Broyden-Fletcher-Goldfarb-Shanno (BFGS)algorithm in which
Sk+1 = Sk +(
1 + γ Tk Sk γk
γ Tk δk
)δk δT
k
γ Tk δk
− (δk γ Tk Sk + Sk γk δT
k )
γ Tk δk
Frame # 16 Slide # 29 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 30: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/30.jpg)
Choice between L2 and L∞ Solutions
� L2 solutions are easier to obtain and filters based on thesesolutions will reject more signal power in stopbands.
However, the passband error tends to have large peaksnear passband edges, which are undesirable in practice.
Frame # 17 Slide # 30 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 31: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/31.jpg)
Choice between L2 and L∞ Solutions
� L2 solutions are easier to obtain and filters based on thesesolutions will reject more signal power in stopbands.
However, the passband error tends to have large peaksnear passband edges, which are undesirable in practice.
� L∞ solutions are more difficult to obtain because theyrequire the application of minimax algorithms which entailmuch more computation.
However, they offer the advantage that the approximationerror tends to be uniformly distributed in passbands, i.e., ittends to be equiripple as in elliptic filters.
Frame # 17 Slide # 31 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 32: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/32.jpg)
Minimax Algorithms
� Minimax algorithms are essentially sequential algorithmsthat involve a series of unconstrained optimizations.
� A representative algorithm of this class is the so-calledleast-pth algorithm.
� Another popular minimax algorithm is one proposed byCharalambous (see References).
Frame # 18 Slide # 32 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 33: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/33.jpg)
Least-pth Minimax Algorithm
1. Input�x 0 and ε1. Set k = 1, p = 2, μ = 2,
�
E 0 = 1099.
Frame # 19 Slide # 33 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 34: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/34.jpg)
Least-pth Minimax Algorithm
1. Input�x 0 and ε1. Set k = 1, p = 2, μ = 2,
�
E 0 = 1099.
2. Initialize frequencies ω1, ω2, . . . , ωK .
Frame # 19 Slide # 34 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 35: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/35.jpg)
Least-pth Minimax Algorithm
1. Input�x 0 and ε1. Set k = 1, p = 2, μ = 2,
�
E 0 = 1099.
2. Initialize frequencies ω1, ω2, . . . , ωK .
3. Using�x k−1 as initial value, minimize
�k (x) = �
E (x)
{K∑
i=1
[|ei (x)|�
E (x)
]p}1/p
where �
E (x) = max1≤i≤K
|ei (x)|
with respect to x, to obtain�x k . Set
�
E k = �
E (�x ).
Frame # 19 Slide # 35 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 36: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/36.jpg)
Least-pth Minimax Algorithm
1. Input�x 0 and ε1. Set k = 1, p = 2, μ = 2,
�
E 0 = 1099.
2. Initialize frequencies ω1, ω2, . . . , ωK .
3. Using�x k−1 as initial value, minimize
�k (x) = �
E (x)
{K∑
i=1
[|ei (x)|�
E (x)
]p}1/p
where �
E (x) = max1≤i≤K
|ei (x)|
with respect to x, to obtain�x k . Set
�
E k = �
E (�x ).
4. If |�E k−1 − �
E k | < ε1, then output�x k and
�
E k , and stop.Otherwise, set p = μp, k = k + 1 and go to step 3.
Frame # 19 Slide # 36 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 37: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/37.jpg)
Stability
� Least-squares or minimax algorithms will often yielddiscrete-time transfer functions with poles outside the unitcircle |z | = 1, and such transfer functions representunstable filters.
Frame # 20 Slide # 37 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 38: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/38.jpg)
Stability
� Least-squares or minimax algorithms will often yielddiscrete-time transfer functions with poles outside the unitcircle |z | = 1, and such transfer functions representunstable filters.
� Fortunately, it is possible to eliminate this problem througha stabilization technique that has been known for a numberof years.
Frame # 20 Slide # 38 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 39: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/39.jpg)
Stability Cont’d
� Let us assume that an optimization algorithm has produceda discrete-time transfer function
H (z) = N (z)
D(z)= N (z)
D ′(z)∏k
i=1(z − p̃i )
with k poles p̃1, p̃2, . . . , p̃k that lie outside the unit circle.
Frame # 21 Slide # 39 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 40: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/40.jpg)
Stability Cont’d
� Let us assume that an optimization algorithm has produceda discrete-time transfer function
H (z) = N (z)
D(z)= N (z)
D ′(z)∏k
i=1(z − p̃i )
with k poles p̃1, p̃2, . . . , p̃k that lie outside the unit circle.
� A stable transfer function that yields the same amplituderesponse can be obtained as
H ′(z) = H0N (z)
D ′(z)∏k
i=1(z − 1/p̃i)=
∑Ni=0 a′
i zi
1 +∑Ni=1 b ′
i zi
where
H0 = 1∏ki=1 p̃i
Frame # 21 Slide # 40 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 41: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/41.jpg)
Prescribed Specifications
� Optimization algorithms in general tend to yield filters inwhich the maximum values of the approximation error in thevarious passbands and stopbands are of the same order.
Frame # 22 Slide # 41 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 42: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/42.jpg)
Prescribed Specifications
� Optimization algorithms in general tend to yield filters inwhich the maximum values of the approximation error in thevarious passbands and stopbands are of the same order.
� In practice, the prescribed passband ripples and minimumstopband attenuations vary wildly from band to band andfrom one application to the next, which means that therequired maximum values of the passband and stopbanderrors also vary wildly.
Frame # 22 Slide # 42 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 43: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/43.jpg)
Prescribed Specifications
� Optimization algorithms in general tend to yield filters inwhich the maximum values of the approximation error in thevarious passbands and stopbands are of the same order.
� In practice, the prescribed passband ripples and minimumstopband attenuations vary wildly from band to band andfrom one application to the next, which means that therequired maximum values of the passband and stopbanderrors also vary wildly.
� In order to achieve desired specifications, we need be ableto control the relative magnitudes of the maximumpassband and stopband errors, and this is achievedthrough the use of a weighting.
Frame # 22 Slide # 43 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 44: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/44.jpg)
Prescribed Specifications Cont’d
� Weighting essentially involves constructing a modified errorfunction of the form
e ′i (x) = W (ω)ei(x)
= W (ω)[M (x, ωi) − M0(ωi )]where W (ω) is a scalar weighting function which is used toemphasize or deemphasize the approximation error inselected passbands or stopbands so as to decrease orincrease its magnitude in those bands.
Frame # 23 Slide # 44 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 45: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/45.jpg)
Prescribed Specifications Cont’d
� Weighting essentially involves constructing a modified errorfunction of the form
e ′i (x) = W (ω)ei(x)
= W (ω)[M (x, ωi) − M0(ωi )]where W (ω) is a scalar weighting function which is used toemphasize or deemphasize the approximation error inselected passbands or stopbands so as to decrease orincrease its magnitude in those bands.
� Typically, W (ω) is a piecewise constant function which isassigned a value greater or less than one to decrease orincrease the magnitude of the approximation error is agiven band.
Frame # 23 Slide # 45 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 46: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/46.jpg)
Prescribed Specifications Cont’d
Arbitrary filter specifications can be achieved as follows:
1. Choose a suitable weighting function on the basis of theprescribed specifications.
Frame # 24 Slide # 46 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 47: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/47.jpg)
Prescribed Specifications Cont’d
Arbitrary filter specifications can be achieved as follows:
1. Choose a suitable weighting function on the basis of theprescribed specifications.
2. Design filters for increasing filter orders until a filter order isfound that would satisfy the required specifications.
Evidently, this is a cut-and-try method and it could entail aconsiderable amount of computation.
Frame # 24 Slide # 47 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 48: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/48.jpg)
Prescribed Specifications Cont’d
� Assuming a filter with m bands, the weighting function canbe deduced from the specified passband ripples and theminimum stopband attenuations as detailed below.
Frame # 25 Slide # 48 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 49: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/49.jpg)
Prescribed Specifications Cont’d
� Assuming a filter with m bands, the weighting function canbe deduced from the specified passband ripples and theminimum stopband attenuations as detailed below.
� Let δ1, δ2, . . ., δk , . . ., δm be the maximum passband andstopband errors imposed by the filter specifications where
δi = 100.05Api −1
100.05Api + 1
for a passband with ripple Api dB and
δj = 10−0.05Aaj
for a stopband with a minimum attenuation Aaj dB.
Frame # 25 Slide # 49 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 50: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/50.jpg)
Prescribed Specifications Cont’d
� Now assume that the weighting function W (ω) is apiecewise-constant function with positive values W1, W2,. . ., Wk , . . ., Wm for bands 1, 2, . . ., k , . . ., m, respectively.
Frame # 26 Slide # 50 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 51: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/51.jpg)
Prescribed Specifications Cont’d
� Now assume that the weighting function W (ω) is apiecewise-constant function with positive values W1, W2,. . ., Wk , . . ., Wm for bands 1, 2, . . ., k , . . ., m, respectively.
� Suitable scaling constants can be assigned as
W1 = δk
δ1, W2 = δk
δ2, . . . , Wk−1 = δk
δk−1
Wk = 1
Wk+1 = δk
δk+1, Wk+2 = δk
δk+2, . . . , Wm = δk
δm
From example, in a BP filter we could assign W1 = δ2/δ1,
W2 = 1, W3 = δ2/δ3 or W1 = 1, W2 = δ1/δ2, W3 = δ1/δ3 orW1 = δ3/δ1, W2 = δ3/δ2, W3 = 1.
Frame # 26 Slide # 51 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 52: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/52.jpg)
Design Examples
A variety of design examples can be found in Antoniou, DigitalSignal Processing, Chap. 14, and in Antoniou and Lu, PracticalOptimization, Chaps. 9 and 16.
Frame # 27 Slide # 52 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 53: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/53.jpg)
Summary
� A great variety of optimization algorithms can be used forthe design of IIR filters.
Frame # 28 Slide # 53 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 54: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/54.jpg)
Summary
� A great variety of optimization algorithms can be used forthe design of IIR filters.
� Quasi-Newton algorithms work very well.
Frame # 28 Slide # 54 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 55: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/55.jpg)
Summary
� A great variety of optimization algorithms can be used forthe design of IIR filters.
� Quasi-Newton algorithms work very well.� The optimization approach is very flexible in that it can be
used to design filters with arbitrary amplitude and/or phaseresponses.
Frame # 28 Slide # 55 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 56: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/56.jpg)
Summary
� A great variety of optimization algorithms can be used forthe design of IIR filters.
� Quasi-Newton algorithms work very well.� The optimization approach is very flexible in that it can be
used to design filters with arbitrary amplitude and/or phaseresponses.
� In FIR filters as well as IIR filters based on the bilineartransformation, techniques are availalble that can be usedto predict the required filter order to achieve prescribedspecifications.
Unfortunately, in IIR filters designed by optimization thefilter order can only be deduced through a cut-and-tryapproach.
Frame # 28 Slide # 56 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 57: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/57.jpg)
Summary
� A great variety of optimization algorithms can be used forthe design of IIR filters.
� Quasi-Newton algorithms work very well.� The optimization approach is very flexible in that it can be
used to design filters with arbitrary amplitude and/or phaseresponses.
� In FIR filters as well as IIR filters based on the bilineartransformation, techniques are availalble that can be usedto predict the required filter order to achieve prescribedspecifications.
Unfortunately, in IIR filters designed by optimization thefilter order can only be deduced through a cut-and-tryapproach.
� As in any other methodology based on optimization, a largeamount of computation is required to complete a design.
Frame # 28 Slide # 57 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 58: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/58.jpg)
References
� A. Antoniou, Digital Signal Processing: Signals, Systems,and Filters, Chap. 16, McGraw-Hill, 2005.
� A. Antoniou and W.-S. Lu Practical Optimization:Algorithms and Engineering Applications, Chaps. 9 and 16,Springer, 2007.
� C. Charalambous, “Acceleration of the least pth algorithmfor minimax optimization with engineering applications,”Mathematical Programming, vol. 17, pp. 270–297, 1979.
� A. Antoniou, “Improved minimax optimisation algorithmsand their application in the design of recursive digitalfilters,” Proc. Inst. Elect. Eng., Part G, vol. 138,pp. 724–730, Dec. 1991.
Frame # 29 Slide # 58 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 59: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/59.jpg)
References
� A. Antoniou and W.-S. Lu, “Design of two-dimensionaldigital filters by using the singular value decomposition,”IEEE Trans. Circuits Syst., vol. 34, pp. 1191–1198,Oct. 1987.
� W.-S. Lu and A. Antoniou, “Design of digital filters and filterbanks by optimization: A state of the art review,” inProc. 2000 European Signal Processing Conference,vol. 1, pp. 351–354, Tampere, Finland, Sept. 2000.
Frame # 30 Slide # 59 A. Antoniou Part4: IIR Filters – Optimizatio n Method
![Page 60: Part 4: IIR Filters – Optimization Approach …andreas/ISCASTutorial/Part 4-IIR Filters...Introduction IIR filters that would satisfy prescribed specifications can be designed](https://reader030.vdocuments.us/reader030/viewer/2022013002/5cedd28988c993306d8d7729/html5/thumbnails/60.jpg)
This slide concludes the presentation.Thank you for your attention.
Frame # 31 Slide # 60 A. Antoniou Part4: IIR Filters – Optimizatio n Method