introduction to estimation theory
DESCRIPTION
Introduction to estimation theory. Seoul Nat’l Univ. Contents. What is estimator for signal models estimator application Signal models Design objectives Options of estimators Objectives and design procedure Options for estimator : smoothing, filtering, and predicting FIR structure - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/1.jpg)
Introduction to estimation theory
C o n tro l In fo rm a tio n S ys tem L a b .Seoul Nat’l Univ.
![Page 2: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/2.jpg)
Contents
C o n tro l In fo rm a tio n S ys tem L a b .
What is estimator for signal models estimator application Signal models Design objectives Options of estimators Objectives and design procedure Options for estimator : smoothing, filtering, and predicting FIR structure Initial state dependency Performance criterion Extension to Control
Seoul Nat’l Univ.
![Page 3: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/3.jpg)
1.1 What is estimator for signal models (1/1)
1.Introduction
C o n tro l In fo rm a tio n S ys tem L a b .Seoul Nat’l Univ.
Parameter estimation
State estimationestimator: Parameter
: State
as small as possible
![Page 4: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/4.jpg)
Other methodology Fault detection
parameter estimation
state observer/estimation
signal separation
spectrum analysis
Output feedback control : state feedback control + estimator
C o n tro l In fo rm a tio n S ys tem L a b .
?
?)(
tx
1u2u
3u 4u
1y2y
3y
),,,,,,()(ˆ 3214321 yyyuuuuftx
1.2 estimator application (1/3)
1.Introduction
Seoul Nat’l Univ.
![Page 5: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/5.jpg)
C o n tro l In fo rm a tio n S ys tem L a b .
Control
Output feedback control = state feedback control + estimator
1.Introduction1.2 estimator application (2/3)
Seoul Nat’l Univ.
plant
estimator
![Page 6: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/6.jpg)
Practical areas Speech
- speech enhancement Image
- medical imaging- denoising
aerospace - target tracking- navigation- flight pass reconstruction
chemical process- distillation columns
mechanical system - motor system
biological area- cardiac arrhythmia detection
C o n tro l In fo rm a tio n S ys tem L a b .
1.Introduction1.2 estimator application ( 3/3)
Seoul Nat’l Univ.
![Page 7: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/7.jpg)
Modele
d
Unmodeled
State s
pace
Generi
c linea
r model
Linear
Nonlinea
r
Stochast
ic
Determ
inistic
Time i
nvarian
t
Time v
arying
Discret
e-tim
e
Continuous-t
ime
Categories of signal models
C o n tro l In fo rm a tio n S ys tem L a b .Seoul Nat’l Univ.
1.Introduction1.3 Signal models (1/3)
![Page 8: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/8.jpg)
State space signal model
In case of stochastic model :
In case of deterministic model :
and are random process
and are deterministic signal Choice of model is important for model-based signal processing
C o n tro l In fo rm a tio n S ys tem L a b .Seoul Nat’l Univ.
1.Introduction1.3 Signal models (2/3)
![Page 9: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/9.jpg)
Modelled vs unmodelled signal
velocity
VoltageInput :
velocityOutput :
C o n tro l In fo rm a tio n S ys tem L a b .Seoul Nat’l Univ.
1.Introduction1.3 Signal models (3/3)
Unmodeled signal
Model based signal
![Page 10: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/10.jpg)
Stability of the filter Estimation error ( often called performance )
unbiasedness : convergence :
efficiency : Robustness
estimation error w.r.t signal model uncertainties Computation load
C o n tro l In fo rm a tio n S ys tem L a b .Seoul Nat’l Univ.
1.Introduction1.4 Design objectives (1/1)
![Page 11: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/11.jpg)
Estimator structure
Performance
CriterionSignal
Models
IIR
(infinite horizon)
FIR
(receding horizon)
Initial state
dependent
stochastic
deterministicleast square
minimax
C o n tro l In fo rm a tio n S ys tem L a b .Seoul Nat’l Univ.
1.Introduction
Given OptionsInitial state
independent
Minimum
variance
NonlinearLinear
1.5 Options for estimators (1/1)
FilterSmoothing Prediction
generic linear
state space
receding horizon
infinite
horizon
![Page 12: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/12.jpg)
Signal models Optimal estimator Does it satisfy
Estimator structure
. . .
Performance criterion
. . .
desired properties
Desired properties
Stability
Robustness
Small error Yes
No
C o n tro l In fo rm a tio n S ys tem L a b .
1.Introduction
Seoul Nat’l Univ.
1.6 Objectives and design procedure (1/2)
![Page 13: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/13.jpg)
Stability FIR
1.Introduction
Small error
Robustness w.r.t uncertainties
w.r.t disturbance
Performance criterion
Objectives : Options
1.6 Objectives and design procedure (2/2)
![Page 14: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/14.jpg)
1.Introduction1.7 Options for estimator : smoothing, filtering, and predicting
C o n tro l In fo rm a tio n S ys tem L a b .Seoul Nat’l Univ.
Categories of estimators
Current time
Smoothing Filtering Predicting
![Page 15: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/15.jpg)
C o n tro l In fo rm a tio n S ys tem L a b .
Which one do
you think better ?
Case 1 (IIR)
Case 2 (FIR)
1.Introduction
Seoul Nat’l Univ.
1.8 FIR structure (1/ 9)
![Page 16: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/16.jpg)
C o n tro l In fo rm a tio n S ys tem L a b .
Case 1 (FIR)
Case 1 (IIR)
1.Introduction
Seoul Nat’l Univ.
1.8 FIR structure (2/9)
BIBO stability of FIR estimators
![Page 17: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/17.jpg)
0 100 200 300 400 500 600 700 800 900 1000-80
-70
-60
-50
-40
-30
-20
-10
0
10Real state Estimate state Diverged estimate state
Divergence of IIR filter (Kalman filter)
C o n tro l In fo rm a tio n S ys tem L a b .Seoul Nat’l Univ.
1.Introduction1.8 FIR structure (3/9)
Robustness to model uncertainty
![Page 18: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/18.jpg)
C o n tro l In fo rm a tio n S ys tem L a b .
Robustness to round off error : comparison of error covariance
Observation : Though rounding at the 4th digit are not serious, rounding of 3rd and 2 nd digit makes
difference between the FIR filter and IIR filter.
FIR filter Kalman Filter
4th digit 0.018184 0.02188
3rd digit 3.019853 12.3217
2nd digit 4.240283 15.47567
Round-off digitFilter Structure
5.0,5.0
01
1
1
1.0
1.0
9950.00998.0
0998.09950.01
RQ
vxy
wuxx
kkk
kkkk
Simulation environments We assume that the filter gain is previously known by off-line calculation Rounding off error is applied when updated Model
1.Introduction
Seoul Nat’l Univ.
1.8 FIR structure (4/9)
![Page 19: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/19.jpg)
1.Introduction
Seoul Nat’l Univ. C o n tro l In fo rm a tio n S ys tem L a b .
1. Stabilization the nominal system2. Stabilization the disturbed systems
In case of Control
Nominal systems
1. Be sure to be deadbeat using FIR structure for nominal systems.
2. Small error for noise or disturbance corrupted systems.
In case of Filter
1.8 FIR structure (5/9)
Require to be deadbeat using nominal systems Nominal systems = zero disturbance / noise system
Deadbeat property
![Page 20: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/20.jpg)
1.Introduction
Seoul Nat’l Univ. C o n tro l In fo rm a tio n S ys tem L a b .
1.8 FIR structure (6/9)
)(tx
)(ˆ tx
Exact filter (deadbeat phenomenon)
Noise
State & estim. trajectory
Horizon size
Deadbeat property
![Page 21: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/21.jpg)
C o n tro l In fo rm a tio n S ys tem L a b .
Original
Filtered
IIR filter
1.Introduction
Seoul Nat’l Univ.
1.8 FIR structure (7/9)
0 0.2 0.3 1-1
0
1Chebyshev-1 Lowpass Filter
frequency (pi units)
PH
AS
E
0 0.2 0.3 1-1
0
1FIR Lowpass Filter
frequency (pi units)
PH
AS
E
0 0.2 0.3 1
50
0
FIR Lowpass Filter
frequency (pi units)
Ma
gn
itu
de
|H
(w)|
in
dB
0 0.2 0.3 1
50
0.25 0
Chebyshev-1 Lowpass Filter
frequency (pi units)
Ma
gn
itu
de
|H
| in
dB
Magnitude
Phase
Time
Frequency
FIR filter
Heavy distortion of
phase at band gap
![Page 22: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/22.jpg)
C o n tro l In fo rm a tio n S ys tem L a b .
cf. Infinite impulse response(IIR) : Nonlinear phase Not always stable Easy to obtain from analog filter Suitable for sharp cutoff characteristic and high speed
1.Introduction
Seoul Nat’l Univ.
1.8 FIR structure (8/9) Advantage & disadvantage
• Advantage of FIR Use of DFT Robustness to round off error Linear phase Guaranteed stability Good for adaptive filter
• Disadvantage of FIR Computation load H/W complexity
![Page 23: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/23.jpg)
C o n tro l In fo rm a tio n S ys tem L a b .
F I RF I R I I R
1.Introduction
Seoul Nat’l Univ.
1.8 FIR structure (9/9)
![Page 24: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/24.jpg)
C o n tro l In fo rm a tio n S ys tem L a b .
Infinite impulse response (IIR) : dependent of
IIR Linear Initial state dependent
1.Introduction
Seoul Nat’l Univ.
1.9 Initial state dependency (1/2)
FIR Linear Initial state free
Finite impulse response (FIR) : Independent of
![Page 25: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/25.jpg)
Filter is to estimate stateThe initial state is also a state
It is not logical to assume the initial state
C o n tro l In fo rm a tio n S ys tem L a b .
1.Introduction
Seoul Nat’l Univ.
1.9 Initial state dependency (2/2)
Example : Try to guess who he is.
1. First case(our approach)
2. Second case(ex. Kalman filter) ,
Given, then guess
Given, then guess
Original picture
![Page 26: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/26.jpg)
Performance criterion
Minimum variance
Least square
Maximum Likelihood
C o n tro l In fo rm a tio n S ys tem L a b .
1.Introduction1.10 Performance criterion (1/3)
Seoul Nat’l Univ.
![Page 27: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/27.jpg)
Performance criterion for deterministic models- filter
- Minimax filter
- Least squares
H
C o n tro l In fo rm a tio n S ys tem L a b .
1.Introduction1.10 Performance criterion (2/3)
Seoul Nat’l Univ.
Performance criterion for Stochastic models- Minimum variance
- Minimax variance
- Minimum Entropy
![Page 28: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/28.jpg)
1.Introduction1.10 Performance criterion (3/3)
C o n tro l In fo rm a tio n S ys tem L a b .Seoul Nat’l Univ.
Stability FIR
Small error
Robustness w.r.t uncertainties
w.r.t disturbance
Minimization
Objectives : Options
Minimization of maxima
![Page 29: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/29.jpg)
1.Introduction1.11 Extension to control : receding horizon control
C o n tro l In fo rm a tio n S ys tem L a b .Seoul Nat’l Univ.
Which one do
you think better ?
What is the receding horizon control?
![Page 30: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/30.jpg)
1.Introduction
C o n tro l In fo rm a tio n S ys tem L a b .Seoul Nat’l Univ.
Stability of the closed-loop systems Small tracking error Robustness
stabilitytracking error
1.11 Extension to control : desired property
![Page 31: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/31.jpg)
Control structure
Performance
CriterionSignal
Models
infinite horizon
receding horizon
output feedback
stochastic
deterministic LQ
minimax
Given Optionsstate feedback
LQG
Finite memory control(including static control)
Dynamic(IIR control)
I/O model
state space
C o n tro l In fo rm a tio n S ys tem L a b .Seoul Nat’l Univ.
1.Introduction1.11 Extension to control : options for controls
![Page 32: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/32.jpg)
1.Introduction
Seoul Nat’l Univ. C o n tro l In fo rm a tio n S ys tem L a b .
Signal models Optimal control Does it satisfy
Control structure
Performance criterionLQG
LQ
Minimum entropy
……
desired properties
Desired properties
Stability
Robustness
Small tracking error Yes
No
State feedback control
Output feedback controlDynamic controlFinite memory control
……
1.11 Extension to control : objectives and design procedures
![Page 33: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/33.jpg)
1.Introduction
Seoul Nat’l Univ. C o n tro l In fo rm a tio n S ys tem L a b .
1.11 extension to control : performance criterion with receding horizon
LQ
LQG
![Page 34: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/34.jpg)
1.Introduction1.11 extension to control : receding horizon output feedback control
Seoul Nat’l Univ. C o n tro l In fo rm a tio n S ys tem L a b .
State feedback receding horizon controlLQC Control……
FilterKalman filter filterMixed filer……
+
Question : Is it optimal ?
FMC (finite memory control)
method 1
method 2 Global optimal output feedback control
cf) LQG
![Page 35: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/35.jpg)
1.Introduction
Seoul Nat’l Univ. C o n tro l In fo rm a tio n S ys tem L a b .
Computation
1.11 extension to control : receding horizon output feedback control
![Page 36: Introduction to estimation theory](https://reader033.vdocuments.us/reader033/viewer/2022061315/5681355f550346895d9cc473/html5/thumbnails/36.jpg)
Contents of standard textbook on optimal control and estimation
1.Introduction
C o n tro l In fo rm a tio n S ys tem L a b .Seoul Nat’l Univ.
1. LQ control Finite horizon Infinite horizon
2. Kalman filter
Finite horizon Infinite horizon
3. LQG control
Finite horizon Infinite horizon
4. Full information control Finite horizon Infinite horizon
5. filter
Finite horizon Infinite horizon
6. Output feedback control
Finite horizon Infinite horizon
Receding horizon
Receding horizon
Receding horizon
Receding horizon
Receding horizon
Receding horizon
Covered in this class