recursive estimation - matematikcentrum · intronaiverlsrplrrpemrmlfilter sp/fd stochastic...
TRANSCRIPT
![Page 1: Recursive estimation - Matematikcentrum · IntroNaiveRLSRPLRRPEMRMLFilter SP/FD stochastic approximation I The gradient can be approximated by nite di erence at the cost of slower](https://reader034.vdocuments.us/reader034/viewer/2022050100/5f3f52c2da4183724326d32b/html5/thumbnails/1.jpg)
Intro Naive RLS RPLR RPEM RML Filter
Recursive estimation
Erik Lindström
Centre for Mathematical Sciences
Lund University
LU/LTH & DTU
Erik Lindström - [email protected] Recursive estimation
![Page 2: Recursive estimation - Matematikcentrum · IntroNaiveRLSRPLRRPEMRMLFilter SP/FD stochastic approximation I The gradient can be approximated by nite di erence at the cost of slower](https://reader034.vdocuments.us/reader034/viewer/2022050100/5f3f52c2da4183724326d32b/html5/thumbnails/2.jpg)
Intro Naive RLS RPLR RPEM RML Filter
Overview
Introduction
Naive recursive estimators
Recursive LS
Recursive Pseudo-Linear Regression
Recursive Prediction Error Method
Recursive Maximum Likelihood
Filtering
Erik Lindström - [email protected] Recursive estimation
![Page 3: Recursive estimation - Matematikcentrum · IntroNaiveRLSRPLRRPEMRMLFilter SP/FD stochastic approximation I The gradient can be approximated by nite di erence at the cost of slower](https://reader034.vdocuments.us/reader034/viewer/2022050100/5f3f52c2da4183724326d32b/html5/thumbnails/3.jpg)
Intro Naive RLS RPLR RPEM RML Filter
Di�erent types
I Forgetting type estimators
I Converging estimators
Ex: Zi ∈ N (µ, 1). Estimate the mean (µ) as
µN =1
N
∑Zi
or asµN = ZN?
Di�erent properties and applications!
Erik Lindström - [email protected] Recursive estimation
![Page 4: Recursive estimation - Matematikcentrum · IntroNaiveRLSRPLRRPEMRMLFilter SP/FD stochastic approximation I The gradient can be approximated by nite di erence at the cost of slower](https://reader034.vdocuments.us/reader034/viewer/2022050100/5f3f52c2da4183724326d32b/html5/thumbnails/4.jpg)
Intro Naive RLS RPLR RPEM RML Filter
Naive approaches
I Windowed estimation
I Use [t − u : t] to estimate parameters
θt = argmaxt∑
n=t−ulog p(yn|yt−u, . . . yn−1)
I Followed by
θt+1 = argmaxt+1∑
n=t−u+1
log p(yn|yt−u+1, . . . yn−1)
Properties?
Erik Lindström - [email protected] Recursive estimation
![Page 5: Recursive estimation - Matematikcentrum · IntroNaiveRLSRPLRRPEMRMLFilter SP/FD stochastic approximation I The gradient can be approximated by nite di erence at the cost of slower](https://reader034.vdocuments.us/reader034/viewer/2022050100/5f3f52c2da4183724326d32b/html5/thumbnails/5.jpg)
Intro Naive RLS RPLR RPEM RML Filter
Recursive LS
I Linear models can be written as
Y = Xθ + e
I Estimate is given by
θ = (XTX )−1(XTY )
Can be written in recursive form!
Erik Lindström - [email protected] Recursive estimation
![Page 6: Recursive estimation - Matematikcentrum · IntroNaiveRLSRPLRRPEMRMLFilter SP/FD stochastic approximation I The gradient can be approximated by nite di erence at the cost of slower](https://reader034.vdocuments.us/reader034/viewer/2022050100/5f3f52c2da4183724326d32b/html5/thumbnails/6.jpg)
Intro Naive RLS RPLR RPEM RML Filter
Recursive LS
I Optimize
θt = argmint∑
s=p
(Ys − XTs θ)2
I whereXTt = [−Yt−1, . . . ,−Yt−p]
andθT = [θ1, . . . , θp]
I This can be written as
θt = R−1t ht
Rt =∑
XsXTs
ht =∑
XsYs (1)
Erik Lindström - [email protected] Recursive estimation
![Page 7: Recursive estimation - Matematikcentrum · IntroNaiveRLSRPLRRPEMRMLFilter SP/FD stochastic approximation I The gradient can be approximated by nite di erence at the cost of slower](https://reader034.vdocuments.us/reader034/viewer/2022050100/5f3f52c2da4183724326d32b/html5/thumbnails/7.jpg)
Intro Naive RLS RPLR RPEM RML Filter
Recursive LS
I We can now write Rt = Rt−1 + XtXTt
I and ht = ht−1 + XtYt
and also
θt = R−1t ht
= R−1t (ht−1 + XtYt)
= R−1t (Rt−1θt−1 + XtYt)
= R−1t (Rt θt−1 − XtXTt θt−1 + XtYt)
= θt−1 + R−1t Xt(Yt − XTt θt−1)
(2)
This is the standard Recursive LS (RLS)
Erik Lindström - [email protected] Recursive estimation
![Page 8: Recursive estimation - Matematikcentrum · IntroNaiveRLSRPLRRPEMRMLFilter SP/FD stochastic approximation I The gradient can be approximated by nite di erence at the cost of slower](https://reader034.vdocuments.us/reader034/viewer/2022050100/5f3f52c2da4183724326d32b/html5/thumbnails/8.jpg)
Intro Naive RLS RPLR RPEM RML Filter
Recursive LS
I We have that Rt = Rt−1 + XtXTt
I but are interested in R−1t
The matrix inversion lemma
[A + BCD]−1 = A−1 − A−1B(DA−1B + C−1)−1DA−1
gives
R−1t = R−1t−1 − R−1t−1Xt(XTt R−1t−1Xt + I )−1XT
t R−1t−1
The RLS algorithm is then given by two simple matrix expressions!
Erik Lindström - [email protected] Recursive estimation
![Page 9: Recursive estimation - Matematikcentrum · IntroNaiveRLSRPLRRPEMRMLFilter SP/FD stochastic approximation I The gradient can be approximated by nite di erence at the cost of slower](https://reader034.vdocuments.us/reader034/viewer/2022050100/5f3f52c2da4183724326d32b/html5/thumbnails/9.jpg)
Intro Naive RLS RPLR RPEM RML Filter
Adaptive Recursive LS
Optimize
θt = argmint∑
s=p
β(t, s)(Ys − XTs θ)2
where
β(t, s) = λ(t)β(t − 1, s)
β(t, t) = 1 (3)
Hence is β(t, s) =∏t
j=s+1 λ(j).Again, recursive equations can be found!
Erik Lindström - [email protected] Recursive estimation
![Page 10: Recursive estimation - Matematikcentrum · IntroNaiveRLSRPLRRPEMRMLFilter SP/FD stochastic approximation I The gradient can be approximated by nite di erence at the cost of slower](https://reader034.vdocuments.us/reader034/viewer/2022050100/5f3f52c2da4183724326d32b/html5/thumbnails/10.jpg)
Intro Naive RLS RPLR RPEM RML Filter
Adaptive Recursive LS
The solution is given by
θt = R−1t ht
where
I Rt = λ(t)Rt−1 + XtXTt
I ht = λ(t)ht−1 + XtYt
And the rest is identical to the standard RLS.
I Interpretation of λ.
Erik Lindström - [email protected] Recursive estimation
![Page 11: Recursive estimation - Matematikcentrum · IntroNaiveRLSRPLRRPEMRMLFilter SP/FD stochastic approximation I The gradient can be approximated by nite di erence at the cost of slower](https://reader034.vdocuments.us/reader034/viewer/2022050100/5f3f52c2da4183724326d32b/html5/thumbnails/11.jpg)
Intro Naive RLS RPLR RPEM RML Filter
Recursive Pseudo-Linear Regression
I ExtendY = Xθ + e
I ToY = X (θ)θ + e
Includes e.g. ARMA and non-linear models!
Erik Lindström - [email protected] Recursive estimation
![Page 12: Recursive estimation - Matematikcentrum · IntroNaiveRLSRPLRRPEMRMLFilter SP/FD stochastic approximation I The gradient can be approximated by nite di erence at the cost of slower](https://reader034.vdocuments.us/reader034/viewer/2022050100/5f3f52c2da4183724326d32b/html5/thumbnails/12.jpg)
Intro Naive RLS RPLR RPEM RML Filter
(Adaptive) RPLR
Letθt = argmin St(θ)
where
St(θ) =t∑s
β(t, s)(Ys − XTs (θ)θ)2
I St(θ) = λ(t)St−1(θ) + (Yt − XTt (θ)θ)2
I Taylor expand around θt−1
Erik Lindström - [email protected] Recursive estimation
![Page 13: Recursive estimation - Matematikcentrum · IntroNaiveRLSRPLRRPEMRMLFilter SP/FD stochastic approximation I The gradient can be approximated by nite di erence at the cost of slower](https://reader034.vdocuments.us/reader034/viewer/2022050100/5f3f52c2da4183724326d32b/html5/thumbnails/13.jpg)
Intro Naive RLS RPLR RPEM RML Filter
(Adaptive) RPLR
I Taylor expansion
St(θ) ≈ St(θt−1) +∇St(θt−1)(θ − θt−1)
+1
2(θ − θt−1)THt(θt−1)(θ − θt−1), (4)
where Ht is the Hessian.
I ∇St(θt−1) ≈ −2Xt(Yt − XTt θt−1)
I Rt = 12Ht = λ(t)Rt−1 + XtX
Tt
I This gives the estimators as
θt = θt−1 + R−1t Xt(Yt − XTt θt−1)
Erik Lindström - [email protected] Recursive estimation
![Page 14: Recursive estimation - Matematikcentrum · IntroNaiveRLSRPLRRPEMRMLFilter SP/FD stochastic approximation I The gradient can be approximated by nite di erence at the cost of slower](https://reader034.vdocuments.us/reader034/viewer/2022050100/5f3f52c2da4183724326d32b/html5/thumbnails/14.jpg)
Intro Naive RLS RPLR RPEM RML Filter
(Adaptive) RPEM
Letθt = argmin St(θ)
where
St(θ) =t∑s
β(t, s)(Ys − Ys|s−1(θ))2
I Approximate by a 2nd order polynomial
I Optimize using Newton-Raphson
Erik Lindström - [email protected] Recursive estimation
![Page 15: Recursive estimation - Matematikcentrum · IntroNaiveRLSRPLRRPEMRMLFilter SP/FD stochastic approximation I The gradient can be approximated by nite di erence at the cost of slower](https://reader034.vdocuments.us/reader034/viewer/2022050100/5f3f52c2da4183724326d32b/html5/thumbnails/15.jpg)
Intro Naive RLS RPLR RPEM RML Filter
(Adaptive) RPEM
I Taylor expansion
St(θ) ≈ St(θt−1) +∇St(θt−1)(θ − θt−1)
+1
2(θ − θt−1)THt(θt−1)(θ − θt−1), (5)
where Ht is the Hessian.
I Solution is given by
θt = θt−1 − Ht(θt−1)∇St(θt−1)
Erik Lindström - [email protected] Recursive estimation
![Page 16: Recursive estimation - Matematikcentrum · IntroNaiveRLSRPLRRPEMRMLFilter SP/FD stochastic approximation I The gradient can be approximated by nite di erence at the cost of slower](https://reader034.vdocuments.us/reader034/viewer/2022050100/5f3f52c2da4183724326d32b/html5/thumbnails/16.jpg)
Intro Naive RLS RPLR RPEM RML Filter
(Adaptive) RPEM
Note that
I St(θ) = λ(t)St−1(θ) + (Yt − Yt|t−1(θ))2
I ∇St(θ) = λ(t)∇St−1(θ) + (Yt − Yt|t−1(θ))∇Yt|t−1(θ)
I ∇St(θt−1) ≈ (Yt − Yt|t−1(θt−1))∇Yt|t−1(θt−1)
I The Hessian is given by
Ht(θ) =2∑
β(t, s)∇Ys|s−1(θ)∇Y Ts|s−1(θ)
−2∑
β(t, s)∇∇Ys|s−1(θ)(Ys − Ys|s−1(θ)) (6)
I Ht(θt−1) ≈ λ(t)Ht−1 + 2∇Yt|t−1(θt−1)∇Y Tt|t−1(θt−1)
Erik Lindström - [email protected] Recursive estimation
![Page 17: Recursive estimation - Matematikcentrum · IntroNaiveRLSRPLRRPEMRMLFilter SP/FD stochastic approximation I The gradient can be approximated by nite di erence at the cost of slower](https://reader034.vdocuments.us/reader034/viewer/2022050100/5f3f52c2da4183724326d32b/html5/thumbnails/17.jpg)
Intro Naive RLS RPLR RPEM RML Filter
(Adaptive) RPEM
This gives
I Rt = 12Ht
I θt = θt−1 + Rt(θt−1)(Yt − Yt|t−1(θt−1))∇Yt|t−1(θt−1)
I Rt = λ(t)Rt−1 +∇Yt|t−1(θt−1)∇Y Tt|t−1(θt−1)
Use matrix inversion lemma to obtain an e�cient recursion.
Erik Lindström - [email protected] Recursive estimation
![Page 18: Recursive estimation - Matematikcentrum · IntroNaiveRLSRPLRRPEMRMLFilter SP/FD stochastic approximation I The gradient can be approximated by nite di erence at the cost of slower](https://reader034.vdocuments.us/reader034/viewer/2022050100/5f3f52c2da4183724326d32b/html5/thumbnails/18.jpg)
Intro Naive RLS RPLR RPEM RML Filter
Recursive ML
It is possible to construct recursive estimators for non-Gaussianmodels
I θt = argmax∑t
n=1 log p(yn|y1:n−1, θ) = argmax `t(θ)
Taylor expand and maximize
∇`t(θt) ≈ ∇`t(θt−1) +∇∇`t(θt−1)(θt − θt−1) (7)
= ∇`t−1(θt−1) +∇ log p(yn|y1:n−1, θt−1) (8)
+∇∇`t(θt−1)(θt − θt−1) = 0. (9)
Simpli�cation gives
θt = θt−1 +1
tI (θt−1)−1∇ log p(yn|y1:n−1, θt−1)
Erik Lindström - [email protected] Recursive estimation
![Page 19: Recursive estimation - Matematikcentrum · IntroNaiveRLSRPLRRPEMRMLFilter SP/FD stochastic approximation I The gradient can be approximated by nite di erence at the cost of slower](https://reader034.vdocuments.us/reader034/viewer/2022050100/5f3f52c2da4183724326d32b/html5/thumbnails/19.jpg)
Intro Naive RLS RPLR RPEM RML Filter
Robbins-Monro stochastic approximation
I This is a special case of the Robbins-Monro stochasticapproximation algorithm
I Problem: x? = argminG (x)
I Introduce xn+1 = xn + a(1+n+A)α g(xn)
I where x is a parameter, a some positive def. matrix, g(x) is anoisy gradient of G and α ∈ (.5, 1].
I It then holds that
xna.s.→ x? (10)
Nα/2(xn − x?)d→ N(0,Σ) (11)
Interpretations
Erik Lindström - [email protected] Recursive estimation
![Page 20: Recursive estimation - Matematikcentrum · IntroNaiveRLSRPLRRPEMRMLFilter SP/FD stochastic approximation I The gradient can be approximated by nite di erence at the cost of slower](https://reader034.vdocuments.us/reader034/viewer/2022050100/5f3f52c2da4183724326d32b/html5/thumbnails/20.jpg)
Intro Naive RLS RPLR RPEM RML Filter
SP/FD stochastic approximation
I The gradient can be approximated by �nite di�erence at thecost of slower convergence.
I but clever methods (SPSA) is still fairly fast
I Idea: Many steps are taken, and the gradient is being averagedover the iterations.
I SPSA only evaluates a single central �nite di�erence (randomlyselected) per iteration and averages again over the iterations.
Result: Computational gain is asymp. equal to the dimension of x(which can be huge).
Erik Lindström - [email protected] Recursive estimation
![Page 21: Recursive estimation - Matematikcentrum · IntroNaiveRLSRPLRRPEMRMLFilter SP/FD stochastic approximation I The gradient can be approximated by nite di erence at the cost of slower](https://reader034.vdocuments.us/reader034/viewer/2022050100/5f3f52c2da4183724326d32b/html5/thumbnails/21.jpg)
Intro Naive RLS RPLR RPEM RML Filter
Filtering
I Recursive estimation using non-linear �lters
I Augment
xn+1 = f (xn) + en+1 (12)
yn+1 = h(xn+1) + wn+1 (13)
I to (xn+1
θn+1
)=
(f (xn)θn
)+
(exn+1
eθn+1
)(14)
yn+1 = h(xn+1, θn+1) + wn+1 (15)
Estimation "trivial", cf. computer exercise 2 and slides on stochapprox.
Erik Lindström - [email protected] Recursive estimation
![Page 22: Recursive estimation - Matematikcentrum · IntroNaiveRLSRPLRRPEMRMLFilter SP/FD stochastic approximation I The gradient can be approximated by nite di erence at the cost of slower](https://reader034.vdocuments.us/reader034/viewer/2022050100/5f3f52c2da4183724326d32b/html5/thumbnails/22.jpg)
Intro Naive RLS RPLR RPEM RML Filter
Consistent estimates in the �ltering setup
I Estimate is often biased.
I Idea. Let Var [eθ]→ 0
I Formalized in the 'iterated �ltering framework'
I Can show consistency θn+1 → θ0
Erik Lindström - [email protected] Recursive estimation