prediction of estsp competition time series by unscented ...ssarkka/pub/estsp-kalvot.pdfab optimal...

AB

Optimal Filtering and SmoothingTime Series Model

Estimation of Parameters and Prediction ResultSummary

Prediction of ESTSP Competition TimeSeries by Unscented Kalman Filter and RTS

Smoother

Simo Särkkä Aki Vehtari

Jouko Lampinen

Laboratory of Computational EngineeringHelsinki University of Technology, Finland

February 7, 2007

Särkkä et al. Prediction of ESTSP Competition Time Series by UKF and . . .

AB

Contents

1 Optimal Filtering and SmoothingState Space ModelsOptimal Filters and PredictorsOptimal Smoothers

2 Time Series ModelIdea of the ApproachBias and Periodic ComponentsNon-linear Correction TermAutocorrelation Compensation

3 Estimation of Parameters and Prediction ResultEstimation of ParametersFinal Prediction Result

4 Summary

AB

Contents

4 Summary

AB

Contents

4 Summary

AB

Contents

4 Summary

AB

State Space ModelsOptimal Filters and PredictorsOptimal Smoothers

State Space Model

State space model with state xk and measurements yk :

xk = f(xk−1,qk−1)yk = h(xk , rk ),

where qk−1 is the process noise and rk is themeasurement noise.The state xk is the hidden internal dynamic state of thesystem on the time step kThe measurements yk model the output of the systemWe want to estimate the state from the measurements anduse it for model based prediction of the time series

AB

State Space Model

AB

State Space Model

AB

State Space Model

AB

Optimal Filters and Predictors

Given measurements y1, . . . ,yT optimal filter producesMMSE optimal online estimate:

x̂(tk ) = E(x(tk ) |y1, . . . ,yk ).

for each k = 1, . . . ,T .Can be also used for computing the optimal predictions:

x̂(t) = E(x(t) |y1, . . . ,yk ).

for t > tk .If the state space model is linear (i.e., Gaussian process),then Kalman filter provides the optimal solutionIf it is non-linear, then unscented Kalman filter can be usedfor approximating the optimal solution

AB

x̂(tk ) = E(x(tk ) |y1, . . . ,yk ).

x̂(t) = E(x(t) |y1, . . . ,yk ).

AB

x̂(tk ) = E(x(tk ) |y1, . . . ,yk ).

x̂(t) = E(x(t) |y1, . . . ,yk ).

AB

x̂(tk ) = E(x(tk ) |y1, . . . ,yk ).

x̂(t) = E(x(t) |y1, . . . ,yk ).

AB

Optimal Smoothers

Optimal smoother produces the optimal batch estimate:

x̂(tk ) = E(x(tk ) |y1, . . . ,yT ).

If the dynamic model is linear, then Rauch-Tung-Striebel(RTS) smoother provides the optimal solutionApproximate smoothers for nonlinear problems exists alsoIn this article, the dynamic model is linear (i.e., Gaussianprocess) given the parameters and thus the linear RTSsmoother suffices

AB

Optimal Smoothers

x̂(tk ) = E(x(tk ) |y1, . . . ,yT ).

AB

Optimal Smoothers

x̂(tk ) = E(x(tk ) |y1, . . . ,yT ).

AB

Optimal Smoothers

x̂(tk ) = E(x(tk ) |y1, . . . ,yT ).

AB

Idea of the ApproachBias and Periodic ComponentsNon-linear Correction TermAutocorrelation Compensation

Idea of the Approach

Model the time series as consisting of periodic and biascomponents with a linear state space modelModel the signal-residual dependence by including anon-linear correction term into the modelModel the remaining residual autocorrelation with anautoregressive (AR) modelEstimate the parameters and predict the time series withthe model using the unscented Kalman filter (UKF) andRauch-Tung-Striebel (RTS) smoother as the numericalmethods

AB

Bias and Periodic Components 1/3

0 100 200 300 400 500 600 700 800 90018

20

22

24

26

28

30

Signal model:

yk = xb(k)+xr (k)+rk

Bias component xb(t) ismodeled as integral of a whitenoise process wb(t)

dxbdt

= wb(t)

Periodic component xr (t) ismodeled as a white noisedriven stochastic resonator

d2xrdt2

= −ω2 xr + wr (t)

AB

0 100 200 300 400 500 600 700 800 90018

20

22

24

26

28

30

Signal model:

dxbdt

= wb(t)

d2xrdt2

= −ω2 xr + wr (t)

AB

0 100 200 300 400 500 600 700 800 90018

20

22

24

26

28

30

Signal model:

dxbdt

= wb(t)

d2xrdt2

= −ω2 xr + wr (t)

AB

Can be written as discretely measured continuous-timevector process x(t) = (xb(t) xr (t) dxr (t)/dt)T as follows:

dxdt

= F x(t) + L w(t)

Linear system theory⇒ equivalent discrete time model:

xk = A xk−1 + qk−1

Measurement model is of the form

yk = H xk + rk

Linear state space model⇒ Kalman filter can be applied.

AB

dxdt

= F x(t) + L w(t)

xk = A xk−1 + qk−1

yk = H xk + rk

AB

dxdt

= F x(t) + L w(t)

xk = A xk−1 + qk−1

yk = H xk + rk

AB

dxdt

= F x(t) + L w(t)

xk = A xk−1 + qk−1

yk = H xk + rk

AB

Finding the parameters that minimize the 50 step predictionerror, results in the following kind of prediction:

100 200 300 400 500 600 700 800 90018

20

22

24

26

28

30PredictionBiasMeasurement

AB

Non-linear Correction Term

−4 −3 −2 −1 0 1 2 3 4−4

−3

−2

−1

0

1

2

3

4

5

Resonator state

Resid

ual

Residual and periodiccomponent still have anon-linear relationship5th degree polynomial gives asuitable fitMeasurement model becomesnon-linear

yk = xb(r)+5∑

i=0

ci(

xr (k))i

+ rk

Use unscented Kalman filter(UKF) instead of Kalman filter

AB

−4 −3 −2 −1 0 1 2 3 4−4

−3

−2

−1

0

1

2

3

4

5

Resonator state

Resid

ual

yk = xb(r)+5∑

i=0

ci(

xr (k))i

+ rk

AB

−4 −3 −2 −1 0 1 2 3 4−4

−3

−2

−1

0

1

2

3

4

5

Resonator state

Resid

ual

yk = xb(r)+5∑

i=0

ci(

xr (k))i

+ rk

AB

−4 −3 −2 −1 0 1 2 3 4−4

−3

−2

−1

0

1

2

3

4

5

Resonator state

Resid

ual

yk = xb(r)+5∑

i=0

ci(

xr (k))i

+ rk

AB

Autocorrelation Compensation

0 10 20 30 40 50 60 70 80−0.2

0

0.2

0.4

0.6

0.8

1

Residual has non-deltaautocorrelation, indicating aperiodic componentCan be modeled as secondorder AR-model:

ek =2∑

i=1

ai ek−i + rark

Can be estimated withRauch-Tung-Striebel smoother

AB

0 10 20 30 40 50 60 70 80−0.2

0

0.2

0.4

0.6

0.8

1

ek =2∑

i=1

ai ek−i + rark

AB

0 10 20 30 40 50 60 70 80−0.2

0

0.2

0.4

0.6

0.8

1

ek =2∑

i=1

ai ek−i + rark

AB

Estimation of ParametersFinal Prediction Result

Estimation of parameters

The unknown parameters are the spectral densities ofprocess noises and the angular velocity of the resonatorForm a discrete grid of sensible parameter valuesEvaluate parameters by computing 50 step predictionerrors in known parts of time seriesFirst find roughly the location of minimum and form densergrid on that areaFind the final smoothed estimate of the time series andmake final prediction with the parameter values giving theminimum error

AB

Prediction

The final estimate of the signal and the prediction result:

100 200 300 400 500 600 700 800 90018

20

22

24

26

28

30PredictionBiasMeasurement

AB

Summary

First the bias and periodic components are modeled aslinear state space modelNon-linear dependence between residual and periodiccomponent is modeled with 5th degree polynomialRemaining autocorrelation is modeled with second orderautoregressive (AR) modelThe estimation and prediction is done with unscentedKalman filter and Rauch-Tung-Striebel smootherQuite classical model based (Bayesian) approach, wherethe uncertainties are modeled as stochastic processes

AB

Summary

AB

Summary

AB

Summary

AB

Summary

Optimal Filtering and SmoothingState Space ModelsOptimal Filters and PredictorsOptimal Smoothers

Time Series ModelIdea of the ApproachBias and Periodic ComponentsNon-linear Correction TermAutocorrelation Compensation

Estimation of Parameters and Prediction ResultEstimation of ParametersFinal Prediction Result

Summary

prediction of estsp competition time series by unscented ...ssarkka/pub/estsp-kalvot.pdfab optimal...

Documents