marcel dettling -...
TRANSCRIPT
![Page 1: Marcel Dettling - stat.ethz.chstat.ethz.ch/education/semesters/ss2011/atsa/ATSA-FS11-Slides-We… · FS 2011 – Week 11 Gas Furnace: Final Remarks • In the previous slide, we see](https://reader033.vdocuments.us/reader033/viewer/2022052021/6035408c77296246936fce74/html5/thumbnails/1.jpg)
1Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
Marcel DettlingInstitute for Data Analysis and Process Design
Zurich University of Applied Sciences
http://stat.ethz.ch/~dettling
ETH Zürich, May 9, 2011
![Page 2: Marcel Dettling - stat.ethz.chstat.ethz.ch/education/semesters/ss2011/atsa/ATSA-FS11-Slides-We… · FS 2011 – Week 11 Gas Furnace: Final Remarks • In the previous slide, we see](https://reader033.vdocuments.us/reader033/viewer/2022052021/6035408c77296246936fce74/html5/thumbnails/2.jpg)
2Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
Prewhitening the Gas Furnace DataWhat to do:
- AR(p)-models are fitted to the differenced series
- The residual time series are Ut and Vt, white noise
- Check the sample cross correlation (see next slide)
- Model the output as a linear combination of pastinput values: transfer function model.
![Page 3: Marcel Dettling - stat.ethz.chstat.ethz.ch/education/semesters/ss2011/atsa/ATSA-FS11-Slides-We… · FS 2011 – Week 11 Gas Furnace: Final Remarks • In the previous slide, we see](https://reader033.vdocuments.us/reader033/viewer/2022052021/6035408c77296246936fce74/html5/thumbnails/3.jpg)
3Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
Prewhitening the Gas Furnace Data
0 5 10 15 20
-0.5
0.0
0.5
1.0
Lag
AC
F
PrW.Out
0 5 10 15 20
-0.5
0.0
0.5
1.0
Lag
PrW.Out & PrW.Inp
-20 -15 -10 -5 0
-0.5
0.0
0.5
1.0
Lag
AC
F
PrW.Inp & PrW.Out
0 5 10 15 20
-0.5
0.0
0.5
1.0
Lag
PrW.Inp
![Page 4: Marcel Dettling - stat.ethz.chstat.ethz.ch/education/semesters/ss2011/atsa/ATSA-FS11-Slides-We… · FS 2011 – Week 11 Gas Furnace: Final Remarks • In the previous slide, we see](https://reader033.vdocuments.us/reader033/viewer/2022052021/6035408c77296246936fce74/html5/thumbnails/4.jpg)
4Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
Transfer Function ModelsProperties:
- Transfer function models are an option to describe the dependency between two time series.
- The first (input) series influences the second (output) one, but there is no feedback from output to input.
- The influence from input to output only goes „forward“.
The model is:
2, 2 1, 10
( )t j t j tj
X X E
![Page 5: Marcel Dettling - stat.ethz.chstat.ethz.ch/education/semesters/ss2011/atsa/ATSA-FS11-Slides-We… · FS 2011 – Week 11 Gas Furnace: Final Remarks • In the previous slide, we see](https://reader033.vdocuments.us/reader033/viewer/2022052021/6035408c77296246936fce74/html5/thumbnails/5.jpg)
5Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
Transfer Function ModelsThe model is:
- E[Et]=0.
- Et and X1,s are uncorrelated for all t and s.
- Et and Es are usually correlated.
- For simplicity of notation, we here assume that the series have been mean centered.
2, 2 1, 10
( )t j t j tj
X X E
![Page 6: Marcel Dettling - stat.ethz.chstat.ethz.ch/education/semesters/ss2011/atsa/ATSA-FS11-Slides-We… · FS 2011 – Week 11 Gas Furnace: Final Remarks • In the previous slide, we see](https://reader033.vdocuments.us/reader033/viewer/2022052021/6035408c77296246936fce74/html5/thumbnails/6.jpg)
6Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
Cross CovarianceWhen plugging-in, we obtain for the cross covariance:
- If only finitely many coefficients are different from zero, we could generate a linear equation system, plug-inand to obtain the estimates .
This is not a statistically efficient estimation method.
21 2, 1, 1, 1, 110 0
( ) ( , ) , ( )t k t j t k j t jj j
k Cov X X Cov X X k j
1̂21̂ ˆ j
![Page 7: Marcel Dettling - stat.ethz.chstat.ethz.ch/education/semesters/ss2011/atsa/ATSA-FS11-Slides-We… · FS 2011 – Week 11 Gas Furnace: Final Remarks • In the previous slide, we see](https://reader033.vdocuments.us/reader033/viewer/2022052021/6035408c77296246936fce74/html5/thumbnails/7.jpg)
7Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
Special Case: X1,t UncorrelatedIf X1,t was an uncorrelated series, we would obtain thecoefficients of the transfer function model quite easily:
However, this is usually not the case. We can then:
- transform all series in a clever way- the transfer function model has identical coefficients- the new, transformed input series is uncorrelated
see blackboard for the transformation…
21
11
( )(0)kk
![Page 8: Marcel Dettling - stat.ethz.chstat.ethz.ch/education/semesters/ss2011/atsa/ATSA-FS11-Slides-We… · FS 2011 – Week 11 Gas Furnace: Final Remarks • In the previous slide, we see](https://reader033.vdocuments.us/reader033/viewer/2022052021/6035408c77296246936fce74/html5/thumbnails/8.jpg)
8Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
Gas Furnace Transformed
0 5 10 15 20
-0.5
0.0
0.5
1.0
Lag
AC
F
pr.Out
0 5 10 15 20
-0.5
0.0
0.5
1.0
Lag
pr.Out & pr.Inp
-20 -15 -10 -5 0
-0.5
0.0
0.5
1.0
Lag
AC
F
pr.Inp & pr.Out
0 5 10 15 20
-0.5
0.0
0.5
1.0
Lag
pr.Inp
![Page 9: Marcel Dettling - stat.ethz.chstat.ethz.ch/education/semesters/ss2011/atsa/ATSA-FS11-Slides-We… · FS 2011 – Week 11 Gas Furnace: Final Remarks • In the previous slide, we see](https://reader033.vdocuments.us/reader033/viewer/2022052021/6035408c77296246936fce74/html5/thumbnails/9.jpg)
9Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
Gas Furnace: Final Remarks• In the previous slide, we see the empirical cross correlations
of the two series and .
• The coefficients from the transfer function model will beproportional to the empirical cross correlations. We can al-ready now conjecture that the output is delayed by 3-7 times, i.e. 27-63 seconds.
• The formula for the transfer function model coefficients is:
21ˆˆ ˆ ( )ˆ
Zk
D
k
21ˆ ( )k tD tZ
ˆk
![Page 10: Marcel Dettling - stat.ethz.chstat.ethz.ch/education/semesters/ss2011/atsa/ATSA-FS11-Slides-We… · FS 2011 – Week 11 Gas Furnace: Final Remarks • In the previous slide, we see](https://reader033.vdocuments.us/reader033/viewer/2022052021/6035408c77296246936fce74/html5/thumbnails/10.jpg)
10Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
State Space ModelsBasic idea: There is a stochastic process/time series
which we cannot directly observe, but only under the addition of some white noise.
Thus: We observe the time series
where , i.i.d.
Example: = # of fish in a lake= # estimated number of fish from a sample
tZ
t t tY Z U 2~ (0, )tU N
tZtY
![Page 11: Marcel Dettling - stat.ethz.chstat.ethz.ch/education/semesters/ss2011/atsa/ATSA-FS11-Slides-We… · FS 2011 – Week 11 Gas Furnace: Final Remarks • In the previous slide, we see](https://reader033.vdocuments.us/reader033/viewer/2022052021/6035408c77296246936fce74/html5/thumbnails/11.jpg)
11Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
Notation and Terminology for an AR(1)We assume that the true underlying process is an AR(1), i.e.
,
where
are i.i.d. innovations, „process noise“.
In practice, we only observe , as realizations of the process
, with , i.i.d.
and additionally, the are independent of , for all s,t, thus they are independent „observation white noise“.
1 1t t tZ a Z E
t t tY Z U 2~ (0, )tU N
ty
2~ (0, )t EE N
tU sEsZ
![Page 12: Marcel Dettling - stat.ethz.chstat.ethz.ch/education/semesters/ss2011/atsa/ATSA-FS11-Slides-We… · FS 2011 – Week 11 Gas Furnace: Final Remarks • In the previous slide, we see](https://reader033.vdocuments.us/reader033/viewer/2022052021/6035408c77296246936fce74/html5/thumbnails/12.jpg)
12Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
More TerminologyWe call
the „state equation“, and
the „observation equation“.
On top of that, we remember once again that the „process noise“ is an innovation that affects all future values and thus also , whereas only influences the currentobservation , but no future ones.
1 1t t tZ a Z E
t t tY Z U
tYtU
tE t kZ
t kY
![Page 13: Marcel Dettling - stat.ethz.chstat.ethz.ch/education/semesters/ss2011/atsa/ATSA-FS11-Slides-We… · FS 2011 – Week 11 Gas Furnace: Final Remarks • In the previous slide, we see](https://reader033.vdocuments.us/reader033/viewer/2022052021/6035408c77296246936fce74/html5/thumbnails/13.jpg)
13Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
AR(1)-Example with α=0.7
Time
yt1
0 20 40 60 80 100
-2-1
01
2
Zustands-Prozess X_tBeobachtungs-Prozess Y_t
AR(1)-Example
![Page 14: Marcel Dettling - stat.ethz.chstat.ethz.ch/education/semesters/ss2011/atsa/ATSA-FS11-Slides-We… · FS 2011 – Week 11 Gas Furnace: Final Remarks • In the previous slide, we see](https://reader033.vdocuments.us/reader033/viewer/2022052021/6035408c77296246936fce74/html5/thumbnails/14.jpg)
14Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
ACF/PACF of Zt
Time
serie
s
0 20 40 60 80 100
-1.0
-0.5
0.0
0.5
1.0
-0.2
0.4
1.0
Lag k
Aut
o-K
orr.
0 5 10 15 20
-0.2
0.2
0.6
Lag k
part.
Aut
okor
r
1 5 10 15 20
![Page 15: Marcel Dettling - stat.ethz.chstat.ethz.ch/education/semesters/ss2011/atsa/ATSA-FS11-Slides-We… · FS 2011 – Week 11 Gas Furnace: Final Remarks • In the previous slide, we see](https://reader033.vdocuments.us/reader033/viewer/2022052021/6035408c77296246936fce74/html5/thumbnails/15.jpg)
15Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
ACF/PACF of Yt
Time
serie
s
0 20 40 60 80 100
-2-1
01
2-0
.20.
41.
0
Lag k
Aut
o-K
orr.
0 5 10 15 20
-0.2
0.1
Lag k
part.
Aut
okor
r
1 5 10 15 20
![Page 16: Marcel Dettling - stat.ethz.chstat.ethz.ch/education/semesters/ss2011/atsa/ATSA-FS11-Slides-We… · FS 2011 – Week 11 Gas Furnace: Final Remarks • In the previous slide, we see](https://reader033.vdocuments.us/reader033/viewer/2022052021/6035408c77296246936fce74/html5/thumbnails/16.jpg)
16Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
AR(1)-Example with α=1
Time
yt1
0 20 40 60 80 100
-20
24
6
Zustands-Prozess X_tBeobachtungs-Prozess Y_t
AR(1)-Example
![Page 17: Marcel Dettling - stat.ethz.chstat.ethz.ch/education/semesters/ss2011/atsa/ATSA-FS11-Slides-We… · FS 2011 – Week 11 Gas Furnace: Final Remarks • In the previous slide, we see](https://reader033.vdocuments.us/reader033/viewer/2022052021/6035408c77296246936fce74/html5/thumbnails/17.jpg)
17Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
ACF/PACF of Zt
Time
serie
s
0 20 40 60 80 100
-10
12
34
5-0
.20.
41.
0
Lag k
Aut
o-K
orr.
0 5 10 15 20
-0.2
0.4
1.0
Lag k
part.
Aut
okor
r
1 5 10 15 20
![Page 18: Marcel Dettling - stat.ethz.chstat.ethz.ch/education/semesters/ss2011/atsa/ATSA-FS11-Slides-We… · FS 2011 – Week 11 Gas Furnace: Final Remarks • In the previous slide, we see](https://reader033.vdocuments.us/reader033/viewer/2022052021/6035408c77296246936fce74/html5/thumbnails/18.jpg)
18Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
What is the goal?The goal of State Space Modeling/Kalman Filtering is:
To uncover the „de-noised“ process Zt from theobserved process Yt.
• The algorithm of Kalman Filtering works with non-stationary time series, too.
• The algorithm is based on a maximum-likelihood-principle where one assume normal distortions.
• There are extensions to multi-dimensional state spacemodels. This is partly discussed in the exercises.
![Page 19: Marcel Dettling - stat.ethz.chstat.ethz.ch/education/semesters/ss2011/atsa/ATSA-FS11-Slides-We… · FS 2011 – Week 11 Gas Furnace: Final Remarks • In the previous slide, we see](https://reader033.vdocuments.us/reader033/viewer/2022052021/6035408c77296246936fce74/html5/thumbnails/19.jpg)
19Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
Summary of Kalman FilteringSummary:
1) The Kalman Filter is a recursive algorithm
2) It relies on an update idea, i.e. we update theforecast with the difference .
3) The weight of the update is determined by therelation between the process variance and theobservation white noise .
4) This relies on the knowledge of . In practicewe have procedures for simultaneous estimation.
1,ˆ
t tZ 1 1,ˆ( )t t ty Y
2E
22 2, , ,Eg h
![Page 20: Marcel Dettling - stat.ethz.chstat.ethz.ch/education/semesters/ss2011/atsa/ATSA-FS11-Slides-We… · FS 2011 – Week 11 Gas Furnace: Final Remarks • In the previous slide, we see](https://reader033.vdocuments.us/reader033/viewer/2022052021/6035408c77296246936fce74/html5/thumbnails/20.jpg)
20
Applied Time Series AnalysisFS 2011 – Week 11Additional Remarks1) For the recursive approach of Kalman filtering, initial
values are necessary. Their choice is not crucial, theirinfluence cancels out rapidly.
2) The procedures yield forecast and filter intervals: and
3) State space models are a very rich class. Every ARIMA(p,d,q) can be written in state space form, andthe Kalman filter can be used for estimating thecoefficients.
4) We can also use Kalman filtering for smoothing, i.e.providing with T>t.
1, 1,ˆ 1.96t t t tZ R 1, 1 1, 1
ˆ 1.96t t t tZ R
1| TtZ Y
![Page 21: Marcel Dettling - stat.ethz.chstat.ethz.ch/education/semesters/ss2011/atsa/ATSA-FS11-Slides-We… · FS 2011 – Week 11 Gas Furnace: Final Remarks • In the previous slide, we see](https://reader033.vdocuments.us/reader033/viewer/2022052021/6035408c77296246936fce74/html5/thumbnails/21.jpg)
21Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
AR(1)-Example with α=0.7
Time
yt1
0 20 40 60 80 100
-2-1
01
2
Zustands-Prozess X_tBeobachtungs-Prozess Y_tFilter-Werte mit 95%-Filter-Intervall
AR(1)-Example with alpha=0.7
![Page 22: Marcel Dettling - stat.ethz.chstat.ethz.ch/education/semesters/ss2011/atsa/ATSA-FS11-Slides-We… · FS 2011 – Week 11 Gas Furnace: Final Remarks • In the previous slide, we see](https://reader033.vdocuments.us/reader033/viewer/2022052021/6035408c77296246936fce74/html5/thumbnails/22.jpg)
22Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
AR(1)-Example with α=1.0
Time
yt2
0 20 40 60 80 100
-3-2
-10
12
3
Zustands-Prozess X_tBeobachtungs-Prozess Y_tFilter-Werte mit 95%-Filter-Intervall
AR(1)-Example with alpha=1.0
![Page 23: Marcel Dettling - stat.ethz.chstat.ethz.ch/education/semesters/ss2011/atsa/ATSA-FS11-Slides-We… · FS 2011 – Week 11 Gas Furnace: Final Remarks • In the previous slide, we see](https://reader033.vdocuments.us/reader033/viewer/2022052021/6035408c77296246936fce74/html5/thumbnails/23.jpg)
23Marcel Dettling, Zurich University of Applied Sciences
Applied Time Series AnalysisFS 2011 – Week 11
AR(1)-Smoothing with α=1.0
Time
yt2
0 20 40 60 80 100
-3-2
-10
12
3
Zustands-Prozess X_tBeobachtungs-Prozess Y_tFilter-Werte mit 95%-Filter-Intervall
Smoothing instead of filtering