time series analysis -- an introduction -- ams 586zhu/ams586/time_series_introduction.pdf · time...
TRANSCRIPT
![Page 1: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/1.jpg)
Time Series Analysis -- An Introduction --
1
AMS 586
![Page 2: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/2.jpg)
Objectives of time series analysis
Data description
Data interpretation
Modeling
Control
Prediction & Forecasting
2
![Page 3: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/3.jpg)
Time-Series Data
• Numerical data obtained at regular time intervals
• The time intervals can be annually, quarterly, monthly, weekly, daily, hourly, etc.
• Example:
Year: 2005 2006 2007 2008 2009
Sales: 75.3 74.2 78.5 79.7 80.2
3
![Page 4: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/4.jpg)
Time Plot
• the vertical axis measures the variable of interest
• the horizontal axis corresponds to the time periods
U.S. Inflation Rate
0.00
2.00
4.00
6.00
8.00
10.00
12.00
14.00
16.00
19
75
19
77
19
79
19
81
19
83
19
85
19
87
19
89
19
91
19
93
19
95
19
97
19
99
20
01
Year
Infl
ati
on
Rate
(%
)
A time-series plot (time plot) is a two-dimensional plot of time series data
4
![Page 5: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/5.jpg)
Time-Series Components
Time Series
Cyclical Component
Irregular /Random Component
Trend Component Seasonal Component
Overall, persistent, long-term movement
Regular periodic fluctuations,
usually within a 12-month period
Repeating swings or movements over more than
one year
Erratic or residual fluctuations
5
![Page 6: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/6.jpg)
Trend Component
• Long-run increase or decrease over time (overall upward or downward movement)
• Data taken over a long period of time
Sales
Time 6
![Page 7: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/7.jpg)
Downward linear trend
Trend Component
• Trend can be upward or downward
• Trend can be linear or non-linear
Sales
Time Upward nonlinear trend
Sales
Time
(continued)
7
![Page 8: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/8.jpg)
Seasonal Component
• Short-term regular wave-like patterns • Observed within 1 year • Often monthly or quarterly
Sales
Time (Quarterly)
Winter
Spring
Summer
Fall
Winter
Spring
Summer
Fall
8
![Page 9: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/9.jpg)
Cyclical Component
• Long-term wave-like patterns
• Regularly occur but may vary in length
• Often measured peak to peak or trough to trough
Sales
1 Cycle
Year 9
![Page 10: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/10.jpg)
Irregular/Random Component
• Unpredictable, random, “residual” fluctuations
• “Noise” in the time series
• The truly irregular component may not be estimated – however, the more predictable random component can be estimated – and is usually the emphasis of time series analysis via the usual stationary time series models such as AR, MA, ARMA etc after we filter out the trend, seasonal and other cyclical components
10
![Page 11: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/11.jpg)
Two simplified time series models
• In the following, we present two classes of simplified time series models 1. Non-seasonal Model with Trend 2. Classical Decomposition Model with Trend and
Seasonal Components
• The usual procedure is to first filter out the trend and seasonal component – then fit the random component with a stationary time series model to capture the correlation structure in the time series
• If necessary, the entire time series (with seasonal, trend, and random components) can be re-analyzed for better estimation, modeling and prediction.
11
![Page 12: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/12.jpg)
odelsMeasonal s-Non with Trend
trend Stochastic process
random noise
Xt = mt + Yt
12
![Page 13: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/13.jpg)
odelMecomposition DClassical with Trend and Season
seasonal component
trend Stochastic process
random noise
Xt = mt + st + Yt
13
![Page 14: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/14.jpg)
odelsMeasonal s-Non with Trend
14
There are two basic methods for estimating/eliminating trend:
Method 1: Trend estimation (first we estimate the trend either by moving average smoothing or regression analysis – then we remove it)
Method 2: Trend elimination by differencing
![Page 15: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/15.jpg)
Method 1: Trend Estimation by Regression Analysis
Estimate a trend line using regression analysis
Year
Time Period
(t)
Sales (X)
2004
2005
2006
2007
2008
2009
0
1
2
3
4
5
20
40
30
50
70
65
Use time (t) as the independent variable:
In least squares linear, non-linear, and exponential modeling, time periods are numbered starting with 0 and increasing by 1 for each time period.
15
![Page 16: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/16.jpg)
Least Squares Regression
The estimated linear trend equation is:
Sales trend
0
10
20
30
40
50
60
70
80
0 1 2 3 4 5 6
Year
sale
s
Year
Time
Period
(t)
Sales
(X)
2004
2005
2006
2007
2008
2009
0
1
2
3
4
5
20
40
30
50
70
65
Without knowing the exact time series random error correlation structure, one often resorts to the ordinary least squares regression method, not optimal but practical.
16
![Page 17: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/17.jpg)
Linear Trend Forecasting
• Forecast for time period 6 (2010):
Year
Time
Period
(t)
Sales
(X)
2004
2005
2006
2007
2008
2009
2010
0
1
2
3
4
5
6
20
40
30
50
70
65
??
One can even performs trend forecasting at this point – but bear in mind that the forecasting may not be optimal.
Sales trend
0
10
20
30
40
50
60
70
80
0 1 2 3 4 5 6
Year
sale
s
17
![Page 18: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/18.jpg)
Method 2: Trend Elimination by Differencing
18
![Page 19: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/19.jpg)
Trend Elimination by Differencing
19
If the operator ∇ is applied to a linear trend function:
Then we obtain the constant function:
In the same way any polynomial trend of degree k can be removed by the operator:
![Page 20: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/20.jpg)
odelMecomposition DClassical (Seasonal Model)
with trend and season
20
where
![Page 21: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/21.jpg)
odel Mecomposition DClassical
21
we remove the seasonal First Differencing:: 2Method component by differencing. We then remove the trend by differencing as well.
can fit a Alternatively, wefit method: -Joint: 3Method combined polynomial linear regression and harmonic functions to estimate and then remove the trend and seasonal component simultaneously as the following:
we estimate and remove the First Filtering:: 1Method trend component by using moving average method; then we estimate and remove the seasonal component by using suitable periodic averages.
![Page 22: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/22.jpg)
Method 1: Filtering
22
(1). We first estimate the trend by the moving average: • If d = 2q (even), we use:
• If d = 2q+1 (odd), we use:
(2). Then we estimate the seasonal component by using the average
, k = 1, …, d, of the de-trended data:
To ensure:
we further subtract the mean of
(3). One can also re-analyze the trend from the de-seasonalized data in order to obtain a polynomial linear regression equation for modeling and prediction purposes.
![Page 23: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/23.jpg)
Method 2: Differencing
23
Define the lag-d differencing operator as:
We can transform a seasonal model to a non-seasonal model:
Differencing method can then be further applied to eliminate the trend component.
![Page 24: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/24.jpg)
Method 3: Joint Modeling
24
As shown before, one can also fit a joint model to analyze both components simultaneously:
![Page 25: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/25.jpg)
Detrended series
P. J. Brockwell, R. A. Davis, Introduction to Time Series and Forecasting, Springer, 1987 25
![Page 26: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/26.jpg)
Time series – Realization of a stochastic process
{Xt} is a stochastic time series if each component takes a value according to a certain probability
distribution function.
A time series model specifies the joint distribution of the sequence of random variables.
26
![Page 27: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/27.jpg)
White noise - example of a time series model
27
![Page 28: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/28.jpg)
Gaussian white noise
28
![Page 29: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/29.jpg)
Stochastic properties of the process
STATIONARITY
.1Once we have removed the seasonal and trend components of a time series (as in the classical decomposition model), the remainder (random) component – the residual, can often be modeled by a stationary time series.
* System does not change its properties in time
* Well-developed analytical methods of signal analysis and stochastic processes
29
![Page 30: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/30.jpg)
WHEN A STOCHASTIC PROCESS IS STATIONARY?
{Xt} is a strictly stationary time series if
f(X1,...,Xn)= f(X1+h,...,Xn+h),
where n1, h – integer
Properties:
* The random variables are identically distributed.
* An idependent identically distributed (iid) sequence is strictly stationary.
30
![Page 31: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/31.jpg)
Weak stationarity
{Xt} is a weakly stationary time series if
• EXt = and Var(Xt) = 2 are independent of time t
• Cov(Xs, Xr) depends on (s-r) only, independent of t
31
![Page 32: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/32.jpg)
Autocorrelation function (ACF)
32
![Page 33: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/33.jpg)
ACF for Gaussian WN
33
![Page 34: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/34.jpg)
ARMA models
Time series is an ARMA(p,q) process if Xt is stationary and if for every t:
Xt 1Xt-1 ... pXt-p= Zt + 1Zt-1 +...+ pZt-p where Zt represents white noise with mean 0 and variance 2
The Left side of the equation represents the Autoregressive AR(p) part, and the right side the Moving Average MA(q) component.
34
![Page 35: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/35.jpg)
Examples
35
![Page 36: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/36.jpg)
Exponential decay of ACF
MA(1) sample ACF
AR(1)
36
![Page 37: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/37.jpg)
More examples of ACF
37
![Page 38: Time Series Analysis -- An Introduction -- AMS 586zhu/ams586/Time_Series_Introduction.pdf · Time Series: Theory and Methods. Springer-Verlag. Brockwell, Peter J. and Davis, Richard](https://reader030.vdocuments.us/reader030/viewer/2022040611/5ed9187a6714ca7f4769240a/html5/thumbnails/38.jpg)
Reference
38
Box, George and Jenkins, Gwilym (1970) Time series analysis: Forecasting and control, San Francisco: Holden-Day. Brockwell, Peter J. and Davis, Richard A. (1991). Time Series: Theory and Methods. Springer-Verlag.
Brockwell, Peter J. and Davis, Richard A. (1987, 2002). Introduction to Time Series and Forecasting. Springer.
We also thank various on-line open resources for time series analysis.