introduction statistics of stochastic processes · introduction statistics of stochastic processes...
Post on 26-Aug-2018
243 Views
Preview:
TRANSCRIPT
Introduction
Statistics of stochastic processes
Generally statistics is performed on observations
y1, . . . , yn
assumed to be realizations of independent random variables
Y1, . . . ,Yn.
In statistics of stochastic processes (= time series analysis) we will assume
y1, . . . , yn
to be realizations of a stochastic process
. . .Y1, . . . ,Yn, . . .
with some rules for dependence .
14 settembre 2014 1 / 31
Introduction
Statistics of stochastic processes
Generally statistics is performed on observations
y1, . . . , yn
assumed to be realizations of independent random variables
Y1, . . . ,Yn.
In statistics of stochastic processes (= time series analysis) we will assume
y1, . . . , yn
to be realizations of a stochastic process
. . .Y1, . . . ,Yn, . . .
with some rules for dependence .14 settembre 2014 1 / 31
Introduction
Aims of time series analysis
Drawing inference from available data, but first we need to find anappropriate model.Once a model has been selected:
provide a compact and correct description of data (trend, seasonaland random terms)
adjust data (filtering, missing values) [separating noise from signal]
test hypotheses (increasing trend? influence of factors) → understandcauses
predict future values
Remarks
- Random terms generally described as a stationary process.
- Linear analysis (additive decomposition of trend, seasonaland stationary process)
14 settembre 2014 2 / 31
Introduction
Aims of time series analysis
Drawing inference from available data, but first we need to find anappropriate model.Once a model has been selected:
provide a compact and correct description of data (trend, seasonaland random terms)
adjust data (filtering, missing values) [separating noise from signal]
test hypotheses (increasing trend? influence of factors) → understandcauses
predict future values
Remarks
- Random terms generally described as a stationary process.
- Linear analysis (additive decomposition of trend, seasonaland stationary process)
14 settembre 2014 2 / 31
Introduction
Aims of time series analysis
Drawing inference from available data, but first we need to find anappropriate model.Once a model has been selected:
provide a compact and correct description of data (trend, seasonaland random terms)
adjust data (filtering, missing values) [separating noise from signal]
test hypotheses (increasing trend? influence of factors) → understandcauses
predict future values
Remarks
- Random terms generally described as a stationary process.
- Linear analysis (additive decomposition of trend, seasonaland stationary process)
14 settembre 2014 2 / 31
Introduction
Aims of time series analysis
Drawing inference from available data, but first we need to find anappropriate model.Once a model has been selected:
provide a compact and correct description of data (trend, seasonaland random terms)
adjust data (filtering, missing values) [separating noise from signal]
test hypotheses (increasing trend? influence of factors) → understandcauses
predict future values
Remarks
- Random terms generally described as a stationary process.
- Linear analysis (additive decomposition of trend, seasonaland stationary process)
14 settembre 2014 2 / 31
Introduction
Aims of time series analysis
Drawing inference from available data, but first we need to find anappropriate model.Once a model has been selected:
provide a compact and correct description of data (trend, seasonaland random terms)
adjust data (filtering, missing values) [separating noise from signal]
test hypotheses (increasing trend? influence of factors) → understandcauses
predict future values
Remarks
- Random terms generally described as a stationary process.
- Linear analysis (additive decomposition of trend, seasonaland stationary process)
14 settembre 2014 2 / 31
Introduction
Aims of time series analysis
Drawing inference from available data, but first we need to find anappropriate model.Once a model has been selected:
provide a compact and correct description of data (trend, seasonaland random terms)
adjust data (filtering, missing values) [separating noise from signal]
test hypotheses (increasing trend? influence of factors) → understandcauses
predict future values
Remarks
- Random terms generally described as a stationary process.
- Linear analysis (additive decomposition of trend, seasonaland stationary process)
14 settembre 2014 2 / 31
Introduction
Stationary process
Definition
A stochastic process Xtt∈Z is (strictly) stationary if
the joint distribution of (Xt1 ,Xt2 , . . . ,Xtk ) is equal to
the distribution of (Xt1+h,Xt2+h, . . . ,Xtk+h)
∀ k ∈ N, h ∈ Z, t1, t2, . . . , tk ∈ Z.
In particular, if a stationary stochastic process has finite second moment,then
E(Xt) and Cov(Xt ,Xt+h) do not depend on t.
14 settembre 2014 3 / 31
Introduction
Stationary process. 2
Linear time series analysis looks only at second-order properties. Then
Definition
A stochastic process Xtt∈Z is stationary if it is in L2 and
E(Xt) = µ Cov(Xt ,Xt+h) = γ(h).
If a Gaussian process is stationary, then it is strictly stationary.A Gaussian process is such that all finite-dimensional distributions aremultivariate normal.
14 settembre 2014 4 / 31
Introduction
Stationary process. 2
Linear time series analysis looks only at second-order properties. Then
Definition
A stochastic process Xtt∈Z is stationary if it is in L2 and
E(Xt) = µ Cov(Xt ,Xt+h) = γ(h).
If a Gaussian process is stationary, then it is strictly stationary.
A Gaussian process is such that all finite-dimensional distributions aremultivariate normal.
14 settembre 2014 4 / 31
Introduction
Stationary process. 2
Linear time series analysis looks only at second-order properties. Then
Definition
A stochastic process Xtt∈Z is stationary if it is in L2 and
E(Xt) = µ Cov(Xt ,Xt+h) = γ(h).
If a Gaussian process is stationary, then it is strictly stationary.A Gaussian process is such that all finite-dimensional distributions aremultivariate normal.
14 settembre 2014 4 / 31
Introduction
Reminders on multivariate normal
Definition
Y = (Y1, . . . ,Yn) is multivariate normal if, ∀a ∈ Rn, atY is a univariatenormal.
Equivalently, Y is multivariate normal ⇐⇒ there exists b ∈ Rn A(n ×m) matrix, X = (X1, . . . ,Xm) independent standard normal r.v. suchthat Y = AX + b. =⇒ E(Y ) = b, Cov(Y ) = AAt , i.e. Y ∼ N(b,AAt).
Alternative characterization via characteristic function.
If Cov(Y ) = S positive definite (i.e. invertible), Y ∼ N(µ, S) has density
fY (y) = (2π)−n/2|S |−1/2 exp−(y − µ)tS−1(y − µ)/2.
(non-singular distribution)
14 settembre 2014 5 / 31
Introduction
Reminders on multivariate normal
Definition
Y = (Y1, . . . ,Yn) is multivariate normal if, ∀a ∈ Rn, atY is a univariatenormal.
Equivalently, Y is multivariate normal ⇐⇒ there exists b ∈ Rn A(n ×m) matrix, X = (X1, . . . ,Xm) independent standard normal r.v. suchthat Y = AX + b.
=⇒ E(Y ) = b, Cov(Y ) = AAt , i.e. Y ∼ N(b,AAt).
Alternative characterization via characteristic function.
If Cov(Y ) = S positive definite (i.e. invertible), Y ∼ N(µ, S) has density
fY (y) = (2π)−n/2|S |−1/2 exp−(y − µ)tS−1(y − µ)/2.
(non-singular distribution)
14 settembre 2014 5 / 31
Introduction
Reminders on multivariate normal
Definition
Y = (Y1, . . . ,Yn) is multivariate normal if, ∀a ∈ Rn, atY is a univariatenormal.
Equivalently, Y is multivariate normal ⇐⇒ there exists b ∈ Rn A(n ×m) matrix, X = (X1, . . . ,Xm) independent standard normal r.v. suchthat Y = AX + b. =⇒ E(Y ) = b, Cov(Y ) = AAt , i.e. Y ∼ N(b,AAt).
Alternative characterization via characteristic function.
If Cov(Y ) = S positive definite (i.e. invertible), Y ∼ N(µ, S) has density
fY (y) = (2π)−n/2|S |−1/2 exp−(y − µ)tS−1(y − µ)/2.
(non-singular distribution)
14 settembre 2014 5 / 31
Introduction
Reminders on multivariate normal
Definition
Y = (Y1, . . . ,Yn) is multivariate normal if, ∀a ∈ Rn, atY is a univariatenormal.
Equivalently, Y is multivariate normal ⇐⇒ there exists b ∈ Rn A(n ×m) matrix, X = (X1, . . . ,Xm) independent standard normal r.v. suchthat Y = AX + b. =⇒ E(Y ) = b, Cov(Y ) = AAt , i.e. Y ∼ N(b,AAt).
Alternative characterization via characteristic function.
If Cov(Y ) = S positive definite (i.e. invertible), Y ∼ N(µ, S) has density
fY (y) = (2π)−n/2|S |−1/2 exp−(y − µ)tS−1(y − µ)/2.
(non-singular distribution)
14 settembre 2014 5 / 31
Introduction
Reminders on multivariate normal
Definition
Y = (Y1, . . . ,Yn) is multivariate normal if, ∀a ∈ Rn, atY is a univariatenormal.
Equivalently, Y is multivariate normal ⇐⇒ there exists b ∈ Rn A(n ×m) matrix, X = (X1, . . . ,Xm) independent standard normal r.v. suchthat Y = AX + b. =⇒ E(Y ) = b, Cov(Y ) = AAt , i.e. Y ∼ N(b,AAt).
Alternative characterization via characteristic function.
If Cov(Y ) = S positive definite (i.e. invertible), Y ∼ N(µ, S) has density
fY (y) = (2π)−n/2|S |−1/2 exp−(y − µ)tS−1(y − µ)/2.
(non-singular distribution)
14 settembre 2014 5 / 31
Introduction
Gaussian processes
Definition
A process Xt is Gaussian, if for any n > 0 and any (t1, . . . , tn) the vectorX = (Xt1 , . . . ,Xtn) has a non-singular multivariate normal distribution.
Then let µ = (µt1 , . . . , µtn) = E(X) andCov(X) = Γ = γ(ti , tj), i , j = 1 . . . n. X has density function
g(x , µ, Γ) = (2π)−n/2|Γ|−1/2 exp
−1
2〈Γ−1(x − µ), x − µ〉
.
Xt is (weakly) stationary if µt ≡ µ and γ(ti , tj) = γ(|ti − tj |); then isalso strictly stationary, as the distribution depends only on µ and Γ.
Linear time series analysis is very well suited for Gaussian processes; less sofor non-Gaussian ones.
14 settembre 2014 6 / 31
Introduction
Gaussian processes
Definition
A process Xt is Gaussian, if for any n > 0 and any (t1, . . . , tn) the vectorX = (Xt1 , . . . ,Xtn) has a non-singular multivariate normal distribution.
Then let µ = (µt1 , . . . , µtn) = E(X) andCov(X) = Γ = γ(ti , tj), i , j = 1 . . . n. X has density function
g(x , µ, Γ) = (2π)−n/2|Γ|−1/2 exp
−1
2〈Γ−1(x − µ), x − µ〉
.
Xt is (weakly) stationary if µt ≡ µ and γ(ti , tj) = γ(|ti − tj |); then isalso strictly stationary, as the distribution depends only on µ and Γ.
Linear time series analysis is very well suited for Gaussian processes; less sofor non-Gaussian ones.
14 settembre 2014 6 / 31
Introduction
Gaussian processes
Definition
A process Xt is Gaussian, if for any n > 0 and any (t1, . . . , tn) the vectorX = (Xt1 , . . . ,Xtn) has a non-singular multivariate normal distribution.
Then let µ = (µt1 , . . . , µtn) = E(X) andCov(X) = Γ = γ(ti , tj), i , j = 1 . . . n. X has density function
g(x , µ, Γ) = (2π)−n/2|Γ|−1/2 exp
−1
2〈Γ−1(x − µ), x − µ〉
.
Xt is (weakly) stationary if µt ≡ µ and γ(ti , tj) = γ(|ti − tj |); then isalso strictly stationary, as the distribution depends only on µ and Γ.
Linear time series analysis is very well suited for Gaussian processes; less sofor non-Gaussian ones.
14 settembre 2014 6 / 31
Introduction
Gaussian processes
Definition
A process Xt is Gaussian, if for any n > 0 and any (t1, . . . , tn) the vectorX = (Xt1 , . . . ,Xtn) has a non-singular multivariate normal distribution.
Then let µ = (µt1 , . . . , µtn) = E(X) andCov(X) = Γ = γ(ti , tj), i , j = 1 . . . n. X has density function
g(x , µ, Γ) = (2π)−n/2|Γ|−1/2 exp
−1
2〈Γ−1(x − µ), x − µ〉
.
Xt is (weakly) stationary if µt ≡ µ and γ(ti , tj) = γ(|ti − tj |); then isalso strictly stationary, as the distribution depends only on µ and Γ.
Linear time series analysis is very well suited for Gaussian processes; less sofor non-Gaussian ones.
14 settembre 2014 6 / 31
Introduction
Hilbert spaces
Many time series problems can be solved using Hilbert space theory.Indeed space L2(Ω) is a Hilbert space with
〈X ,Y 〉 = E(XY ), ‖X − Y ‖2 = E(|X − Y |2).
Restricting to the 0-mean subspace 〈X ,Y 〉 = Cov(X ,Y ).
14 settembre 2014 7 / 31
Introduction
Detrending data
Often data do not appeat as arising from stationary processes.
Estimating trend, and then study residuals (differences from trend)
smoothingpolynomial (esp. line) fitting
Study differenced series
In all cases, trasformations may be useful
More systematic model fitting in the future.
14 settembre 2014 8 / 31
Examples of time series
Johnson & Johnson quarterly earnings
J & J
Time
Ear
ning
s pe
r Sha
re
1960 1965 1970 1975 1980
05
1015
data3 points smoothing5 points smoothing
14 settembre 2014 9 / 31
Examples of time series
Johnson & Johnson data: deviations from trend
Time
Dev
iatio
ns fr
om m
ovin
g av
erag
e
1960 1965 1970 1975 1980
-3-2
-10
12
14 settembre 2014 10 / 31
Examples of time series
Johnson & Johnson data: deviations in log-scale
Time
Dev
iatio
ns (i
n lo
g sc
ale)
from
mov
ing
aver
age
1960 1965 1970 1975 1980
-0.4
-0.3
-0.2
-0.1
0.0
0.1
0.2
0.3
14 settembre 2014 11 / 31
Examples of time series
Sunspots data 1700-1980
1700 1750 1800 1850 1900 1950
050
100
150
year
sunspots
14 settembre 2014 12 / 31
Examples of time series
Sunspots data 1700-1980: square-root transformation
1700 1750 1800 1850 1900 1950
02
46
810
1214
year
sunspots
14 settembre 2014 13 / 31
Examples of time series
PanAm international air passengers 1949-60
Time
Pas
seng
ers
(100
0's)
1950 1952 1954 1956 1958 1960
100
200
300
400
500
600
14 settembre 2014 14 / 31
Examples of time series
PanAm yearly data
Annual air passengers
Time
aggregate(AP)
1950 1952 1954 1956 1958 1960
2000
3000
4000
5000
14 settembre 2014 15 / 31
Examples of time series
PanAm monthly variation
1 2 3 4 5 6 7 8 9 10 11 12
100
200
300
400
500
600
Seasonal component in air passengers
Month
14 settembre 2014 16 / 31
Examples of time series
Level of Lake Huron 1875-1972
Level of lake Huron
Time
ft
1880 1900 1920 1940 1960
67
89
1011
12
14 settembre 2014 17 / 31
Examples of time series
Lake Huron level: deviations from trend
Deviations from trend in level of lake Huron
Time
ft
0 20 40 60 80 100
-2-1
01
2
14 settembre 2014 18 / 31
Examples of time series
sales of red wine in Australia 1980-91
Red wine sales in Australia
Time
kilolitres
1980 1982 1984 1986 1988 1990 1992
500
1000
1500
2000
2500
3000
14 settembre 2014 19 / 31
Examples of time series
Deviation from trend in wine sales
Deviations from trend in sales of red wine
Time
kilolitres
0 20 40 60 80 100 120 140
-1000
-500
0500
1000
14 settembre 2014 20 / 31
Examples of time series
PanAm monthly variation
1 2 3 4 5 6 7 8 9 10 11 12
500
1000
1500
2000
2500
3000
Seasonal variation in wine sales (AUS)
14 settembre 2014 21 / 31
Examples of time series
Global temperature data 1856-2005
Global temperature data
Time
Ano
mal
ies
from
196
1-90
mea
n
1900 1950 2000
-1.0
-0.5
0.0
0.5
Monthly averagesYearly averages
14 settembre 2014 22 / 31
Examples of time series
Global temperature: recent years and trend
Global temperatures 1971-2005 (regression line in blue)
Time
Anomalies
1970 1975 1980 1985 1990 1995 2000 2005
-0.4
-0.2
0.0
0.2
0.4
0.6
0.8
14 settembre 2014 23 / 31
Examples of time series
Measles data in England 1944-1967
1945 1950 1955 1960 1965
05000
10000
15000
20000
25000
Measles in England
year
case
s pe
r biw
eek
14 settembre 2014 24 / 31
Examples of time series
EEG data from a subject with epilepsy
0 1000 2000 3000 4000
-150
-50
50150
time (arbitrary unit)
EEG
14 settembre 2014 25 / 31
Examples of time series
De-trend and de-seasonalize (period T = 2q)
yearly average: mt =1
T
(12xt−q +
q−1∑j=−(q−1)
xt+j + 12xt+q
).
seasonal deviation: wk =1
n
n−1∑j=0
(xjT+k −mjT+k) , k = 1 . . .T .
seasonal component: sk = wk − 1T
T∑i=1
wi , k = 1 . . .T .
st = st−[ t−1T ]T , t > T .
deseasonalized data dt = xt − st .
mt trend component on deseasonalized data.
Yt = xt − mt − st random component.
Otherwise, difference data: ∇TXt := Xt − Xt−T .
∇TXt are de-seasonalized; then a trend can be eliminated from these.
14 settembre 2014 26 / 31
Examples of time series
De-trend and de-seasonalize (period T = 2q)
yearly average: mt =1
T
(12xt−q +
q−1∑j=−(q−1)
xt+j + 12xt+q
).
seasonal deviation: wk =1
n
n−1∑j=0
(xjT+k −mjT+k) , k = 1 . . .T .
seasonal component: sk = wk − 1T
T∑i=1
wi , k = 1 . . .T .
st = st−[ t−1T ]T , t > T .
deseasonalized data dt = xt − st .
mt trend component on deseasonalized data.
Yt = xt − mt − st random component.
Otherwise, difference data: ∇TXt := Xt − Xt−T .
∇TXt are de-seasonalized; then a trend can be eliminated from these.
14 settembre 2014 26 / 31
Examples of time series
De-trend and de-seasonalize (period T = 2q)
yearly average: mt =1
T
(12xt−q +
q−1∑j=−(q−1)
xt+j + 12xt+q
).
seasonal deviation: wk =1
n
n−1∑j=0
(xjT+k −mjT+k) , k = 1 . . .T .
seasonal component: sk = wk − 1T
T∑i=1
wi , k = 1 . . .T .
st = st−[ t−1T ]T , t > T .
deseasonalized data dt = xt − st .
mt trend component on deseasonalized data.
Yt = xt − mt − st random component.
Otherwise, difference data: ∇TXt := Xt − Xt−T .
∇TXt are de-seasonalized; then a trend can be eliminated from these.
14 settembre 2014 26 / 31
Examples of time series
De-trend and de-seasonalize (period T = 2q)
yearly average: mt =1
T
(12xt−q +
q−1∑j=−(q−1)
xt+j + 12xt+q
).
seasonal deviation: wk =1
n
n−1∑j=0
(xjT+k −mjT+k) , k = 1 . . .T .
seasonal component: sk = wk − 1T
T∑i=1
wi , k = 1 . . .T .
st = st−[ t−1T ]T , t > T .
deseasonalized data dt = xt − st .
mt trend component on deseasonalized data.
Yt = xt − mt − st random component.
Otherwise, difference data: ∇TXt := Xt − Xt−T .
∇TXt are de-seasonalized; then a trend can be eliminated from these.
14 settembre 2014 26 / 31
Examples of time series
De-trend and de-seasonalize (period T = 2q)
yearly average: mt =1
T
(12xt−q +
q−1∑j=−(q−1)
xt+j + 12xt+q
).
seasonal deviation: wk =1
n
n−1∑j=0
(xjT+k −mjT+k) , k = 1 . . .T .
seasonal component: sk = wk − 1T
T∑i=1
wi , k = 1 . . .T .
st = st−[ t−1T ]T , t > T .
deseasonalized data dt = xt − st .
mt trend component on deseasonalized data.
Yt = xt − mt − st random component.
Otherwise, difference data: ∇TXt := Xt − Xt−T .
∇TXt are de-seasonalized; then a trend can be eliminated from these.
14 settembre 2014 26 / 31
Examples of time series
De-trend and de-seasonalize (period T = 2q)
yearly average: mt =1
T
(12xt−q +
q−1∑j=−(q−1)
xt+j + 12xt+q
).
seasonal deviation: wk =1
n
n−1∑j=0
(xjT+k −mjT+k) , k = 1 . . .T .
seasonal component: sk = wk − 1T
T∑i=1
wi , k = 1 . . .T .
st = st−[ t−1T ]T , t > T .
deseasonalized data dt = xt − st .
mt trend component on deseasonalized data.
Yt = xt − mt − st random component.
Otherwise, difference data: ∇TXt := Xt − Xt−T .
∇TXt are de-seasonalized; then a trend can be eliminated from these.
14 settembre 2014 26 / 31
Examples of time series
De-trend and de-seasonalize (period T = 2q)
yearly average: mt =1
T
(12xt−q +
q−1∑j=−(q−1)
xt+j + 12xt+q
).
seasonal deviation: wk =1
n
n−1∑j=0
(xjT+k −mjT+k) , k = 1 . . .T .
seasonal component: sk = wk − 1T
T∑i=1
wi , k = 1 . . .T .
st = st−[ t−1T ]T , t > T .
deseasonalized data dt = xt − st .
mt trend component on deseasonalized data.
Yt = xt − mt − st random component.
Otherwise, difference data: ∇TXt := Xt − Xt−T .
∇TXt are de-seasonalized; then a trend can be eliminated from these.
14 settembre 2014 26 / 31
Examples of time series
Autocovariance and autocorrelation functions
If a process Xt is stationary,
γ(h) := Cov(Xt ,Xt+h) is the Autocovariance function (ACVF).
Recall the correlation ρ(X ,Y ) =Cov(X ,Y )√V (X )V (Y )
.
For a stationary process V (Xt) = V (Xt+h) = γ(0). Hence
ρ(h) = ρ(Xt ,Xt+h) =γ(h)
γ(0)is the Autocorrelation function (ACF).
First properties of ACVF:
γ(h) = γ(−h) [stationarity =⇒ Cov(Xt ,Xt+h) = Cov(Xt−h,Xt)]
|γ(h)| ≤ γ(0) [as |ρ(X ,Y ) ≤ 1]
14 settembre 2014 27 / 31
Examples of time series
Autocovariance and autocorrelation functions
If a process Xt is stationary,
γ(h) := Cov(Xt ,Xt+h) is the Autocovariance function (ACVF).
Recall the correlation ρ(X ,Y ) =Cov(X ,Y )√V (X )V (Y )
.
For a stationary process V (Xt) = V (Xt+h) = γ(0). Hence
ρ(h) = ρ(Xt ,Xt+h) =γ(h)
γ(0)is the Autocorrelation function (ACF).
First properties of ACVF:
γ(h) = γ(−h) [stationarity =⇒ Cov(Xt ,Xt+h) = Cov(Xt−h,Xt)]
|γ(h)| ≤ γ(0) [as |ρ(X ,Y ) ≤ 1]
14 settembre 2014 27 / 31
Examples of time series
Autocovariance and autocorrelation functions
If a process Xt is stationary,
γ(h) := Cov(Xt ,Xt+h) is the Autocovariance function (ACVF).
Recall the correlation ρ(X ,Y ) =Cov(X ,Y )√V (X )V (Y )
.
For a stationary process V (Xt) = V (Xt+h) = γ(0). Hence
ρ(h) = ρ(Xt ,Xt+h) =γ(h)
γ(0)is the Autocorrelation function (ACF).
First properties of ACVF:
γ(h) = γ(−h) [stationarity =⇒ Cov(Xt ,Xt+h) = Cov(Xt−h,Xt)]
|γ(h)| ≤ γ(0) [as |ρ(X ,Y ) ≤ 1]
14 settembre 2014 27 / 31
Examples of time series
Autocovariance and autocorrelation functions
If a process Xt is stationary,
γ(h) := Cov(Xt ,Xt+h) is the Autocovariance function (ACVF).
Recall the correlation ρ(X ,Y ) =Cov(X ,Y )√V (X )V (Y )
.
For a stationary process V (Xt) = V (Xt+h) = γ(0). Hence
ρ(h) = ρ(Xt ,Xt+h) =γ(h)
γ(0)is the Autocorrelation function (ACF).
First properties of ACVF:
γ(h) = γ(−h) [stationarity =⇒ Cov(Xt ,Xt+h) = Cov(Xt−h,Xt)]
|γ(h)| ≤ γ(0) [as |ρ(X ,Y ) ≤ 1]
14 settembre 2014 27 / 31
Examples of time series
Autocovariance and autocorrelation functions
If a process Xt is stationary,
γ(h) := Cov(Xt ,Xt+h) is the Autocovariance function (ACVF).
Recall the correlation ρ(X ,Y ) =Cov(X ,Y )√V (X )V (Y )
.
For a stationary process V (Xt) = V (Xt+h) = γ(0). Hence
ρ(h) = ρ(Xt ,Xt+h) =γ(h)
γ(0)is the Autocorrelation function (ACF).
First properties of ACVF:
γ(h) = γ(−h) [stationarity =⇒ Cov(Xt ,Xt+h) = Cov(Xt−h,Xt)]
|γ(h)| ≤ γ(0) [as |ρ(X ,Y ) ≤ 1]
14 settembre 2014 27 / 31
Examples of time series
Simple stationary processes and their ACVF
IID(0, σ2): Xtt∈Z independent and identically distributed r. v. withE(Xt) = 0, V(Xt) = σ2: γ(0) = σ2, γ(h) = 0 for |h| > 0.
WN(0, σ2) [white noise] Xtt∈Z uncorrelated random variables withmean 0 and variance σ2: γ(0) = σ2, γ(h) = 0 for |h| > 0.
WN(0, σ2) need not be independent. For instance if Ztt∈Z are IID andN(0,1) [normal r.v.], then
Xt =
Zt t odd
(Z 2t−1 − 1)/
√2 t even
is WN(0, 1) but not IID(0, 1).
It is not IID, since (e.g.) X1 and X2 are obviously not independent. Leftfor exercise that Xt is WN.
Less contrived examples of Xtt∈Z WN but not IID will be seen later inthe course.
14 settembre 2014 28 / 31
Examples of time series
Simple stationary processes and their ACVF
IID(0, σ2): Xtt∈Z independent and identically distributed r. v. withE(Xt) = 0, V(Xt) = σ2: γ(0) = σ2, γ(h) = 0 for |h| > 0.
WN(0, σ2) [white noise] Xtt∈Z uncorrelated random variables withmean 0 and variance σ2: γ(0) = σ2, γ(h) = 0 for |h| > 0.
WN(0, σ2) need not be independent. For instance if Ztt∈Z are IID andN(0,1) [normal r.v.], then
Xt =
Zt t odd
(Z 2t−1 − 1)/
√2 t even
is WN(0, 1) but not IID(0, 1).
It is not IID, since (e.g.) X1 and X2 are obviously not independent. Leftfor exercise that Xt is WN.
Less contrived examples of Xtt∈Z WN but not IID will be seen later inthe course.
14 settembre 2014 28 / 31
Examples of time series
Simple stationary processes and their ACVF
IID(0, σ2): Xtt∈Z independent and identically distributed r. v. withE(Xt) = 0, V(Xt) = σ2: γ(0) = σ2, γ(h) = 0 for |h| > 0.
WN(0, σ2) [white noise] Xtt∈Z uncorrelated random variables withmean 0 and variance σ2: γ(0) = σ2, γ(h) = 0 for |h| > 0.
WN(0, σ2) need not be independent. For instance if Ztt∈Z are IID andN(0,1) [normal r.v.], then
Xt =
Zt t odd
(Z 2t−1 − 1)/
√2 t even
is WN(0, 1) but not IID(0, 1).
It is not IID, since (e.g.) X1 and X2 are obviously not independent. Leftfor exercise that Xt is WN.
Less contrived examples of Xtt∈Z WN but not IID will be seen later inthe course.
14 settembre 2014 28 / 31
Examples of time series
Simple stationary processes and their ACVF
IID(0, σ2): Xtt∈Z independent and identically distributed r. v. withE(Xt) = 0, V(Xt) = σ2: γ(0) = σ2, γ(h) = 0 for |h| > 0.
WN(0, σ2) [white noise] Xtt∈Z uncorrelated random variables withmean 0 and variance σ2: γ(0) = σ2, γ(h) = 0 for |h| > 0.
WN(0, σ2) need not be independent. For instance if Ztt∈Z are IID andN(0,1) [normal r.v.], then
Xt =
Zt t odd
(Z 2t−1 − 1)/
√2 t even
is WN(0, 1) but not IID(0, 1).
It is not IID, since (e.g.) X1 and X2 are obviously not independent. Leftfor exercise that Xt is WN.
Less contrived examples of Xtt∈Z WN but not IID will be seen later inthe course.
14 settembre 2014 28 / 31
Examples of time series
Moving average processes and their ACVF . 2
MA(1): moving average Xtt∈Z is MA(1) if
Xt = Zt + ϑZt−1, t ∈ Z where ϑ ∈ R, Zt ∼WN(0, σ2).
A simple computation:
γ(0) = σ2(1 + ϑ2), γ(1) = ϑσ2, γ(h) = 0 for |h| > 1.
Similarly Xtt∈Z ∼ MA(q) if
Xt = Zt + ϑ1Zt−1 + · · ·ϑqZt−q, t ∈ Z,with ϑ1, . . . , ϑq ∈ R, Zt ∼WN(0, σ2).
Another simple computation leads to γ(h) = 0 for |h| > q.
14 settembre 2014 29 / 31
Examples of time series
Moving average processes and their ACVF . 2
MA(1): moving average Xtt∈Z is MA(1) if
Xt = Zt + ϑZt−1, t ∈ Z where ϑ ∈ R, Zt ∼WN(0, σ2).
A simple computation:
γ(0) = σ2(1 + ϑ2), γ(1) = ϑσ2, γ(h) = 0 for |h| > 1.
Similarly Xtt∈Z ∼ MA(q) if
Xt = Zt + ϑ1Zt−1 + · · ·ϑqZt−q, t ∈ Z,with ϑ1, . . . , ϑq ∈ R, Zt ∼WN(0, σ2).
Another simple computation leads to γ(h) = 0 for |h| > q.
14 settembre 2014 29 / 31
Examples of time series
AutoRegressive processes
AR(1) [AutoRegressive] Xtt∈Z is AR(1) if is stationary and
Xt = φXt−1 + Zt , t ∈ Z where φ ∈ R, Zt ∼WN(0, σ2). (1)
(1) is an (infinite set of) equation. It is not obvious that a stationaryprocess exists satisfying them (this will be discussed later).We are not saying Xtt∈N is the Markov chain defined throughXt = φXt−1 + Zt , t > 0 with X0 some prescribed r.v.
Now, assume a stationary process Xtt∈Z exists satisfying (1) andE(XtZs) = 0 for t < s (this latter property seems natural as Xt should bedefined in terms of Zt and the previous ones).
Then γ(0) = V(Xt) = E((φXt−1 + Zt)2)
= φ2V(Xt−1) + σ2 + 2φE(Xt−1Zt) = φ2γ(0) + σ2.
Hence γ(0) =σ2
1− φ2(makes sense only if φ2 < 1 ).
14 settembre 2014 30 / 31
Examples of time series
AutoRegressive processes
AR(1) [AutoRegressive] Xtt∈Z is AR(1) if is stationary and
Xt = φXt−1 + Zt , t ∈ Z where φ ∈ R, Zt ∼WN(0, σ2). (1)
(1) is an (infinite set of) equation. It is not obvious that a stationaryprocess exists satisfying them (this will be discussed later).
We are not saying Xtt∈N is the Markov chain defined throughXt = φXt−1 + Zt , t > 0 with X0 some prescribed r.v.
Now, assume a stationary process Xtt∈Z exists satisfying (1) andE(XtZs) = 0 for t < s (this latter property seems natural as Xt should bedefined in terms of Zt and the previous ones).
Then γ(0) = V(Xt) = E((φXt−1 + Zt)2)
= φ2V(Xt−1) + σ2 + 2φE(Xt−1Zt) = φ2γ(0) + σ2.
Hence γ(0) =σ2
1− φ2(makes sense only if φ2 < 1 ).
14 settembre 2014 30 / 31
Examples of time series
AutoRegressive processes
AR(1) [AutoRegressive] Xtt∈Z is AR(1) if is stationary and
Xt = φXt−1 + Zt , t ∈ Z where φ ∈ R, Zt ∼WN(0, σ2). (1)
(1) is an (infinite set of) equation. It is not obvious that a stationaryprocess exists satisfying them (this will be discussed later).We are not saying Xtt∈N is the Markov chain defined throughXt = φXt−1 + Zt , t > 0 with X0 some prescribed r.v.
Now, assume a stationary process Xtt∈Z exists satisfying (1) andE(XtZs) = 0 for t < s (this latter property seems natural as Xt should bedefined in terms of Zt and the previous ones).
Then γ(0) = V(Xt) = E((φXt−1 + Zt)2)
= φ2V(Xt−1) + σ2 + 2φE(Xt−1Zt) = φ2γ(0) + σ2.
Hence γ(0) =σ2
1− φ2(makes sense only if φ2 < 1 ).
14 settembre 2014 30 / 31
Examples of time series
AutoRegressive processes
AR(1) [AutoRegressive] Xtt∈Z is AR(1) if is stationary and
Xt = φXt−1 + Zt , t ∈ Z where φ ∈ R, Zt ∼WN(0, σ2). (1)
(1) is an (infinite set of) equation. It is not obvious that a stationaryprocess exists satisfying them (this will be discussed later).We are not saying Xtt∈N is the Markov chain defined throughXt = φXt−1 + Zt , t > 0 with X0 some prescribed r.v.
Now, assume a stationary process Xtt∈Z exists satisfying (1) andE(XtZs) = 0 for t < s (this latter property seems natural as Xt should bedefined in terms of Zt and the previous ones).
Then γ(0) = V(Xt) = E((φXt−1 + Zt)2)
= φ2V(Xt−1) + σ2 + 2φE(Xt−1Zt) = φ2γ(0) + σ2.
Hence γ(0) =σ2
1− φ2(makes sense only if φ2 < 1 ).
14 settembre 2014 30 / 31
Examples of time series
AutoRegressive processes
AR(1) [AutoRegressive] Xtt∈Z is AR(1) if is stationary and
Xt = φXt−1 + Zt , t ∈ Z where φ ∈ R, Zt ∼WN(0, σ2). (1)
(1) is an (infinite set of) equation. It is not obvious that a stationaryprocess exists satisfying them (this will be discussed later).We are not saying Xtt∈N is the Markov chain defined throughXt = φXt−1 + Zt , t > 0 with X0 some prescribed r.v.
Now, assume a stationary process Xtt∈Z exists satisfying (1) andE(XtZs) = 0 for t < s (this latter property seems natural as Xt should bedefined in terms of Zt and the previous ones).
Then γ(0) = V(Xt) = E((φXt−1 + Zt)2)
= φ2V(Xt−1) + σ2 + 2φE(Xt−1Zt) = φ2γ(0) + σ2.
Hence γ(0) =σ2
1− φ2(makes sense only if φ2 < 1 ).
14 settembre 2014 30 / 31
Examples of time series
AutoRegressive processes. 2
Remarks: we have found φ2 < 1 ⇐⇒ |φ| < 1 as a necessary conditionfor an AR(1) satisfying E(XtZs) = 0 for t < s. It will also be sufficient.
Implicit assumption in the computations: E(Xt) = 0 (this can be provedanalogously).
More simply, one can then compute γ(h) for h > 0 (left for exercise).
14 settembre 2014 31 / 31
top related