trend analysis: methodology victor shatalov meteorological synthesizing centre east
Post on 22-Dec-2015
221 Views
Preview:
TRANSCRIPT
Trend analysis: methodology
Victor Shatalov
Meteorological Synthesizing Centre East
TFMM trend analysis workshop, 17-18 November 2014
Main topics
Trend analysis of annual averages of concentration/deposition fluxes
Trend analysis of monthly averages (with seasonal variations)
TFMM trend analysis workshop, 17-18 November 2014
B[a]P concentrations in Germany
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
ng
/m3
Trend analysis: generalities
Aim: investigation of general tendencies in time series such as:
Measured and calculated pollutant concentrations at monitoring sites
Average concentrations/deposition fluxes in EMEP countries
…Method: trend analysis – decomposition of the considered series into regular component (trend) and random component (residue)
B[a]P concentrations in Germany
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
ng
/m3
TrendResidue
Residue (random component)
-0.2
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
ng
/m3
TFMM trend analysis workshop, 17-18 November 2014
Main steps
Detection of trend and its character: increasing decreasing mixed
Identification of trend type: linear quadratic exponential other
Quantification of trend: total reduction annual reduction magnitude of seasonal variations magnitude of random component other
Interpretation of the obtained results
Presentation by Markus Wallasch, 15 TFMM meeting, April 2014
TFMM trend analysis workshop, 17-18 November 2014
Determination of trend existence
B[a]P measurements: SE12
0.00
0.02
0.04
0.06
0.08
0.10
0.12
0.14
19
95
19
96
19
97
19
98
19
99
20
00
20
01
20
02
20
03
20
04
20
05
20
06
20
07
20
08
20
09
20
10
ng/m
3
Decreasing pair
Increasing pair
Mann-Kendall test:Z = (number of increasing pairs) – (number of decreasing pairs) with normalization.Critical values: ± 1.44 at 85% level
± 1.65 at 90% level± 1.96 at 95% level
Z = - 1.49Decreasing trend at 85% significance level
B[a]P concentrations in Germany
0
0.2
0.4
0.6
0.8
1
1.2
19
90
19
92
19
94
19
96
19
98
20
00
20
02
20
04
20
06
20
08
20
10
ng
/m3
Z = - 4.05 Z = 1.8Mixed trend character:
In the period from 1990 to 2000 – statistically significant (at 95% level) decreasing trend
In the period from 2004 to 2010 – statistically significant (at 90% level) increasing trend
Typical situation for HMs and POPs
TFMM trend analysis workshop, 17-18 November 2014
Random component
-0.3
-0.2
-0.1
0.0
0.1
0.2
0.3
1990
1992
1994
1996
1998
2000
2002
2004
2006
2008
2010
ng
/m3
B[a]P concentrations in Germany
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1990
1992
1994
1996
1998
2000
2002
2004
2006
2008
2010
ng
/m3
Calculations
Linear trend
Determination of trend type: linear trend
Conc = A · Time + B + ω Calculation of A and B:regression or Sen’s slope
Z = - 3.1 decrease
Z = 3.8 increase
Residual trend exists
Criterion of the choice of trend type: Mann-Kendall test should not show statistically significant trend on all sub-periods of the time series
ω – residues (random component)
TFMM trend analysis workshop, 17-18 November 2014
B[a]P concentrations in Germany
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
ng
/m3
Air concentrations
Trend
Criterion of non-linearityCriterion of non-linearity of the obtained trend in time:
NL = max[abs(Δi /Cichord)] · 100%
Supposed threshold value: 10%
C ichord
Δ i
i
Chord
B[a]P concentrations in Finland
0.0000.0100.020
0.0300.0400.0500.0600.070
0.0800.0900.100
19
90
19
91
19
92
19
93
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
20
02
20
03
20
04
20
05
20
06
20
07
20
08
20
09
20
10
ng
/m3
Air concentrations Trend
NL = 15.6%
Non-linear trend
B[a]P concentrations in Belgium
0.000
0.100
0.200
0.300
0.400
0.500
0.600
0.700
19
90
19
91
19
92
19
93
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
20
02
20
03
20
04
20
05
20
06
20
07
20
08
20
09
20
10
ng/m
3
Air concentrations Trend
NL = 8.1%
Linear trend
Fraction of non-linear trends
Heavy metals (Pb) 87%
POPs (B[a]P) 62%
TFMM trend analysis workshop, 17-18 November 2014
Residual (random component)
-0.2
-0.1
0.0
0.1
0.2
0.3
1990
1992
1994
1996
1998
2000
2002
2004
2006
2008
2010
ng
/m3
Determination of trend type: mono-exponential trend
Conc = A · exp(- Time / ) + ω,
– characteristic time
Z = - 3.3 Z = 3.2 decrease increase
Residual trend exists
Calculation of A and :least square method
B[a]P concentrations in Germany
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1990
1992
1994
1996
1998
2000
2002
2004
2006
2008
2010
ng
/m3
Calculations
Exponential trend
TFMM trend analysis workshop, 17-18 November 2014
Random component
-0.2
-0.1
0.0
0.1
0.2
1990
1992
1994
1996
1998
2000
2002
2004
2006
2008
2010
ng
/m3
Determination of trend type: polynomial trend
Conc = A · Time2 + B · Time + C + ω
Z = 0.5 Z = -2.3 no trend decrease
Residual trend exists
Calculation of A, B and C:least square method
B[a]P concentrations in Germany
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1990
1992
1994
1996
1998
2000
2002
2004
2006
2008
2010
ng
/m3
Calculations
Polynomial trend
TFMM trend analysis workshop, 17-18 November 2014
Residual (random component)
-0.2
-0.1
0.0
0.1
0.2
1990
1992
1994
1996
1998
2000
2002
2004
2006
2008
2010
ng
/m3
B[a]P concentrations in Germany
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1990
1992
1994
1996
1998
2000
2002
2004
2006
2008
2010
ng
/m3
Calculations
Bi-exponential trend
Determination of trend type: bi-exponential trend
Calculated byleast square method
Z = 0 Z = -1.4 no trend no trend
See [Smith, 2002]
Conc = A1 · exp(- Time /1) + A2 · exp(- Time /2)
Ai – amplitudes, i – characteristic times
No statistically significant residual trend obtained
TFMM trend analysis workshop, 17-18 November 2014
B[a]P concentrations in Germany
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1990
1992
1994
1996
1998
2000
2002
2004
2006
2008
2010
ng
/m3
Calculations
Bi-exponential trend
Statistical significance of increasing trend
Z = 1.8
Mann-Kendall test for 2004 – 2010:
does not confirm statistically significant increasing trend
does not claim the absence of increasing trend
Confidence interval for trend slope:
[TS0 + A, TS0 + B]
TS0 – slope of calculated trend
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
1990
1992
1994
1996
1998
2000
2002
2004
2006
2008
2010
[A, B] – confidence interval for slope of random component
Typical situation for B[a]P: increase in the end of the period
0
0.2
0.4
0.6
0.8
1
1.2
1990
1992
1994
1996
1998
2000
2002
2004
2006
2008
2010
ng
/m3
0
0.2
0.4
0.6
0.8
1
1.2
1990
1992
1994
1996
1998
2000
2002
2004
2006
2008
2010
ng
/m3
Increase is statistically significant
TFMM trend analysis workshop, 17-18 November 2014
Non-linear trend analysis
Conc = A1 · exp(- Year / 1) + A2 · exp(- Year / 2) + ω
Regression model, non-linear in the parameters 1 and 2
Non-linear regression models are widely investigated, for example:
Nonlinear regression, Gordon K. Smith, in Encyclopedia on Environmetrics, ISBN 0471899976, Wiley&Sons, 2002, vol 3, pp. 1405 – 1411
Estimating and Validating Nonlinear Regression Metamodels in Simulation, I. R. dos Santos and A. M. O. Porta Nova, Communications in Statistics, Simulation and Computation, 2007, vol. 36: pp. 123 – 137
Nonlinear regression, G. A. F. Seber and C. J. Wild, Wiley-Interscience, 2003
TFMM trend analysis workshop, 17-18 November 2014
B[a]P concentrations in Germany
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
ng
/m3
Calculations
Bi-exponential trend
Parameters for trend characterization: reduction/growth
ΔCi
Negative values of reduction mean growth
Relative annual reductionRi = ΔCi / Ci = (1 – Ci+1 / Ci)
Total reduction per periodRtot = (Сbeg–Cend)/Cbeg=1–Cend/Cbeg
Average annual reductionRav = 1 – (Cend / Cbeg) 1/(N-1)
where N – number of years
Reduction parametersRmin = min (Ri)Rmax = max (Ri)Rav
Rtot
For the considered example:Rmin = - 6% (growth)Rmax = 15%Rav = 6%Rtot = 69%
Cbeg
Cend
TFMM trend analysis workshop, 17-18 November 2014
B[a]P concentrations in Germany
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
ng
/m3
Calculations
Bi-exponential trend
Δ
Residue (random component)
-0.2
-0.1
0.0
0.1
0.2
0.3
0.4
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
ng
/m3
Parameters for trend characterization: random component
Parameter: standard deviation of random component normalized by trend values Frand = σ(Δ/Ctrend)
-20%
-10%
0%
10%
20%
30%
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
Normalized random component
For the considered example: Frand = 11%
Frand
TFMM trend analysis workshop, 17-18 November 2014
Seasonal variations of pollution
Seasonal variations are characteristic of heavy metals and (particularly)
for POPs
B[a]P concentrations measured at EMEP site CZ3 from 1996 to 2010. Pronounced seasonal variations are seen.
B[a]P concentrations at CZ3 site
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
19
96
19
97
19
98
19
99
20
00
20
01
20
02
20
03
20
04
20
05
20
06
20
07
20
08
20
09
20
10
ng
/m3
Pb concentrations measured at EMEP site DE7 from 1990 to 2008. Seasonal variations are also seen.
Pb concentrations at DE7 site
0
10
20
30
40
50
60
70
19
90
19
91
19
92
19
93
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
20
02
20
03
20
04
20
05
20
06
20
07
20
08
ng
/m3
TFMM trend analysis workshop, 17-18 November 2014 Possible approaches to description of seasonal
variations
t – time
– chatracteristic times,
A, B – constants,
φ – phase shifts.
Bi-exponential approximation
Mono-exponential approximation *)
Conc = A · exp(– t / + B · cos(2 · t – φ))
or
Log(Conc) = A’ – t / + B · cos(2 · t – φ)
*) Kong et al., Statistical analysis of long-term monitoring data… Environ. Sci. Techn., 10/2014
Conc = A1 · exp(– t / 1) · (1 + B1 · cos(2 · t – φ1)) + A2 · exp(– t / 2) · (1 + B2 · cos(2 · t – φ2))
TFMM trend analysis workshop, 17-18 November 2014
Usage of higher harmonics
Trend calculated by bi-exponential approach. Possible artifact: negative trend values
Measurement data at CZ3 from 1996 to 2010
Statistical significance of second harmonic: Fisher’s test F
-0.5
0
0.5
1
1.5
2
2.5
3
3.5
19
96
19
97
19
98
19
99
20
00
20
01
20
02
20
03
20
04
20
05
20
06
20
07
20
08
20
09
20
10
ng/m
3
Measurements trend standard
Possibility to avoid negative values: usage of higher harmonicsConc = Tr1 + Tr2 , Tri = Ai·exp(– t / i)·(1+Bi·cos(2·t–φi)+Ci·cos(4·t–ψi))
0
0.5
1
1.5
2
2.5
3
3.5
19
96
19
97
19
98
19
99
20
00
20
01
20
02
20
03
20
04
20
05
20
06
20
07
20
08
20
09
20
10
ng
/m3
Measurements two harmonics
TFMM trend analysis workshop, 17-18 November 2014
One harmonic
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
19
90
19
91
19
92
19
93
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
20
02
20
03
20
04
20
05
20
06
20
07
20
08
20
09
20
10
ng
/m3
Calculations Trend Main component
Average B[a]P concentrations in Europe from 1990 to 2010 (main harmonic only)
Poor approximation for small values of concentrations
Pronounced harmonic trend with doubled frequency
Residues for one-harmonic approximationResidues, one harmonic
-0.06
-0.04
-0.02
0
0.02
0.04
0.06
0.08
0.1
19
90
19
91
19
92
19
93
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
20
02
20
03
20
04
20
05
20
06
20
07
20
08
20
09
20
10
ng
/m3
Usage of higher harmonics
TFMM trend analysis workshop, 17-18 November 2014
One harmonic
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
19
90
19
91
19
92
19
93
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
20
02
20
03
20
04
20
05
20
06
20
07
20
08
20
09
20
10
ng
/m3
Calculations Trend Main component
Significance of second harmonic is confirmed by Fisher’s test
Average B[a]P concentrations in Europe from 1990 to 2010 (main harmonic only)
Poor approximation for small values of concentrations
Trend including two harmonics
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
19
90
19
91
19
92
19
93
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
20
02
20
03
20
04
20
05
20
06
20
07
20
08
20
09
20
10
ng/m
3
Calculations Trend Main component
Usage of higher harmonics
TFMM trend analysis workshop, 17-18 November 2014
Full trend
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
19
90
19
91
19
92
19
93
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
20
02
20
03
20
04
20
05
20
06
20
07
20
08
20
09
20
10
ng
/m3
Concentrations
Trend
Main component
Splitting trends to particular componentsExample: average B[a]P concentrations for Germany from 1990 to 2010.
Main component
0
0.2
0.4
0.6
0.8
1
1.2
19
90
19
91
19
92
19
93
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
20
02
20
03
20
04
20
05
20
06
20
07
20
08
20
09
20
10
ng/m
3 Cmain
Ctot = Cmain + Cseas + Crand
Seasonal component
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
19
90
19
91
19
92
19
93
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
20
02
20
03
20
04
20
05
20
06
20
07
20
08
20
09
20
10
ng
/m3
Full
trend
CseasRelative annual reductions (as above): Rmin, Rmax, Rav, Rtot
Ctot
Random component
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
19
90
19
91
19
92
19
93
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
20
02
20
03
20
04
20
05
20
06
20
07
20
08
20
09
20
10
ng
/m3
Crand
TFMM trend analysis workshop, 17-18 November 2014
Full trend
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
19
90
19
91
19
92
19
93
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
20
02
20
03
20
04
20
05
20
06
20
07
20
08
20
09
20
10
ng
/m3
Concentrations
Trend
Main component
Splitting trends to particular componentsExample: average B[a]P concentrations for Germany from 1990 to 2010.
Main component
0
0.2
0.4
0.6
0.8
1
1.2
19
90
19
91
19
92
19
93
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
20
02
20
03
20
04
20
05
20
06
20
07
20
08
20
09
20
10
ng/m
3 Cmain
Ctot = Cmain + Cseas + Crand
Seasonal component
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
19
90
19
91
19
92
19
93
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
20
02
20
03
20
04
20
05
20
06
20
07
20
08
20
09
20
10
ng
/m3
Full
trend
Cseas
Ctot
Random component
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
19
90
19
91
19
92
19
93
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
20
02
20
03
20
04
20
05
20
06
20
07
20
08
20
09
20
10
ng
/m3
Crand
Seasonal component, normalized
-100%
-80%
-60%
-40%
-20%
0%
20%
40%
60%
80%
100%
19
90
19
91
19
92
19
93
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
20
02
20
03
20
04
20
05
20
06
20
07
20
08
20
09
20
10
Normalization: Cseas/Cmain
Average value of the annual amplitude of the normalized seasonal component Fseas
Threshold value: 10%
Fraction of trends with essential seasonality
Heavy metals (Pb) 93%
POPs (B[a]P) 100%
TFMM trend analysis workshop, 17-18 November 2014
Full trend
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
19
90
19
91
19
92
19
93
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
20
02
20
03
20
04
20
05
20
06
20
07
20
08
20
09
20
10
ng
/m3
Concentrations
Trend
Main component
Splitting trends to particular componentsExample: average B[a]P concentrations for Germany from 1990 to 2010.
Main component
0
0.2
0.4
0.6
0.8
1
1.2
19
90
19
91
19
92
19
93
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
20
02
20
03
20
04
20
05
20
06
20
07
20
08
20
09
20
10
ng/m
3 Cmain
Ctot = Cmain + Cseas + Crand
Seasonal component
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
19
90
19
91
19
92
19
93
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
20
02
20
03
20
04
20
05
20
06
20
07
20
08
20
09
20
10
ng
/m3
Full
trend
Cseas
Ctot
Random component
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
19
90
19
91
19
92
19
93
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
20
02
20
03
20
04
20
05
20
06
20
07
20
08
20
09
20
10
ng
/m3
Crand
Random component, normalized
-100%-80%-60%-40%-20%
0%20%40%60%80%
100%
19
90
19
91
19
92
19
93
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
20
02
20
03
20
04
20
05
20
06
20
07
20
08
20
09
20
10
Normalization: Crand/Cmain
Standard deviation of normalized random component Frand
TFMM trend analysis workshop, 17-18 November 2014
Phase shift as a fingerprint of source type
Trends for PB concentrations at CZ1
0
2
4
6
8
10
12
14
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
Air
conc
entr
atio
ns, n
g/m
3
Anthropogenic
Secondary
Δφ
Difference Δφ of phase shift φ between Pb pollution at CZ1 location due to anthropogenic and secondary sources.
Phase shift can be used to determine which source type (anthropogenic or secondary) mainly contributes to the pollution at given location (in a particular country).
TFMM trend analysis workshop, 17-18 November 2014
List of trend parametersParameters for trend characterization:
Relative reduction over the whole period (Rtot),
Relative annual reductions of contamination:
average over the period (Rav),
maximum (Rmax),
minimum (Rmin).
Relative contribution of seasonal variability (Fseas).
Relative contribution of random component (Frand).
Phase shift of maximum values of contamination with respect to the beginning of the year (φ).
Statistical tests:
Non-linearity parameter (NL) 10%
Relative contribution of seasonal variability (Fseas) 10%
top related