mos forecasts: the role of model and initial condition errorsdswc09/contributions/... · two...
TRANSCRIPT
-
The correction of forecast errorsbased on MOS: The impact of
initial condition and model errors
S. Vannitsem
Institut Royal Météorologique de Belgique
Dresden, July 30, 2009
-
Outline
1. Introduction2. The MOS technique3. The role of initial and model errors: a dynamical view4. Analysis of operational forecasts5. A unified MOS scheme for deterministic AND ensemble forecasts:
EVMOS6. Conclusions and perspectives
-
The quality of atmospheric forecasts decreases as a function of time
NCEP ensemble forecasts
-
Model error dynamics
Eta model runs withTwo different convectiveschemes
0.0001
0.001
0.01
0.1
1
10
0 2 4 6 8 10 12
Mea
n sq
uare
err
or
Time (hours)
Kain-Fritch vs Betts-Miller-Janjic convective scheme
U 500 hPaV 500 hPaU 850 hPaV 850 hPalinearquadratic
-
Two sources of errors affecting the forecasts and the data assimilation process:
-initial condition errors-model errors
See C. Nicolis, Rui A. P. Perdigao, and S. Vannitsem Dynamics of Prediction Errors under the Combined Effect of Initial Condition and Model Errors, Journal of the Atmospheric Sciences , 66, 766–778, 2009.
How can we improve the forecast quality (Besides the improvement of the modeland the observational system)?
Use of techniques to post-process the forecasts in order to improve their quality.
What is corrected by the post-processing method?
-
The Model Output Statistics technique
• ‘Deterministic’ approach: Linear techniques (Classical MOS, perfectprog), adaptative Kalman filtering. Ref: Glahn and Lowry, 1972, JAM, 11, 1203; Wilks, 2006, Academic Press;….
• ‘Deterministic’ approach: Nonlinear techniques (Neural networks, nonlinear fits). Ref: Marzban, 2003, MWR, 131, 1103; Casaioli et al, 2003, NPG, 10, 373;….
• Probabilistic approaches: correction of the (deterministic) precipitationforecasts. Ref: Lemcke and Kruizinga, 1988, MWR, 116, 1077;….
• (True) Probabilistic approaches: Calibration of the ensemble forecasts. Ref: Roulston and Smith, 2003, Tellus, 55A, 16; Gneiting et al, 2005, MWR, 133, 1098; Wilks, 2006, MA, 13, 243; Hamill and Wilks, 2007, MWR, 135, 2379;…….
Post-processing of forecasts in order to improve their qualitybased on information gathered from past forecasts
-
The Linear MOS technique
Linear regression between a set of observables of forecasts and observations
∑=
+=n
iiic tVtttX
1
)()()()( βα
∑=
−=M
kkkc tXtXtJ
1
2, ))()(()( M past forecasts
n=1
22 )t(V)t(V)t(V)t(X)t(V)t(X)t(
)t(V)t()t(X)t(
><>>=
-
Example of fit:
Training:
2 Summer seasons2002-2003
Verification:
2004-2005280
285
290
295
300
280 285 290 295 300
Lead time: 6 hours
Obs
erva
tion,
Cor
rect
ed fo
reca
sts
Forecasts
280
285
290
295
300
305
310
280 285 290 295 300 305
lead time: 60 hours
Obs
erva
tins,
Cor
rect
ed fo
reca
sts
Forecasts
280
285
290
295
300
275 280 285 290 295 300
Lead time: 120 hours
Obs
erva
tions
, Cor
rect
ed fo
reca
sts
Forecasts
2-m Temperature, Uccle
-
The mean square error
Properties of the MOS forecast
2)(
2222
)(
222 )())(),(())()(()(
)()(
tVXXtV
cc
c
ttVtXCtXtXt
tXtX
σβσσσ
σ =>=>>=−>=
-
Short term dynamics of MOS forecasts (Formalism based on Nicolis, 2004, JAS)
The real system{ }
{ }),,(
),,(
λ
μ
YXSdtYd
YXFdtXd
rrrr
rrrr
=
=
Xr
Vr
and span the same phase space
Variablesnot describedby the model
{ })',V(GdtVd
μ=rr
r
The model
{ }
{ })),(),(()0()(
)'),(()0()(
0
0
∫
∫
+=
+=
t
t
YXFdXtX
VGdVtV
μτττ
μττ
rrrrr
rrrrFormal sols
-
)(21))()(( 32210
2 tOtStSStt VC +++=−σσ
Typical initial errors
)0()0(X)0(V ε+= rrr Gaus random noise
+Systematic error
22
2)0(1
2)0(
2)0(
4)0(
0
))0(())0((),0((2
)))0(())0((),0((2
XFXGVCS
XFXGVCS
SX
rr
rr
−∝
−∝
+=
ε
ε
ε
σ
σσσ
>−>
-
))))0(())0((()()0()0((2)))0(())0(((2
)))0(())0(()()0()0((2
))0()0((
0
2'2
'1
22'0
>⎥⎦⎤−−−=
-
Summary
A. Correction of initial condition errors
B. Correction of model errors
- The systematic part is corrected.
- The time-dependent part can be corrected provided thatthe covariance of the model error with the predictors is high.
- Systematic initial error is corrected.
- Random initial errors are only partially corrected.
2)0(
2)0(
4)0(
0X
Sσσ
σ
ε
ε
+=
-
The Lorenz system
zxzbxydtdz
Gybxzxydtdy
aFaxzydtdx
−+−=
+−−=
+−−−= 22
- Initial condition errors: N(0,s), s small- Model errors (parametric error on a, b, F or G)
Chaotic for a=0.25, b=6, F=16, G=3
Results obtained with 100,000 realizations starting from differentinitial conditions
-
The Lorenz system (continued)
10-7
10-6
10-5
10-4
10-3
10-2
10-1
100
101
2 4 6 8 10
Mea
n S
quar
e E
rror
Time
(a)
10-7
10-6
10-5
10-4
10-3
10-2
10-1
100
0.01 0.1 1
10-1610-1410-1210-1010-810-610-410-2100
0.01 0.1 1 10
Mea
n s
quar
e er
ror
Time
(a)
Initial cond
ition error o
nly
Model error
only
Error in parameter a
Correction of variable x
-
The Lorenz system (continued)
10-12
10-10
10-8
10-6
10-4
10-2
100
0.01 0.1 1 10
( Ym- Yt)2
( Yc- Yt)2, MOS X, Y
( Yc- Yt)2, MOS Y, XZ
Mea
n S
quar
e E
rror
Time
Partial conclusions
- IC not well corrected by MOS- MOS can correct model errors when model predictors well chosen
Model errors only in the parameter b
Correction of variable y
Different MOS schemes
- Predictors: X, Y- Predictors: Y, XZ
-
Application to the ECMWF forecasting system: temperature
Data: temperature for the period December 1 2001 to November 30 2005
- Training period: December 1 2001 to November 30 2003
- Independent evaluation period: December 1 2003 to November 30 2005
-Temperature 500 hPa, 850 hPa-Temperature at 2 meters
Evaluation on a grid covering Belgium (verification using the set of analyses)
(52 N, 2 E) to (49 N, 7 E)
Evaluation at some synoptic stations: model forecast given by the closestgridpoint
-
0
1
2
3
4
5
6
0 20 40 60 80 100 120
Raw forecastsMOS, T 850 hPa
MS
E
Time (hours)
Temperature at 850 hPa
Results averaged over the wholegrid
0
1
2
3
4
5
6
7
0 20 40 60 80 100 120
Corrected forecastsRaw forecasts
RM
SE
Time (hours)
(a)
2002-2003
Temperature at 2 meters
0
1
2
3
4
5
6
0 20 40 60 80 100 120
Corrected forecasts
Raw forecasts
RM
SE
Time (hours)
(c)
2004-2005
-
1
2
3
4
5
6
7
0 20 40 60 80 100 120
Raw forecastsMOS, T2mMOS, T2m, T850hPaMOS, T2m, SH
MS
E
Time (hours)
(a)
Uccle
Uccle-Ukkel
1
2
3
4
5
6
7
8
9
0 20 40 60 80 100 120
Raw forecastsMOS, T2m
MS
E
Time (hours)
(a)
Uccle
0
0.2
0.4
0.6
0.8
1
1.2
1.4
0 20 40 60 80 100 120
V CD C
Cor
rect
ion
s
Time (hours)
(b)Uccle
Training + verif on 2002-2003
Training: 2002-2003Analysis on 2004-2005
-
Other stations
1
2
3
4
5
6
7
8
0 20 40 60 80 100 120
Raw forecastsMOS, T2m
MS
E
Time (hours)
(a)
Middelkerke
0
2
4
6
8
10
12
0 20 40 60 80 100 120
Raw forecastsMOS, T2m
MS
E
Time (hours)
(a)
Elsenborn
Training + verif on 2002-2003
A second predictor?
Sensible heat flux
A second predictor?
TMP 850 hPa
-
New Linear MOS
)()(
)()()()()(
)()()()(
2
2
22
22
tt
tVtVtXtXt
tVttXt
V
X
σσβ
βα
=><><
=
>=<
)(, tX kc
Intermediate cost function:
Minimization
∑= +
−+=
M
k
kk
ttttXtVtttJ
1222
2
)()()())())()()((()(
δκ σβσβα
κβςα ++=X δς +=V
One ‘free’ parameter: 2
2
2
2
tofixed X
V
σσ
σσλ
κ
δ=
∑∑==
+−+
−=
M
k
kkM
k
kk
tttX
tttVtJ
12
2
12
2
)()))(()((
)())()(()(
κδ σβςα
σς
Needs some knowledge aboutthe sources of errors
-
2)(
2)(
222 )())()(()(
)()(
tXtVCC
C
ttXtXt
tXtX
σσβσ =>=>>=>
-
268
270
272
274
276
278
280
282
284
286
288
268 270 272 274 276 278 280 282 284 286 288
Observations
Forecasts
Lead time: 24 hours, Initial time: 12
RAWEVMOSLMOS
268
270
272
274
276
278
280
282
284
286
288
268 270 272 274 276 278 280 282 284 286 288
Observations
Forecasts
Lead time: 120 hours, Initial time: 12
RAWEVMOSLMOS
265
270
275
280
285
290
266 268 270 272 274 276 278 280 282 284 286
Observations
Forecasts
Lead time: 240 hours, Initial time: 12
RAWEVMOSLMOS
Control Forecast is used.
Training: DJF 2002-2004
Verif: DJF 2002
MOS at Station ‘Uccle’.
-
0
0.5
1
1.5
2
2.5
3
3.5
4
0 50 100 150 200 250
RMSE, SPREAD
Time (hours)
(a)
DMO, RMSEEVMOS-1, RMSE
DMO, SPREADEVMOS-1, SPREAD
0
0.5
1
1.5
2
2.5
3
3.5
4
0 50 100 150 200 250
RMSE, SPREAD
Time (hours)
(b)
DMO, RMSENGR, RMSE
DMO, SPREADNGR, SPREAD
0
0.5
1
1.5
2
2.5
3
3.5
4
0 50 100 150 200 250
RMSE, SPREAD
Time (hours)
(c)
DMO, RMSEEVMOS-2, RMSE
DMO, SPREADEVMOS-2, SPREAD
0
0.5
1
1.5
2
2.5
3
3.5
4
0 50 100 150 200 250
RMSE, SPREAD
Time (hours)
(d)
DMO, RMSELMOS-2, RMSEDMO, SPREAD
LMOS-2, SPREAD
Application to an operational ensemble: ECMWF, Winter, Uccle
2 meter Temperature
-
Conclusions on MOS
A dynamical analysis of the MOS correction has been undertaken with emphasison the correction of
-initial condition errors-Model errors
Both partly corrected butcorrection more sensitiveto model errors.
)))0(())0((),0(( XFXGVCrr
−One central quantity
An additional model observable can improveconsiderably the correction if proportional to G-F
-
For the ECMWF deterministic forecast:
TMP 2m
- Substantial corrections with MOS with 1 observable - MOS with 2 observables:
In the central part of Belgium: Tmp 850 hPaOn the coast: Sensible Heat Flux
TMP 850 hPa
- No correction. Mainly initial condition errors
More infos:
Vannitsem S. and C. Nicolis, Dynamical properties of Model Output Statistics forecasts. Mon. Wea. Rev., 136, 405-419, 2008.
Vannitsem S., Dynamical properties of MOS forecasts. Analysis of the ECMWF operational forecasting system. Weather and Forecasting, 23, 1032-1043, 2008.
-
Correction of Ensemble forecasts based on EVMOS
A unified Scheme for both deterministic and ensemble forecasts is proposed,based on the assumption that the forecast displays errors.
-Do not damp the ensemble spread anymore.
-The correction provides a mean and a variabilityin agreement with the observations
-The scheme can be trained on the control forecast only.
References:
Vannitsem, S., A unified Linear Model Output Statistics scheme for both deterministic and ensemble forecasts. Accepted inQuart. J. Roy. Met. Soc., 2009
The correction of forecast errors based on MOS: The impact of initial condition and model errors The Model Output Statistics techniqueSummaryThe Lorenz systemThe Lorenz system (continued)New Linear MOS